At First
In The Beginning
expression
You know 初 means first, and に can mark a point in time. So 初めに means at first or in the beginning.
初めに has a lot of overlap with 始めに. But while 始めに is for the beginning of some larger process, 初めに refers more to doing something first or to the initial point of a certain time frame. You might say 初めにアクビをしたのは私です (I was the one who yawned first), for example, or 今年の初めに (at the start of this year).
This word uses the kun'yomi, which you didn't learn with the kanji, so here's a mnemonic to help you out. What are you beginning? You are beginning Haj (はじ). (Haj is the fifth pillar of Islam, and is a pilgrimage to Mecca during the month of Dhu al-Hijja).
花さんとは、4月の初めに出会いました。
I met Hana-san at the beginning of April.
うちは毎年、年の初めに家族みんなで朝日を見に行きます。
Our family goes to see the sunrise together at the beginning of each year.
初めに口を開いたのはカオルだった。
It was Kaoru who first opened her mouth.
初めになぐりかかってきたのはそっちだろ!
You're the one who started out with the punching bag!