Preheat the oven to 325℉. Spray a muffin pan with nonstick spray or line with paper liners.
In a large bowl, use an electric mixer to combine the butter and sugar until light and fluffy. Add the milk, eggs, lemon extract and lemon zest; beat well. Mix in the dry ingredients and beat just until combined.
Use an ice cream scoop or large spoon to divide the batter among the muffin cups, filling each about 3/4 full. Bake 16 to 20 minutes, until a toothpick inserted into the center comes out clean. Let the muffins cool for about 20 minutes, then remove the muffins from the tin and let them cool completely on a wire rack.
ADD THE GLAZE:
In a medium bowl, whisk together the powdered sugar and enough lemon juice to create a thick glaze. Place a piece of wax paper underneath the cooling rack. Drizzle the glaze over the tops of the muffins. It will run down the sides a little bit. Let the muffins sit until the glaze firms up. Enjoy! Keep extra muffins in a covered container for up to two days.