Heat oven to 350°F. Line two 12-cup muffin tins with paper liners. In a medium bowl, whisk together the flour, cocoa, baking powder, baking soda, and salt; set aside.
Using an electric mixer, beat the butter and sugar on medium-high until fluffy, 2 to 3 minutes. Beat in the vanilla, then the eggs one at a time, scraping down the sides of the bowl as necessary.
Reduce mixer speed to low. Add the flour mixture in 3 additions and the milk in 2 additions, beginning and ending with the flour mixture. Mix just until combined (do not overmix).
Divide the batter among the prepared muffin tins and bake until a toothpick inserted in the center of a cupcake comes out clean, 20 to 25 minutes. Let cool.
Spread the frosting on the cupcakes and sprinkle with the marshmallows.