How to Make It
Prepare the Cupcakes: Preheat oven to 350°F. Line a 12-cup standard muffin tin with paper liners. Beat butter and granulated sugar in a large bowl on medium speed with an electric mixer until light and fluffy, about 5 minutes. Add eggs, 1 at a time, beating well after each addition. Beat in vanilla.
Whisk together flour, baking powder, baking soda, and salt in a bowl. Starting and ending with flour, add flour mixture and sour cream in 5 additions, beating just until combined after each addition. Scoop half of batter into a separate bowl. Stir pumpkin and pumpkin pie spice into 1 bowl of batter; stir cocoa and milk into the second bowl.
Layer pumpkin and chocolate batters alternately to fill prepared muffin cups; swirl each with a wooden pick. Bake until a toothpick inserted in the center comes out clean, 20 to 22 minutes. Cool in pan on a wire rack 10 minutes. Transfer to a wire rack to cool completely, about 20 minutes.
While cupcakes are cooling, prepare the Icing: Using an electric mixer, beat cream cheese and butter on medium speed until smooth. Add cocoa, milk, vanilla, and salt and beat on low speed until smooth, about 3 minutes. Add powdered sugar and beat until smooth, about 3 minutes. If icing is too thick, beat in additional milk, 1 teaspoon at a time, to reach desired consistency. Top cooled cupcakes with icing.