In a small saucepan, bring 4 cups water to a boil. Remove from heat, add the tea bags, and let steep for 10 minutes; discard the bags and let the tea cool.
Serve over ice and top off each glass with 2 tablespoons sweetened condensed milk.