In a medium saucepan, whisk together the cornstarch and 2 tablespoons of milk until smooth; whisk in the rest of the milk, cream, sugar, cocoa and salt. Bring to a boil over medium-high, stirring constantly, and cook 1 minute.
Transfer to a large measuring cup and stir in the vanilla. Pour into 4 popsicle molds and insert sticks. Freeze until the fudgsicles are solid, about 6 hours (or up to 1 week). To get them out of the molds, turn the molds upside down and run them under hot water to loosen the fudgsicles from the molds and pull them out. Enjoy!