Combine brown sugars, butter, cream, milk, corn syrup, and salt in a medium-sized saucepan over medium-low heat. Cook, stirring frequently, until the butter is melted and sugar has dissolved.

Increase heat to medium and bring mixture to a full boil, stirring constantly.

Once mixture reaches a full boil, set a timer for 4 minutes and stir occasionally while caramel cooks (or use a candy thermometer and cook until caramel sauce reaches 230F/110C).

Remove from heat and pour into a large heatproof bowl. Allow caramel to cool completely before proceeding.

Place softened butter in a large mixing bowl and use an electric mixer to beat until smooth and creamy.

With mixer on low-speed, gradually add about half of the powdered sugar until combined. Drizzle in caramel and stir until completely incorporated, then gradually add remaining powdered sugar as needed until frosting reaches desired consistency (you may not need all 4 cups).

Stir in vanilla extract.

Spread or pipe over cake or cupcakes, sprinkle with fine sea salt if desired.