Melt butter in a medium saucepan over medium heat; stir in brown sugar and milk. Bring to a boil and boil for 2 minutes, stirring frequently.
Remove the pot from heat; stir in peanut butter and vanilla.
Place confectioners' sugar in a large mixing bowl. Pour in peanut butter mixture and beat with an electric mixer until smooth.
Pour peanut butter mixture into an 8x8-inch dish.
Chill until firm before slicing, about 1 hour.