It seems like the button is not shown for themes that don’t have a config for it. This is probably due to how theming works internally (and how much the default theme says about ST’s layout) and there is no way around it besides updating the theme.
Would be nice if the icon was shown even if the theme did not have it specified, but it wouldn’t make sense to rework theme logic for just that.
I like the panel API changes. I won’t be using them over the current create_panel
workaround anytime soon because the changes are recent, but I can certainly imagine using them in my OutputPanel abstraction class soon.