The initial small size is due to a 'Connecting to sound system..." label that is shown for a split second before the real UI comes in. Better to just not do that, I think.
Copied from gnome-media. See that module for git history.