We were leaking various references, leading to failures to populate the list upon reloading, or to segfaults. With this commit, all created user objects, as well as the user-manager object are finalized when the panel is unloaded.
Copied from the accounts-dialog git module.