Keep track of the connection and conference objects so we can leave and / or disconnect early, before the connection is established or the conference joined.