Synapse is unable to cleanly `shutdown` homeserver after failing to `start`

[MadLittleMods]

Synapse is unable to cleanly shutdown homeserver after failing to start. Like when it fails to connect to the database for whatever reason.

Discovered while writing some Complement tests for Synapse Pro for small hosts: https://github.com/element-hq/synapse-small-hosts/pull/204

Sister-issue to #19188

Reproduction

# XXX: Use a bad port like `9999999` in the `listeners` homeserver config
homeserver_config = HomeServerConfig.load_config(
    "Synapse Homeserver", argv_options
)
hs = create_homeserver(homeserver_config)
hs_ref = weakref.ref(hs)

setup(hs)

try:
    # We expect this to explode at this point because of the bad port
    await start(
        hs,
        # We set `freeze` to `False` here so the homeserver can be properly
        # garbage collected if it is removed later.
        freeze=False,
    )
except Exception as exc:
    # Cleanup on failure
    await hs.shutdown()

# Cleanup the internal reference in our test case
del hs

# Force garbage collection.
gc.collect()

hs_after_shutdown = hs_ref()
if hs_after_shutdown is not None:
    self.fail("HomeServer reference should not be valid at this point ")