Characters stay online after the player logs out and closes the client ("zombie sessions")

[meketreve]

Status: TODO — Priority: MEDIUM — Area: Networking / Sessions

Reported by the lead developer on Discord, 2026-08-15. Filed here so it has a thread; the matching backlog entry is ISSUE-041 in docs/backlog/issues.md.

Problem

Players are sometimes shown as online after they have logged out and closed the game client. Reported from live server experience; previous attempts were made to address it and none is confirmed to have fixed it. Not reproduced in a development environment.

A mechanism that produces exactly this symptom is already documented inside ISSUE-038 (the memory-growth investigation), which found it while looking for something else:

• TcpConnection (src/Perpetuum/Network/TcpConnection.cs:32) sets the OS-level TCP keepalive, and that keepalive is the only backstop for a peer that disappears without a clean TCP close.
• There is no application-level heartbeat or idle timeout. ZoneSession tracks _lastReceivedPacketTime / InactiveTime, but it is read in exactly one place — Player.cs:1305, for the "was this player AFK at time of death" loot calculation — and never compared against a threshold to force a disconnect.
• Until the keepalive fires, SessionManager keeps the entry, Character.IsOnline stays true, and the ghost's Player stays InZone and is still processed by the zone tick every frame.

ISSUE-038 shipped a mitigation for that path — keepalive time reduced from 24 hours to 2 hours, confirmed live in TcpConnection.cs:32 — and explicitly deferred the robust fix, recording an application-level idle timeout as "Deliberately not done ... user was offered this as an option and did not request it. Worth reconsidering if 2h keepalive still isn't tight enough after real-world observation." That observation has now been reported, so the deferral is due for review.

One detail does not fit and should be settled first. The keepalive mechanism above explains ungraceful disconnects — a crash, a force-kill, a dropped network, a closed laptop lid. The report describes a graceful exit: the player logged out and closed the client, which should send a clean TCP close and run the normal session teardown. Either the reports also cover ungraceful cases, or the logout path itself sometimes fails to tear the session down. Those are different defects with different fixes, and the evidence to tell them apart has not been gathered.

Impact

• Players appear online when they are not. This is visible to everyone and misinforms corporation coordination and PvP decisions.
• Every surviving ghost holds its Player in the zone and keeps it inside the ProcessManager tick loop, so it costs CPU as well as memory for as long as it lives. This is the mechanism ISSUE-038 identified as a likely contributor to the reported memory growth.
• Anything gated on online state acts on stale information for the lifetime of the ghost.

Proposed fix

1. Establish which case is actually being reported — graceful logout or ungraceful disconnect — before changing anything. A session that survives a clean logout is a different defect from one that survives a dropped connection.
2. If graceful logouts are affected, audit the session teardown path end to end: the client's logout command, SessionManager removal, Character.IsOnline, and the removal of the Player from the zone. A teardown that is skipped or that fails partway would leave exactly this state.
3. If only ungraceful disconnects are affected, implement the application-level idle timeout that ISSUE-038 deferred, using the InactiveTime that ZoneSession already tracks. This does not depend on OS keepalive behaviour, which NATs and firewalls can interfere with.
4. Cover the teardown at the unit tier with a fake session, and the surviving-state question at the integration tier.

Notes

• Priority is a judgement made when filing; the report did not assign one. MEDIUM rather than higher because a partial mitigation already shipped. Raise it if the graceful-logout path turns out to be the cause, since that would mean an ordinary logout can leave a ghost.
• Filed separately from ISSUE-038 rather than folded into it: that issue is about memory growth and would close on a memory measurement, while the visible-online symptom is what players actually report and needs to survive that issue closing.
• Every line number above was checked against 4e6d697 and is anchored to it. Line numbers drift.

[meketreve]

Traced this in the code today. It answers the question the report left open — whether these are clean
logouts or dropped connections — and the answer is that the distinction may not matter, because there
is only one teardown path and both kinds of exit reach it identically.

The path

Session.Disconnect(safeLogout) calls ForceQuit, which ends in _connection.Disconnect() — the same
call a dropped socket makes. Below that point nothing knows how the client left.

• TcpConnection.Disconnect() runs the whole teardown as
  Task.Run(OnDisconnected).ContinueWith(t => Dispose()).LogExceptions() —
  src/Perpetuum/Network/TcpConnection.cs:68
• Session.OnDisconnected wraps SignOut() in a transaction, completes it, and then raises
  Disconnected on the next line, outside the transaction —
  src/Perpetuum/Services/Sessions/Session.cs:300-308
• SessionManager subscribes to that event twice, in this order: OnSessionDisconnected
  (SessionManager.cs:67), then Remove (SessionManager.cs:124)

What that means

A single exception thrown anywhere inside SignOut() produces exactly the reported symptom:

• The transaction rolls back, so the inuse = 0 write in DeselectCharacter is undone and the
  character stays online. Character.IsOnline is the characters.inuse column
  (Character.cs:195).
• Session.cs:308 is never reached, so SessionManager.Remove never runs. The session stays in
  _sessions and the Player stays in the zone tick.
• LogExceptions (TaskExtensions.cs:9) logs the exception and swallows it, so nothing else surfaces.

There is a second variant that needs no rollback at all: Disconnected is a plain multicast invoke
with no per-subscriber guard, so if OnSessionDisconnected throws, Remove — subscribed after it —
never runs.

It also fits the reports saying sometimes rather than always: SignIn runs
update characters set inuse=0 where accountid=@id (Session.cs:209), so a ghost clears itself the
next time that player signs in.

What I could not establish, and the cheap way to settle it

I do not know what throws. The candidates are the three Character database writes in
DeselectCharacter and the ThrowIfNull(AccountNotFound) against the account repository. That is not
derivable from the code.

The server log can answer it with no code change, because the teardown leaves a marker.
[Relay] client disconnected. (SessionManager.cs:99) is written by a subscriber to Disconnected,
so it can only appear after that event has fired. A normal disconnect logs it. A ghost produced this
way logs an exception and no such line. Character deselected /M\ (Session.cs:289) behaves the same
way, since it runs from the transaction's commit callback.

So the question is: in a window where a ghost was reported, is there a logged exception with no
matching [Relay] client disconnected. line?

If yes, this is confirmed, and the fix is to make the teardown survive a failing SignOut() — the
session leaving _sessions and the character going offline should not depend on that method never
throwing — rather than only hunting for whatever threw. If no, this mechanism is wrong and the
keepalive story recorded in ISSUE-038's investigation notes is the whole of it.

Either way the application-level idle timeout that ISSUE-038 deferred still looks worth doing, since it
covers the ungraceful case without depending on OS keepalive behaviour that NATs and firewalls
interfere with. The two address different halves.

One unrelated thing found in passing

Not filed separately and deliberately not fixed: the .ThrowIfZero(ErrorCodes.SQLExecutionError)
guards after accountonlinetimestart and accountonlinetimestop (Session.cs:218 and
Session.cs:263) cannot fire. Both procedures begin with SET NOCOUNT ON, so ExecuteNonQuery()
returns -1 rather than a row count, and ThrowIfZero compares against 0 (Guard.cs:12). It was my
first suspect here and it is ruled out for that reason. Making those guards live would turn a currently
silent no-op into a thrown exception, which seems like your call rather than a drive-by change.

No code was changed. Line numbers are against 1e68c4a, and the files cited are byte-identical to
4e6d697. The same findings are going into the ISSUE-041 entry in docs/backlog/issues.md, per the
Tracking: pairing.