Standalone: delete snapshotted history by default

[cloutiertyler]

Description of Changes

Closes #5542.

SpacetimeDB Standalone previously kept the entire commitlog and all snapshots on disk forever, so the data directory grew without bound. As @gefjon explained on the issue, commitlog segments whose entire contents precede the most recent snapshot are not needed to restart the database, and neither are older snapshots. SpacetimeDB-cloud archives this historical data to cold storage, but standalone had no lifecycle management at all.

This PR adds a retention policy for historical data and makes deletion the default for standalone.

Behavior

A new background task (spawned by LocalPersistenceProvider, replacing the commitlog compressor when deletion is enabled) runs once at startup and then whenever a new snapshot is created. It:

1. retains the retain-snapshots most recent snapshots (default 2) and deletes all older ones, and
2. deletes all commitlog segments whose entire contents precede the oldest retained snapshot, using the same segment-boundary arithmetic as the existing snapshot_watching_commitlog_compressor (a segment is named for the first transaction it contains, so the last segment starting at or before the snapshot offset is kept).

The commitlog is retained from the oldest retained snapshot onwards, so the database can be restored from any retained snapshot even if the newest one turns out to be unreadable. The initial run at startup means existing deployments reclaim disk space on upgrade without waiting for the next snapshot.

Because snapshots are captured from committed, not necessarily durable, state, and restore only considers snapshots at or below the durable commitlog offset, the pruner ignores snapshots above the durable offset entirely: they neither count towards the retention limit nor are they ever deleted. This mirrors the "decided snapshot" gating that cloud archival applies, and guarantees a restorable snapshot plus its commitlog suffix always survive a crash.

Snapshot deletion renames the directory to .archived_snapshot before removing it (the same mechanism cloud archival uses), so a prune interrupted mid-deletion can never leave a partially deleted directory that looks like a valid snapshot; leftovers are swept on the next run. When deletion is enabled, local snapshot compression is disabled, since compressing snapshots that will be deleted after the next snapshot is wasted work.

Configuration

A new [retention] section in the standalone config.toml, documented in the config template and in the standalone configuration reference docs:

[retention]
# "delete": delete historical data whenever a new snapshot is taken (default).
# "keep":   keep historical data on disk indefinitely; historical commitlog
#           segments are compressed (the previous behavior).
policy = "delete"

# Number of most recent snapshots to retain when policy = "delete".
retain-snapshots = 2

Note that this changes default behavior: standalone users who want to preserve the full transaction history must now set policy = "keep". Cloud is unaffected, as it does not use LocalPersistenceProvider outside of tests.

Implementation notes

• spacetimedb-commitlog: new Commitlog::remove_segments, mirroring compress_segments. It refuses to remove the writable head segment, removes oldest first so a failure partway through cannot leave a gap in the log, and keeps the in-memory segment list consistent.
• spacetimedb-durability: Local::remove_segments delegating to the commitlog.
• spacetimedb-engine: RetentionConfig/RetentionPolicy next to the existing DurabilityConfig, plus prune_history and the snapshot_watching_history_pruner task in relational_db.
• spacetimedb-standalone: config plumbing, config template, and docs.

API and ABI breaking changes

None. StandaloneOptions gains a retention field.

Expected complexity level and risk

3. This deletes user data by design, so the risk is in deleting too much. Mitigations: the pruner only acts when more than retain-snapshots durable snapshots exist, never touches the segment containing the oldest retained snapshot or anything after it, never touches snapshots above the durable offset, and the head segment cannot be removed at the commitlog layer.

Testing

• New remove_segments unit test in the commitlog crate: refuses the writable segment, removes exactly the requested segments, and the log remains traversable afterwards.
• New prune_history test in the engine crate against a real fs-backed Local durability with tiny segments: no-op below the retention threshold, deletes exactly the stale snapshots and segments, cleans up leftover .archived_snapshot directories, ignores snapshots above the durable offset, and the remaining commitlog covers the oldest retained snapshot through the latest transaction.
• Standalone config tests: [retention] parses, and the default (both an empty config and the shipped config.toml template) is delete with retain-snapshots = 2.
• Full test suites of the commitlog, durability, engine, and standalone crates pass; clippy is clean.