feat(worker): compute a spectrogram once, and delete it when the audio goes

[TheMeinerLP]

A spectrogram is computed per request today and kept nowhere: GET /api/sessions/{id}/tracks/{user}/spectrogram streams the whole ciphertext past an FFT, answers a fixed 600 × 128 matrix, and throws the work away. Every view of the same track pays the same full decrypt again.

spectrograms_by_default — per guild, default false — moves that work to the worker, which draws each track's picture at job completion out of the plaintext WAV it already has on disk, and stores it beside the recording. A view then costs two small object reads and no arithmetic.

Storing it is only defensible with one rule attached, and the rule is the design:

A stored spectrogram is deleted when its audio is deleted.

Without it, switching this on would create a retained rendering of a person's voice activity that outlives the retention window their recording was subject to — because the sweep deletes the S3 object and nothing else. The console's own design argues that a spectrogram "is less than the audio and it is not nothing"; it must not become the thing that survives. So the artefact and the audio share a lifetime, and the sweep that deletes one deletes the other in the same pass.

How the sweep deletes both

sweep_expired_audio reads spectrogram_key alongside s3_key on every candidate, deletes the recording, deletes the picture if the row names one, and only then stamps audio_deleted_at. One pass, one loop iteration, one except — not a second sweep that could be forgotten, disabled, or fail on its own schedule. The stamp is the durable claim that this recording is gone, so it is written only when both objects are, and a partial failure leaves the row unstamped and retried next hour; a repeated DELETE of an object already gone is what S3 answers successfully anyway. mark_audio_deleted clears spectrogram_key in the same UPDATE, because a row that went on naming a deleted object would claim a picture exists.

The key is selected, not derived. The naming rule (sessions/{s}/speakers/{u}.spectrogram.enc) can change; the objects already in the bucket cannot, and the sweep has to delete what was written rather than what would be written today.

That ordering is also why the worker records the key before it uploads the object, which is the one part of SpectrogramStore that is not negotiable. The reverse order has a failure mode this design cannot afford: an object in the bucket that no row names is an object nothing will ever delete. Written this way round, the worst case is the harmless one — a row naming an object that is not there, which the read path answers by drawing the track.

And the property that was true before is still true, with a test on it: after the sweep the track is neither playable nor visualisable. test_a_recording_that_is_gone_has_no_picture_either sets the artefact up in the bucket, takes the recording away, and asserts 404 from both routes.

When storing fails

It does not fail the job. _store_spectrogram swallows every exception it can raise, logs spectrogram.failed at WARNING, and returns.

A transcription that succeeded and could not be drawn has done the valuable part. It runs after queue.complete, so by then the transcript is stored and the job is done — letting a failure travel to process_one's handler would call queue.fail on an already-transcribed job and spend minutes of inference again to retry a picture the console draws for itself in a second. Both directions are tested: a record that fails, a put that fails, and a track that cannot be parsed at all.

Its position is fixed at both ends. Any earlier and a failure re-queues a transcribed job; any later and process_one's finally has already deleted the plaintext. That finally still removes everything, artefact included — it is written into the same scratch directory as the WAV, and a test asserts the directory is gone afterwards.

What it costs

76.8 kB of matrix per track, about 100 kB stored. The extra third is base64 inside a small self-describing JSON envelope: the artefact records the shape it was drawn at, so a build whose COLUMNS moved refuses it and redraws instead of rendering somebody's meeting under somebody else's axes. That is worth 25 kB.

Fixed means fixed — a three-hour workshop costs exactly what a two-minute stand-up does, because the column count does not depend on the recording's length. A six-person meeting is about 0.6 MB.

For a busy guild — four such meetings every working day — a year is about 1.2 GB if nothing were ever deleted. Nothing here lives that long. An artefact lives exactly as long as its recording, so the steady state is one audio_retention_days window of them: at the default thirty days, roughly 100 MB, against the tens of gigabytes that guild's audio occupies over the same window. In storage the picture is a rounding error beside the recording. It is not a rounding error in what it says, which is why the retention rule is the interesting half of this change and the storage figure is the boring one.

Backfill: no

Jobs already transcribed have no artefact and will not acquire one by this setting being switched on. There is no backfill job, and adding one would mean fetching and decrypting every recording in the bucket to redraw pictures the console can produce on demand.

The read path is what makes that a non-issue rather than a gap: it prefers the artefact and falls back to drawing the track — for a job from before the setting was on, for a guild that never turned it on, for an artefact that is missing, and for one this build cannot read. The endpoint's contract does not depend on which answered.

A re-queue does produce one, incidentally rather than by design: the job is transcribed again by a worker that reads the setting again. That is the only way an old session acquires a picture, and it costs a full re-transcription, so it is worth doing for a bad transcript and not for a spectrogram.

Turning the setting off mid-life

Existing artefacts stay. Turning it off says "stop drawing new ones", not "destroy what has been drawn".

The defence is that the retention rule above still governs every one of them: each stored picture is still deleted with its own recording, so nothing outlives the window it was created under, and turning the setting off changes what happens next rather than reaching backwards. The alternative — deleting them on a config change — would be a second deletion path for the same objects, unreviewable against the first, running at the moment somebody is editing a form. That is the wrong moment for a bulk delete of anything, and it would be the only place in this system where a settings write erases data.

What changes immediately is the cost: jobs finishing from now on store nothing, and views of tracks that already have a picture are still answered from it until retention takes both away. An operator who wants the pictures gone sooner shortens audio_retention_days, which moves the recording and the picture together — the only way this system moves them.

Authorisation did not move

_authorised_track runs first and runs on every request: the same session_participant query, decided again, never cached. Only then does anything look in the bucket. A cheap answer is exactly the kind that grows a cache in front of it, and the point of the ordering is that a cache of the payload cannot become a cache of the permission. Cache-Control: private, no-store is unchanged, on both paths, with a test on each. A stranger gets 404 whether or not an artefact exists.

Running the authorisation first is also what refuses a swept track before the artefact is ever considered.

The artefact is encrypted, under the job's own wrapped data key, in the same envelope format as the recording — so the console reads it through the same stream_wav, a master-key rotation reaches both together, and this bucket does not acquire its single readable object. The picture is a rendering of somebody's voice activity; it is behind the same authorisation rule as the audio for that reason, and storing it in the clear beside the ciphertext would have undone that at rest.

Two moves, and why

The FFT and the picture's stored form now live in sturnus.application.spectrogram. Two processes draw the same picture — the console from S3, the worker from disk — and the brief for this change was explicitly not to grow a second FFT path. A shared definition has to sit below both callers, so it takes a stream of plaintext WAV bytes, which is all either caller can promise. sturnus.console.spectrogram is what is left: the two ways the console obtains one. The static test that forbids anything on the serving path from writing plaintext to disk now covers the moved module too — a module on the serving path is on the serving path wherever it lives.

CorruptRecording moves to sturnus.domain.errors for the same reason in miniature: two layers raise it now, and application may not import the console, so domain is the only place below both.

Not here

No console change — and none is needed: settingGroups.ts anticipated this key by name and files it on the catch-all tab deliberately, where it works exactly as every other key does until somebody who knows what it is moves it. No migration; spectrogram_key arrived with 0013 and nothing wrote it until now. No change to the picture's dimensions or the endpoint's response shape.

docs/operations.md gains §6.2.13 with the storage arithmetic, the retention rule, the no-backfill statement and the turning-it-off answer, and §6 now says the sweep deletes both.

---

Rebased onto main at b831c05, past #142, #143, #144, #145, #147 and #148. 2656 tests pass (2624 on main, so +32), mypy, ruff check and ruff format --check clean.

Three notes from those rebases. #142's WritesARealWav arrived while this branch was open and is the same fixture this branch had added for itself, so the spectrogram tests now use its one and FakeCrypto goes back to the single-purpose fake it was — one WAV builder in the file rather than two. #147 collapsed the worker entrypoint's duplicated master-key decodes into a single KeyWrapper binding, so _SpectrogramArtefacts takes that binding rather than decoding the key a third time. And #144 stores its export artefacts unencrypted, which this branch does not: the picture is sealed under the job's own data key, so the bucket still holds no readable object. That gap is #144's to close and nothing here depends on it.