feat(worker): compute a spectrogram once, and delete it when the audio goes

docs/operations.md

@@ -700,6 +700,14 @@ in — from the console. Turning it on asserts that the document at
 6.2.10 before setting it**: unlike `video_consent_offered`, this one grants
 access rather than withholding it.
 
+`spectrograms_by_default` is `true` or `false`, defaults to `false`, and
+is not about consent at all: it decides whether the worker draws each
+track's spectrogram once and stores it beside the recording, instead of
+the console drawing one per view. It stores something new about every
+recording, so it starts off; what it stores is deleted with the audio it
+was drawn from. **Read section 6.2.13 before setting it** — it has a
+storage cost and a rule attached.
+
 ### 4.0 `voice_channel_ids`, and the key it replaced
 
 `voice_channel_ids` names every voice channel Sturnus is allowed to record
@@ -778,8 +786,8 @@ when it detects this.
 invocation, per API request, and by the consent cache with a five-second
 TTL), and `transcription_language`,
 `transcription_prompt`, `document_target`, `document_provider`,
-`merge_gap_seconds` (read per job by the *worker* process, not the bot at
-all), and `max_parallel_tracks` (read by the worker *inside* the claim
+`merge_gap_seconds`, `spectrograms_by_default` (read per job by the
+*worker* process, not the bot at all), and `max_parallel_tracks` (read by the worker *inside* the claim
 query itself, so it governs the very next claim — see 4.2). The two
 transcription keys apply to the next job the worker claims,
 which means a session already recording is still transcribed with the new
@@ -1280,6 +1288,16 @@ needs to: a bucket lifecycle rule is a second, independent line of
 defence, never a substitute for that database record, and never a
 replacement for this sweep.
 
+**The sweep deletes the recording's spectrogram in the same pass**, when
+the guild asked for stored ones (`spectrograms_by_default`, §6.2.13) and
+there is one. A spectrogram shows when somebody spoke and for how long;
+leaving it behind would make this setting a way for something about a
+person's voice to outlive the window their recording was subject to, and
+this sweep is the only thing in the system that ends a recording's life.
+`audio_deleted_at` is stamped only once both objects are gone, so a
+partial failure is retried on the next sweep rather than recorded as
+done.
+
 Because recordings outlive their transcription by weeks, not minutes, the
 retention period is not merely an implementation detail — **it belongs in
 the privacy policy shown to participants** (the `policy_url` document),
@@ -2233,6 +2251,90 @@ clearing its secret, each emit `console.oauth_client_changed` at
 `registered`, `secret_set`, `secret_cleared` or `removed`. Neither half
 of the credential is on the line — not the secret, and not the client id.
 
+### 6.2.13 `spectrograms_by_default`: drawing each track's picture once
+
+The console can show a track as a spectrogram — where the speech is, and
+whether the capture worked at all (§6.3 explains what the picture is for).
+Drawing one costs a full streamed decrypt of the recording plus 600
+transforms, and by default that happens on **every view**: nothing is
+kept.
+
+`spectrograms_by_default` moves the work to the worker. When it is `true`,
+a job that finishes transcribing draws its own track's picture — from the
+plaintext WAV it already has on disk, which exists nowhere else and for no
+longer than that job — and stores it beside the recording in the same
+bucket, encrypted under the same session data key. A view then costs two
+small object reads and no arithmetic.
+
+#### What it stores, and for how long
+
+A picture is a fixed 600 × 128 cells, one byte each: **76.8 kB of matrix,
+about 100 kB stored** once it is base64'd into its small self-describing
+envelope. Fixed means fixed — a three-hour workshop costs exactly what a
+two-minute stand-up does, because the number of columns does not depend on
+the length of the recording.
+
+Per meeting that is about 100 kB × the number of people who spoke: a
+six-person meeting is 0.6 MB. A guild holding four such meetings every
+working day accumulates roughly 1.2 GB over a year — *if* nothing were
+ever deleted. Nothing here is kept that long: an artefact lives exactly as
+long as the recording it was drawn from, so the steady-state cost is a
+`audio_retention_days` window of them, not a year of them. At the default
+thirty days that is about 100 MB for the guild above, against the tens of
+gigabytes its audio occupies over the same window. The picture is a
+rounding error beside the recording, in storage. It is not a rounding
+error in what it says.
+
+#### The rule this setting is subject to
+
+> **A stored spectrogram is deleted when its audio is deleted.**
+
+The retention sweep (§6) deletes both objects in the same pass, and stamps
+`audio_deleted_at` only once both are gone. This is not an optimisation
+and it is not optional: a spectrogram shows when a person spoke and for
+how long, and a picture left behind by a sweep that erased the recording
+would be a record of somebody's voice activity outliving the retention
+window their recording — and their consent — was subject to. After a
+sweep the track is neither playable nor visualisable, exactly as it was
+before this setting existed.
+
+#### Turning it on
+
+```
+/config set key:spectrograms_by_default value:true
+```
+
+or the console's settings page. Values are `true` and `false` and nothing
+else, for the reason §6.2.10 gives. `/config clear` restores the default,
+which is `false`. It takes effect **immediately**: the worker reads it per
+job, and nothing caches it.
+
+**Nothing is backfilled.** Jobs already transcribed have no artefact and
+will not acquire one by this setting being switched on; the console goes
+on drawing those tracks on demand, at the old cost, indefinitely. A
+session that is re-queued (§6.2.9) is transcribed again by a worker that
+reads the setting again, so a redo does produce one. If you want pictures
+for a specific old session, re-queueing it is the way, and it costs a full
+re-transcription — which is rarely worth it for a picture.
+
+#### Turning it off
+
+Existing artefacts **stay**, and this is deliberate. Turning the setting
+off says "stop drawing new ones", not "destroy what has been drawn": the
+pictures already stored are still governed by the rule above, so each one
+is still deleted with its own recording and none of them outlives the
+window it was created under. Deleting them on a config change would be a
+second, unreviewable deletion path for the same objects — and one that
+runs at the moment somebody is editing a form, which is the wrong moment
+for a bulk delete of anything.
+
+What changes immediately is the cost: jobs finishing from now on store
+nothing, and views of tracks that already have a picture go on being
+answered from it until retention takes both away. If you need the stored
+pictures gone sooner than their recordings, shorten
+`audio_retention_days` — that moves the recording and the picture
+together, which is the only way this system moves them.
+
 ### 6.3 Listening to a recording by hand
 
 Every automated check this system has can describe a track — its level,

src/sturnus/application/recording.py

@@ -46,6 +46,30 @@ def audio_key(session_id: int, discord_user_id: int) -> str:
     return f"sessions/{session_id}/speakers/{discord_user_id}.enc"
 
 
+def spectrogram_key(session_id: int, discord_user_id: int) -> str:
+    """Object key for the stored picture of one speaker's recording.
+
+    Beside the audio and derived from the same two ids, so an operator
+    looking at the bucket can see at a glance which recording an artefact
+    belongs to -- and so that a prefix listing of a session shows
+    everything that session put in the bucket, which is what makes "is
+    anything left?" a question the bucket can answer after a retention
+    sweep.
+
+    Still `.enc`, because it is: the picture is sealed under the same
+    session data key the recording is. What is stored is a rendering of
+    somebody's voice activity, and an object in this bucket that is
+    readable without the master key would be the only one.
+
+    The key the worker actually used is nonetheless written down on the
+    job (`transcription_job.spectrogram_key`) rather than recomputed from
+    this rule at deletion time. The rule can change; a bucket full of
+    objects written under the old one cannot, and the retention sweep must
+    delete what was written rather than what would be written today.
+    """
+    return f"sessions/{session_id}/speakers/{discord_user_id}.spectrogram.enc"
+
+
 class SessionRecorder(Protocol):
     """What `RecordingService` needs to persist about a session's rows."""
 

src/sturnus/application/retention.py

@@ -8,8 +8,13 @@
 
 `sweep_expired_audio` is the periodic sweep that actually calls it: it reads
 candidates through `JobStore`, filters them with `expired_jobs`, then --
-for each -- deletes the object through `AudioDeleter` and stamps
-`audio_deleted_at` as the durable evidence that the deletion happened. The
+for each -- deletes the recording *and its stored spectrogram* through
+`AudioDeleter` and stamps `audio_deleted_at` as the durable evidence that
+the deletion happened. Both objects, in one pass, because a spectrogram
+that outlived the recording it was drawn from would be a rendering of
+somebody's voice activity surviving the retention window that voice was
+subject to -- see `sweep_expired_audio` and
+`sturnus.domain.settings.SPECTROGRAMS_BY_DEFAULT`. The
 bucket lifecycle rule (Spec 12.2) is a second line of defence, never a
 substitute for that record. `JobStore`/`AudioDeleter` are narrow local
 `Protocol`s rather than concrete types, the same pattern
@@ -63,40 +68,83 @@ async def candidates_for_retention(self) -> list[dict[str, object]]:
         of it anywhere in the codebase, the same reasoning
         `sturnus.application.publishing.sessions_to_announce`'s caller
         follows for `status`/`announced_at`/`document_url`.
+
+        Every candidate carries `spectrogram_key` -- `None` for the jobs
+        that have no stored picture, which is most of them and all of them
+        for a guild that never switched `spectrograms_by_default` on. It
+        is selected rather than derived from the job's ids because the
+        naming rule may change and the objects already in the bucket
+        cannot: this sweep must delete what was written.
         """
         ...
 
-    async def mark_audio_deleted(self, job_id: int, now: datetime) -> None: ...
+    async def mark_audio_deleted(self, job_id: int, now: datetime) -> None:
+        """Stamps the deletion, and forgets where the picture was.
+
+        `spectrogram_key` is cleared in the same statement. The column
+        says where this job's artefact is, and once the sweep has deleted
+        it there is no artefact for it to point at; leaving the key behind
+        would make every later sweep re-delete an object that is already
+        gone and would leave the row claiming a picture exists.
+        """
+        ...
 
 
 class AudioDeleter(Protocol):
-    """Where an expired recording's object is actually removed."""
+    """Where an expired recording's objects are actually removed.
+
+    One method for both, because both are objects in the same bucket
+    under the same credentials, and a port with a second method for the
+    artefact would invite a caller to delete one kind and not the other.
+    """
 
     async def delete(self, key: str) -> None: ...
 
 
 async def sweep_expired_audio(jobs: JobStore, store: AudioDeleter, now: datetime) -> None:
-    """Deletes every expired job's audio object and stamps `audio_deleted_at`.
+    """Deletes every expired job's audio, and its spectrogram, and stamps the row.
+
+    **A stored spectrogram is deleted when its audio is deleted, in the
+    same pass.** That rule is the whole reason storing one is defensible.
+    A spectrogram is a rendering of when somebody spoke and for how long;
+    it is less than the audio and it is not nothing, and this sweep is the
+    only thing in the system that ends a recording's life. If it deleted
+    the object and left the picture, `spectrograms_by_default` would be a
+    switch that quietly makes something about a person's voice outlive the
+    retention window that person's recording was subject to -- which is
+    what the window is for. So the picture goes with it, here, rather than
+    in a second sweep that could be forgotten, disabled, or fail on its
+    own.
 
     Survives its own errors per job: a failure deleting one job's audio
-    (or stamping it afterwards) is logged and does not stop the sweep from
-    handling the rest -- one unreachable object must not block every other
-    job's retention from being enforced.
+    (or its picture, or stamping it afterwards) is logged and does not
+    stop the sweep from handling the rest -- one unreachable object must
+    not block every other job's retention from being enforced.
 
-    `audio_deleted_at` is stamped only after `store.delete` actually
-    succeeds, so a failed deletion is retried on the next sweep instead of
+    `audio_deleted_at` is stamped only after both deletions actually
+    succeed, so a failed deletion is retried on the next sweep instead of
     being silently recorded as done. The reverse order (stamp then delete)
     would risk exactly the outcome `audio_deleted_at` exists to rule out --
     a stamp claiming deletion happened when it did not. The cost of the
     chosen order is a delete that succeeds but whose stamp then fails
     being retried once more next sweep; `store.delete` on an
     already-missing key is idempotent (an S3 `DELETE` on a missing object
     still succeeds), so that retry costs nothing.
+
+    The audio goes first and the picture second, which matters only in the
+    one case where the sweep is interrupted between them: what is left
+    behind is then a picture whose audio is gone, and a row still asking
+    to be swept. The console refuses a track whose object has been erased
+    before it looks for a picture at all, so that interval is invisible
+    from outside and ends on the next sweep.
     """
     for job in expired_jobs(await jobs.candidates_for_retention(), now):
         job_id = cast(int, job["id"])
         try:
             await store.delete(cast(str, job["s3_key"]))
+            picture = cast("str | None", job["spectrogram_key"])
+            if picture is not None:
+                await store.delete(picture)
             await jobs.mark_audio_deleted(job_id, now)
         except Exception as exc:
             log_exception(