feat(worker): transcribe for the language and the vocabulary of the room

charts/sturnus/values.yaml

@@ -267,7 +267,13 @@ worker:
   replicaCount: 1
   env:
     STURNUS_MODEL_CACHE_DIR: /data/model-cache
-    STURNUS_WHISPER_MODEL: large-v3-turbo
+    # Not `large-v3-turbo`: turbo is a distilled decoder, four layers where
+    # this has thirty-two, and the accuracy it trades away lands outside
+    # English -- which is the only place these meetings happen. It is a
+    # latency trade, and nothing here is waiting on latency: transcription
+    # is offline, per speaker, after the meeting. `worker.resources` below
+    # is where that decision is actually paid for.
+    STURNUS_WHISPER_MODEL: large-v3
   # Larger than bot/link's: `process_one` downloads the encrypted object and
   # decrypts it to a WAV before transcribing, both under its work directory
   # (`sturnus.application.worker.process_one`, which creates a
@@ -278,25 +284,80 @@ worker:
   # Setting STURNUS_WORK_DIR to anything outside this mount points the worker
   # at the read-only root filesystem, so the two have to move together.
   tmpSizeLimit: 4Gi
+  # Sized for large-v3 at int8_float32 with beam_size 8, which is a
+  # different budget from the large-v3-turbo/beam 5 these numbers were set
+  # for. Three parts, added up rather than guessed at:
+  #
+  #   * The weights, resident for the life of the process. 1.55B
+  #     parameters quantised to one byte each is ~1.55GB, against ~0.81GB
+  #     for turbo's 809M. (The checkpoint on the cache volume below is
+  #     float16 and twice that; it is quantised while loading, not kept.)
+  #   * The decoding buffers, which exist only while a job runs and are
+  #     the part that actually scales: the cached attention keys and
+  #     values are held per beam, per decoder layer, and large-v3 has 32
+  #     decoder layers where turbo has 4. faster-whisper's own published
+  #     CPU benchmark for this architecture (large-v2 -- same 1.55B
+  #     parameters, same layer counts -- int8, beam 5) peaks at ~2.9GB
+  #     all in, i.e. ~1.35GB on top of the weights. beam_size 8 scales
+  #     that share by 8/5, so ~2.2GB.
+  #   * The process itself -- interpreter, numpy, soxr, boto3, SQLAlchemy,
+  #     one decrypted WAV read through it. A few hundred MB; call it 0.5GB.
+  #
+  # ~4.25GB at peak, and measured at **4.11GB** on an x86 CPU pinned to
+  # four threads, transcribing 4.7 minutes of German speech -- close enough
+  # to the arithmetic above to trust it for longer jobs, since only the
+  # second item grows with the audio. For comparison the previous
+  # configuration (large-v3-turbo, int8, beam 5) peaked at 1.96GB, which is
+  # why the old 2560Mi limit held.
+  #
+  # The request is 4Gi (steady state between jobs is
+  # closer to 2GB -- the second item is transient), and the limit 6Gi. The
+  # limit is the number worth being generous with: exceeding it is an
+  # OOMKill in the middle of a transcription, which fails the job, and
+  # headroom on a node that has the memory costs nothing until it is
+  # needed. cpu stays at 4: large-v3 needs more *time* than turbo, not
+  # more parallelism, and the ceiling that time has to stay under is
+  # STURNUS_JOB_LEASE_SECONDS (1800s), not this.
+  #
+  # Measured, that ceiling is further away than it looks. The same run
+  # transcribed 282s of speech in 145s -- 1.94x faster than real time,
+  # against 4.6x for the old configuration. `vad_filter` means only the
+  # *speech* in a track is decoded, not its padded length, so 1800s of
+  # lease covers roughly 58 minutes of one person actually talking. Beyond
+  # that the lease expires mid-job; with `replicas: 1` (hardcoded in the
+  # Deployment template) and the worker processing one job at a time,
+  # nothing else can claim it, so the only consequence today is that a
+  # restart may reclaim a job that was still running. Raising the worker
+  # beyond one replica is what would turn this into two workers
+  # transcribing the same recording, and is the moment to revisit the
+  # lease.
   resources:
     requests:
       cpu: "4"
-      memory: 2Gi
+      memory: 4Gi
     limits:
       cpu: "4"
-      memory: 2560Mi
-  # The model (faster-whisper large-v3-turbo) downloads on first start on a
-  # cold cache volume, which takes minutes -- not seconds. A startupProbe
-  # gives that its own, generous budget: liveness and readiness are not
+      memory: 6Gi
+  # The model (faster-whisper large-v3) downloads on first start on a cold
+  # cache volume, which takes minutes -- not seconds. A startupProbe gives
+  # that its own, generous budget: liveness and readiness are not
   # evaluated at all until it succeeds, so a slow download cannot be mistaken
   # for a hung process and cannot trigger a restart loop. Once the startup
   # probe succeeds, liveness/readiness take over on their normal cadence.
+  # The budget is what it is because `sturnus.entrypoints.worker` builds
+  # the engine -- download, then quantise 1.55B parameters down to int8 --
+  # *before* it starts the health server, so nothing answers /healthz until
+  # that has finished.
   startupProbe:
     httpGet:
       path: /healthz
       port: health
     periodSeconds: 10
-    failureThreshold: 60 # 10 minutes to cover a cold-cache model download
+    # 20 minutes. Twice what it was, because large-v3's float16 checkpoint
+    # is ~3.1GB against large-v3-turbo's ~1.6GB and the quantisation at the
+    # end of it is longer too. This only ever costs anything on a cold
+    # cache volume: it is a ceiling, not a wait.
+    failureThreshold: 120
   livenessProbe:
     httpGet:
       path: /healthz
@@ -309,7 +370,11 @@ worker:
     periodSeconds: 10
   persistence:
     # Holds the downloaded model weights so a restart doesn't repeat the
-    # download; ~1.6GB for large-v3-turbo in int8 (Spec 7) plus headroom.
+    # download. What lands here is the float16 checkpoint as published,
+    # ~3.1GB for large-v3 (Spec 7) -- the int8 quantisation happens in
+    # memory at load time and is never written back. Plenty of headroom at
+    # 10Gi, and worth keeping: a resize is far more disruptive than the
+    # unused gigabytes are expensive.
     size: 10Gi
     storageClassName: ""
     accessMode: ReadWriteOnce

docs/first-deployment.md

@@ -201,6 +201,15 @@ In Discord, as an administrator:
    the target collection id, and `policy_url`, which must point at a real
    privacy policy naming the retention period. Participants consent to
    what that document says.
+
+   Two more keys are worth a look here even though they default to
+   something sensible: `transcription_language` (default `de`) and
+   `transcription_prompt`, which defaults to OneLiteFeather's own project
+   names. Both decide what the protocol actually *says*, and both are
+   cheap to get right now and awkward to notice later — a wrong language
+   is pinned per speaker for a whole session, and a name the model has
+   never seen comes out as the nearest word it has. Section 4 of
+   `operations.md` explains both.
 3. Have one person run `/consent grant` and `/link start` to confirm both
    flows end where they should.
 

docs/operations.md

@@ -83,8 +83,8 @@ in the bot would read it.
 | `STURNUS_MASTER_KEY_ID` | **yes** | no | Label recorded as `encryption_key_id` when a data key is wrapped. Not key material. |
 | `STURNUS_OUTLINE_BASE_URL` | **yes** | no | Base URL of the Outline instance the finished protocol is posted to. |
 | `STURNUS_OUTLINE_SERVICE_KEY` | **yes** | **yes** | Outline API token `OutlineSink` authenticates with when creating documents. Note the name — it is `OUTLINE_SERVICE_KEY`, not an `API_TOKEN` variant. A token that is invalid, lacks access, or points at a collection that does not exist surfaces as `PermanentDocumentError`; see section 5. |
-| `STURNUS_WHISPER_MODEL` | `large-v3-turbo` | no | faster-whisper model to load. Larger models are more accurate and markedly slower, and this deployment transcribes on CPU (see the chart's `worker.resources`), so the difference is measured in minutes per recording rather than seconds. |
-| `STURNUS_WHISPER_DEFAULT_LANGUAGE` | `en` | no | Language reported when faster-whisper's own detection comes up empty. It matters more than a fallback usually does: the first transcription for a speaker in a session pins that speaker's language, and every later job for them reuses it. |
+| `STURNUS_WHISPER_MODEL` | `large-v3` | no | faster-whisper model to load. Larger models are more accurate and markedly slower, and this deployment transcribes on CPU (see the chart's `worker.resources`), so the difference is measured in minutes per recording rather than seconds. It is deliberately not `large-v3-turbo`: turbo is a distilled decoder with four layers instead of thirty-two, and what it gives up is concentrated outside English. Transcription happens offline, per speaker, after the meeting, so the time it costs is time nobody is waiting on — the memory it costs is real, and the chart's `worker.resources` comment works it out. |
+| `STURNUS_WHISPER_DEFAULT_LANGUAGE` | `de` | no | Language reported when faster-whisper's own detection comes up empty. This is the floor under the per-guild `transcription_language` (section 4.1), not the usual setting to reach for — it is consulted only for a guild that asked for detection (`transcription_language auto`) and got nothing back. It still matters more than a fallback usually does: the first transcription for a speaker in such a session pins that speaker's language, and every later job for them reuses it. |
 | `STURNUS_MODEL_CACHE_DIR` | unset | no | Where model weights are cached. When set, the worker exports it as `HF_HOME` before loading the model, so the download lands on a persistent volume; left unset, every cold start re-downloads several gigabytes of weights. |
 | `STURNUS_WORK_DIR` | `/tmp` | no | Scratch directory the encrypted recording is downloaded and decrypted into before transcription. It must be large enough for the biggest single recording — the chart sizes the corresponding volume with `worker.tmpSizeLimit`. |
 | `STURNUS_MAX_JOB_ATTEMPTS` | `3` | no | How many failed attempts a job gets before `JobQueue.fail` marks it `dead`. See section 5 for what a `dead` job means for the rest of its session. |
@@ -94,11 +94,27 @@ in the bot would read it.
 | `STURNUS_SENTRY_ENVIRONMENT` | `production` | no | Value Sentry files events under in its environment filter. Ignored when no DSN is set. |
 
 Whisper's device and compute type are deliberately *not* environment-driven:
-the worker constructs `WhisperEngine` with `"cpu"` and `int8` hardcoded,
-because Spec 7 sizes this deployment for CPU inference. There is no
-`STURNUS_WHISPER_DEVICE` to set — moving to GPU is a code change, not a
+the worker constructs `WhisperEngine` with `"cpu"` and `int8_float32`
+hardcoded, because Spec 7 sizes this deployment for CPU inference. There is
+no `STURNUS_WHISPER_DEVICE` to set — moving to GPU is a code change, not a
 configuration change.
 
+`int8_float32` rather than plain `int8`: the weights are quantised to int8
+either way, and the suffix names the type everything else runs in.
+CTranslate2 treats bare `int8` as an alias and picks that float type for
+whichever machine it finds itself on, which would leave transcription
+quality depending on the node the pod was scheduled to. It also falls back
+silently rather than refusing a compute type it cannot provide, so a wrong
+value here costs accuracy with nothing in the logs to say so.
+
+Neither the decoding parameters (`beam_size`, `condition_on_previous_text`,
+the VAD filter and the two hallucination thresholds) is configurable
+either. They are quality decisions with one right answer for this workload,
+argued in `sturnus/infrastructure/whisper.py` and pinned by
+`tests/infrastructure/test_whisper.py`; what *is* per-guild is the language
+and the vocabulary, and those are runtime configuration rather than
+environment variables — see section 4.1.
+
 ### 1.3 `sturnus-link` (`sturnus.entrypoints.link.LinkSettings`)
 
 | Variable | Required | Secret | Purpose |
@@ -376,6 +392,40 @@ naming the guild, rather than costing the protocol.
 /config set timezone Europe/Berlin
 ```
 
+Worth setting for the same reason, and for a bigger one: `transcription_language`
+decides what language the recordings are transcribed as. It defaults to
+`de`. The alternative is not "no language" but detection, and detection is
+weak exactly where it is used here — it runs on one speaker's track with
+the silence already cut out of it, so a participant whose first
+contribution is a three-second "ja, genau" gives it almost nothing to work
+with. Whatever it guesses is then pinned for that speaker for the rest of
+the session, so one unlucky guess is not one bad job, it is every job for
+that person from then on. Naming the language removes the guess.
+
+```
+/config set transcription_language de
+```
+
+A guild that genuinely meets in more than one language sets it to `auto`,
+which is what asks for detection-and-pinning explicitly. There is no third
+state: clearing the key restores the `de` default rather than removing it.
+
+`transcription_prompt` is the vocabulary Whisper is biased towards while
+decoding — Whisper's `initial_prompt`. It defaults to OneLiteFeather's own
+project names and stack, written as an ordinary German sentence so the
+style it biases towards is punctuated prose as well. Proper nouns are both
+what a general model reliably gets wrong and what a protocol is read for: a
+decision minuted about the wrong project is worse than no minutes. Set it
+if your names are different ones:
+
+```
+/config set transcription_prompt "Protokoll eines Meetings über Foo, Bar und Baz."
+```
+
+Keep it a sentence rather than a word list, keep it in the transcription
+language, and keep it short — Whisper only sees the last ~224 tokens of it,
+and a long prompt bleeds its own wording into the transcript.
+
 Until every required key (`voice_channel_id`, `consent_role_id`,
 `document_target`, `policy_version`, `policy_url`, `admin_role_id`) is set,
 the bot logs a warning naming the guild and skips building that guild's
@@ -404,9 +454,14 @@ when it detects this.
 
 **Live immediately, and never were stale.** `admin_role_id`,
 `policy_version`, `policy_url` (read per command invocation, and by the
-consent cache with a five-second TTL), and `document_target`,
-`document_provider`, `merge_gap_seconds` (read per job by the *worker*
-process, not the bot at all).
+consent cache with a five-second TTL), and `transcription_language`,
+`transcription_prompt`, `document_target`, `document_provider`,
+`merge_gap_seconds` (read per job by the *worker* process, not the bot at
+all). The two transcription keys apply to the next job the worker claims,
+which means a session already recording is still transcribed with the new
+value — and a job that has already run is not redone. Changing them
+because a protocol came out wrong therefore affects the next meeting, not
+the one you are looking at.
 
 **Deferred until the recording in progress ends.** `voice_channel_id` and
 `consent_role_id`. These decide which channel a session's row names and

docs/verification/end-to-end-checklist.md

@@ -248,7 +248,10 @@ measured, not a plausibility check against the estimate.
 - [ ] CPU and memory for the bot pod under load: ____________
 - [ ] CPU and memory for the worker pod under load (this is the one most
   likely to differ sharply from the spec's estimate, since transcription
-  is the heaviest step): ____________
+  is the heaviest step). Expect ~4.1GB at peak for large-v3 at
+  int8_float32 with beam_size 8 — measured off-cluster on four CPU threads,
+  so this is the confirmation that it holds on a worker node, not a first
+  look. The limit is 6Gi: ____________
 - [ ] **[Plan 4]** CPU and memory for the link-service pod under load:
   ____________
 - [ ] Actual recording size per speaker-hour (extrapolate from this

src/sturnus/application/reconfigure.py

@@ -59,8 +59,9 @@ class GuildRuntimeConfig:
     """Everything about a guild that the bot process holds in memory.
 
     Deliberately *not* every configuration key: `admin_role_id`,
-    `policy_url`, `policy_version`, `document_target`, `merge_gap_seconds`
-    and `document_provider` are read per use (by a permission check, by the
+    `policy_url`, `policy_version`, `document_target`, `merge_gap_seconds`,
+    `document_provider`, `transcription_language` and
+    `transcription_prompt` are read per use (by a permission check, by the
     consent cache, or by the worker process entirely) and were never stale
     to begin with. Only what the bot caches needs reconciling.
     """

src/sturnus/application/transcription.py

@@ -30,11 +30,21 @@ class TranscriptionResult:
 
 
 class TranscriptionEngine(Protocol):
-    async def transcribe(self, path: Path, language: str | None) -> TranscriptionResult:
+    async def transcribe(
+        self, path: Path, language: str | None, initial_prompt: str | None
+    ) -> TranscriptionResult:
         """Transcribe one speaker's recording.
 
         `language` pins the language; `None` asks the engine to detect it and
         report what it found.
+
+        `initial_prompt` is vocabulary and style for the engine to lean
+        towards — an organisation's project names, the words a general
+        model has never seen and will otherwise replace with something it
+        has. It is deliberately a required argument rather than one with a
+        default: the guild's configured prompt (Spec 11) is worth nothing
+        if a call site can quietly leave it out, and a caller that really
+        has no vocabulary to offer says so by passing `None`.
         """
         ...