Skip to content

feat(worker): publish a protocol to more than one place, in more than one shape - #144

Merged
TheMeinerLP merged 1 commit into
mainfrom
feat/api-export-targets
Aug 23, 2026
Merged

feat(worker): publish a protocol to more than one place, in more than one shape#144
TheMeinerLP merged 1 commit into
mainfrom
feat/api-export-targets

Conversation

@TheMeinerLP

@TheMeinerLP TheMeinerLP commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Note: this description was briefly wrong. Two agents working in parallel staged their pull-request text under the same scratchpad path, and this PR was published with #146's body (the spectrogram change). Only the description was affected — never a commit, never the code. Rewritten below from the change itself after the merge.


Outline was not one of several places a protocol could go — it was the only shape the pipeline had. DocumentSink was already the right port, but entrypoints/worker constructed OutlineSink unconditionally and document_provider selected nothing at all.

Two seams, not one

The registry half is the obvious one. The second is easy to miss and would have shipped broken: render_transcript emits Outline-flavoured Markdownmention:// chips — and escape_markdown escapes Markdown specials. A PDF or HTML sink handed that string gets mention syntax as literal text and Markdown-escaped HTML.

So a format is a pair: a renderer and a sink. That pairing is what the registry holds, and it is why document_provider alone was never enough.

src/sturnus/application/export_formats.py holds FORMATS: Mapping[str, ExportFormat] — a strategy table read through a factory, not a branch. render is a callable; sink names a family (OUTLINE_SINK / OBJECT_STORE_SINK); target_pattern says what that format's target column may contain. Adding a format is one entry. Nothing outside the module writes a format name — format_named and supported_formats are the only way to ask.

The sink half resolves one layer out, in infrastructure/documents/sinks.py::DocumentSinks.sink_for(destination), because sturnus.application may not import sturnus.infrastructure. It takes the whole destination: an object-store sink writes one object per session per target and so cannot be a singleton. It branches per family, never per format, and None is a real answer — one unbuildable destination must not take a guild's working Outline document down with it.

Implemented: outline (unchanged — a test asserts its output byte-for-byte against the packaged template), markdown, html. The HTML renderer runs in a second SandboxedEnvironment with autoescape=True and no md filter registered, so a template reaching for escape_markdown fails to render rather than quietly emitting backslashes.

pdf and confluence are absent, not inert. Configuring one answers 400 listing what is supported. §3.4 of the phase-2 spec explains why PDF is deferred: every route to one is a large native dependency in an image that currently holds Python and a Whisper model, and that is a decision about attack surface this change does not pre-empt.

One failing destination must not lose the others

session_document is read before publishing, not written after, so a destination that stays down brings the session back every 300 s without reprinting what already worked. Renders are cached per format, so two Markdown targets cost one Jinja pass.

That needed a second candidate set. closed_undocumented_sessions cannot see a session whose Outline document succeeded — it is documented — so a failed secondary would never have been retried at all. SessionRepository.sessions_with_unpublished_targets closes that: closed-or-documented, jobs terminal, an enabled target with no row for this session. The sweep unions the two, so a session answering both publishes once.

The artefact needs a URL somebody can open

CreatedDocument(id, url) is the contract and the announcement posts document_url, so an object-store artefact is served by a console route under the session's own participant rule — not a presigned S3 URL, which would outlive the access rules and cannot be revoked. HTML additionally gets default-src 'none'; style-src 'unsafe-inline'; sandbox and nosniff.

The primary target is the oldest enabled one (lowest id) — stable under a rename, and the one a single-destination guild always had. session.document_url is stamped from it alone.

What the rebase changed, and it is worth reading

Rebasing onto #142 revealed that this branch had built a second implementation of the participant rule: ConsoleSessionDocuments carried its own EXISTS on session_participant. #142 had meanwhile established the opposite convention — gate on SessionReads.session_for, literally the call /api/sessions/{id} is served from. A protocol is the same meeting the transcript is, so two rules for it was one too many. The port lost requested_by, routes_documents gained one _may_read helper both handlers begin with, and the route tests now express "Ben was not in it" through FakeReads rather than a participant set on the document double — that set would have been a third implementation, inside the tests meant to prove there is only one.

Known gap, flagged deliberately

Stored Markdown and HTML artefacts are unencrypted. Envelope encryption needs a wrapped-data-key column and session_document has none; migrations were out of scope here. It is the same content the database already holds in clear, but it is new plaintext in a bucket that until now held only ciphertext. #146 does this correctly for spectrograms and is the pattern for the follow-up.

Also: STURNUS_CONSOLE_ORIGIN was added to WorkerSettings (same default as ApiSettings). The two must agree or protocol links posted into Discord point nowhere — documented in docs/operations.md §1.2.

Checks

pytest -q -m 'not slow' 2400 passed (+141) · mypy clean, 275 files · ruff check · ruff format --check — all clean, on the rebased branch.

… one shape

`document_provider` was read on every publish and selected nothing: one code
path constructed `OutlineSink` unconditionally. The obvious fix -- a mapping
from provider to sink -- is only half of one, because `render_transcript`
emits Outline's own `mention://` chips and puts every value through
`escape_markdown`. An HTML sink handed that string publishes the mention
scheme as literal text with backslashes in front of every full stop.

So a format is a pair, and `sturnus.application.export_formats` is the
registry that holds the pairing: `outline` (today's template and sink,
unchanged), `markdown` and `html` (their own templates, the object store).
`pdf` and `confluence` are specified and deliberately absent -- a target
naming a format nothing implements is refused where an administrator can
read the refusal rather than skipped silently after every meeting.

A guild may enable several destinations. `publish_session` writes to each,
survives one failing, and records each success in `session_document`; the
retry sweep reads that record and publishes only what is missing, so a
destination that stays down cannot reprint the Outline document every five
minutes. `session.document_url` is stamped from the primary -- the oldest
enabled target -- which keeps the announcement path and every existing
reader of a session working unchanged. A guild with no rows in
`guild_export_target` still publishes exactly where `document_target` says.

An object-store artefact's URL points at a new console route serving it
under the session's own participant rule, checked per request, rather than
at a presigned S3 URL that would outlive the access rules that issued it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant