feat(worker): publish a protocol to more than one place, in more than one shape - #144
Merged
Conversation
… one shape `document_provider` was read on every publish and selected nothing: one code path constructed `OutlineSink` unconditionally. The obvious fix -- a mapping from provider to sink -- is only half of one, because `render_transcript` emits Outline's own `mention://` chips and puts every value through `escape_markdown`. An HTML sink handed that string publishes the mention scheme as literal text with backslashes in front of every full stop. So a format is a pair, and `sturnus.application.export_formats` is the registry that holds the pairing: `outline` (today's template and sink, unchanged), `markdown` and `html` (their own templates, the object store). `pdf` and `confluence` are specified and deliberately absent -- a target naming a format nothing implements is refused where an administrator can read the refusal rather than skipped silently after every meeting. A guild may enable several destinations. `publish_session` writes to each, survives one failing, and records each success in `session_document`; the retry sweep reads that record and publishes only what is missing, so a destination that stays down cannot reprint the Outline document every five minutes. `session.document_url` is stamped from the primary -- the oldest enabled target -- which keeps the announcement path and every existing reader of a session working unchanged. A guild with no rows in `guild_export_target` still publishes exactly where `document_target` says. An object-store artefact's URL points at a new console route serving it under the session's own participant rule, checked per request, rather than at a presigned S3 URL that would outlive the access rules that issued it.
TheMeinerLP
force-pushed
the
feat/api-export-targets
branch
from
August 23, 2026 13:57
7c79363 to
f61c74b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Outline was not one of several places a protocol could go — it was the only shape the pipeline had.
DocumentSinkwas already the right port, butentrypoints/workerconstructedOutlineSinkunconditionally anddocument_providerselected nothing at all.Two seams, not one
The registry half is the obvious one. The second is easy to miss and would have shipped broken:
render_transcriptemits Outline-flavoured Markdown —mention://chips — andescape_markdownescapes Markdown specials. A PDF or HTML sink handed that string gets mention syntax as literal text and Markdown-escaped HTML.So a format is a pair: a renderer and a sink. That pairing is what the registry holds, and it is why
document_provideralone was never enough.src/sturnus/application/export_formats.pyholdsFORMATS: Mapping[str, ExportFormat]— a strategy table read through a factory, not a branch.renderis a callable;sinknames a family (OUTLINE_SINK/OBJECT_STORE_SINK);target_patternsays what that format'stargetcolumn may contain. Adding a format is one entry. Nothing outside the module writes a format name —format_namedandsupported_formatsare the only way to ask.The sink half resolves one layer out, in
infrastructure/documents/sinks.py::DocumentSinks.sink_for(destination), becausesturnus.applicationmay not importsturnus.infrastructure. It takes the whole destination: an object-store sink writes one object per session per target and so cannot be a singleton. It branches per family, never per format, andNoneis a real answer — one unbuildable destination must not take a guild's working Outline document down with it.Implemented:
outline(unchanged — a test asserts its output byte-for-byte against the packaged template),markdown,html. The HTML renderer runs in a secondSandboxedEnvironmentwithautoescape=Trueand nomdfilter registered, so a template reaching forescape_markdownfails to render rather than quietly emitting backslashes.pdfandconfluenceare absent, not inert. Configuring one answers 400 listing what is supported. §3.4 of the phase-2 spec explains why PDF is deferred: every route to one is a large native dependency in an image that currently holds Python and a Whisper model, and that is a decision about attack surface this change does not pre-empt.One failing destination must not lose the others
session_documentis read before publishing, not written after, so a destination that stays down brings the session back every 300 s without reprinting what already worked. Renders are cached per format, so two Markdown targets cost one Jinja pass.That needed a second candidate set.
closed_undocumented_sessionscannot see a session whose Outline document succeeded — it isdocumented— so a failed secondary would never have been retried at all.SessionRepository.sessions_with_unpublished_targetscloses that: closed-or-documented, jobs terminal, an enabled target with no row for this session. The sweep unions the two, so a session answering both publishes once.The artefact needs a URL somebody can open
CreatedDocument(id, url)is the contract and the announcement postsdocument_url, so an object-store artefact is served by a console route under the session's own participant rule — not a presigned S3 URL, which would outlive the access rules and cannot be revoked. HTML additionally getsdefault-src 'none'; style-src 'unsafe-inline'; sandboxandnosniff.The primary target is the oldest enabled one (lowest id) — stable under a rename, and the one a single-destination guild always had.
session.document_urlis stamped from it alone.What the rebase changed, and it is worth reading
Rebasing onto #142 revealed that this branch had built a second implementation of the participant rule:
ConsoleSessionDocumentscarried its ownEXISTSonsession_participant. #142 had meanwhile established the opposite convention — gate onSessionReads.session_for, literally the call/api/sessions/{id}is served from. A protocol is the same meeting the transcript is, so two rules for it was one too many. The port lostrequested_by,routes_documentsgained one_may_readhelper both handlers begin with, and the route tests now express "Ben was not in it" throughFakeReadsrather than a participant set on the document double — that set would have been a third implementation, inside the tests meant to prove there is only one.Known gap, flagged deliberately
Stored Markdown and HTML artefacts are unencrypted. Envelope encryption needs a wrapped-data-key column and
session_documenthas none; migrations were out of scope here. It is the same content the database already holds in clear, but it is new plaintext in a bucket that until now held only ciphertext. #146 does this correctly for spectrograms and is the pattern for the follow-up.Also:
STURNUS_CONSOLE_ORIGINwas added toWorkerSettings(same default asApiSettings). The two must agree or protocol links posted into Discord point nowhere — documented indocs/operations.md§1.2.Checks
pytest -q -m 'not slow'2400 passed (+141) ·mypyclean, 275 files ·ruff check·ruff format --check— all clean, on the rebased branch.