feat(i18n): automated translation pipeline with machine review gate

[Timur Tukaev (tym83)]

Summary

Builds the automated translation pipeline on top of the i18n proof-of-concept in #593. Where #593 established the multi-language site structure and a few hand-checked pages, this turns translation into a repeatable, reviewable process: it discovers what changed in the English source, translates it through a machine review gate, and opens a weekly PR for a maintainer to merge. Nothing reaches production without that merge.

This targets main; the multi-language site structure from #593 has since landed there, so this PR stands on its own.

What

• Change detection. source_digest (sha256 of the English source, the same convention hack/check-i18n.sh already enforces) drives a worklist of missing/stale pages per language. Scope is the latest docs version only, plus recent blog posts — older docs versions are noindex, so translating them would spend budget on pages search engines ignore.
• Per-page review gate. Each page runs translate → back-translate (meaning-drift check) → two virtual native reviewers (a technical editor for fluency, a Cozystack maintainer for technical correctness) → a bounded revise loop. Code, shortcodes, and inline code are masked during translation and restored after, so they stay byte-for-byte. caption/alt text inside shortcodes is translated (it renders to readers); src/width are not.
• Honest stamping. A page that clears the gate is stamped translation_review: auto-reviewed; one that runs out of revise rounds with findings still open is stamped auto-reviewed-with-findings and its findings are posted to the weekly PR so a maintainer can triage them. Only a human sets ratified.
• Reader-facing disclosure. Documentation pages carry a machine-translation banner linking to the English original until a native speaker marks them ratified. The banner is docs-only by design — the blog, marketing pages, and homepage hero do not carry it.
• Cadence. The runner translates daily (to use each day's model budget) but accumulates into one i18n/week-<ISO week> branch, so maintainers review and merge a single translation PR per week. publish_mode: pr_only — CODEOWNERS and branch protection are untouched.
• Sample output. Two real pipeline translations of the same docs page (ru + de, docs/v1.5/getting-started/install-kubernetes.md) are included so quality can be judged directly.

Why

The English docs are the source of truth and change constantly; native localization takes months. This lets translated docs ship and be indexed immediately (publish-then-ratify), with the banner keeping that honest, while native review happens asynchronously and is tracked per page. The review gate is not a substitute for native ratification — both virtual reviewers are the same model as the translator, so the gate measures self-consistency and catches the obvious failures, no more. It is deliberately conservative: fail-closed on unparseable reviewer verdicts, refuses to write a page with a dropped code block, and never publishes without a maintainer merge.

Notes for reviewers

• Auth. Bootstrapped on a maintainer's Claude subscription via the Claude Agent SDK (no metered billing). The intent is to move the backlog burst to an organization-owned API key; that is a one-line config switch (auth: api-key), no code change. See README "Ownership and continuity".
• Backlog size. 183 pages × 6 languages ≈ 1084 jobs. Measured on the pilot, a page is ~3 min when it clears the gate and ~11 min when it runs the full revise loop, so the first pass is best done on an API key rather than a personal subscription.
• es/pt-br are translated by the pipeline but not yet declared in hugo.yaml (no content shipped). Declaring a language before its content exists publishes empty indexable pages; a test enforces the ordering.
• Tests for the pure functions: python3 hack/i18n/test_i18n.py.

Preview

The included sample pages render under /ru/docs/v1.5/getting-started/install-kubernetes/ and /de/… in the deploy preview, each with the machine-translation banner.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 10acf86c-3097-4037-95b5-e9588b661a17

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/i18n-pipeline

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[gemini-code-assist]

Code Review

This pull request introduces an automated, machine-reviewed translation pipeline for the Cozystack website, adding localization scripts, prompts, style guides, and tests under hack/i18n/, alongside initial support for Spanish and Brazilian Portuguese. It also integrates a machine-translation disclaimer banner into the documentation layout and adds hreflang alternates for SEO. Feedback on these changes highlights a non-standard Hugo API usage (.Sites.Default) in the translation banner, a duplicate entry in .gitignore, and formatting conflicts in the translation and revision prompts where the instructions ask for key: value lines instead of the JSON format expected by the parsing script.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

.Sites.Default is not a standard Hugo API and may cause template evaluation errors depending on the Hugo version. The standard and safe way to retrieve the default content language is .Site.DefaultContentLanguage.

Suggested change

	{{- $src := .Sites.Default.Language.Lang -}}
	{{- if and (ne .Lang $src) (ne .Params.translation_review "ratified") -}}
	{{- $src := .Site.DefaultContentLanguage -}}
	{{- if and (ne .Lang $src) (ne .Params.translation_review "ratified") -}}

[gemini-code-assist]

The directory .venv-i18n/ is duplicated in the .gitignore file (on lines 32 and 36). Removing the duplicate entry keeps the file clean.

# i18n pipeline venv (bootstrapped by hack/i18n/run-daily.sh)
.venv-i18n/

# i18n pipeline: per-run artifacts, not content
hack/i18n/last-run-findings.md

[gemini-code-assist]

The system prompt instructs the model to output the front matter as key: value lines, but the translation pipeline (translate.py via _FM_PROTOCOL) expects a JSON object. This conflict can confuse the model and lead to protocol parsing errors. Updating the prompt to match the JSON protocol will improve reliability.

Suggested change

	===FRONTMATTER===
	<the translated values, one `key: value` per line, same keys, same order>
	===BODY===
	<the translated body>
	===FRONTMATTER===
	<the translated values as a JSON object, same keys, same order>
	===BODY===
	<the translated body>

[gemini-code-assist]

Similar to the translation prompt, the revision system prompt instructs the model to output key: value lines for the front matter, which conflicts with the JSON object format expected by _FM_PROTOCOL in translate.py. Updating this to instruct the model to return a JSON object will prevent protocol errors during the revision loop.

Suggested change

	===FRONTMATTER===
	<corrected key: value lines, same keys, same order>
	===BODY===
	<corrected body>
	===FRONTMATTER===
	<corrected JSON object, same keys, same order>
	===BODY===
	<corrected body>

[Andrei Kvapil (kvaps)]

Reviewed this PR's own diff (36 files) against its base, with #593 now merged. Ran hack/i18n/test_i18n.py (49 pass) and hack/check-i18n.sh (clean). The design is careful and the README is unusually honest about what the gate is and isn't — nearly all of my comments are about the runner, not the translation logic.

Blocking: the daily runner wedges on its second run

run-daily.sh translates on whatever branch the clone happens to be on, publishes on a different one, and the EXIT trap restores the original branch (hack/i18n/run-daily.sh:79-81, 115-126). In order:

1. Day 1 commits translations onto i18n/week-<N> and pushes. The trap then checks out START_REF, which deletes those files from the working tree — they now only exist on the week branch.
2. Day 2 therefore sees them as missing again (build_worklist checks the filesystem, lib.py:197-208) and re-translates the exact same pages, spending a day of quota redoing day 1.
3. Publishing then fails hard: the re-translated files are untracked, origin/i18n/week-<N> already contains them, and git checkout -B "$BRANCH" origin/$BRANCH (run-daily.sh:121) aborts with "The following untracked working tree files would be overwritten by checkout". Under set -e the run dies leaving the tree dirty, so every later run stops at the clean-tree preflight (run-daily.sh:74-78).

I reproduced this in a scratch repo following the exact command sequence: day 2 exits 1 at the checkout and leaves ?? content/ru/ behind. The first run of a new week hits the same wall through the origin/main path once the previous week's PR is merged.

Root cause is that the working tree is never synced to the remote — git fetch (line 115) updates refs only, so both the English sources and the already-published translations stay frozen at clone time, and worklist correctness depends on that tree.

Suggestion: check out the week branch before translating — fetch, git checkout -B "$BRANCH" origin/$BRANCH (or origin/main), then run translate.py, then commit in place. That makes the tree the branch the work belongs to, keeps day 2's worklist accurate, removes the untracked-file collision, and refreshes content/en in the same move.

Worth fixing

The revise loop's last round is never re-reviewed. translate.py:271-319 — on the final iteration a page with findings is revised and the loop then exits. The text written is post-revision, but findings (stamped, returned, and posted to the weekly PR) describe the pre-revision text. So the finding list in the PR comment doesn't correspond to the pages actually shipped, and the last Opus call for each failing page produces output nothing ever checks. Either re-check after the last revise, or skip revising on it.

"A page is only written once it passes the gate" (README.md:35, config.yaml:115) contradicts the code and the rest of the README: pages that exhaust the revise loop are written, stamped -with-findings (translate.py:342-346, README.md:249-252). The diagram at README.md:13-33 has no edge for that path either. This is the sentence a reviewer leans on, so it's worth making it say what happens.

auto_merge exists. The default is pr_only and the claim in the description holds today, but run-daily.sh:170-173 will call gh pr merge --auto --squash after a one-word config change, with no second signal required. If "nothing reaches production without a maintainer merge" is meant as a property rather than a default, consider dropping that branch — nothing in this PR uses it.

Failures in the publish stage are swallowed. gh pr create ... || true and gh pr comment ... || true (lines 152-154, 167). A failed gh call leaves commits pushed, no PR opened, and exit 0 — indistinguishable from a good run in cron output.

Unpinned dependencies. pip install --quiet claude-agent-sdk pyyaml runs on every invocation (line 34), upgrading in place on a machine that holds a maintainer's CLAUDE_CODE_OAUTH_TOKEN and gh credentials. A pinned requirements.txt costs nothing here.

An i18n problem already visible in the shipped strings

The banner builds one sentence out of four keys plus link text (layouts/partials/translation-banner.html:39-42). That hardcodes English word order and punctuation, and two of the six languages already read wrong:

• hi: "देखें अंग्रेज़ी मूल" — Hindi is verb-final; the natural order is "अंग्रेज़ी मूल देखें".
• de / ru: "Einen Fehler entdeckt? eröffnen Sie ein Issue." and "Нашли ошибку? создайте issue." — lowercase after a question mark.

One key per sentence, with the link as a placeholder, lets a translator move it. Worth noting these UI strings never pass through the pipeline's review gate — only content/ does.

Smaller things

• .gitignore:32 and :36 — .venv-i18n/ listed twice.
• README.md:93 says 183 × 4 = 720 jobs; config.yaml lists six languages and the description says ~1084.
• Nothing removes a translation whose English source was deleted, and check-i18n.sh only checks digests of files that exist, so orphans accumulate silently.
• blog_since: "2026-05-17" is a fixed date that will quietly rot into "nothing new is in scope" — the config comment already suggests computing now-60d.

On the security posture and the gate

No workflow is added, so the usual automation risks don't apply: no pull_request_target, no repo-held secret, no untrusted checkout. Model calls use allowed_tools=[] and max_turns=1 (translate.py:117-122), so the translator has no filesystem or tool access, and git add -- content/<lang> genuinely bounds what a run can commit. What's left is that everything runs from one person's machine on their personal subscription, with their gh token and their DCO sign-off on machine-generated commits — and that the human merge gate is procedural, since no maintainer can meaningfully review Hindi or Chinese docs in a bulk weekly PR. The README says as much and the banner is the right mitigation; I'm noting it, not objecting.

The gate is real machinery rather than a stamp: verdict parsing fails closed (translate.py:149-172), an explicit "revise with no findings" verdict still blocks (lines 288-290, 300-302), and a lost or duplicated placeholder is a hard refusal to write (lines 213-219). Its ceiling is the one the description already admits — same model, so it measures self-consistency. Worth being precise that it never blocks publication, only stamps: it's a triage classifier, not a gate.

Requesting changes for the runner bug alone; the rest is small. Noting for the record that this is a draft and was stacked on #593, which has since merged.

[netlify]

❌ Deploy Preview for cozystack failed.

Name	Link
🔨 Latest commit	d38db12
🔍 Latest deploy log	https://app.netlify.com/projects/cozystack/deploys/6aaf5b8e530b2d0008982850

[Timur Tukaev (tym83)]

All findings addressed in 4653ba7..ae79b45 (8 commits). Point by point:

Runner wedge (blocking). Fixed the way you suggested: the week branch is now checked out before translating (fetch + checkout -B from origin/i18n/week-N or fallback base), and the run commits in place — 1f7f304. Reproduced your two-day scenario in a scratch repo before and after: day 2 now sees day 1's pages as done and the untracked-file collision cannot occur. Going further down that path surfaced three more holes in the reordering, fixed in 025d2a1 and ae79b45:

• a new week always based on origin/main would re-translate an unmerged previous week and open a conflicting PR — a new week now bases on the newest surviving i18n/week-* branch;
• checkout -B origin/... silently discarded commits stranded by a failed push — a local branch ahead of (or unknown to) origin is kept, and stranded branches are re-pushed at the start of every run;
• the re-push logic in turn had to discriminate zombies: a merged-and-deleted branch whose local copy survived is deleted (by PR state via gh pr view, since ancestor checks break under squash merges), not resurrected. One residual edge remains: a morning where gh is broken but git push works could re-plant a zombie once; cheap hardening later is to skip that leg when gh auth status fails.

Revise loop. The final round no longer revises — 5c1269d. The text written is always the text the reviewers last saw, and the findings posted to the weekly PR describe the pages actually shipped.

README vs code. Reworded to say the gate triages rather than blocks, added the missing -with-findings edge to the diagram, fixed the job math (183 × 6 ≈ 1100), and noted in Governance that it's a triage classifier — 62395fc.

auto_merge is gone as a code path, not a default — b522046. || true on gh pr create/comment dropped in the same commit: commits are pushed by then, so a failed publish now fails the run. Deps pinned in hack/i18n/requirements.txt.

Banner. One i18n key per sentence with the link as a template placeholder, all seven languages — b90a72f. Hindi is verb-final now («अंग्रेज़ी मूल देखें»), de/ru capitalize after the question mark, zh-cn uses full-width punctuation. Verified with a full Hugo build rendering the banner in ru/de/hi/zh-cn. Agreed these strings bypass the content gate — they're hand-reviewed.

Smaller things — 45326b6, 62395fc: .gitignore dup removed; orphaned translations are now removed by the pipeline (only source_digest-stamped files, with a mass-deletion floor of 5 English pages and a hard refusal to run against a missing content/en); blog_since accepts a rolling "60d" window and the config uses it. One note: check-i18n.sh did already report orphans (references English source that does not exist) — what was missing was removal.

Tests: 54 pass (5 new), check-i18n.sh clean. Unrelated but spotted along the way: head-end.html drops the latest_version_id fallback — if that param ever disappears, all docs go noindex silently; happy to fix here or separately.

[IvanHunters]

Verdict

LGTM with non-blocking notes

The core safety claims (code/shortcode byte-for-byte preservation, fail-closed reviewer-verdict parsing, no direct push to main, no auto-merge, es/pt-br declaration ordering) were independently re-derived from the code and executed, not just read, and all held up. Remaining items are non-blocking process/observability gaps.

Findings

[MINOR] hack/i18n/run-daily.sh:85-87,76-84 — no lock against overlapping invocations

The script's only defence against concurrent runs is a documented convention and a dirty-working-tree check. Two invocations against the same clone (a manual run overlapping a slow cron run) can both pass the clean-tree check, then race on git checkout -B "$BRANCH", commits, and git push. Worst case is a confusing local git state (no production impact, since nothing reaches main without an explicit PR merge), but it is silent until the second run's git push fails. A flock/pidfile guard at the top of the script would make this fail loudly and immediately instead of racing.

[MINOR] hack/i18n/translate.py (usage-limit stop / per-page skip-after-retry) — reported to stdout/stderr only, no durable record

RateLimited (daily quota hit) and repeated ProtocolError (page skipped after PROTOCOL_ATTEMPTS) are reported only via print(...). There is no GitHub Action (by design) and no log file recommended in the README's cron/launchd setup, so if the invoking job doesn't itself capture and monitor output, a page that fails every day, or a run that silently stops early on quota, has no durable signal for a maintainer. Not destructive (the page stays in the worklist and retries), but worth a one-line README addition (redirect to a rotated log, or a periodic worklist.py summary check).

Caveats

• Hugo mount-for-undeclared-language mechanism verified in an isolated minimal Hugo build: a module.mounts entry for a language absent from languages: produces no output directory, so es/pt-br cannot publish empty indexable pages. A full hugo --gc --minify of the real tree could not complete in this environment (stalled on Hugo Modules resolution, unrelated to this diff); the targeted repro is what the verdict rests on.
• bash hack/check-i18n.sh clean (key parity + freshness). python3 hack/i18n/test_i18n.py: 54/54 pass offline, including test_dropped_placeholder_is_rejected, test_duplicated_placeholder_is_rejected, test_unparseable_fails_closed, test_every_served_language_has_content.
• "never publishes without a maintainer merge" traced end to end: every git push targets refs/heads/i18n/week-<ISO week>, gh pr create opens a PR, no gh pr merge/--auto anywhere. .github/CODEOWNERS unchanged in the final diff.
• No committed/logged secrets: no workflow ships in this diff; CLAUDE_CODE_OAUTH_TOKEN/ANTHROPIC_API_KEY are read from environment only and never printed. .gitignore correctly ignores .venv-i18n/ and last-run-findings.md.
• The two included sample pages contain no fenced code blocks, so they don't empirically exercise the "code preserved byte-for-byte" claim on real content — that guarantee rests on lib.py's protect/restore plus the placeholder-count check in translate.py, both read and both covered by passing unit tests. Samples do demonstrate inline-code and {{% ref %}}/{{% alert %}} shortcode preservation.
• Prompt-injection surface: translate.py embeds raw English source into reviewer prompts unsanitized; mitigated by CODEOWNERS gating content/en/** behind human review before the pipeline sees it.

Recommended follow-ups

• Add a flock/pidfile guard to run-daily.sh so a second concurrent invocation fails fast instead of racing on the same clone.
• Document (README "Cadence") how cron/launchd output should be captured so a stuck run or a page repeatedly failing ProtocolError isn't missed.
• Re-request review / dismiss the stale CHANGES_REQUESTED — the blocking runner bug and all secondary points from the prior review are addressed in the current HEAD.

[Timur Tukaev (tym83)]

IvanHunters thanks for the deep pass — re-deriving the safety claims by execution is exactly the review this pipeline needed. Both follow-ups landed in 25732ae:

Overlap guard. Went with a pidfile rather than flock(1): the runner may well be a macOS/launchd machine, where flock doesn't exist. Per-clone lock in $TMPDIR (keyed by clone path, so two different clones on one box don't block each other), second invocation fails fast with the holder's pid; a lock whose pid is dead is taken over instead of wedging every subsequent cron run. Smoke-tested both branches: live pid blocks, dead pid takes over. Lock cleanup is folded into the existing EXIT trap (bash keeps only the last trap, so a separate one would have silently dropped the branch restore).

Log capture. README "Cadence" now shows the cron line with an append-only log and the two weekly health signals: grep -c '::warning::skipped' (a page stuck on protocol errors) and worklist.py | head (backlog must trend to zero). Agreed there's no durable channel by design — the log file is the durable channel now, and the README says so explicitly.

54/54 tests still pass, bash -n clean.

On the prompt-injection note: agreed the mitigation is CODEOWNERS on content/en/** — worth keeping in mind if the pipeline ever grows a path that translates content a human didn't merge (e.g. community PRs pre-merge previews). It doesn't have one today.

Andrei Kvapil (@kvaps) the blocking runner bug and all secondary points from your review are addressed in the current HEAD (4653ba7..25732ae, summary in my comment above) — could you re-review / lift the CHANGES_REQUESTED when you get a chance?

[Aleksei Sviridkin (lexfrei)]

NOT LGTM — two write paths corrupt or silently downgrade pages, and both are reachable on the next run.

Business context: the English docs change constantly and native localization takes months, so this turns translation into a daily pipeline with a machine review gate and a weekly maintainer-merged PR, letting localized docs ship immediately with honest disclosure.

Ran the pipeline tests (74 pass), hack/check-i18n.sh (clean), and a production Hugo build to check the SEO changes. The runner rework holds up: the week branch is checked out before translating, the stranded-push and zombie-branch paths are handled, gh failures are no longer swallowed, and dependencies are pinned. My findings are in the translation core rather than the runner.

Blockers

B1: .html pages reach the model with their markup unprotected and unchecked

File: hack/i18n/lib.py:347

protect() masks fenced code, shortcodes, HTML comments and inline code. Raw HTML tags are not on that list, and .html pages are mostly raw HTML.

Evidence: running lib.protect() over the in-scope sources, content/en/docs/v1.5/roadmap.html yields zero protected spans — all three <a href="..."> links, the target="_blank", the mailto: and the <br /> go to the model verbatim. content/en/_index.html gets 25 spans (the shortcodes) but every <div class="hero-gitops">, </div> and <h2 class="section-label"> stays exposed. _split_payload_response (translate.py:217-223) only counts §§...§§ tokens, so with no tokens present there is nothing to fail closed on. worklist.py --lang ru lists docs/v1.5/roadmap.html today, so this is the next run and not a hypothetical.

Impact: a translated class= attribute, a dropped </div>, or a mangled URL ships silently. The gate does not check markup, check-i18n.sh only compares digests, and Hugo renders broken HTML without complaint. README.md:179 says URLs are preserved structurally, which holds for markdown links but not for these.

Fix: add an HTML-tag span to protect(), or drop .html from translate_globs and translate only their front matter.

B2: hand-written transcreations are overwritten with literal machine output

File: hack/i18n/translate.py:350

Evidence: eight pages carry l10n: transcreate from the proof-of-concept — content/{de,ru,hi,zh-cn}/_index.html plus four blog posts. _index.html is explicitly in translate_globs (config.yaml:107), and neither iter_source_files nor build_worklist consults l10n, so the page is re-translated the moment its English source changes. I checked that ru/_index.html's digest matches today, so staleness is one edit away. translate_page then replaces the body wholesale and writes out_fm["l10n"] = "mt" unconditionally, after the merge_target_only_keys call on line 344 — the marker the README lists as the human triage signal is the thing that gets clobbered.

Impact: four hand-crafted homepages become literal machine translation. Because the banner is wired into docs/baseof.html only, the localized homepage then carries that machine output with no disclosure at all. B1 applies to the same pages, so the markup is at risk in the same write.

Fix: skip a target page whose existing front matter says l10n: transcreate (or list those paths in exclude_globs) and report them in the weekly PR instead of overwriting them.

Non-blocking follow-ups

1. run-daily.sh:212-214 — PREV_WEEK picks the newest remote week branch with no PR-state check. Merged branches are safe (delete_branch_on_merge is on and fetch --prune drops them), but closing a PR never deletes its branch, so rejecting a week's translations means next week bases on them and re-proposes the lot. The local-zombie path just above already asks gh pr view for state.
2. run-daily.sh never checks gh up front. The first call is gh pr view ... || echo NONE (line 174), so an unauthenticated runner burns a full day of quota and pushes a branch before failing at gh pr create (line 292). A gh auth status alongside the claude/API-key preflight would fail in the first second instead.
3. layouts/partials/hooks/head-end.html:50 uses .Language.LanguageCode, deprecated in Hugo v0.158.0 in favour of .Language.Locale; the pinned 0.160.1 warns on every build. Related: the two new module.mounts entries use lang:, deprecated in v0.153.0 in favour of sites.matrix. The lang: pattern is pre-existing, but these are new mounts.
4. hack/i18n/config.yaml declares hreflang: per language while the template reads .Language.LanguageCode from hugo.yaml. Nothing consumes the config field, so either wire it up or drop it before it drifts.
5. worklist.py has no --path, though its docstring claims flag parity and run-daily.sh forwards the same "$@" to both. The argparse error is swallowed by 2>/dev/null, so --path silently prints no preview.
6. has_ref_shortcode runs after the whole gate, but it is a pure function of the English source since ref shortcodes are masked byte-for-byte. An unsupported shape therefore costs three full gates per language per day before anyone notices. All 315 refs across the 182 in-scope pages deref cleanly today, so this is a cost question rather than a correctness one.
7. Once content/es/ lands without es in hugo.yaml, Hugo silently drops the mount — I built with a probe page and got no /es/ output and an untouched English tree. That is safe, but nothing signals that translated pages are accumulating unserved, and test_every_served_language_has_content only guards the other direction.
8. The description still says this targets poc/i18n-multilang pending #593. That merged on 2026-07-20 and the base is already main.

Checked and dismissed: .Sites.Default is correct (it is the defaultContentLanguage site, available since Hugo 0.123, and resolves fine in the build); the duplicate .venv-i18n/ entry is gone; both prompts now specify the JSON front-matter protocol the code parses; and hreflang emits relative URLs in a bare local build but absolute ones under the production --baseURL, which is what Google needs.

[Aleksei Sviridkin (lexfrei)]

protect() masks fences, shortcodes, HTML comments and inline code, but not raw HTML tags — and .html pages are mostly raw HTML.

Running this over the in-scope tree, content/en/docs/v1.5/roadmap.html produces zero protected spans: its three <a href="..."> links, the target="_blank", the mailto: and the <br /> all reach the model verbatim. content/en/_index.html gets 25 spans for its shortcodes, but every <div class="hero-gitops">, </div> and <h2 class="section-label"> stays exposed.

The placeholder guard in _split_payload_response only counts §§...§§ tokens, so on a page with no tokens there is nothing to fail closed on. A translated class= attribute, a dropped </div> or a mangled URL ships silently — the gate does not check markup, check-i18n.sh only compares digests, and Hugo renders broken HTML without complaint.

docs/v1.5/roadmap.html is in the worklist right now, so this lands on the next run.

Either add an HTML-tag span here, or drop .html from translate_globs and translate only their front matter.

[Timur Tukaev (tym83)]

Thanks — you were right, and it was worse than "markup reaches the model": the placeholder guard in _split_payload_response only counts §§…§§ tokens, so on a page that produces none there was nothing to fail closed on at all. Reproduced exactly what you described:

page	protected spans	raw tags exposed
docs/v1.5/roadmap.html	0	10
_index.html	25	23

Fixed in #630 (41255d0) by masking the tags rather than dropping .html from translate_globs — losing the roadmap pages from translation seemed the worse trade. Only the tag is masked; the text between tags stays exposed so it still gets translated. Autolinks are masked first, since <https://…> starts with a letter and would otherwise be swallowed by the tag pattern, and a bare < in prose (a value < 10) is deliberately not matched.

After the fix, on the same two pages:

page	protected spans	raw tags exposed	round-trip
docs/v1.5/roadmap.html	10	0	lossless
_index.html	48	0	lossless

So those pages now carry tokens, which means the existing fail-closed guard actually has something to check. Six regression tests added, including the < -in-prose case and the autolink-ordering trap.

[Aleksei Sviridkin (lexfrei)]

This overwrites l10n unconditionally, after the merge_target_only_keys call on line 344 — so the marker the README lists as the human triage signal is exactly what gets clobbered.

Eight pages carry l10n: transcreate from the proof-of-concept: content/{de,ru,hi,zh-cn}/_index.html plus four blog posts. _index.html is explicitly in translate_globs (config.yaml:107), and neither iter_source_files nor build_worklist consults l10n, so the page is re-translated as soon as its English source changes. ru/_index.html's digest matches today, which puts staleness one homepage edit away.

At that point four hand-crafted homepages become literal machine translation — and since the banner is wired into docs/baseof.html only, the localized homepage carries that output with no disclosure at all.

Suggest skipping a target page whose existing front matter says l10n: transcreate (or listing those paths in exclude_globs) and reporting them in the weekly PR instead.

[Timur Tukaev (tym83)]

Thanks — good catch, and the ordering detail you pointed at is exactly what made it bite: the assignment runs after merge_target_only_keys, so the explicit write always won and the marker the README names as the human triage signal was the one thing guaranteed to be destroyed.

Confirmed the eight pages you found:

content/{de,ru,hi,zh-cn}/_index.html
content/{de,ru,hi,zh-cn}/blog/2026-05-19-introducing-cozystack-wizard/index.md

And confirmed the exposure: _index.html is in translate_globs, neither iter_source_files nor build_worklist consulted l10n, and the banner is wired into docs/baseof.html only — so a localized homepage would have carried machine output with no disclosure at all.

Fixed in #630 (41255d0), taking your first suggestion rather than exclude_globs, so the rule travels with the page instead of living in a path list someone has to remember to update:

• build_worklist skips any target whose front matter carries l10n: transcreate (PRESERVED_L10N_VALUES, so the set is one constant to extend);
• translate_page no longer downgrades l10n — belt and braces, in case a page ever reaches it another way;
• drifted hand-localized pages are reported in the weekly PR under "Hand-localized pages that drifted", since silently skipping them would just trade one invisible failure for another — they'd quietly diverge from an English source nobody is tracking.

Verified against the tree: all 8 pages are now out of the worklist (worklist is 1078 items, none of them these). Zero are currently drifted — their digests still match — so the guard is in place before the first homepage edit rather than after it. Four regression tests added, including a stale-digest transcreated page that must stay out of the worklist and show up in the drift report.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[Timur Tukaev (tym83)]

Aleksei Sviridkin (@lexfrei) your two blockers here — unescaped raw HTML in .html pages, and the transcreate overwrite — are both fixed in #630, verified on the current stack head. Once #630 lands in feat/i18n-pipeline this branch inherits the fixes and is ready to clear. (The runner review is separate and already addressed upstream.)

[myasnikovdaniil]

Pipeline looks solid, placeholder guard and orphan removal especially. Before this lands I want three things resolved, two of them inline.

Main one is not in this diff at all: there is no navigation path to translated docs. hreflang is added in head-end.html, but version links are not language aware. layouts/docs/docs-landing.html:40 renders href="{{ .url }}" where .url is /docs/v1.6/, and version-switcher.html:7 does hasPrefix $currentPath .url, which never matches /ru/docs/..., so on russian pages the switcher shows literal Versions and every item in it goes to english. On prod curl -s https://cozystack.io/ru/docs/ | grep '<td><a' returns eight version links and none of them has /ru/. So we tell crawlers the language versions exist and give the reader no way to click there. I think it needs separate PR before this one.

Same area: /ru/docs/v1.4/ is 404 because language tree has no version _index.md. v1.6 will get one from the pipeline, older versions stay 404.

[myasnikovdaniil]

This is what makes PoC leftovers permanent. Page that left the scope is not orphan because english source still exists, and worklist doesn't see it either, so content/ru/docs/v1.4/getting-started/_index.md and its 3 siblings on main will never be refreshed and never removed. They are unreachable from navigation too, so nobody will notice them rotting.

Either count out of scope pages as orphans and delete them, or clean the v1.4 leftovers in this PR and say in README that this is known debt. Right now it is neither.

[myasnikovdaniil]

Branch is 56 commits behind main and main is already on latest_version_id: v1.6. These two pages land out of scope on the day they merge, pipeline won't refresh them, orphan removal won't take them, so they become exactly the leftovers I described in the lib.py comment. Rebase and regenerate for v1.6, or drop them from this PR.

[myasnikovdaniil]

This is single point of failure and I don't think we should ship it this way. There is no Action, so the pipeline runs from one maintainer clone (dedicated one, per run-daily.sh header), on a personal Max subscription, with token that expires in about a year, and the weekly PR is merged by hand as well. If that person is on vacation or leaves, translations just stop, silently. No failed run to see, only worklist growing that somebody has to run manually.

And it doesn't stay contained in translations: english docs keep updating automatically from the upstream tags workflow, digests drift, and i18n-lint starts failing PRs for everybody else, see my comment in #630.

auth: api-key is already implemented right here and README calls org key the plan, so what blocks doing that now, as a GitHub Action? If the key is not provisioned yet, then README should at least name who owns this and what to do while they are not around.

[Aleksei Sviridkin (lexfrei)]

NOT LGTM — B1 is closed, B2 only half, the branch conflicts with main, and the new commits add an orphan-cleanup regression and a nesting fix the pipeline never reaches.

I re-reviewed the three commits since 12e90d5c9 at 2652726. Tests: 87 pass (74 before). hack/check-i18n.sh is clean at this head.

My two blockers

B1 (raw HTML sent unprotected) is closed. At this head lib.protect() gives content/en/docs/v1.5/roadmap.html 10 spans, all tags, and content/en/_index.html 48 (22 shortcode, 3 comment, 23 tag). No tag is left exposed, and both pages round-trip byte-for-byte. Restore fails closed: _split_payload_response raises ProtocolError when I drop one tag token from the reply, and again when I duplicate one, while the unchanged reply goes through. The Markdown versions that main now carries give the same result (_index.md: 63 spans, 33 tags). Deleting the tag pass turns three TestHtmlProtection tests red.

B2 (hand-written pages overwritten) is half closed. Only the worklist refuses a transcreate page now, and three gaps remain:

1. The writer still sets out_fm["l10n"] = "mt" unconditionally (translate.py:350). Called directly on a stale l10n: transcreate page with a stubbed model, translate_page() returns l10n: mt. No caller skips the worklist today, but nothing pins this either: delete line 350 and all 87 tests stay green.
2. l10n_mode() uses a regex that accepts only the bare and double-quoted forms. l10n: 'transcreate' is the same YAML value, yet it returns None, so that page goes back into the worklist and is not reported.
3. The report never fires once the backlog drains. With only a stale transcreate page left, main() takes if not items: (translate.py:497), deletes the report and returns before find_transcreated runs (line 586), so run-daily.sh has nothing to post. test_empty_worklist_clears_a_prior_run_report pins that early return.

Fix: check the existing page's l10n inside translate_page too, read it by parsing the front matter as YAML instead of with a regex, and build the report on the empty-worklist path. #630 already has the first and the third, see the last section.

New blockers

N1: conflicts with main, and the merged tree fails the i18n lint

git merge-tree against current main conflicts in hugo.yaml and .gitignore. No GitHub Actions run has happened since 18b140aa2, and that i18n lint run failed. With the conflict resolved in a scratch copy, hack/check-i18n.sh fails: i18n/es.toml and i18n/pt-br.toml lack the five home_use_cases* keys that main added to en.toml.

For hugo.yaml, take the security block from main and drop allowContent. In 43f8571 main converted its fourteen HTML content files to Markdown, because Hugo 0.162.0 disallows text/html content by default as a security hardening (per the release notes; the comment here says 0.163). After the merge nothing under content/ is .html, and the merged tree builds on 0.164.0, the version main pins, without allowContent. Keeping the key re-allows text/html content while no page needs it. The _index.html entry in translate_globs also matches nothing after the rebase, and *.md already covers the homepages.

N2: retiring a docs version stops all orphan removal

hack/i18n/lib.py:318. Since 18b140aa2, every stamped translation of a non-latest version is an orphan, and these orphans count against the same five-page ORPHAN_PAGE_FLOOR that guards against a moved English tree. Once the latest version is translated, the next release orphans all of it at once. I built that case: latest v1.7, six stamped ru pages under docs/v1.6/, plus a stamped translation of a blog post whose English source was deleted. translate.py --lang ru --dry-run prints the floor warning and removes nothing, including the deleted post's translation. Every later run does the same until someone deletes the files by hand. Before this commit a version bump created no orphans, so deleted pages kept being cleaned up. The unit test uses one page, which is under the floor. Retired versions need a path that does not count toward the floor.

N3: the nested-placeholder fix is unreachable

hack/i18n/translate.py:217. restore() now unwinds a shortcode stashed inside inline code, but _split_payload_response rejects the reply before that: it wants every stored token exactly once in the body, and the inner token is never there because it sits inside the outer token's stored text. As a positive control I fed each in-scope page its own masked body back. Two of 172 pages fail: docs/v1.5/install/ansible.md (line 180, the §§SC_23§§ from the restore() docstring) and docs/v1.5/install/talos/boot-to-talos.md (line 29). Their v1.6 copies fail the same way after the merge. Neither page can ever be translated, and each daily run burns three translate calls per language on each of them. test_shortcode_inside_inline_code_survives never calls _split_payload_response, so it passes anyway. Either check only the tokens present in the masked text, or stop nesting at masking time.

N4: commit and PR text

This repository allows only merge commits, and the merge commit message is the PR body, so every commit and the body reach main as written.

• 31 of the 38 commits, including cb0e2e7f4 and 2652726e9, carry Co-Authored-By: Claude <noreply@anthropic.com>. A trailer should not name a model or vendor; I'd ask for Assisted-by: LLM instead. I missed this last time.
• cb0e2e7f4 says "Addresses the two blocking findings from review." and "Fix a nesting bug found while verifying the above", and a2677d21b is titled "address maintainer review". The messages should explain the change without pointing at the review.
• 18b140aa2 switches the banner to .Site.DefaultContentLanguage, which breaks the 0.160.1 build (can't evaluate field DefaultContentLanguage in type page.Site), and 2652726e9 repairs it. With merge commits main would keep a commit that does not build, so fold the repair into 18b140aa2.
• The body still says the PR targets poc/i18n-multilang and is stacked on #593.

N1 needs a rebase anyway, and squashing into a few logical commits during it fixes this list as well.

The build commit

At 12e90d5c9 the banner used .Sites.Default. 18b140aa2 swapped in .Site.DefaultContentLanguage, the bot's suggestion, and 2652726e9 went back to site.Sites.Default. So the last commit fixes a break from earlier in this branch, and my July check still stands. The production build at this head on 0.160.1 succeeds with 1598 pages, and the banner shows on translated docs pages only. It warns about module.mounts.lang (deprecated since 0.153.0) and about .Site.Sites and .Page.Sites (since 0.156.0). The second warning comes from banner line 39, and .Sites.Default raised it too; I missed that in July. With hugo.Sites.Default.Language.Lang the warning is gone, and the ru and en pages I compared are byte-identical. A correction to my follow-up 3: on 0.160.1 the .Language.LanguageCode deprecation is logged at INFO, and it becomes a WARN on 0.164.0.

Non-blocking

• Two new tests cannot fail for what their names claim. test_prose_comparison_is_not_swallowed stays green with the tag regex set to <[^\n]*, since any mask round-trips. test_tag_inside_inline_code_is_protected_once stays green when the tag pass runs before inline code and nests. Assert on the masked text or the store instead.
• The first run after the merge deletes both v1.5 sample pages as a superseded version, along with the four v1.4 proof-of-concept pages: six files, two distinct pages, under the floor. myasnikovdaniil's thread on install-kubernetes.md asks to regenerate or drop them. None of his three threads has a reply.
• My earlier follow-ups: 5 (worklist.py --path) is fixed. 1 (PREV_WEEK ignores PR state), 2 (no gh auth status preflight), 3 (Hugo deprecations), 4 (unused hreflang:), 6 (has_ref_shortcode after the gate), 7 (content/es without es) and 8 (the description) are unchanged and have no reply.
• The code now covers everything kvaps raised on 20 July; his review is still marked as requesting changes at 4653ba7.

Overlap with #630

In the B1 and B2 threads you wrote that both were fixed in #630 (41255d0). cb0e2e7f4 then fixes both again on this branch under other names: find_transcreated, _format_transcreated and l10n_mode here, find_hand_localized, _format_hand_drift and PRESERVED_L10N_VALUES there. That is where the new conflicts of #630 in lib.py, translate.py and README.md come from. The #630 version also has the writer guard and the empty-worklist report that B2 still lacks. This PR has to merge first, so I'd move the #630 versions down into this branch in place of cb0e2e7f4 and rebase #630 on top.

[Aleksei Sviridkin (lexfrei)]

Still unconditional, so a transcreate page that reaches the writer loses its marker. See B2 in the review body.

[Aleksei Sviridkin (lexfrei)]

With only a stale transcreate page left, the worklist is empty and this return runs before the skipped-pages report is built. See B2.

[Aleksei Sviridkin (lexfrei)]

Retired-version orphans go through the five-page floor in main(). After the next release the floor trips on every run, and translations of deleted pages stop being removed too. See N2.

[Aleksei Sviridkin (lexfrei)]

A token nested in another token's stored text never appears in body, so this refuses even an unchanged reply for ansible.md and boot-to-talos.md. See N3.