feat(i18n): deterministic integrity/typography checks and real per-language style guides

[Timur Tukaev (tym83)]

Summary

Three quality improvements to the translation pipeline. No architectural change — the glossary + style-guide + back-translation + two-reviewer + digest design is kept and extended.

Stacked on feat/i18n-pipeline, since that is where the pipeline lives.

What

• Link destinations are masked. Masking previously covered code, shortcodes and comments, so a URL sitting in ordinary prose ([text](https://…)) was defended by a prompt rule alone — the model could mutate or localize it and nothing would catch it. URLs are now placeholders like any other protected span, while link text stays exposed and still gets translated.

• Two deterministic checks feed the existing revise loop.

  • integrity_findings() compares versions, bare CLI flags and do-not-translate terms between source and translation, catching a localized version separator (v1.5 → v1,5), a dropped --flag, or a transliterated brand.
  • check_typography() enforces the rules the style guides state: Russian guillemets, German „…“, Spanish inverted marks, Chinese full-width punctuation, European-Portuguese vocabulary leaking into pt-BR, Devanagari digits.

  Both look at prose only — markup, code, link targets and list markers are exempt, so the checks do not fire on ASCII punctuation that is correct inside an HTML attribute or a path.

• The six per-language style guides were 3–6 lines each, yet the entire fluency and typography strategy rests on injecting them into the translate prompt and both reviewer prompts. They are now 80–100 lines each and work as an instruction set and a review rubric: register and address form, heading conventions, a decision rule for terms outside the glossary, typography, number/date formatting, the grammar traps of translating from English into that language, calque patterns with fixes, false friends, and a per-language checklist of MT failure modes. Every original decision is preserved and expanded around.

• Fixed a live contradiction. Hard rule 5 in prompts/translate.md said to keep numbers unchanged, while several style guides require a decimal comma or a different date order in prose — the model was reading two incompatible instructions. The rule now separates a number's value (and any version or identifier: literal, reproduced character-for-character) from its formatting in prose (follows the language).

• lint_translations.py applies the typography rules to already-published pages, where a hand edit is otherwise never re-checked. Advisory by default, --strict to gate.

Why

Only back-ticked code was structurally guaranteed; bare-prose commands, identifiers and link URLs were soft. On a docs site this dense with code, a silently mutated URL or flag is the most expensive failure mode available. The style guides were the second gap: the architecture already routes them into three prompts, but there was almost nothing in them to route.

Validation

• 91 tests pass (74 existing + 17 new covering link masking, integrity findings and typography).
• The typography lint reports 0 findings across the entire existing translated tree after false-positive tuning.

Deliberately left as follow-ups

Glossary growth loop (harvesting recurring terminology findings into glossary.yaml), a needs_human reviewer flag distinct from revise, and a severity-based soft hold on the auto-reviewed stamp.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c3359bb9-e336-4a1b-9f4e-51bc81ee8d45

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/i18n-quality-improvements

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Aleksei Sviridkin (lexfrei)]

NOT LGTM. The new masking passes make almost half of the in-scope docs untranslatable, and the deterministic checks contradict the style guides they are supposed to enforce.

Business context: the pipeline defended URLs, bare flags, versions and per-language typography with prompt rules only; this PR makes those guarantees deterministic (link/HTML masking plus integrity and typography checks feeding the revise loop) and turns the stub style guides into real per-language contracts.

Blockers

B1: pages with [text]({{< ref ... >}}) links can no longer be translated

protect() step 4 re-stashes placeholders created by earlier passes (hack/i18n/lib.py:58). In [text]({{% ref "/x" %}}) the shortcode pass stashes the ref as SC_0, then _LINKDEST_RE stashes that token again as URL_1 whose stored value is the SC_0 placeholder. The model only ever sees URL_1, so the placeholder guard counts SC_0 at 0 in every reply and raises ProtocolError on every attempt. restore() (lib.py:475) is a single pass in insertion order and cannot unwind the nesting either, and the back-translation path (lib.restore(back_en, tr_store)) restores with no guard at all. Same mechanism for [text](<https://...>) and [id]: <https://...>, legal CommonMark, none in the corpus today.

Reproduced: protect() on [install guide]({{% ref "/docs/install" %}}) yields a store where the URL token's value is itself a placeholder, and the guard on a token-preserving reply reports the inner token at 0. Running protect() over the whole configured scope: 82 of 181 pages produce nested placeholders (]({{< ref ... >}}) occurs about 279 times in scope). Each of those pages deterministically fails every protocol attempt on every run and burns the full retry budget of model calls first.

Fix in the same layer: skip destinations that contain the placeholder marker in the _LINKDEST_RE/_REFDEF_RE substitutions, iterate restore() in reverse insertion order (later stashes can only reference earlier tokens), and add a fail-closed residual-placeholder scan after restore for the unguarded back-translation path. Round-trip tests for [t]({{< ref "x" >}}) and [t](<https://x>) belong in TestLinkDestinationMasking.

B2: the version-integrity check makes the style guides' own mandated conversions gate-fatal

_VERSION_RE (lib.py:553) counts every bare decimal in prose and demands byte-for-byte survival as a major finding, while four style guides rewritten in this same PR mandate decimal comma in prose ("3.14" to "3,14", "0.5 vCPU" to "0,5 vCPU") and the prompt fix in this PR explicitly permits prose reformatting. The German thousands rule ("10,000" to "10.000") additionally trips the invented-token minor. gate_passed requires an empty findings list (translate.py:316) and deterministic checks refire identically every round, so a page with any prose decimal quantity can never pass: it burns all revise rounds on every source change and ships -with-findings forever. This re-creates at the checker level the exact contradiction the headline commit resolves in the prompt. Also, style-guides/pt-br.md contradicts itself on consecutive lines (37 and 38: "0.5 vCPU" must become "0,5 vCPU" and must also "stay exactly as written, including when quoted in prose").

Reproduced: integrity_findings("It is 2.5 times faster.", "Es ist 2,5-mal schneller.") returns a major demanding "2.5" verbatim; integrity_findings("It runs 10,000 pods.", "Es betreibt 10.000 Pods.") returns a minor "do not invent versions". Live in scope: docs/v1.5/getting-started/deploy-app.md carries "2.5 GB" and "1.5 GB" in bare prose.

Fix: restrict _VERSION_RE to unambiguous version tokens (v-prefixed and/or three-component), or accept the localized decimal form as equivalent for bare d.d while keeping v-prefixed and three-component tokens exact; that preserves the v1.5 to v1,5 catch the PR description advertises. Align pt-br.md lines 37-38 and the prompt's "resource quantities" wording with whichever rule wins. Tests in TestIntegrityFindings: ("It is 3.14 wide.", "Es ist 3,14 breit.") must be clean while v1.5 to v1,5 stays caught.

B3: the Spanish question/exclamation rules flag correct text by construction

The rule at lib.py:617 anchors at any capitalized word, not at sentence start, so a correctly opened question that contains a capitalized brand matches from the brand onward. This corpus capitalizes Cozystack/Kubernetes/Talos in nearly every sentence, and the exclamation rule has the same defect. Any finding blocks gate_passed, so most Spanish pages with a question can never clear the gate. There is no content/es/ yet; the first full es run hits this at scale.

Reproduced: check_typography() on the correct question "¿Qué es Cozystack?" returns a finding, matching from "Cozystack?".

Fix: anchor to sentence start (string/line start or after sentence-ending punctuation). Test: the exact string above must return [].

B4: the pipeline README no longer matches the gate it documents

The per-page pipeline diagram in hack/i18n/README.md lists every gate stage (translate, back-translate, two reviewers, revise) but not the new deterministic integrity/typography stage that feeds the same findings loop, and the file inventory table has no row for lint_translations.py. A maintainer triaging a weekly PR sees findings from: integrity-check / typography-check that the pipeline's own documentation says don't exist. Add the stage to the diagram and a lint_translations.py row to the table.

Non-blocking follow-ups

1. _REFDEF_RE masks the first word of footnote definitions: [^1]: Some note comes back with the first word replaced by a placeholder (reproduced), leaving it untranslated. No footnotes in the corpus today; exclude [^...] labels.
2. The lint_translations.py docstring says "this makes them enforceable on every PR", but .github/workflows/i18n-lint.yml runs only check-i18n.sh and test_i18n.py. Wire it in as an advisory step or soften the claim.
3. The belt-and-braces path in translate.py:362-363 writes a machine-translated body while keeping l10n: transcreate. If that currently unreachable path ever fires, machine output ships labeled as human transcreation with no disclaimer. Raising instead of writing is strictly safer.
4. integrity_findings do-not-translate counting is case-sensitive substring matching; a term that is also an ordinary English word will raise majors when a generic use is legitimately translated. Word-boundary matching would cut noise that blocks the gate via the B2 mechanism.
5. _LINKDEST_RE stops a destination at the first ), so a balanced-paren URL (.../Foo_(bar)) is partially masked with a stray ) left in prose. None in the corpus today.

[Aleksei Sviridkin (lexfrei)]

B1 (nested placeholders): this pass re-stashes tokens created by the shortcode and autolink passes. [text]({{% ref "/x" %}}) becomes a URL placeholder wrapping the shortcode placeholder; the model never sees the inner token, so the placeholder guard raises ProtocolError on every attempt. 82 of 181 in-scope pages hit this. restore() cannot unwind the nesting either (single pass, insertion order), and the back-translation path restores unguarded. Fix: skip destinations that contain the placeholder marker here and in _REFDEF_RE, reverse the restore() iteration order, add a residual-placeholder check after restore. Details in the review body.

[Aleksei Sviridkin (lexfrei)]

B2 (gate-fatal contradiction): this counts every bare d.d decimal in prose and demands it byte-for-byte, while the style guides in this same PR mandate "0.5 vCPU" to "0,5 vCPU" and the prompt fix permits prose reformatting. Deterministic findings refire every round and gate_passed needs an empty list, so a page with any prose decimal can never pass (docs/v1.5/getting-started/deploy-app.md has two). Restrict this to v-prefixed and/or three-component tokens, or treat the localized decimal form as equivalent for bare d.d. Details in the review body.

[Aleksei Sviridkin (lexfrei)]

B3 (false-positive anchor): the pattern matches at any capitalized word, so the correct question "¿Qué es Cozystack?" is flagged from "Cozystack?" onward (reproduced). With brands capitalized in nearly every sentence, most es pages with a question can never clear the gate. Anchor to sentence start (string/line start or after sentence-ending punctuation); same fix for the exclamation rule.

[Aleksei Sviridkin (lexfrei)]

NOT LGTM — the round-1 fixes hold, but one of them rebuilt the placeholder guard so a hallucinated token now becomes silent content duplication, and the transcreate change made l10n machine-read without the README following.

Blockers

B1: weakened placeholder guard accepts tokens the model was never sent

File: hack/i18n/translate.py:223
Issue: The tok in masked_src filter correctly stopped requiring inner tokens of a nested stash, but it also stopped forbidding them.
Evidence: Reproduced on this tree: protect() on a backticked shortcode stores §§SC_0§§ inside §§INLINECODE_1§§'s value; a reply carrying the legit outer token plus a hallucinated §§SC_0§§ passes the guard, reverse restore expands both, the page ships with the shortcode twice, and the final residual §§ check sees nothing. At the previous head the same reply produced a visible literal token; this diff makes the outcome invisible.
Impact: silent content duplication in a published page — the exact failure class the protocol guard exists to refuse.
Fix: one line — body.count(tok) != (1 if tok in masked_src else 0): a token the model never saw must occur zero times. Plus a TestPayloadProtocol case: a stray token the model was never sent is rejected.

B2: the front-matter table no longer tells the truth about l10n

File: hack/i18n/README.md:191
Issue: The row says l10n is read by "humans triaging review". After this PR the marker is load-bearing for machines: build_worklist skips transcreate pages, translate_page refuses to write them, check-i18n.sh downgrades their drift to a warning, and update-digests skips them.
Evidence: hack/i18n/lib.py:250 (PRESERVED_L10N_VALUES), translate.py:258-262, and is_transcreate in check-i18n.sh — all introduced or rewired by this PR; the README row predates them unchanged.
Impact: a maintainer marking or unmarking transcreate cannot learn from the README that the marker changes pipeline behavior — which is the point of the feature.
Fix: update the "Read by" cell and add one sentence describing the cycle: mark transcreate → pipeline leaves the page alone → CI warns on drift → refresh by hand → update-digests <file>.

Non-blocking follow-ups

1. is_transcreate in check-i18n.sh greps ^l10n: file-wide while recorded_l10n is bounded to front matter — same divergence class as the known file-wide ^source_digest: quirk, unreachable today. Neither accepts single-quoted YAML (l10n: 'transcreate'), and the failure direction there is overwriting a transcreation with machine output. Cheap fix: bound the shell grep to the front-matter block, parse via split_frontmatter in Python.
2. check_digest_freshness prints "translation freshness: OK (N pages match)" immediately after a ::warning:: about a drifted transcreate page — the summary contradicts the warning it just printed.

Verified fixed from round 1, each with a repro on this tree: nested placeholder masking (protect/restore round-trips identity, 122 tests pass), decimal-comma and thousands formatting no longer trip the version check while v1.5 → v1,5 is still caught, the Spanish inverted-mark rule anchors to sentence start ("¿Qué es Cozystack?" is clean), the README documents the deterministic-checks stage and lint_translations.py, the typography lint runs in CI as an advisory step, footnote definitions are excluded from ref-def masking, and do-not-translate matching is word-bounded.

[Aleksei Sviridkin (lexfrei)]

This filter correctly stopped requiring inner tokens of a nested stash, but it also stopped forbidding them: a reply carrying a hallucinated inner token (§§SC_0§§ when the model was only sent §§INLINECODE_1§§) passes the guard, reverse restore expands both, and the page ships the content twice — the residual §§ check sees nothing. One line closes it: body.count(tok) != (1 if tok in masked_src else 0) — a token the model never saw must occur zero times.

[Timur Tukaev (tym83)]

Aleksei Sviridkin (@lexfrei) both blockers from round 2 are addressed on feat/i18n-quality-improvements:

B1 — placeholder guard. The guard now enforces body.count(tok) != (1 if tok in masked_src else 0): a token the model was actually sent must survive exactly once, and a token it was never sent (an inner token of a nested stash, or a hallucination) must not appear at all. The error message no longer hardcodes expected 1 — it reports the real expected count and labels the case injected/lost/duplicated. Added TestPayloadProtocol.test_inner_token_the_model_never_saw_is_rejected, mutation-checked: it fails against the previous guard and passes now. 123 tests green.

B2 — README front-matter table. The l10n row's "Read by" cell now lists worklist.py, translate.py, hack/check-i18n.sh alongside the human triage, and a new sentence spells out the transcreate cycle (mark → pipeline leaves the page alone → CI warns on drift → refresh by hand → update-digests <file>).

The two non-blocking follow-ups (front-matter-bounded grep for is_transcreate, and the freshness summary that prints right after a drift warning) are noted for a separate pass so they don't hold this PR. Ready for re-review.

[myasnikovdaniil]

Checks themselves look good. One thing about the freshness guard, inline.

[myasnikovdaniil]

This error path knows nothing about translation scope, and that turns old translations into CI blocker. Any l10n: mt page with drifted digest fails the lint, lint runs on every PR touching content/**, but the pipeline only refreshes the latest version, so content/*/docs/v1.4/getting-started/_index.md cannot be fixed by rerunning it.

Digests match right now, I checked all four languages on main, so nothing is red yet. It goes red as soon as the upstream tags workflow pushes make update-all RELEASE_TAG=v1.4.x into content/en/docs/v1.4/, and from that moment every PR in this repo is blocked until somebody translates that page by hand or deletes it. Either downgrade out of scope pages to warning like transcreate ones, or remove them.

[Aleksei Sviridkin (lexfrei)]

NOT LGTM — both blockers from my last review are fixed, but the branch now conflicts with its base, and most of the conflicting code is the same fixes written twice, so the rebase decides what ships.

Blockers

B1: conflict with feat/i18n-pipeline

Files: hack/i18n/lib.py, hack/i18n/translate.py, hack/i18n/README.md

Issue: this branch forked the base at 12e90d5c9. The base then got cb0e2e7f4, which reimplements three things this PR already has:

1. Skipping l10n: transcreate pages: recorded_l10n/is_hand_localized/find_hand_localized here, l10n_mode/find_transcreated there, same _L10N_RE. Git merges both skips into build_worklist without a conflict, and keeping both sides of the report hunk in translate.py gives two sections listing the same pages.
2. Raw HTML tag masking, with a different design. Here the whole tag is masked, one line only. The base also matches multi-line tags and leaves alt/title/caption values open for translation. The two _HTMLTAG_RE definitions merge without a conflict marker, and the later one (this PR's) silently wins.
3. Nested placeholder restore: reverse order here, a fixed-point loop there. Either works, keep one.

Evidence: git merge-tree --write-tree 8133286ac 2652726e9 reports content conflicts in exactly these three files. With a union resolution (both sides kept, one syntax break fixed) the suite runs 136 tests and fails one: the base's test_visible_attributes_still_translate, because this PR's tag pass masks alt="A running cluster" before the base's pass sees it.

Impact: an approval now would cover lib.py and translate.py in a form that won't be merged, and a mechanical resolution either keeps duplicate code or drops the base's attribute handling without any error.

Fix: rebase and pick one implementation of each. Keep this PR's _split_payload_response (it merges cleanly). The base version still requires every stored token, so there a correct reply for a backticked shortcode fails as "lost". Keep this PR's l10n row in the README table too.

Since the history gets rewritten anyway: this repo merges with merge commits, so commit messages land on main as written. df696750e, 97dd5fc31, c2ba45e88 and 41255d05a have Co-Authored-By: Claude <noreply@anthropic.com>, and my own commits a9af5f3c5 through 345f9983e have Assisted-By: Claude <noreply@anthropic.com>. Both should be Assisted-by: LLM. 41255d05a ("Two gaps found in review by Aleksei Sviridkin (@lexfrei)") and 005aa461a ("The round-1 fix that...") talk about the review, not the change. All of this was already there at my last review, I should have raised it then.

Checked

• Placeholder guard: the reply from my last repro (the sent §§INLINECODE_1§§ plus an unsent §§SC_0§§) is rejected as "injected". At 345f9983e the same reply was accepted and the shortcode ended up in the page twice. Reverting the guard line fails test_inner_token_the_model_never_saw_is_rejected and nothing else.
• l10n docs: the row and the new paragraph match the code. On a copy of the tree a drifted transcreate homepage warns, update-digests content/ru/_index.html clears only that warning, and bare update-digests skips all eight transcreate pages.
• 8133286ac: the classifier matches _docs_out_of_scope in lib.py. With the real stamped pages, drift in v1.5 fails, drift in v1.4 gives four warnings and exit 0, drift in docs/_index.md fails, and a missing latest_version_id fails closed.
• 123 tests pass, check-i18n.sh and lint_translations.py are clean.

Non-blocking

1. hack/check-i18n.sh:105: latest_version_id: 'v1.5' parses as 'v1.5' with the quotes, so every versioned page, v1.5 included, counts as superseded. Current-version drift becomes a warning and the lint exits 0, while lib.py reads the same value correctly. register_version.sh writes double quotes, so it takes a hand edit, but nothing would show it. This line right after LATEST_DOCS="$(latest_docs_version)" fails closed on any bad parse (checked on the same copy): [ -d "$CONTENT_DIR/$DEFAULT_LANG/docs/$LATEST_DOCS" ] || LATEST_DOCS=""
2. No test covers the classifier: with != flipped to = in is_superseded_docs all 123 tests stay green. The shell lint has no tests at all, so this is only a suggestion.
3. hack/i18n/test_i18n.py:447: assertIn("injected", ...) passes with any label, because the fixed tail of the message ("dropped, duplicated or injected code") always has the word. With the label forced to "duplicated" the suite stays green. "injected by the model" would check it. Same for "duplicated" in test_duplicated_placeholder_is_rejected.
4. The new warning says the pipeline removes superseded translations on its next run. At this PR's base find_orphan_translations keeps them on purpose, and only 18b140aa2 in the base changes that, so the message becomes true after the rebase. The same reason is also written in three comments, one is enough.
5. Both non-blocking items from my last review are still there, and the summary one now also follows the new warnings: four superseded warnings, then "translation freshness: OK (22 translated pages match their English source)".
6. The PR body becomes the merge commit message here. It still says "91 tests pass" and doesn't mention HTML masking, transcreate handling or the new check-i18n.sh warnings.

delete_branch_on_merge is on, so merging #623 retargets this PR to main.