Skip to content

feat: convert Markdown math to native Word OMath - #154

Open
kiki830621 wants to merge 7 commits into
mainfrom
idd/141-md-docx-omath
Open

feat: convert Markdown math to native Word OMath#154
kiki830621 wants to merge 7 commits into
mainfrom
idd/141-md-docx-omath

Conversation

@kiki830621

@kiki830621 kiki830621 commented Aug 13, 2026

Copy link
Copy Markdown
Member

Refs #141

Summary

  • Add an opt-in literal|omath Markdown-to-DOCX math mode with a backward-compatible literal default.
  • Parse a conservative dollar-delimited subset through latex-math-swift and emit native inline/display Word OMath.
  • Add typed, located, pre-write failures plus package/compiled-CLI regression coverage and documentation.

Verification

  • R3 frozen candidate: 68c7403b39f3fad5d5da551f02fcc5c9f51bcb8a.
  • Focused package tests: 38 passed, 0 failed; expanded focused review: 43 passed, 0 failed.
  • Compiled CLI route tests: 10 passed, 0 failed.
  • Full MDToWord package: 127 tests with the same 42 pre-existing baseline failures and no new existing-suite failures.
  • Root package: 55 tests, 0 failures, 3 environment skips.
  • Spectra validation passed with 0 Critical / 0 Warning.
  • Independent R3 verification: FAIL; see feat: convert Markdown math to native Word OMath #154 (comment).

R3 blocking

  • Multiline display spans can cross original CommonMark block/container boundaries before AST validation.
  • Incomplete destination/reference grammar can write generated placeholders into relationship targets.
  • Multiline/quoted HTML boundaries are not fully excluded, causing hidden formula failures or visible placeholder leakage.
  • Formatting-node-spanning delimiters conflict with the frozen proposal boundary.

Checklist

  • Diagnose
  • Spectra proposal and design
  • Implement with RED → GREEN evidence
  • Independent IDD verification
  • Verify-gated PASS

No GitHub auto-close trailer is included; issue closure remains a separate gated action.

Copy link
Copy Markdown
Member Author

IDD Verification — FAIL

Frozen SHA: d79e6a5ed9acb5b6b07882ee560a3de5c83b010d

四個直接落在本變更契約內的 correctness blockers:

  1. Archived OMML namespaceword/document.xml<m:oMath>,但 root 未宣告標準 xmlns:m;namespace-aware parser 會報未綁定前綴。
  2. Display logical paragraphBefore\n$$x$$\nAfter 或相同 multiline 形狀會成功輸出內部 MDTOWORDMATHPLACEHOLDER...TOKEN,沒有 OMath、也沒有 located failure。
  3. Reference destination:多行 reference definition 或 container 內的 destination 會被 scanner 改寫;實際 relationship URL 變成含內部 placeholder 的 URL。
  4. Code exclusion:blockquote 內 tilde fence 與四空白 indented code 會被掃描,原始 $x$ 消失並洩漏 placeholder。

驗證閘門另有一項:frozen diff 的 git diff --check 因 21 個新增 CRLF 行回傳 exit 2;這是 merge/verification gate,不列為產品 correctness blocker。

效能 finding 經 Devil's Advocate 降為 P2:placeholder 配對靜態上呈高次方搜尋,debug profile 100/200/400 個公式約 0.71/4.67/36.84 秒;修正時應改為單向 marker walker,並用 release benchmark決定最終 severity。

測試資格:

  • focused package:30/30
  • compiled CLI:7/7
  • public non-@testable client:type-check pass
  • candidate full package:114 tests/42 failures
  • exact parent baseline:89 tests/相同 42 failures
  • root:52 passed/0 failed/3 skipped

因此既有 42 failures 維持由 sister #155 追蹤,並非本次 FAIL 原因。

流程揭露:四個 lens 受本機四槽限制以 3+1 批次執行;外部 cross-model reviewer 因隱私政策拒絕傳送約 100 KB 私有 diff,未規避政策,由 coordinator self-review 補位。Sequenced Devil’s Advocate 已完成,沒有新增非重複 blocker。

下一步:在本 PR 先補 RED regression,再修正上述四項,重跑完整 verification。

@kiki830621

Copy link
Copy Markdown
Member Author

Verify Report — PR #154 — R2

Aggregate

FAIL — 3 blocking findings. Frozen candidate: 3365faf08db6ee400d791e684d9ef181461edea8.

Engine

Four independent lenses (requirements, logic, security/data-integrity, regression), sequenced Devil's Advocate, and primary Codex review. All reports were written before aggregation and the frozen SHA remained unchanged.

Requirements coverage

6/7 requirements are fully addressed. The conservative scanner/one-forward-scan requirement remains only partially addressed.

Findings

# Severity Finding Source Action
1 P1 Valid multiline inline link/image destinations can be tokenized; the generated DOCX succeeds with an internal placeholder in the relationship target, silently losing the original URL. requirements + logic + security + regression + DA + Codex Blocking
2 P1 Display placement uses adjacent physical lines instead of CommonMark block boundaries, so valid standalone display blocks next to headings or in their own list items are rejected. requirements + logic + regression + DA + Codex Blocking
3 P2 Marker allocation still performs source.contains and linear token search for every formula. Release 2,500/5,000/10,000-formula measurements scale about 0.42/1.65/6.33 seconds, violating the normative one-forward-scan contract. requirements + logic + security + regression + DA + Codex Blocking

The optional reference-title case is also a real scanner-boundary defect. DA classifies it as a P2 clarification because the current spec explicitly names destinations rather than titles; it should be handled in the same remediation but is not required to establish this FAIL.

Fresh verification evidence

  • Focused package: 37/37 passed.
  • Compiled CLI: 8/8 passed.
  • Root full: 53 tests, 0 failures, 3 environment skips.
  • Package full: 121 tests, exactly the documented 42 parent-baseline failures (1 E2E + 41 RoundTrip), no new baseline failures.
  • External non-@testable client build: PASS.
  • Dependency graph: one OOXMLSwift 1.5.0 resolution; LaTeXMathSwift 0.2.0.
  • Spectra: valid, 0 Critical, 0 Warning, 1 existing Suggestion.
  • CRLF-aware diff gate: PASS.
  • R1 closures confirmed: archived xmlns:m, known container/reference regressions, destination preservation, and render-time dictionary lookup.

Process gap

The desktop environment provided independent reviewer agents plus the primary Codex family, but no second external model family was allowed to receive the private diff. This is disclosed rather than represented as cross-model verification.

Disposition

PR remains needs-fix. No verified tag is created. The next implementation round must add RED regressions for multiline destinations, positive CommonMark block-boundary display cases, and deterministic linear marker allocation before another frozen verification round.

@kiki830621

Copy link
Copy Markdown
Member Author

IDD Verify R3 — PR #154 / Issue #141

Result

FAIL — frozen SHA 68c7403b39f3fad5d5da551f02fcc5c9f51bcb8a 尚有 CommonMark boundary 與 placeholder data-integrity blockers。不得標 verified、不得合併、不得關閉 issue。

Frozen candidate

  • Branch:idd/141-md-docx-omath
  • PR head/local HEAD:68c7403b39f3fad5d5da551f02fcc5c9f51bcb8a
  • Base/merge-base:f3efc3fa8ca1831a4900e55d6f0578e3fbd1a9b5
  • Worktree:clean
  • PR:OPEN,無 closing issue reference
  • Commit discipline:subject 以 (#141) 結尾,無 auto-close keyword

Lens verdicts

Lens Verdict Key result
Requirements/spec FAIL display跨block、destination/reference grammar、optional title、formatting-node scope drift
Logic/reliability FAIL production CLI資料失真與placeholder外洩反例
Security/data-integrity FAIL multiline HTML block/comment仍被掃描
Regression/test quality FAIL focused綠燈未涵蓋container identity/blank-block negatives
Sequenced Devil's Advocate FAIL 去重為4組直接 #141 blockers
Codex supplemental FAIL 獨立重現B1/B3/B4;跨模型外送遭政策拒絕,process gap已揭露

Blocking findings(去重)

  1. [P1] Multiline display跨原始CommonMark block/container。 Scanner在AST前搜尋任意後續$$,會把blank-separated paragraphs、不同list items或blockquote外內容壓成一個display formula;transformed AST validation無法復原原始邊界。
  2. [P1] Destination/reference grammar仍可讓generated placeholder進入relationship target。 合法multiline reference label與angle destination含)可重現URL被改寫;invalid optional-title另會靜默漏轉後續visible公式。
  3. [P1] HTML boundary未完整shield。 Multiline HTML comment/block內公式仍被解析;quoted > inline HTML可讓internal placeholder出現在DOCX可見文字。
  4. [P1/P2] Formatting-node boundary與proposal漂移。 $*x*$$**y**$跨Text/Emphasis/Strong仍被當成LaTeX,違反frozen non-goal。

共同根因:scanner在原始CommonMark AST/source ranges前改寫source,沒有建立允許carrier ranges,也未驗證每個token恰好在可見Text/display carrier中被消耗一次。

R2 blockers status

  • Marker allocation O(N²):CLOSED。source先線性預索引,10,000-marker debug regression約0.018s;release 2,500/5,000/10,000約0.014/0.023/0.042s。
  • Ordinary multiline inline link regression:CLOSED for covered case,但完整destination grammar仍由B2阻擋。
  • CommonMark AST display validation:PARTIAL,既有heading/list正例與soft-break負例綠,但原始跨block span在AST前被洗平(B1)。

Fresh test evidence

  • Package focused scanner + OMath:38/38 pass;requirements/security擴大focused為43/43 pass。
  • Compiled CLI:10/10 pass。
  • Root:55 executed,0 failures,3 environment skips。
  • Full MDToWord:127 executed;42 failures完全吻合parent baseline(1 E2E + 41 RoundTrip,另案bug: restore a green md-to-word-swift package test baseline #155),無新增既有suite failure。
  • Spectra validate:PASS;analyze:0 Critical/0 Warning/1 Suggestion。
  • CRLF-aware diff check:PASS;plain diff check:exit 2(21個CRLF新增行),task 7.3的ordinary PASS證據須更正。
  • Public external API probe/single OOXMLSwift 1.5.0 identity:PASS。

Production negative probes

  • - $$\n- x\n- $$:exit 0,三個list items被壓成一個m:oMathPara
  • $$\n\nx\n\n$$:exit 0,三個paragraphs被壓成一個display formula。
  • <span title=\"> $x$\">text</span>:exit 0,archive含MDTOWORDMATHPLACEHOLDER...TOKEN
  • HTML comment/<div> block內unsupported formula:合法文件整體失敗;既有KEEP destination仍byte-identical。
  • $*x*$$**y**$:exit 0,OMML text分別成*x***y**
  • Multiline reference label與angle destination反例:relationship target含generated placeholder。

Required remediation before R4

  • 用原始CommonMark AST source ranges建立allowed/opaque spans;multiline display必須同一Paragraph與container identity。
  • destination/reference/HTML/code等由AST range排除,不在scanner重作不完整CommonMark grammar。
  • build後要求每個token恰好由允許carrier消耗一次;落入relationship/HTML/metadata或消失即fail before write。
  • 為上列每個反例補package與compiled CLI RED→GREEN;失敗路徑驗既有destination byte-identical。
  • 若要支援跨formatting-node math,先走Spectra ingest改契約;否則保持literal。
  • 更新tasks 7.3證據(ordinary diff claim)與one-forward-scan文字。

Process disclosure

四個本機獨立lens與sequenced DA均完成且報告非空。另嘗試呼叫外部Claude Code作跨模型審查,但執行環境因未取得針對私有程式碼外傳目的地的明示授權而拒絕;未繞過。此process gap已揭露,不影響本輪FAIL判定。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant