From 08f29f898d9d6b5513870be46434f7820a6931da Mon Sep 17 00:00:00 2001 From: Carlos Scheidegger Date: Fri, 31 Jul 2026 16:32:35 -0500 Subject: [PATCH 1/8] Shortcode extensions port: eager activation + shortcodes: key fix + contract corpus (bd-540a976a) Phase 1 core fix: extension shortcode scripts are now eagerly loaded on the first dispatch that reaches the Lua stage, instead of looked up on demand by extension id == shortcode name. Handler names come from Lua table keys and are decoupled from the extension id (extension quarto-tiers contributes shortcode tier), so the by-name model failed for most real-world extensions. Load order fixes precedence per plan D4: document shortcodes: scripts < extensions in discovery order (built-ins first, more-local last); Rust built-ins always win. Per-script load failures now warn with the extension id and script path named as the cause instead of hijacking the triggering shortcode's result. Phase 2 early fix: extract_shortcode_paths dropped document-level shortcodes: entries stored as ConfigValueKind::PandocInlines (the metadata-as-str failure class); now uses as_plain_text(). New Phase 0 compatibility corpus (smoke-all/extensions/contract-*): six passing fixtures covering table-return + global-fn registration, dash-named shortcodes, block/inline context, kwargs missing-key semantics, return-value coercions, brace escaping, and document-level shortcodes: with quarto.shortcode.error_output. contract-escape-comment is parked via tests.run.skip pending a decision on Q1's Hugo-style {{}} escape form. Verified end-to-end: connect-docs {{< tier Enhanced >}} now renders badge-enhanced spans with zero unknown-shortcode warnings. Plan: claude-notes/plans/2026-07-31-shortcode-extensions-port.md Co-Authored-By: Claude Fable 5 --- .../2026-07-31-shortcode-extensions-port.md | 375 ++++++++++++++++++ .../src/transforms/shortcode_resolve.rs | 139 ++++--- .../contract-doc-shortcodes/shorty.lua | 9 + .../contract-doc-shortcodes/test.qmd | 15 + .../contract-escape-braces/test.qmd | 12 + .../contract-escape-comment/test.qmd | 14 + .../_extensions/greeter/_extension.yml | 5 + .../_extensions/greeter/greeter.lua | 3 + .../extensions/contract-global-fn/test.qmd | 12 + .../_extensions/coerce/_extension.yml | 5 + .../_extensions/coerce/coerce.lua | 11 + .../contract-return-coercions/test.qmd | 22 + .../_extensions/echoer/_extension.yml | 5 + .../_extensions/echoer/echoer.lua | 8 + .../extensions/contract-table-return/test.qmd | 16 + 15 files changed, 602 insertions(+), 49 deletions(-) create mode 100644 claude-notes/plans/2026-07-31-shortcode-extensions-port.md create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/shorty.lua create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/test.qmd create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-escape-braces/test.qmd create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-escape-comment/test.qmd create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-global-fn/_extensions/greeter/_extension.yml create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-global-fn/_extensions/greeter/greeter.lua create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-global-fn/test.qmd create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-return-coercions/_extensions/coerce/_extension.yml create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-return-coercions/_extensions/coerce/coerce.lua create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-return-coercions/test.qmd create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-table-return/_extensions/echoer/_extension.yml create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-table-return/_extensions/echoer/echoer.lua create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-table-return/test.qmd diff --git a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md new file mode 100644 index 000000000..ad8e97b0e --- /dev/null +++ b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md @@ -0,0 +1,375 @@ +# Shortcode extensions: Quarto 1 → Quarto 2 port plan + +**Status:** Reviewed 2026-07-31 — design decisions signed off (see § Design +decisions; Phase 3 deferred). Awaiting go-ahead to implement. +**Braid strand:** bd-540a976a (epic; related: bd-8b0af414, bd-nzdm1wry, bd-u145dg3y, bd-5edooc78, bd-mqk49) +**Date:** 2026-07-31 + +## Overview + +Goal: a Quarto 1 shortcode extension — `_extensions///_extension.yml` +with `contributes: shortcodes: [handler.lua]` — works in Quarto 2, with the +documented Q1 handler contract (`handler(args, kwargs, meta, raw_args, context)`) +honored, and with failures surfaced as source-mapped, `Q-*`-coded diagnostics +instead of Q1's silent-passthrough / guess-what-you-meant behavior. + +**Headline finding from the study:** this is *not* greenfield. Q2 already has a +working end-to-end path: tree-sitter grammar → `Inline::Shortcode` → +`ShortcodeResolveTransform` → Rust handler (`meta`) or Lua handler loaded from +`_extensions/` (five built-in extensions ship embedded: `kbd`, `lipsum`, +`placeholder`, `version`, `video`). The extension epic +(`claude-notes/plans/2026-03-16-extensions-grand-plan.md`, Phases 1–4 complete) +built the discovery/manifest/resolution machinery. This plan is therefore a +**gap-closure plan**, organized around a compatibility test corpus derived from +Q1's contract. + +## Sources studied + +- **Q1 docs (user contract):** `external-sources/quarto-web` — + `docs/extensions/shortcodes.qmd` (authoring contract), + `docs/authoring/_shortcodes.qmd` (built-in table), `docs/extensions/distributing.qmd` + (`_extension.yml` schema), `docs/extensions/lua-api.qmd:367-374` + (`quarto.shortcode.*`), `docs/extensions/_shortcode-escaping.qmd`. +- **Q1 implementation:** `external-sources/quarto-cli` — the LPeg grammar + `src/resources/pandoc/datadir/lpegshortcode.lua` (authoritative syntax), + `src/resources/filters/customnodes/shortcodes.lua` (dispatch + return coercion), + `src/resources/filters/quarto-pre/shortcodes-handlers.lua` (registration + + built-ins), `src/extension/extension.ts` (discovery), + `src/command/render/filters.ts:602-705` (activation), + `src/core/handlers/include.ts`/`embed.ts` (TS-side directives). +- **Q2 current state:** `crates/quarto-core/src/transforms/shortcode_resolve.rs`, + `crates/quarto-core/src/extension/{types,read,discover,mod}.rs`, + `crates/pampa/src/lua/shortcode.rs`, `crates/quarto-core/src/stage/stages/include_expansion.rs`, + `crates/tree-sitter-qmd/tree-sitter-markdown/grammar.js:623-666`. + +## The Quarto 1 contract (condensed) + +What a Q1 shortcode extension author was promised: + +1. **Layout:** `_extensions//` or `_extensions///` containing + `_extension.yml` with `contributes: shortcodes: [.lua, …]`. Everything + above `_extensions/` is not installed. +2. **Activation:** every discovered extension contributing shortcodes is + **automatically active** — no YAML opt-in (unlike filters/formats). Discovery + walks built-ins first, then `_extensions/` dirs from project root down to the + input's directory; later (more local) same-id extensions override earlier ones. +3. **Handler registration:** the Lua file either `return { name = fn, … }` or + defines global functions harvested from the chunk's sandboxed env. Flat + namespace keyed by shortcode *name* (org/name namespacing identifies the + extension, never the invocation). Precedence: document `shortcodes:` YAML < + extension-contributed < built-ins (Q1 built-ins always win). +4. **Handler signature:** `fn(args, kwargs, meta, raw_args, context)`: + - `args`: pandoc.List of positional values (inlines/strings); + - `kwargs`: table whose missing keys yield **empty `pandoc.Inlines`** (not nil); + - `meta`: lazy dotted-path metadata lookup (`meta["a.b"]` works), not raw Meta; + - `raw_args` (≥1.3): all arg values in source order, names stripped; + - `context` (≥1.5): `"block"` | `"inline"` | `"text"`. +5. **Return values:** string | Inline | Block | Inlines | Blocks | plain array | + nil, with documented coercions per context (blocks→inlines via + `blocks_to_inlines` in inline context; inlines wrapped in `Para` in block + context; `"text"` context stringifies). +6. **Contexts in the document:** shortcodes resolve in prose (inline), alone in a + paragraph (block), and in *text positions*: `Code`/`CodeBlock` text, + `RawInline`/`RawBlock`, `Math`, element attributes (single-quoted), + `Link.target`, `Image.src`. Opt-outs: `cell-code` class (engine output) and + `{shortcodes=false}` attribute. +7. **Escaping:** `{{{< … >}}}` renders the literal `{{< … >}}`; `{{}}` + comment form ditto. +8. **Helpers:** `quarto.shortcode.read_arg(args, n)` and + `quarto.shortcode.error_output(name, msg_or_args, context)`; ambient + script-dir state (`withScriptFile`) drives `quarto.utils.resolve_path`, + sibling-module `require`, and relative HTML dependencies. +9. **Unknown shortcode:** warn + pass through the original `{{< … >}}` text as a + raw inline/block (silently, in text context). Documented as a *feature* for + Hugo interop ("shortcodes not recognized by Quarto are passed through + unmodified to Hugo"). +10. **Built-ins:** `meta`, `var`, `env`, `pagebreak`, `kbd`, `video`, `include`, + `embed`, `lipsum`, `placeholder`, `contents`, `version`, `brand` (the last + missing from the docs' own table). `include`/`embed` are **not** Lua handlers + in Q1 — they're TS text-level directives at pre-/post-engine stages. + +Q1 quirks we get to *not* port (see § Design decisions): the undocumented paired +shortcode syntax (`{{< name >}}…{{< /name >}}`, 1.4, zero docs); the shadowed +`local result` bug in `shortcodes.lua:172-173` that drops some nested results; +grid-table non-support caused by Q1's text-level pre-parse (Q2 parses shortcodes +in-grammar, so this restriction may simply not apply — verify in Phase 0). + +## Gap analysis: Q1 contract vs Q2 today + +| # | Q1 contract item | Q2 status | Gap | +|---|---|---|---| +| 1 | Syntax `{{< … >}}` incl. nesting, escapes | ✅ in-grammar (`grammar.js:623-666`), `Inline::Shortcode` | Verify `{{}}` comment-escape form; newlines inside shortcodes (Q1 allows) — Q-2-27/28 currently *reject* line breaks: deliberate strictness, keep, but confirm messaging | +| 2 | Extension discovery + manifest | ✅ `extension/discover.rs`, `read.rs` | **bd-8b0af414**: Q2 hard-requires `title`/`author`/`contributes`; real Q1 extensions omit these and silently fail to load. **bd-nzdm1wry**: load failure → `tracing::warn!` only, then misattributed "unknown shortcode" at use site | +| 3 | Auto-activation of shortcode extensions | ❌ **correctness bug** — on-demand load keyed by shortcode name (`dispatch_shortcode` → `find_extension(&shortcode.name)`, `shortcode_resolve.rs:370`) | Any extension whose shortcode names differ from its extension id is never loaded — the common case (`quarto-tiers` contributes `tier`, `fontawesome` contributes `fa`). Q2's built-ins work only because their ids coincide with their shortcode names. **Confirmed live**: `external-sources/connect-docs/docs-quarto-2` — `{{< tier … >}}` → "Unknown shortcode" despite a valid, discovered `_extensions/quarto-tiers/`. Fix: Q1's eager model — on first shortcode dispatch, load *all* discovered extensions' `contributes.shortcodes` scripts, then dispatch by handler name. Also: conflict/shadowing silent; Q1's `filterBuiltInExtensions` shadow-warning has no analogue | +| 4 | Handler signature 5-tuple | ✅ `shortcode.rs:298` TS-compatible; kwargs empty-Inlines metatable present | `meta` lazy dotted lookup: verify parity; `raw_args` shape: verify | +| 5 | Return-value coercions | ✅ `convert_return_value`/`classify_table_result` | Verify against Q1's table (esp. blocks→inlines in inline ctx, `nil` handling) via corpus | +| 6 | `context = "text"` | ⚠️ `ShortcodeCallContext::Text` exists in pampa, **unreachable** — quarto-core only dispatches Block/Inline | Shortcodes in code blocks, attributes, link targets, image src are **not resolved**. Grammar already parses shortcodes in link destinations + quoted attr strings; resolve wiring missing | +| 7 | Script-dir ambient state | ✅ `push_script_dir` stack, `quarto.utils.resolve_path` | **No `require`/`package.path` at all** (native or WASM) — any extension with a sibling module breaks | +| 8 | Built-in shortcodes | `meta` (Rust), `include` (Rust stage), `kbd`/`lipsum`/`placeholder`/`version`/`video` (Lua) | **Missing: `var` (+ `_variables.yml`), `env`, `pagebreak`, `brand`, `contents`, `embed`** | +| 9 | Unknown-shortcode behavior | warning (uncoded) + visible `?name` inline | No `Q-*` code, no source-mapped location shown, no passthrough option; HTML writer **silently drops** any surviving `Inline::Shortcode` (`html.rs:1059`); `shortcode_to_span` has a `process::exit(1)` on nested kv args (`pampa/src/pandoc/shortcode.rs:95-98`) | +| 10 | Precedence built-ins > user | Inverted-ish: Rust built-ins win, but Lua built-in extensions can be shadowed by user extensions (`find_extension` rfind) | Decide + document + diagnose (see D3) | +| 11 | `shortcodes:` YAML key | ✅ `extract_shortcode_paths` | Also accepts only paths; Q1 also allowed extension *names* in format-contributed `shortcodes:` — defer | +| 12 | `quarto add/remove/list/update` | ❌ 8-line `NotImplemented` stubs | CLI installation story (bd-5edooc78 pins the remove-guard requirement) | +| 13 | `quarto.shortcode.{read_arg,error_output}` | ✅ `shortcode.rs:384` | Verify exact semantics against `init.lua:1002-1032` | +| 14 | Escaped shortcode round-trip | ✅ `is_escaped` → Preserve | Corpus-verify writer output renders literal `{{< … >}}` | + +## Design decisions (Q2-native improvements) + +These are the places where we deliberately diverge from Q1, following the +project's porting principles: *strictness is acceptable when the diagnostic is +source-mapped and actionable; prefer explicit declaration over inference.* +Each is flagged **[decided]** (follows an existing Q2 policy) or **[needs user +sign-off]**. + +**D1. Unknown shortcode → coded, source-mapped diagnostic; no silent drop. +[decided 2026-07-31]** +Q1 warns and passes the raw text through (silently, in text context) — partly a +Hugo-interop feature. Decision: warning-level diagnostic with a new `Q-*` code, +`.with_location()` pointing at the invocation (we have `SourceInfo` on every +`Shortcode` node — Q1 could never do this), plus the visible `?name` marker in +output. For Hugo-style passthrough, require explicitness: a +`shortcodes: passthrough: [ref, figure]` (or similar) config key that names +foreign shortcodes, rather than Q1's pass-everything-unknown. +Unknown-shortcode-as-*error* needs no dedicated flag: `q2 render --strict` +(warnings-as-errors, already shipped) composes with this warning. + +**D2. Extension load failure is a real diagnostic, never a downstream +misattribution. [decided — this is bd-nzdm1wry]** +A malformed `_extension.yml` or a Lua file that fails to load must produce a +coded diagnostic naming the extension file and cause, at load/first-use time. +The current behavior (silent `tracing::warn!`, then "unknown shortcode ?greet" +pointing at the *user's document*) is precisely the Q1-style misattribution this +port should eliminate. + +**D3. Manifest strictness: relax to Q1-compat intake, validate loudly. +[decided 2026-07-31 — proposal approved as written]** +Q1 requires no named fields in `_extension.yml`. Q2 hard-requires +`title`/`author`/`contributes` (bd-8b0af414), so real Q1 extensions +(julia-engine, marimo) fail to load — and per D2 today they fail *silently*. +Proposal: only `contributes` is structurally required (an extension contributing +nothing is an error, matching Q1's `validateExtension`); missing +`title`/`author` become warnings at most; `version`/`quarto-required` validated +as semver *when present*, with a source-mapped error into the YAML file when +malformed (we have quarto-yaml source locations; Q1 didn't). + +**D4. Handler-name conflicts are diagnosed, not silent. [decided 2026-07-31, +conditional on practicality]** +Keep a deterministic precedence (matching Q1: built-ins win; among extensions, +more-local wins; document `shortcodes:` files lowest). The shadowing diagnostic +(naming both files) is approved *if practical* — user flagged a feasibility +concern: at the point where registration overwrites a name, we may not have +good source attribution for both definitions. Assess during Phase 1: if the +engine's handler registry records `(name, script_path)` per registration (it +already tracks the script being loaded), a file-level (not span-level) +diagnostic should be cheap; if it turns out invasive, ship the precedence rule +documented but undiagnosed and file a follow-on strand. + +**D5. `include` stays a Rust pre-stage, not a Lua handler. [decided]** +Q2's `include_expansion.rs` already mirrors Q1's TS-side design (and Q1 itself +never had a Lua `include`). Keep circular-include detection and source-mapped +missing-file errors as coded diagnostics (verify they have `Q-*` codes; add if +not). + +**D6. `embed` is out of scope for this plan. [confirmed 2026-07-31]** +Q1 `embed` drags in notebook rendering, `notebook-links`/`notebook-view`, and +the jupyter-embed placeholder machinery. User: `{{< embed >}}` needs a more +drastic redesign for Q2 — deferred to its own strand/epic, dependent on Q2's +engine story. + +**D7. Paired shortcodes not ported. [confirmed 2026-07-31 — deferred]** +Shipped in Q1 1.4 (`#5902`), never documented, zero occurrences in quarto-web. +User: these likely exist purely for Hugo passthrough — which means if we ever +implement the D1 passthrough config for Hugo interop, paired syntax belongs to +*that* feature (pass the paired form through verbatim), not to the handler +dispatch machinery. File a backlog strand recording this framing. + +**D8. In-grammar parsing is the single source of truth. [decided — already Q2 +reality; text-position scanning deferred 2026-07-31]** +Q1 has *four* shortcode parsers (LPeg grammar, sentinel encoder, AST-level +metadata re-parser, TS regex parser) because it had to smuggle shortcodes past +Pandoc's reader. Q2 parses them in the tree-sitter grammar with real +`SourceInfo`. Consequences to verify in the corpus: grid tables (Q1-documented +restriction should just vanish); metadata-position shortcodes (Q1 needed +`astshortcode.lua`; where does Q2 stand? — Phase 0 must answer this); +attribute-position quoting rules. User note: the one place Q1's mess may need +partial re-doing is detecting shortcodes inside *opaque text positions* (code +block contents, URL targets, attributes) — that whole area (Phase 3) is +**deferred**; Phase 0 still records the current behavior as known-gap baseline +probes so the deferral is documented, not accidental. + +## Work plan + +TDD throughout: every phase starts by adding failing tests/corpus entries, per +CLAUDE.md. Each phase is a candidate braid child strand once the plan is +approved. + +### Phase 0 — Compatibility corpus + behavioral baseline (the test plan) + +The deliverable is a fixture suite that encodes the Q1 contract, so every later +phase has failing tests to turn green, and so we discover *actual* Q2 behavior +where the study only has static reads. + +- [ ] Port Q1's smoke fixtures (`external-sources/quarto-cli/tests/docs/shortcodes/` + — `shorty.lua`, `custom.qmd`, the `?meta:…`/`?var:…` error expectations) + into local fixtures (copy, never reference `external-sources/` from tests). +- [ ] Author a "contract extension" fixture exercising: table-return and + global-fn registration; dash-named shortcode; all five handler params; + kwargs missing-key → empty Inlines; `meta` dotted lookup incl. `\\.` + escape and 1-based array index; each return-value coercion row; both + escape forms; nested shortcode as arg and as kwarg value. +- [ ] Integration tests driving the real binary path (`render_document_to_file` + / `q2 render` on fixtures) — not `HtmlRenderConfig::default()` shortcuts. +- [ ] Baseline probes (tests that *document* current behavior, marked + known-gap): shortcode in code block / attribute / link target / image src; + shortcode in YAML metadata values (title etc.); shortcode in grid table; + extension with missing `title`; extension whose Lua `require`s a sibling. +- [ ] Pick 2–3 real published Q1 shortcode extensions (e.g. `quarto-ext/fontawesome`, + `shafayetShafee/bsicons` class) and add them as fixtures; record what + breaks. These are the acceptance tests for the whole plan. +- [ ] Real-world acceptance target: `external-sources/connect-docs/docs-quarto-2` + (Posit Connect docs). Known failure today: `{{< tier … >}}` from + `_extensions/quarto-tiers/` → "Unknown shortcode" (gap row 3). Copy the + minimal extension shape into a local fixture (never reference + `external-sources/` from tests); use the full project as a manual + end-to-end check. + +### Phase 1 — Extension loading: eager activation, Q1-compat intake, loud failures (D2, D3, gap row 3) + +- [ ] **Fix the name-keyed activation bug (gap row 3)** — need not land + first, but must land as part of this phase's work: + test with an extension whose shortcode name ≠ extension id (fixture: + a `quarto-tiers`-shaped extension contributing `tier`); replace the + `find_extension(&shortcode.name)` on-demand path with Q1's eager + semantics — on first shortcode dispatch, load every discovered + extension's `contributes.shortcodes` scripts into the engine, then + dispatch by registered handler name. Keep document-level laziness + (no Lua unless the doc contains shortcodes). +- [ ] Tests: malformed `_extension.yml` (bad YAML, bad semver, empty + contributes) → coded, source-mapped diagnostics; minimal Q1 manifest + (no title/author) loads. +- [ ] Relax `read.rs` required fields per D3; add `Q-*` codes for manifest + errors (extension subsystem: decide `Q-5-*` vs new subsystem number). +- [ ] `discover_extensions` returns structured failures; `dispatch_shortcode`'s + unknown-name fallthrough distinguishes "no such extension" from + "extension found but failed to load" (closes bd-nzdm1wry). +- [ ] Shadowing diagnostic per D4 (closes the silent-`rfind` gap). + +### Phase 2 — Handler contract parity + `require` (gap rows 4, 5, 7, 13) + +- [ ] Corpus rows from Phase 0 for calling convention + coercions green. +- [ ] Sandboxed, script-dir-relative `require` (native + WASM via + `SystemRuntime`), scoped to the extension's directory; test with a + sibling-module fixture. (This is the highest-risk item for real-world + extensions; likely a `package.preload`-style loader rather than exposing + the C `package` lib.) +- [ ] `quarto.shortcode.read_arg`/`error_output` parity vs `init.lua:1002-1032`. +- [ ] Fix `shortcode_to_span`'s `process::exit(1)` (nested kv arg) → diagnostic. + +### Phase 3 — DEFERRED (2026-07-31): `text` context — shortcodes in code, attributes, targets (gap row 6) + +Deferred per user review (see D8): text-position detection is the one place +Q1's parsing mess may partly return, and it is not needed for the current +acceptance targets. Phase 0's baseline probes document the gap. Content kept +below for the eventual follow-on strand: + +- Tests: `{{< meta k >}}` in CodeBlock text, Code inline, element attribute + (single-quoted, per Q1), `Link.target`, `Image.src`; `{shortcodes=false}` + and `cell-code` opt-outs; unknown shortcode in text context. +- Add `ResolutionContext::Text` in quarto-core; wire traversal over the + text positions; dispatch with `ShortcodeCallContext::Text`; stringify + results (Q1 `shortcodes.lua:248`). +- Decide grammar vs post-hoc scan for positions the grammar doesn't reach + (code block *contents* are opaque to the inline grammar — this likely + needs a targeted text-level scan; keep it in one module, single parser). +- Q1's `Image.src` default-extension fixup (#14583) — check whether Q2's + pipeline has the same failure mode before porting it. +- bd-u145dg3y's block-shortcode-used-inline warning (`Q-2-x` request) now + folds into Phase 5 instead. + +### Phase 4 — Missing built-ins: `var`, `env`, `pagebreak` (gap row 8, easy tier) + +- [ ] `var`: `_variables.yml` loading (project-scoped, values parsed as qmd + inlines), dotted lookup, unknown-var diagnostic (coded, source-mapped — + improvement over Q1's `?var:name`); `quarto.variables.get` Lua API. +- [ ] `env`: positional name + optional fallback arg (Q1 1.5 `#8316`); decide + unset-and-no-fallback behavior (Q1: `Null`; propose coded warning). +- [ ] `pagebreak`: per-format raw table (html/latex/typst/docx/odt/context/epub, + `\f` fallback) — implement as Rust handler or built-in Lua extension; + follow the existing built-in-extension pattern + (`claude-notes/plans/2026-04-01-builtin-extensions.md`). +- [ ] Each: implement in whichever tier (Rust handler vs embedded Lua ext) + matches its needs; document the choice in the strand. + +### Phase 5 — Unknown-shortcode policy + writer hardening (D1, gap row 9) + +- [ ] Implement the D1 policy as signed off: `Q-*` code, `.with_location()`, + visible marker; passthrough config for foreign shortcodes. +- [ ] HTML writer: surviving `Inline::Shortcode` is never silently dropped — + emit marker + diagnostic (relates to orphaned `Q-3-30`/`Q-3-42` catalog + entries; wire or retire them). +- [ ] Backfill `.with_code()` on the existing uncoded warnings in + `shortcode_resolve.rs` (`:376`, `:398`, `:431`, extract sites). +- [ ] bd-u145dg3y: block-level shortcode used inline → coded warning + (absorbed here from deferred Phase 3). + +### Phase 6 — Deferred / follow-on strands (file, don't implement here) + +- [ ] `text`-context resolution (deferred Phase 3 above — D8). +- [ ] `brand` shortcode (depends on brand.yml support status in Q2). +- [ ] `contents` shortcode (needs the collect-and-move filter design). +- [ ] `embed` (own epic; needs a drastic redesign for Q2, engine-dependent — D6). +- [ ] `q2 add/remove/list/update` CLI (own epic; bd-5edooc78 remove-guard; + network, git, trust prompt). +- [ ] Format-contributed `shortcodes:` naming embedded extensions. +- [ ] Paired shortcodes: backlog strand recording the D7 decision. + +## Related strands (link as `related` on the new strand) + +- bd-8b0af414 — manifest over-strictness (Phase 1 absorbs) +- bd-nzdm1wry — extension load failure misattribution (Phase 1 absorbs) +- bd-u145dg3y — block shortcode used inline, wants `Q-2-x` (Phase 3/5) +- bd-5edooc78 — `q2 remove` must guard built-ins (Phase 6 CLI epic) +- bd-mqk49 — pipeline stages not extension-registrable (context for Phase 6) +- bd-129m3 / bd-36fr9 — provenance anchors for shortcode values (adjacent) + +## Prior art in-repo (read before implementing) + +- `claude-notes/plans/2026-03-16-extensions-grand-plan.md` (+ phase plans 1–4) +- `claude-notes/plans/2026-03-20-extensions-phase3-shortcode-resolution.md` — + documents the TS dispatch internals this plan builds on +- `claude-notes/plans/2026-03-31-shortcode-args-compat.md` — calling convention +- `claude-notes/plans/2026-04-01-builtin-extensions.md` (+ batch2, video) — + the embedded-extension pattern Phase 4 follows +- `claude-notes/designs/provenance-contract.md` — `stamp_shortcode_anchors` +- `claude-notes/designs/transform-pipeline-phases.md` — where the transform sits + +## Appendix: process notes (toward the Q1→Q2 porting-guide skill) + +Captured for the guidance document we'll draft at the end of this effort: + +1. **Three parallel studies, then reconcile:** (a) Q1 *documented* contract + (quarto-web) — what users were promised; (b) Q1 *implementation* + (quarto-cli) — what actually happens, incl. undocumented features and bugs; + (c) Q2 current state — what already exists (it's rarely zero). The + interesting deltas are doc-vs-impl (undocumented features: paired + shortcodes; underdocumented: `raw_args`) and impl-vs-impl (Q1 bug we may + not want: shadowed `local result`). +2. **Spot-check the studies against the tree** before writing the plan + (built-ins list, stub files, unreachable enum variants) — static reading + agents are good but load-bearing claims deserve a grep. +3. **Classify each gap as compat / improve / drop**, with the project's two + levers named explicitly: (i) added strictness is OK iff the diagnostic is + source-mapped + actionable (`Q-*` code, `.with_location()`); (ii) prefer + explicit declaration (e.g. passthrough list) over Q1-style inference + (pass-through-anything-unknown). +4. **Phase 0 is always a compatibility corpus** ported from Q1's own test + fixtures plus real published extensions — Q1's tests encode the de-facto + contract better than its docs; real extensions are the acceptance bar. +5. **Check the braid skein + claude-notes first:** existing strands + (bd-8b0af414, bd-nzdm1wry…) and completed phase plans reframed this from + "port a feature" to "close gaps in a mostly-done port". +6. **Q1's architecture workarounds may evaporate in Q2:** Q1's four shortcode + parsers exist only because Pandoc's reader was in the way; Q2's in-grammar + parse deletes the whole problem class (and its restrictions, e.g. grid + tables). Ask "which Q1 mechanisms were workarounds for infrastructure Q2 + replaced?" before porting mechanism-by-mechanism. diff --git a/crates/quarto-core/src/transforms/shortcode_resolve.rs b/crates/quarto-core/src/transforms/shortcode_resolve.rs index f2ef0c289..6202f2a51 100644 --- a/crates/quarto-core/src/transforms/shortcode_resolve.rs +++ b/crates/quarto-core/src/transforms/shortcode_resolve.rs @@ -52,7 +52,6 @@ use std::sync::Arc; use quarto_analysis::AnalysisContext; use crate::Result; -use crate::extension::discover::find_extension; use crate::extension::types::Extension; use crate::render::RenderContext; use crate::transform::{AstTransform, TransformPhase}; @@ -260,6 +259,19 @@ fn flatten_blocks_to_inlines(blocks: &[Block], value_source: &SourceInfo) -> Vec result } +/// Lua engine plus one-shot extension-activation state. +/// +/// The `extensions_loaded` flag lives with the engine (not the transform) so a +/// freshly created engine can never observe a stale "already loaded" marker +/// from a previous document. +pub struct LuaEngineState { + engine: pampa::lua::LuaShortcodeEngine, + /// Whether every discovered extension's `contributes.shortcodes` scripts + /// have been loaded into this engine. Set on the first dispatch that + /// reaches the Lua stage. + extensions_loaded: bool, +} + /// Transform that resolves shortcodes in the AST. /// /// Supports both built-in Rust handlers and Lua shortcode scripts loaded from @@ -328,10 +340,11 @@ impl ShortcodeResolveTransform { shortcode: &Shortcode, ctx: &ShortcodeContext<'_>, resolution_ctx: ResolutionContext, - lua_engine: &mut Option, + lua_engine: &mut Option, + diagnostics: &mut Vec, ) -> ShortcodeResult { let mut result = self - .dispatch_shortcode(shortcode, ctx, resolution_ctx, lua_engine) + .dispatch_shortcode(shortcode, ctx, resolution_ctx, lua_engine, diagnostics) .await; stamp_shortcode_anchors(&mut result, &shortcode.name, ctx.source_info); result @@ -345,7 +358,8 @@ impl ShortcodeResolveTransform { shortcode: &Shortcode, ctx: &ShortcodeContext<'_>, resolution_ctx: ResolutionContext, - lua_engine: &mut Option, + lua_engine: &mut Option, + diagnostics: &mut Vec, ) -> ShortcodeResult { // Handle escaped shortcodes - preserve as literal text if shortcode.is_escaped { @@ -359,38 +373,44 @@ impl ShortcodeResolveTransform { } } - // 2. Try Lua engine (loaded handlers) - if let Some(engine) = lua_engine.as_mut() { - // If handler is already loaded, call it - if engine.has_handler(&shortcode.name) { - return dispatch_lua_shortcode(engine, shortcode, ctx, resolution_ctx).await; - } - - // 3. Try name-based extension lookup (on-demand loading) - if let Some(ext) = find_extension(&shortcode.name, &self.extensions) - && !ext.contributes.shortcodes.is_empty() - { - for script_path in &ext.contributes.shortcodes { - if let Err(e) = engine.load_script(script_path).await { - let diagnostic = - DiagnosticMessageBuilder::warning("Shortcode script error") - .problem(format!( - "Failed to load shortcode script `{}`: {}", - script_path.display(), - e - )) - .with_location(ctx.source_info.clone()) - .build(); - return ShortcodeResult::Error(ShortcodeError { - key: shortcode.name.clone(), - diagnostic, - }); + // 2. Lua handlers. On the first dispatch that reaches the Lua stage, + // eagerly load every discovered extension's shortcode scripts. + // Handler names come from the Lua table keys (or harvested globals), + // decoupled from the extension id — an extension named + // `quarto-tiers` may contribute a shortcode named `tier`, so + // lookup-by-extension-name cannot work. Load order determines + // same-name precedence (later registration wins): document + // `shortcodes:` scripts (loaded at engine creation), then + // extensions in discovery order (built-ins first, more-local + // last). Rust built-in handlers (step 1) always win. + if let Some(state) = lua_engine.as_mut() { + if !state.extensions_loaded { + state.extensions_loaded = true; + for ext in &self.extensions { + for script_path in &ext.contributes.shortcodes { + if let Err(e) = state.engine.load_script(script_path).await { + // A broken script must not hijack the triggering + // shortcode's result (it may resolve from another + // extension); warn with the extension and script + // named as the cause, and keep loading the rest. + diagnostics.push( + DiagnosticMessageBuilder::warning("Shortcode script error") + .problem(format!( + "Failed to load shortcode script `{}` from extension `{}`: {}", + script_path.display(), + ext.id, + e + )) + .with_location(ctx.source_info.clone()) + .build(), + ); + } } } - // Retry after loading extension scripts - if engine.has_handler(&shortcode.name) { - return dispatch_lua_shortcode(engine, shortcode, ctx, resolution_ctx).await; - } + } + if state.engine.has_handler(&shortcode.name) { + return dispatch_lua_shortcode(&mut state.engine, shortcode, ctx, resolution_ctx) + .await; } } @@ -830,11 +850,7 @@ pub fn extract_shortcode_paths(meta: &ConfigValue, document_dir: &std::path::Pat }; items .iter() - .filter_map(|item| match &item.value { - ConfigValueKind::Path(s) => Some(document_dir.join(s)), - ConfigValueKind::Scalar(_) => item.as_str().map(|s| document_dir.join(s)), - _ => None, - }) + .filter_map(|item| item.as_plain_text().map(|s| document_dir.join(s))) .collect() } @@ -861,7 +877,9 @@ impl AstTransform for ShortcodeResolveTransform { let runtime = runtime.clone(); match pampa::lua::LuaShortcodeEngine::new(&self.target_format, runtime) { Ok(mut engine) => { - // Load scripts from metadata-specified paths + // Load scripts from metadata-specified paths. These load + // before extension scripts, so a same-named extension + // handler overrides a document-level one (Q1 precedence). for path in &self.lua_shortcode_paths { if let Err(e) = engine.load_script(path).await { diagnostics.push( @@ -875,7 +893,10 @@ impl AstTransform for ShortcodeResolveTransform { ); } } - Some(engine) + Some(LuaEngineState { + engine, + extensions_loaded: false, + }) } Err(e) => { diagnostics.push( @@ -901,7 +922,8 @@ impl AstTransform for ShortcodeResolveTransform { .await; // Extract Lua-registered data before the engine is dropped - if let Some(engine) = lua_engine.as_mut() { + if let Some(state) = lua_engine.as_mut() { + let engine = &mut state.engine; // Extract diagnostics from quarto.warn()/quarto.error() match engine.extract_diagnostics() { Ok(lua_diags) => diagnostics.extend(lua_diags), @@ -965,7 +987,7 @@ fn resolve_blocks<'a>( transform: &'a ShortcodeResolveTransform, metadata: &'a ConfigValue, diagnostics: &'a mut Vec, - lua_engine: &'a mut Option, + lua_engine: &'a mut Option, ) -> Pin + 'a>> { Box::pin(async move { let mut i = 0; @@ -978,7 +1000,13 @@ fn resolve_blocks<'a>( source_info: &shortcode_owned.source_info, }; match transform - .resolve_shortcode(&shortcode_owned, &ctx, ResolutionContext::Block, lua_engine) + .resolve_shortcode( + &shortcode_owned, + &ctx, + ResolutionContext::Block, + lua_engine, + diagnostics, + ) .await { ShortcodeResult::Blocks(new_blocks) => { @@ -1051,7 +1079,7 @@ fn resolve_block<'a>( transform: &'a ShortcodeResolveTransform, metadata: &'a ConfigValue, diagnostics: &'a mut Vec, - lua_engine: &'a mut Option, + lua_engine: &'a mut Option, ) -> Pin + 'a>> { Box::pin(async move { match block { @@ -1210,7 +1238,7 @@ fn resolve_inlines<'a>( transform: &'a ShortcodeResolveTransform, metadata: &'a ConfigValue, diagnostics: &'a mut Vec, - lua_engine: &'a mut Option, + lua_engine: &'a mut Option, ) -> Pin + 'a>> { Box::pin(async move { let mut i = 0; @@ -1228,6 +1256,7 @@ fn resolve_inlines<'a>( &shortcode_ctx, ResolutionContext::Inline, lua_engine, + diagnostics, ) .await { @@ -1288,7 +1317,7 @@ fn recurse_inline<'a>( transform: &'a ShortcodeResolveTransform, metadata: &'a ConfigValue, diagnostics: &'a mut Vec, - lua_engine: &'a mut Option, + lua_engine: &'a mut Option, ) -> Pin + 'a>> { Box::pin(async move { match inline { @@ -1651,7 +1680,13 @@ mod tests { }; let result = transform - .resolve_shortcode(&shortcode, &ctx, ResolutionContext::Inline, &mut None) + .resolve_shortcode( + &shortcode, + &ctx, + ResolutionContext::Inline, + &mut None, + &mut Vec::new(), + ) .await; assert!(matches!(result, ShortcodeResult::Preserve)); } @@ -1668,7 +1703,13 @@ mod tests { }; let result = transform - .resolve_shortcode(&shortcode, &ctx, ResolutionContext::Inline, &mut None) + .resolve_shortcode( + &shortcode, + &ctx, + ResolutionContext::Inline, + &mut None, + &mut Vec::new(), + ) .await; match result { ShortcodeResult::Error(err) => { diff --git a/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/shorty.lua b/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/shorty.lua new file mode 100644 index 000000000..19082dc18 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/shorty.lua @@ -0,0 +1,9 @@ +return { + shorty = function(args) + if args[1] == "error" then + return quarto.shortcode.error_output("shorty", "error message", "inline") + else + return pandoc.Strong(args[1]) + end + end +} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/test.qmd new file mode 100644 index 000000000..7636794be --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/test.qmd @@ -0,0 +1,15 @@ +--- +title: Document-level shortcodes key +format: html +shortcodes: + - shorty.lua +_quarto: + tests: + html: + ensureFileRegexMatches: + - ["strong>_bringit_", "Shortcode Error \\(shorty\\): error message"] +--- + +{{< shorty _bringit_ >}} + +{{< shorty error >}} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-escape-braces/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-escape-braces/test.qmd new file mode 100644 index 000000000..6867634e5 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-escape-braces/test.qmd @@ -0,0 +1,12 @@ +--- +title: Escaped shortcode with extra braces +format: html +_quarto: + tests: + html: + noErrors: true + ensureFileRegexMatches: + - ["fakename"] +--- + +Literal: {{{< fakename >}}} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-escape-comment/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-escape-comment/test.qmd new file mode 100644 index 000000000..af2a4252b --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-escape-comment/test.qmd @@ -0,0 +1,14 @@ +--- +title: Escaped shortcode comment form +format: html +_quarto: + tests: + run: + skip: "Known gap: Q1's {{}} comment-escape form is not in the qmd grammar (parse error). Pending decision on bd-540a976a (shortcode-extensions port, Phase 5); unskip when the grammar supports it." + html: + noErrors: true + ensureFileRegexMatches: + - ["othername"] +--- + +Literal: {{}} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-global-fn/_extensions/greeter/_extension.yml b/crates/quarto/tests/smoke-all/extensions/contract-global-fn/_extensions/greeter/_extension.yml new file mode 100644 index 000000000..fced92d6a --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-global-fn/_extensions/greeter/_extension.yml @@ -0,0 +1,5 @@ +title: Greeter +author: Test +contributes: + shortcodes: + - greeter.lua diff --git a/crates/quarto/tests/smoke-all/extensions/contract-global-fn/_extensions/greeter/greeter.lua b/crates/quarto/tests/smoke-all/extensions/contract-global-fn/_extensions/greeter/greeter.lua new file mode 100644 index 000000000..2c87ab7ed --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-global-fn/_extensions/greeter/greeter.lua @@ -0,0 +1,3 @@ +function greet(args, kwargs, meta) + return "GREET-" .. pandoc.utils.stringify(args[1]) +end diff --git a/crates/quarto/tests/smoke-all/extensions/contract-global-fn/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-global-fn/test.qmd new file mode 100644 index 000000000..40080aca6 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-global-fn/test.qmd @@ -0,0 +1,12 @@ +--- +title: Global-function registration +format: html +_quarto: + tests: + html: + noErrors: true + ensureFileRegexMatches: + - ["GREET-World"] +--- + +{{< greet World >}} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-return-coercions/_extensions/coerce/_extension.yml b/crates/quarto/tests/smoke-all/extensions/contract-return-coercions/_extensions/coerce/_extension.yml new file mode 100644 index 000000000..fe6b13fbf --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-return-coercions/_extensions/coerce/_extension.yml @@ -0,0 +1,5 @@ +title: Coerce +author: Test +contributes: + shortcodes: + - coerce.lua diff --git a/crates/quarto/tests/smoke-all/extensions/contract-return-coercions/_extensions/coerce/coerce.lua b/crates/quarto/tests/smoke-all/extensions/contract-return-coercions/_extensions/coerce/coerce.lua new file mode 100644 index 000000000..c0ec884cd --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-return-coercions/_extensions/coerce/coerce.lua @@ -0,0 +1,11 @@ +return { + rstr = function() return "RET-STRING" end, + rinline = function() return pandoc.Strong({pandoc.Str("RET-INLINE")}) end, + rinlines = function() return pandoc.Inlines({pandoc.Str("RET-"), pandoc.Str("INLINES")}) end, + rblock = function() return pandoc.Para({pandoc.Str("RET-BLOCK")}) end, + rblocks = function() return pandoc.Blocks({ + pandoc.Para({pandoc.Str("RET-BLOCKS-1")}), + pandoc.Para({pandoc.Str("RET-BLOCKS-2")}) + }) end, + rarray = function() return {pandoc.Str("RET-"), pandoc.Str("ARRAY")} end +} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-return-coercions/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-return-coercions/test.qmd new file mode 100644 index 000000000..acb8b2b12 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-return-coercions/test.qmd @@ -0,0 +1,22 @@ +--- +title: Return value coercions +format: html +_quarto: + tests: + html: + noErrors: true + ensureFileRegexMatches: + - ["RET-STRING", "strong>RET-INLINE", "RET-INLINES", "RET-BLOCK", "RET-BLOCKS-1", "RET-BLOCKS-2", "RET-ARRAY"] +--- + +String: {{< rstr >}} + +Inline node: {{< rinline >}} + +Inlines list: {{< rinlines >}} + +{{< rblock >}} + +{{< rblocks >}} + +Array: {{< rarray >}} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-table-return/_extensions/echoer/_extension.yml b/crates/quarto/tests/smoke-all/extensions/contract-table-return/_extensions/echoer/_extension.yml new file mode 100644 index 000000000..5c081e100 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-table-return/_extensions/echoer/_extension.yml @@ -0,0 +1,5 @@ +title: Echoer +author: Test +contributes: + shortcodes: + - echoer.lua diff --git a/crates/quarto/tests/smoke-all/extensions/contract-table-return/_extensions/echoer/echoer.lua b/crates/quarto/tests/smoke-all/extensions/contract-table-return/_extensions/echoer/echoer.lua new file mode 100644 index 000000000..177fb73b6 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-table-return/_extensions/echoer/echoer.lua @@ -0,0 +1,8 @@ +return { + ['dash-name'] = function(args) + return "DASH-OK" + end, + ctx = function(args, kwargs, meta, raw_args, context) + return "CTX-" .. context + end +} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-table-return/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-table-return/test.qmd new file mode 100644 index 000000000..129504552 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-table-return/test.qmd @@ -0,0 +1,16 @@ +--- +title: Table-return registration, dash names, context +format: html +_quarto: + tests: + html: + noErrors: true + ensureFileRegexMatches: + - ["DASH-OK", "CTX-block", "CTX-inline"] +--- + +Inline dash: {{< dash-name >}} + +Inline context: {{< ctx >}} here. + +{{< ctx >}} From 3585a6d29069483650ab05609ffb815133f21c5e Mon Sep 17 00:00:00 2001 From: Carlos Scheidegger Date: Fri, 31 Jul 2026 16:43:17 -0500 Subject: [PATCH 2/8] Shortcode handler-contract parity: raw_args, structured meta, nested args, stringify scalars (bd-540a976a) Four Q1-parity fixes in the Lua shortcode calling convention, each with a previously-failing contract fixture or regression test: - raw_args now carries keyword-argument values after positionals (names stripped), matching TS Quarto; Q1's video.lua relies on raw_args[1] as the src fallback. Exact source interleaving of positional-after-keyword args is not recoverable from the q2 AST and is documented as an approximation. - The handler meta parameter is now the full metadata tree as nested Lua tables with native scalar types, plus a dotted-path __index fallback on the top level. Supports Q1's both documented access styles: meta['a.b.c'] and meta.a.b.c, including 1-based array segments (meta['authors.2']). Replaces the flat top-level stringified pairs, which dropped nested and compound values entirely. pampa's ShortcodeArgs.metadata is now a ConfigValue. - Nested shortcode arguments ({{< outer {{< inner >}} >}}) resolve bottom-up before the outer handler runs and arrive as stringified plain args (TS Quarto semantics). They were previously silently dropped. Keyword-value nested shortcodes are handled the same way. - pandoc.utils.stringify now converts booleans/integers/numbers to their string value (pandoc-documented behavior). This also fixes the regression the meta change would have caused in video.lua's auto-stretch gate, which stringifies meta['auto-stretch']. Contract fixtures contract-args-kwargs, contract-meta-dotted (extended with chaining/boolean/array probes), contract-nested-arg now pass; full workspace suite green (10,810 tests). Plan: claude-notes/plans/2026-07-31-shortcode-extensions-port.md Co-Authored-By: Claude Fable 5 --- .../2026-07-31-shortcode-extensions-port.md | 37 ++-- crates/pampa/src/lua/shortcode.rs | 197 ++++++++++++++++-- crates/pampa/src/lua/utils.rs | 32 +++ .../src/transforms/shortcode_resolve.rs | 148 ++++++++++--- .../_extensions/argy/_extension.yml | 5 + .../_extensions/argy/argy.lua | 9 + .../extensions/contract-args-kwargs/test.qmd | 12 ++ .../_extensions/metaget/_extension.yml | 5 + .../_extensions/metaget/metaget.lua | 17 ++ .../extensions/contract-meta-dotted/test.qmd | 30 +++ .../_extensions/nester/_extension.yml | 5 + .../_extensions/nester/nester.lua | 8 + .../extensions/contract-nested-arg/test.qmd | 12 ++ 13 files changed, 451 insertions(+), 66 deletions(-) create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/_extensions/argy/_extension.yml create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/_extensions/argy/argy.lua create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/test.qmd create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-meta-dotted/_extensions/metaget/_extension.yml create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-meta-dotted/_extensions/metaget/metaget.lua create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-meta-dotted/test.qmd create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-nested-arg/_extensions/nester/_extension.yml create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-nested-arg/_extensions/nester/nester.lua create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-nested-arg/test.qmd diff --git a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md index ad8e97b0e..c5fd8d5f4 100644 --- a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md +++ b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md @@ -209,14 +209,20 @@ The deliverable is a fixture suite that encodes the Q1 contract, so every later phase has failing tests to turn green, and so we discover *actual* Q2 behavior where the study only has static reads. -- [ ] Port Q1's smoke fixtures (`external-sources/quarto-cli/tests/docs/shortcodes/` - — `shorty.lua`, `custom.qmd`, the `?meta:…`/`?var:…` error expectations) - into local fixtures (copy, never reference `external-sources/` from tests). -- [ ] Author a "contract extension" fixture exercising: table-return and - global-fn registration; dash-named shortcode; all five handler params; - kwargs missing-key → empty Inlines; `meta` dotted lookup incl. `\\.` - escape and 1-based array index; each return-value coercion row; both - escape forms; nested shortcode as arg and as kwarg value. +- [x] Port Q1's smoke fixtures — `shorty.lua` + error_output ported as + `contract-doc-shortcodes` (passing; Q2 renders `[Shortcode Error + (shorty): error message]` instead of Q1's `?shorty:error message` — + accepted deviation, clearer text). `?var:` expectations wait for + Phase 4 var. +- [x] Author contract fixtures (committed 08f29f89, passing): + `contract-table-return` (table-return registration, dash names, + block/inline context), `contract-global-fn`, `contract-return-coercions` + (string/Inline/Inlines/Block/Blocks/array), `contract-escape-braces`, + `contract-doc-shortcodes`. In-flight (failing = TDD targets, uncommitted): + `contract-args-kwargs` (raw_args), `contract-meta-dotted` (dotted + lookup), `contract-nested-arg`. Parked with `tests.run.skip`: + `contract-escape-comment` (grammar gap, decision pending — recommend + targeted Q-2-x diagnostic over porting the Hugo `/* */` form). - [ ] Integration tests driving the real binary path (`render_document_to_file` / `q2 render` on fixtures) — not `HtmlRenderConfig::default()` shortcuts. - [ ] Baseline probes (tests that *document* current behavior, marked @@ -235,15 +241,12 @@ where the study only has static reads. ### Phase 1 — Extension loading: eager activation, Q1-compat intake, loud failures (D2, D3, gap row 3) -- [ ] **Fix the name-keyed activation bug (gap row 3)** — need not land - first, but must land as part of this phase's work: - test with an extension whose shortcode name ≠ extension id (fixture: - a `quarto-tiers`-shaped extension contributing `tier`); replace the - `find_extension(&shortcode.name)` on-demand path with Q1's eager - semantics — on first shortcode dispatch, load every discovered - extension's `contributes.shortcodes` scripts into the engine, then - dispatch by registered handler name. Keep document-level laziness - (no Lua unless the doc contains shortcodes). +- [x] **Fix the name-keyed activation bug (gap row 3)** — done, commit + 08f29f89. `LuaEngineState` wraps engine + one-shot flag; eager load of + all extensions' scripts on first Lua-stage dispatch; D4 precedence + (doc `shortcodes:` < extensions in discovery order < Rust built-ins); + per-script load failures warn naming extension id + script path. + Verified end-to-end on connect-docs (`tier` renders, 0 warnings). - [ ] Tests: malformed `_extension.yml` (bad YAML, bad semver, empty contributes) → coded, source-mapped diagnostics; minimal Q1 manifest (no title/author) loads. diff --git a/crates/pampa/src/lua/shortcode.rs b/crates/pampa/src/lua/shortcode.rs index 32cca9df0..5925ddf73 100644 --- a/crates/pampa/src/lua/shortcode.rs +++ b/crates/pampa/src/lua/shortcode.rs @@ -10,10 +10,12 @@ */ use mlua::{Function, Lua, Result, Table, Value}; +use quarto_pandoc_types::config_value::{ConfigValue, ConfigValueKind}; use quarto_source_map::{By, SourceInfo}; use std::collections::HashMap; use std::path::Path; use std::sync::Arc; +use yaml_rust2::Yaml; use crate::pandoc::{Block, Inline}; @@ -328,17 +330,74 @@ impl LuaShortcodeEngine { } fn build_meta_table(&self, args: &ShortcodeArgs) -> Result { - let table = self.lua.create_table()?; - for (key, val) in &args.metadata { - table.set(key.as_str(), val.as_str())?; - } + // TS Quarto compat: `meta` supports both native chaining + // (`meta.custom.nested.value`) via real nested tables, and Q1's + // dotted-string lookup (`meta["custom.nested.value"]`, documented as + // `meta["github.owner"]`) via a __index fallback on the top-level + // table. A literal top-level key containing dots wins over the + // dotted-path interpretation because rawget-able keys never reach + // __index. Numeric path segments index arrays 1-based + // (`meta["author.1"]`), matching Q1's `option()` navigation. + let v = config_value_to_lua(&self.lua, &args.metadata)?; + let table = match v { + Value::Table(t) => t, + _ => self.lua.create_table()?, + }; + let mt = self.lua.create_table()?; + mt.set( + "__index", + self.lua + .load( + r#" +function(t, k) + if type(k) ~= "string" or not string.find(k, ".", 1, true) then + return nil + end + local cur = t + for part in string.gmatch(k, "[^.]+") do + if type(cur) ~= "table" then + return nil + end + local v = rawget(cur, part) + if v == nil then + local n = tonumber(part) + if n ~= nil then + v = rawget(cur, n) + end + end + if v == nil then + return nil + end + cur = v + end + return cur +end +"#, + ) + .eval::()?, + )?; + table.set_metatable(Some(mt))?; Ok(Value::Table(table)) } fn build_raw_args(&self, args: &ShortcodeArgs) -> Result { + // TS Quarto compat: raw_args carries ALL argument values in source + // order, with keyword-argument names stripped (a named arg + // contributes only its value). The q2 AST stores positional and + // keyword args separately, so the original interleaving is not + // recoverable; we approximate with positionals first, then keyword + // values in declaration order. Invocations that interleave a + // positional after a keyword arg are the only case that differs + // from Q1, and neither Q1's docs nor its built-ins ever do that. let table = self.lua.create_table()?; - for (i, arg) in args.positional.iter().enumerate() { - table.set(i + 1, arg.as_str())?; + let mut i = 0; + for arg in args.positional.iter() { + i += 1; + table.set(i, arg.as_str())?; + } + for (_key, val) in args.keyword.iter() { + i += 1; + table.set(i, val.as_str())?; } Ok(Value::Table(table)) } @@ -350,7 +409,47 @@ impl LuaShortcodeEngine { pub struct ShortcodeArgs { pub positional: Vec, pub keyword: Vec<(String, String)>, - pub metadata: Vec<(String, String)>, + /// Full document metadata tree; converted to nested Lua tables (with a + /// dotted-path `__index` fallback) for the handler's `meta` parameter. + pub metadata: ConfigValue, +} + +/// Convert a ConfigValue tree into a Lua value for the shortcode `meta` param. +/// +/// - Scalars map to native Lua values (string/integer/number/boolean; null +/// maps to nil) +/// - `PandocInlines` (the storage form of bare YAML strings in front matter) +/// map to their plain-text string — Q1 handlers stringify meta values anyway +/// - Arrays map to 1-based tables, maps to nested tables +/// - `PandocBlocks` and other kinds without a string form map to nil +fn config_value_to_lua(lua: &Lua, v: &ConfigValue) -> Result { + Ok(match &v.value { + ConfigValueKind::Scalar(Yaml::Boolean(b)) => Value::Boolean(*b), + ConfigValueKind::Scalar(Yaml::Integer(i)) => Value::Integer(*i), + ConfigValueKind::Scalar(Yaml::Real(s)) => match s.parse::() { + Ok(f) => Value::Number(f), + Err(_) => Value::String(lua.create_string(s)?), + }, + ConfigValueKind::Scalar(Yaml::Null) => Value::Nil, + ConfigValueKind::Array(items) => { + let t = lua.create_table()?; + for (i, item) in items.iter().enumerate() { + t.set(i + 1, config_value_to_lua(lua, item)?)?; + } + Value::Table(t) + } + ConfigValueKind::Map(entries) => { + let t = lua.create_table()?; + for entry in entries { + t.set(entry.key.as_str(), config_value_to_lua(lua, &entry.value)?)?; + } + Value::Table(t) + } + _ => match v.as_plain_text() { + Some(s) => Value::String(lua.create_string(&s)?), + None => Value::Nil, + }, + }) } /// Errors from the shortcode engine. @@ -524,11 +623,28 @@ mod tests { path } + fn empty_meta() -> ConfigValue { + ConfigValue::new_map(vec![], SourceInfo::generated(By::programmatic_config())) + } + + fn meta_from_pairs(pairs: &[(&str, &str)]) -> ConfigValue { + let si = SourceInfo::generated(By::programmatic_config()); + let entries = pairs + .iter() + .map(|(k, v)| quarto_pandoc_types::config_value::ConfigMapEntry { + key: (*k).to_string(), + key_source: si.clone(), + value: ConfigValue::new_string(*v, si.clone()), + }) + .collect(); + ConfigValue::new_map(entries, si) + } + fn make_empty_args() -> ShortcodeArgs { ShortcodeArgs { positional: vec![], keyword: vec![], - metadata: vec![], + metadata: empty_meta(), } } @@ -817,7 +933,7 @@ return { let args = ShortcodeArgs { positional: vec!["world".to_string()], keyword: vec![], - metadata: vec![], + metadata: empty_meta(), }; let result = engine .call("echo", &args, ShortcodeCallContext::Inline) @@ -851,7 +967,7 @@ return { let args = ShortcodeArgs { positional: vec![], keyword: vec![("greeting".to_string(), "howdy".to_string())], - metadata: vec![], + metadata: empty_meta(), }; let result = engine .call("kwarg", &args, ShortcodeCallContext::Inline) @@ -863,6 +979,44 @@ return { } } + #[tokio::test] + async fn test_raw_args_includes_keyword_values() { + // TS Quarto compat: raw_args carries positional AND keyword values + // (names stripped), so `{{< v src k=fast >}}` yields raw_args of + // {"src", "fast"}. Q1's video.lua relies on raw_args[1] as the src + // fallback. + let tmp = TempDir::new().unwrap(); + let script = write_script( + tmp.path(), + "raw.lua", + r#" +return { + raw = function(args, kwargs, meta, raw_args, context) + return tostring(#raw_args) .. ":" .. table.concat(raw_args, ",") + end +} +"#, + ); + + let runtime = make_runtime(); + let mut engine = LuaShortcodeEngine::new("html", runtime).unwrap(); + engine.load_script(&script).await.unwrap(); + + let args = ShortcodeArgs { + positional: vec!["src".to_string()], + keyword: vec![("k".to_string(), "fast".to_string())], + metadata: empty_meta(), + }; + let result = engine + .call("raw", &args, ShortcodeCallContext::Inline) + .await + .unwrap(); + match result { + LuaShortcodeResult::Text(s) => assert_eq!(s, "2:src,fast"), + other => panic!("Expected Text, got {:?}", other), + } + } + #[tokio::test] async fn test_handler_receives_meta() { let tmp = TempDir::new().unwrap(); @@ -885,7 +1039,7 @@ return { let args = ShortcodeArgs { positional: vec![], keyword: vec![], - metadata: vec![("title".to_string(), "My Doc".to_string())], + metadata: meta_from_pairs(&[("title", "My Doc")]), }; let result = engine .call("meta_reader", &args, ShortcodeCallContext::Inline) @@ -1008,7 +1162,7 @@ return { let args = ShortcodeArgs { positional: vec!["test-value".to_string()], keyword: vec![], - metadata: vec![], + metadata: empty_meta(), }; let result = engine .call("readarg", &args, ShortcodeCallContext::Inline) @@ -1204,7 +1358,7 @@ return { let args = ShortcodeArgs { positional: vec!["5".to_string()], keyword: vec![], - metadata: vec![], + metadata: empty_meta(), }; let result = engine .call("stringify_arg", &args, ShortcodeCallContext::Inline) @@ -1238,7 +1392,7 @@ return { let args = ShortcodeArgs { positional: vec!["hello".to_string()], keyword: vec![("key".to_string(), "val".to_string())], - metadata: vec![], + metadata: empty_meta(), }; let result = engine .call("count_args", &args, ShortcodeCallContext::Inline) @@ -1319,7 +1473,7 @@ return { let args = ShortcodeArgs { positional: vec!["3".to_string()], keyword: vec![], - metadata: vec![], + metadata: empty_meta(), }; let result = engine .call("lipsum_pattern", &args, ShortcodeCallContext::Inline) @@ -1358,7 +1512,7 @@ return { let args = ShortcodeArgs { positional: vec!["icon-name".to_string()], keyword: vec![("title".to_string(), "My Icon".to_string())], - metadata: vec![], + metadata: empty_meta(), }; let result = engine .call("fa", &args, ShortcodeCallContext::Inline) @@ -1403,7 +1557,7 @@ return { let args_with = ShortcodeArgs { positional: vec![], keyword: vec![("width".to_string(), "800".to_string())], - metadata: vec![], + metadata: empty_meta(), }; let result = engine .call("unsplash", &args_with, ShortcodeCallContext::Inline) @@ -1418,7 +1572,7 @@ return { let args_without = ShortcodeArgs { positional: vec![], keyword: vec![], - metadata: vec![], + metadata: empty_meta(), }; let result = engine .call("unsplash", &args_without, ShortcodeCallContext::Inline) @@ -1460,10 +1614,7 @@ return { let args = ShortcodeArgs { positional: vec![], keyword: vec![], - metadata: vec![ - ("title".to_string(), "My Document".to_string()), - ("author".to_string(), "Jane".to_string()), - ], + metadata: meta_from_pairs(&[("title", "My Document"), ("author", "Jane")]), }; let result = engine .call("meta_sc", &args, ShortcodeCallContext::Inline) @@ -1502,7 +1653,7 @@ return { let args = ShortcodeArgs { positional: vec!["hello".to_string(), "world".to_string()], keyword: vec![], - metadata: vec![], + metadata: empty_meta(), }; let result = engine .call("raw", &args, ShortcodeCallContext::Inline) diff --git a/crates/pampa/src/lua/utils.rs b/crates/pampa/src/lua/utils.rs index 9cead0540..41629afae 100644 --- a/crates/pampa/src/lua/utils.rs +++ b/crates/pampa/src/lua/utils.rs @@ -800,6 +800,12 @@ fn stringify_value(value: &Value) -> Result { Ok(result) } Value::String(s) => Ok(s.to_str()?.to_string()), + // Pandoc parity: stringify "converts booleans, numbers, and strings + // to their string value". Metadata booleans/numbers reach handlers + // as native Lua values, so these must not collapse to "". + Value::Boolean(b) => Ok(b.to_string()), + Value::Integer(i) => Ok(i.to_string()), + Value::Number(n) => Ok(n.to_string()), _ => Ok(String::new()), } } @@ -954,6 +960,32 @@ mod tests { assert_eq!(result, "hello"); } + #[test] + fn test_stringify_scalars() { + // Pandoc parity: booleans, numbers, and strings stringify to their + // string value (metadata scalars reach shortcode handlers as native + // Lua values). + let lua = create_test_lua(); + + let result: String = lua + .load("return pandoc.utils.stringify(false)") + .eval() + .unwrap(); + assert_eq!(result, "false"); + + let result: String = lua + .load("return pandoc.utils.stringify(42)") + .eval() + .unwrap(); + assert_eq!(result, "42"); + + let result: String = lua + .load("return pandoc.utils.stringify(2.5)") + .eval() + .unwrap(); + assert_eq!(result, "2.5"); + } + #[test] fn test_stringify_emph() { let lua = create_test_lua(); diff --git a/crates/quarto-core/src/transforms/shortcode_resolve.rs b/crates/quarto-core/src/transforms/shortcode_resolve.rs index 6202f2a51..68750ee02 100644 --- a/crates/quarto-core/src/transforms/shortcode_resolve.rs +++ b/crates/quarto-core/src/transforms/shortcode_resolve.rs @@ -366,6 +366,20 @@ impl ShortcodeResolveTransform { return ShortcodeResult::Preserve; } + // Resolve nested shortcode arguments bottom-up (TS Quarto semantics): + // each `ShortcodeArg::Shortcode` is dispatched and its result + // stringified before the outer handler runs, so the outer handler + // sees a plain string argument. + let resolved_holder; + let shortcode = if has_nested_shortcode_args(shortcode) { + resolved_holder = self + .resolve_nested_args(shortcode, ctx, lua_engine, diagnostics) + .await; + &resolved_holder + } else { + shortcode + }; + // 1. Try built-in Rust handlers first for handler in &self.handlers { if handler.name() == shortcode.name { @@ -425,6 +439,109 @@ impl ShortcodeResolveTransform { diagnostic, }) } + + /// Produce a copy of `shortcode` with every nested shortcode argument + /// dispatched and replaced by its stringified result. + /// + /// Mirrors TS Quarto's bottom-up traversal: the inner shortcode resolves + /// first (in inline context) and the outer handler receives the + /// stringified result as an ordinary string argument. An inner shortcode + /// that errors contributes an empty string (its diagnostic is still + /// emitted); an escaped inner shortcode contributes its literal + /// `{{< … >}}` text. + async fn resolve_nested_args( + &self, + shortcode: &Shortcode, + ctx: &ShortcodeContext<'_>, + lua_engine: &mut Option, + diagnostics: &mut Vec, + ) -> Shortcode { + let mut resolved = shortcode.clone(); + for arg in resolved.positional_args.iter_mut() { + self.resolve_nested_arg(arg, ctx, lua_engine, diagnostics) + .await; + } + // The CST currently coerces keyword-argument values to strings, but + // the AST type permits nested shortcodes here; handle them the same + // way rather than silently dropping (see also the shortcode_to_span + // kv-arg hardening in pampa). + let keys: Vec = resolved.keyword_args.keys().cloned().collect(); + for key in keys { + if let Some(arg) = resolved.keyword_args.get_mut(&key) { + self.resolve_nested_arg(arg, ctx, lua_engine, diagnostics) + .await; + } + } + resolved + } + + /// Replace one `ShortcodeArg::Shortcode` with its stringified resolution. + async fn resolve_nested_arg( + &self, + arg: &mut ShortcodeArg, + ctx: &ShortcodeContext<'_>, + lua_engine: &mut Option, + diagnostics: &mut Vec, + ) { + let ShortcodeArg::Shortcode(inner) = arg else { + return; + }; + let inner = inner.clone(); + if inner.is_escaped { + if let Inline::Str(s) = shortcode_to_literal(&inner) { + *arg = ShortcodeArg::String(s.text); + } + return; + } + let inner_ctx = ShortcodeContext { + metadata: ctx.metadata, + source_info: &inner.source_info, + }; + // Box the recursive call: dispatch_shortcode -> resolve_nested_args + // -> resolve_nested_arg -> dispatch_shortcode is async recursion. + let result = Box::pin(self.dispatch_shortcode( + &inner, + &inner_ctx, + ResolutionContext::Inline, + lua_engine, + diagnostics, + )) + .await; + let text = match result { + ShortcodeResult::Inlines(inlines) => { + crate::transforms::metadata_normalize::inlines_to_plain_text(&inlines) + } + ShortcodeResult::Blocks(blocks) => { + let inlines = flatten_blocks_to_inlines(&blocks, &inner.source_info); + crate::transforms::metadata_normalize::inlines_to_plain_text(&inlines) + } + ShortcodeResult::Error(error) => { + diagnostics.push(error.diagnostic); + String::new() + } + ShortcodeResult::Preserve => { + if let Inline::Str(s) = shortcode_to_literal(&inner) { + s.text + } else { + String::new() + } + } + }; + *arg = ShortcodeArg::String(text); + } +} + +/// Does this shortcode carry any nested shortcode arguments (positional or +/// keyword)? +fn has_nested_shortcode_args(shortcode: &Shortcode) -> bool { + shortcode + .positional_args + .iter() + .any(|a| matches!(a, ShortcodeArg::Shortcode(_))) + || shortcode + .keyword_args + .values() + .any(|a| matches!(a, ShortcodeArg::Shortcode(_))) } impl Default for ShortcodeResolveTransform { @@ -490,35 +607,14 @@ fn shortcode_to_lua_args( }) .collect(); - // Extract top-level metadata as string key-value pairs for Lua. - // - // Forward scalars of every stringifiable kind, not just strings: a handler - // that gates on a boolean/numeric flag (e.g. the `video` shortcode reading - // `auto-stretch: false` to decide reveal stretching — bd-5b21rbaq) needs to - // see it. Booleans/ints are stringified ("false", "16"); string and - // PandocInlines scalars come through `as_plain_text()`. Map/array values - // remain dropped (no flat string form). - let meta_entries: Vec<(String, String)> = if let Some(entries) = metadata.as_map_entries() { - entries - .iter() - .filter_map(|entry| { - let v = &entry.value; - let s = v - .as_bool() - .map(|b| b.to_string()) - .or_else(|| v.as_int().map(|n| n.to_string())) - .or_else(|| v.as_plain_text()); - s.map(|s| (entry.key.clone(), s)) - }) - .collect() - } else { - Vec::new() - }; - + // Forward the full metadata tree; pampa converts it to nested Lua tables + // with native scalar types (so a handler gating on a boolean/numeric flag + // — e.g. `video` reading `auto-stretch: false`, bd-5b21rbaq — sees the + // real value) plus Q1's dotted-string lookup fallback. pampa::lua::ShortcodeArgs { positional, keyword, - metadata: meta_entries, + metadata: metadata.clone(), } } diff --git a/crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/_extensions/argy/_extension.yml b/crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/_extensions/argy/_extension.yml new file mode 100644 index 000000000..72ccb346c --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/_extensions/argy/_extension.yml @@ -0,0 +1,5 @@ +title: Argy +author: Test +contributes: + shortcodes: + - argy.lua diff --git a/crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/_extensions/argy/argy.lua b/crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/_extensions/argy/argy.lua new file mode 100644 index 000000000..f3858cd63 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/_extensions/argy/argy.lua @@ -0,0 +1,9 @@ +return { + argy = function(args, kwargs, meta, raw_args, context) + local a1 = pandoc.utils.stringify(args[1]) + local mode = pandoc.utils.stringify(kwargs['mode']) + local missing = pandoc.utils.stringify(kwargs['nope']) + local mtag = (missing == "") and "EMPTY" or "NONEMPTY" + return "A1=" .. a1 .. ";MODE=" .. mode .. ";MISSING=" .. mtag .. ";NRAW=" .. tostring(#raw_args) + end +} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/test.qmd new file mode 100644 index 000000000..36d534124 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/test.qmd @@ -0,0 +1,12 @@ +--- +title: args, kwargs, raw_args semantics +format: html +_quarto: + tests: + html: + noErrors: true + ensureFileRegexMatches: + - ["A1=hello;MODE=fast;MISSING=EMPTY;NRAW=2"] +--- + +{{< argy hello mode=fast >}} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-meta-dotted/_extensions/metaget/_extension.yml b/crates/quarto/tests/smoke-all/extensions/contract-meta-dotted/_extensions/metaget/_extension.yml new file mode 100644 index 000000000..eaaeca6ab --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-meta-dotted/_extensions/metaget/_extension.yml @@ -0,0 +1,5 @@ +title: Metaget +author: Test +contributes: + shortcodes: + - metaget.lua diff --git a/crates/quarto/tests/smoke-all/extensions/contract-meta-dotted/_extensions/metaget/metaget.lua b/crates/quarto/tests/smoke-all/extensions/contract-meta-dotted/_extensions/metaget/metaget.lua new file mode 100644 index 000000000..25332d089 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-meta-dotted/_extensions/metaget/metaget.lua @@ -0,0 +1,17 @@ +return { + getmeta = function(args, kwargs, meta) + local key = pandoc.utils.stringify(args[1]) + local v = meta[key] + if v == nil then return "META-NIL" end + return "META[" .. pandoc.utils.stringify(v) .. "]" + end, + chainmeta = function(args, kwargs, meta) + local v = meta.custom.nested.value + if v == nil then return "CHAIN-NIL" end + return "CHAIN[" .. tostring(v) .. "]" + end, + boolmeta = function(args, kwargs, meta) + local v = meta.flag + return "BOOL[" .. tostring(v) .. "]" + end +} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-meta-dotted/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-meta-dotted/test.qmd new file mode 100644 index 000000000..6b98e2c25 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-meta-dotted/test.qmd @@ -0,0 +1,30 @@ +--- +title: Meta dotted lookup +format: html +custom: + nested: + value: deep-val +top: top-val +flag: false +authors: + - Ada + - Grace +_quarto: + tests: + html: + noErrors: true + ensureFileRegexMatches: + - ["META\\[top-val\\]", "META\\[deep-val\\]", "META-NIL", "CHAIN\\[deep-val\\]", "BOOL\\[false\\]", "META\\[Grace\\]"] +--- + +Top: {{< getmeta top >}} + +Dotted: {{< getmeta custom.nested.value >}} + +Absent: {{< getmeta no-such-key >}} + +Chained: {{< chainmeta >}} + +Boolean: {{< boolmeta >}} + +Array index: {{< getmeta authors.2 >}} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-nested-arg/_extensions/nester/_extension.yml b/crates/quarto/tests/smoke-all/extensions/contract-nested-arg/_extensions/nester/_extension.yml new file mode 100644 index 000000000..9938c5ba8 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-nested-arg/_extensions/nester/_extension.yml @@ -0,0 +1,5 @@ +title: Nester +author: Test +contributes: + shortcodes: + - nester.lua diff --git a/crates/quarto/tests/smoke-all/extensions/contract-nested-arg/_extensions/nester/nester.lua b/crates/quarto/tests/smoke-all/extensions/contract-nested-arg/_extensions/nester/nester.lua new file mode 100644 index 000000000..5e365f307 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-nested-arg/_extensions/nester/nester.lua @@ -0,0 +1,8 @@ +return { + outer = function(args) + return "OUTER[" .. pandoc.utils.stringify(args[1]) .. "]" + end, + inner = function() + return "IN" + end +} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-nested-arg/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-nested-arg/test.qmd new file mode 100644 index 000000000..dc4f7cef2 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-nested-arg/test.qmd @@ -0,0 +1,12 @@ +--- +title: Nested shortcode as argument +format: html +_quarto: + tests: + html: + noErrors: true + ensureFileRegexMatches: + - ["OUTER\\[IN\\]"] +--- + +{{< outer {{< inner >}} >}} From 5d004a3c9b13bba142df5ae2fb437e2c98035fb6 Mon Sep 17 00:00:00 2001 From: Carlos Scheidegger Date: Fri, 31 Jul 2026 16:55:10 -0500 Subject: [PATCH 3/8] Extension intake: Q1-compat manifests, Q-16 diagnostics, shadowing info (bd-540a976a) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the Phase 1 loud-failure work: - _extension.yml no longer hard-requires title/author (Extension.title/author are now Option, read via as_plain_text). Only contributes remains structurally required. Real Q1 extensions with minimal manifests now load (bd-8b0af414). New smoke fixture q1-compat-minimal-manifest verifies the end-to-end path. - New Q-16 extension subsystem in the error catalog: Q-16-1 Extension Not Loaded, Q-16-2 Shortcode Script Load Failure, Q-16-3 Unknown Shortcode, Q-16-4 Shortcode Handler Shadowed (info). - discover_extensions now returns (extensions, diagnostics); manifest read/parse failures surface as Q-16-1 warnings naming the manifest file and cause, seeded into StageContext diagnostics — no longer a tracing log line followed by a misattributed unknown-shortcode at the use site (bd-nzdm1wry). Verified end-to-end: broken manifest renders with 'Warning [Q-16-1]: Extension not loaded' + Q-16-3 at the invocation. - Backfilled .with_code on shortcode resolution diagnostics (Q-16-2 script load failures, Q-16-3 unknown shortcode / handler not found). - D4 shadowing diagnostic: the Lua engine records cross-script handler-name collisions (same-script reload is not a collision); the resolve transform drains them as Q-16-4 info diagnostics. Info level deliberately does not trip noErrorsOrWarnings or --strict — overriding a built-in is a supported pattern (lipsum-override fixture unchanged). Full workspace suite green (10,813 tests). Plan: claude-notes/plans/2026-07-31-shortcode-extensions-port.md Co-Authored-By: Claude Fable 5 --- .../2026-07-31-shortcode-extensions-port.md | 10 +- crates/pampa/src/lua/mod.rs | 3 +- crates/pampa/src/lua/shortcode.rs | 81 ++++++++++++ crates/quarto-core/src/extension/discover.rs | 115 +++++++++++------- crates/quarto-core/src/extension/read.rs | 77 ++++++------ crates/quarto-core/src/extension/types.rs | 8 +- crates/quarto-core/src/filter_resolve.rs | 4 +- crates/quarto-core/src/stage/context.rs | 4 +- .../src/stage/stages/metadata_merge.rs | 4 +- .../src/transforms/shortcode_resolve.rs | 24 +++- .../quarto-error-catalog/error_catalog.json | 28 +++++ .../.quarto/render-manifest.json | 7 ++ .../.quarto/render-manifest.json | 7 ++ .../_extensions/minimal/_extension.yml | 3 + .../_extensions/minimal/minimal.lua | 5 + .../q1-compat-minimal-manifest/test.qmd | 12 ++ 16 files changed, 299 insertions(+), 93 deletions(-) create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/.quarto/render-manifest.json create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/.quarto/render-manifest.json create mode 100644 crates/quarto/tests/smoke-all/extensions/q1-compat-minimal-manifest/_extensions/minimal/_extension.yml create mode 100644 crates/quarto/tests/smoke-all/extensions/q1-compat-minimal-manifest/_extensions/minimal/minimal.lua create mode 100644 crates/quarto/tests/smoke-all/extensions/q1-compat-minimal-manifest/test.qmd diff --git a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md index c5fd8d5f4..12292cc93 100644 --- a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md +++ b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md @@ -259,7 +259,15 @@ where the study only has static reads. ### Phase 2 — Handler contract parity + `require` (gap rows 4, 5, 7, 13) -- [ ] Corpus rows from Phase 0 for calling convention + coercions green. +- [x] Corpus rows from Phase 0 for calling convention + coercions green + (commit 3585a6d2): raw_args includes kwarg values (documented + approximation: positional-then-keyword order, exact interleaving not in + the AST); meta is now the full metadata tree as nested Lua tables with + native scalars + dotted-path `__index` (both Q1 access styles incl. + `authors.2` array segments); nested shortcode args resolve bottom-up + and arrive stringified; `pandoc.utils.stringify` handles + booleans/numbers (pandoc parity — also fixes video auto-stretch gate + under the new native-boolean meta). - [ ] Sandboxed, script-dir-relative `require` (native + WASM via `SystemRuntime`), scoped to the extension's directory; test with a sibling-module fixture. (This is the highest-risk item for real-world diff --git a/crates/pampa/src/lua/mod.rs b/crates/pampa/src/lua/mod.rs index 9bd14b55e..343a92a93 100644 --- a/crates/pampa/src/lua/mod.rs +++ b/crates/pampa/src/lua/mod.rs @@ -47,5 +47,6 @@ pub use runtime::NativeRuntime; pub use runtime::{RuntimeError, RuntimeResult, SystemRuntime}; #[allow(unused_imports)] pub use shortcode::{ - LuaShortcodeEngine, LuaShortcodeError, LuaShortcodeResult, ShortcodeArgs, ShortcodeCallContext, + LuaShortcodeEngine, LuaShortcodeError, LuaShortcodeResult, ShadowEvent, ShortcodeArgs, + ShortcodeCallContext, }; diff --git a/crates/pampa/src/lua/shortcode.rs b/crates/pampa/src/lua/shortcode.rs index 5925ddf73..fee1cd88e 100644 --- a/crates/pampa/src/lua/shortcode.rs +++ b/crates/pampa/src/lua/shortcode.rs @@ -55,6 +55,15 @@ pub enum LuaShortcodeResult { Error(String), } +/// A same-named shortcode handler was registered by two different scripts; +/// the later registration won. +#[derive(Debug, Clone)] +pub struct ShadowEvent { + pub handler: String, + pub previous_script: String, + pub new_script: String, +} + /// Lua shortcode engine for loading and dispatching handlers. /// /// This is `!Send + !Sync` because it holds a `Lua` state. It must only @@ -63,6 +72,12 @@ pub struct LuaShortcodeEngine { lua: Lua, handlers: HashMap, handler_script_dirs: HashMap, + /// Which script registered each handler (full script path), for + /// shadowing detection. + handler_scripts: HashMap, + /// Recorded handler-name collisions across scripts, drained by the + /// caller for informational diagnostics (Q-16-4). + shadow_events: Vec, runtime: Arc, } @@ -111,6 +126,8 @@ impl LuaShortcodeEngine { lua, handlers: HashMap::new(), handler_script_dirs: HashMap::new(), + handler_scripts: HashMap::new(), + shadow_events: Vec::new(), runtime, }) } @@ -178,6 +195,7 @@ impl LuaShortcodeEngine { .lua .create_registry_value(value) .map_err(LuaShortcodeError::LuaError)?; + self.record_registration(&name, script_path); self.handler_script_dirs .insert(name.clone(), script_dir.clone()); self.handlers.insert(name, key); @@ -200,6 +218,7 @@ impl LuaShortcodeEngine { .lua .create_registry_value(value) .map_err(LuaShortcodeError::LuaError)?; + self.record_registration(&name, script_path); self.handler_script_dirs .insert(name.clone(), script_dir.clone()); self.handlers.insert(name, key); @@ -239,6 +258,29 @@ impl LuaShortcodeEngine { self.handlers.contains_key(name) } + /// Record which script registered `name`; notes a shadow event when a + /// different script had already registered it. Re-registering from the + /// same script (e.g. a script loaded twice) is not shadowing. + fn record_registration(&mut self, name: &str, script_path: &Path) { + let new_script = script_path.to_string_lossy().to_string(); + if let Some(prev) = self + .handler_scripts + .insert(name.to_string(), new_script.clone()) + && prev != new_script + { + self.shadow_events.push(ShadowEvent { + handler: name.to_string(), + previous_script: prev, + new_script, + }); + } + } + + /// Drain recorded handler-shadowing events. + pub fn take_shadow_events(&mut self) -> Vec { + std::mem::take(&mut self.shadow_events) + } + /// Extract diagnostics collected during shortcode execution. pub fn extract_diagnostics( &self, @@ -979,6 +1021,45 @@ return { } } + #[tokio::test] + async fn test_shadow_events_recorded_on_cross_script_override() { + let tmp = TempDir::new().unwrap(); + let first = write_script( + tmp.path(), + "first.lua", + r#"return { dupe = function() return "FIRST" end }"#, + ); + let second = write_script( + tmp.path(), + "second.lua", + r#"return { dupe = function() return "SECOND" end }"#, + ); + + let runtime = make_runtime(); + let mut engine = LuaShortcodeEngine::new("html", runtime).unwrap(); + engine.load_script(&first).await.unwrap(); + // Same script loaded again: not a shadow event. + engine.load_script(&first).await.unwrap(); + assert!(engine.take_shadow_events().is_empty()); + + engine.load_script(&second).await.unwrap(); + let events = engine.take_shadow_events(); + assert_eq!(events.len(), 1); + assert_eq!(events[0].handler, "dupe"); + assert!(events[0].previous_script.ends_with("first.lua")); + assert!(events[0].new_script.ends_with("second.lua")); + + // Later registration wins. + let result = engine + .call("dupe", &make_empty_args(), ShortcodeCallContext::Inline) + .await + .unwrap(); + match result { + LuaShortcodeResult::Text(s) => assert_eq!(s, "SECOND"), + other => panic!("Expected Text, got {:?}", other), + } + } + #[tokio::test] async fn test_raw_args_includes_keyword_values() { // TS Quarto compat: raw_args carries positional AND keyword values diff --git a/crates/quarto-core/src/extension/discover.rs b/crates/quarto-core/src/extension/discover.rs index 2f0939a6e..008b63a8c 100644 --- a/crates/quarto-core/src/extension/discover.rs +++ b/crates/quarto-core/src/extension/discover.rs @@ -10,10 +10,10 @@ use std::path::Path; use quarto_system_runtime::{PathKind, SystemRuntime}; -use tracing::warn; use super::read::{read_extension, read_extension_with_org}; use super::types::Extension; +use quarto_error_reporting::{DiagnosticMessage, DiagnosticMessageBuilder}; /// Discover all extensions available for a document. /// @@ -29,8 +29,9 @@ pub fn discover_extensions( project_dir: Option<&Path>, builtin_extensions_dir: Option<&Path>, runtime: &dyn SystemRuntime, -) -> Vec { +) -> (Vec, Vec) { let mut extensions = Vec::new(); + let mut diagnostics = Vec::new(); let mut dirs_to_search = Vec::new(); // Built-in extensions first (lowest priority) @@ -39,7 +40,7 @@ pub fn discover_extensions( .path_exists(builtin_dir, Some(PathKind::Directory)) .unwrap_or(false) { - scan_extensions_dir(builtin_dir, runtime, &mut extensions); + scan_extensions_dir(builtin_dir, runtime, &mut extensions, &mut diagnostics); } let start_dir = input.parent().unwrap_or(input); @@ -72,10 +73,10 @@ pub fn discover_extensions( continue; } - scan_extensions_dir(ext_dir, runtime, &mut extensions); + scan_extensions_dir(ext_dir, runtime, &mut extensions, &mut diagnostics); } - extensions + (extensions, diagnostics) } /// Scan all entries in an extensions directory. @@ -83,6 +84,7 @@ fn scan_extensions_dir( ext_dir: &Path, runtime: &dyn SystemRuntime, extensions: &mut Vec, + diagnostics: &mut Vec, ) { let entries = match runtime.dir_list(ext_dir) { Ok(entries) => entries, @@ -90,7 +92,7 @@ fn scan_extensions_dir( }; for entry in entries { - scan_extension_entry(&entry, runtime, extensions); + scan_extension_entry(&entry, runtime, extensions, diagnostics); } } @@ -102,6 +104,7 @@ fn scan_extension_entry( entry: &Path, runtime: &dyn SystemRuntime, extensions: &mut Vec, + diagnostics: &mut Vec, ) { let ext_file = entry.join("_extension.yml"); @@ -112,7 +115,7 @@ fn scan_extension_entry( { match read_extension(&ext_file, runtime) { Ok(ext) => extensions.push(ext), - Err(e) => warn!("Failed to read extension {}: {}", ext_file.display(), e), + Err(e) => diagnostics.push(extension_not_loaded_diagnostic(&ext_file, &e)), } return; } @@ -133,12 +136,34 @@ fn scan_extension_entry( { match read_extension_with_org(&sub_ext_file, org_name.as_deref(), runtime) { Ok(ext) => extensions.push(ext), - Err(e) => warn!("Failed to read extension {}: {}", sub_ext_file.display(), e), + Err(e) => diagnostics.push(extension_not_loaded_diagnostic(&sub_ext_file, &e)), } } } } +/// Build the Q-16-1 diagnostic for a manifest that could not be loaded. +/// +/// Surfacing this at discovery time (instead of a bare log line) is what +/// prevents the failure from being misattributed later as an unknown +/// shortcode in the user's document (bd-nzdm1wry). +fn extension_not_loaded_diagnostic( + ext_file: &Path, + err: &crate::error::QuartoError, +) -> DiagnosticMessage { + DiagnosticMessageBuilder::warning("Extension not loaded") + .with_code("Q-16-1") + .problem(format!( + "The extension manifest `{}` could not be loaded: {}", + ext_file.display(), + err + )) + .add_hint( + "Shortcodes, filters, and formats contributed by this extension will be unavailable.", + ) + .build() +} + /// Find a specific extension by name among discovered extensions. /// /// Returns the **last** match so that user extensions (appended after @@ -236,7 +261,7 @@ contributes: let runtime = make_runtime(); let input = tmp.path().join("test.qmd"); - let extensions = discover_extensions(&input, None, None, &runtime); + let (extensions, _diags) = discover_extensions(&input, None, None, &runtime); assert_eq!(extensions.len(), 1); assert_eq!(extensions[0].id.name, "test-ext"); @@ -259,7 +284,7 @@ contributes: let runtime = make_runtime(); let input = tmp.path().join("test.qmd"); - let extensions = discover_extensions(&input, None, None, &runtime); + let (extensions, _diags) = discover_extensions(&input, None, None, &runtime); assert_eq!(extensions.len(), 1); assert_eq!(extensions[0].id.name, "ext"); @@ -301,7 +326,7 @@ contributes: let runtime = make_runtime(); let input = sub_dir.join("test.qmd"); - let extensions = discover_extensions(&input, Some(project_dir), None, &runtime); + let (extensions, _diags) = discover_extensions(&input, Some(project_dir), None, &runtime); assert_eq!(extensions.len(), 2); // Project-level should come first (lower priority) @@ -316,7 +341,7 @@ contributes: let runtime = make_runtime(); let input = tmp.path().join("test.qmd"); - let extensions = discover_extensions(&input, None, None, &runtime); + let (extensions, _diags) = discover_extensions(&input, None, None, &runtime); assert!(extensions.is_empty()); } @@ -327,7 +352,7 @@ contributes: let runtime = make_runtime(); let input = tmp.path().join("test.qmd"); - let extensions = discover_extensions(&input, None, None, &runtime); + let (extensions, _diags) = discover_extensions(&input, None, None, &runtime); assert!(extensions.is_empty()); } @@ -348,24 +373,26 @@ contributes: "#, ); - // Invalid extension (missing title) + // Invalid extension: unparseable YAML. (Missing title/author is NOT + // invalid — Q1-compat intake, bd-8b0af414.) write_extension( &tmp.path().join("_extensions/bad-ext"), - r#" -author: Author -contributes: - shortcodes: - - hello.lua -"#, + "contributes: [unclosed\n nonsense: {{{{", ); let runtime = make_runtime(); let input = tmp.path().join("test.qmd"); - let extensions = discover_extensions(&input, None, None, &runtime); + let (extensions, diags) = discover_extensions(&input, None, None, &runtime); - // Only the valid extension should be discovered + // Only the valid extension should be discovered, and the broken one + // must surface as a Q-16-1 diagnostic naming its manifest file + // (bd-nzdm1wry), not vanish into a log line. assert_eq!(extensions.len(), 1); assert_eq!(extensions[0].id.name, "good-ext"); + assert_eq!(diags.len(), 1); + let rendered = format!("{:?}", diags[0]); + assert!(rendered.contains("Q-16-1"), "diagnostic: {rendered}"); + assert!(rendered.contains("bad-ext"), "diagnostic: {rendered}"); } // === find_extension tests === @@ -374,8 +401,8 @@ contributes: fn test_find_extension_by_name() { let ext = Extension { id: super::super::types::ExtensionId::new("lightbox"), - title: "Lightbox".to_string(), - author: "Author".to_string(), + title: Some("Lightbox".to_string()), + author: Some("Author".to_string()), version: None, quarto_required: None, path: PathBuf::from("/ext"), @@ -391,8 +418,8 @@ contributes: fn test_find_extension_by_org_name() { let ext = Extension { id: super::super::types::ExtensionId::with_organization("acm", "quarto-journals"), - title: "ACM".to_string(), - author: "Author".to_string(), + title: Some("ACM".to_string()), + author: Some("Author".to_string()), version: None, quarto_required: None, path: PathBuf::from("/ext"), @@ -410,8 +437,8 @@ contributes: // Built-in (first in vec) should be overridden by user (last in vec) let builtin = Extension { id: super::super::types::ExtensionId::with_organization("lipsum", "quarto"), - title: "Lipsum Built-in".to_string(), - author: "Built-in Author".to_string(), + title: Some("Lipsum Built-in".to_string()), + author: Some("Built-in Author".to_string()), version: None, quarto_required: None, path: PathBuf::from("/builtin/quarto/lipsum"), @@ -419,8 +446,8 @@ contributes: }; let user = Extension { id: super::super::types::ExtensionId::new("lipsum"), - title: "Lipsum User".to_string(), - author: "User Author".to_string(), + title: Some("Lipsum User".to_string()), + author: Some("User Author".to_string()), version: None, quarto_required: None, path: PathBuf::from("/user/lipsum"), @@ -430,15 +457,15 @@ contributes: // Name-only lookup: should find user (last match) let found = find_extension("lipsum", &extensions).unwrap(); - assert_eq!(found.title, "Lipsum User"); + assert_eq!(found.title.as_deref(), Some("Lipsum User")); } #[test] fn test_find_extension_org_name_returns_last_match() { let builtin = Extension { id: super::super::types::ExtensionId::with_organization("lipsum", "quarto"), - title: "Lipsum Built-in".to_string(), - author: "Built-in Author".to_string(), + title: Some("Lipsum Built-in".to_string()), + author: Some("Built-in Author".to_string()), version: None, quarto_required: None, path: PathBuf::from("/builtin/quarto/lipsum"), @@ -446,8 +473,8 @@ contributes: }; let user = Extension { id: super::super::types::ExtensionId::with_organization("lipsum", "quarto"), - title: "Lipsum User Override".to_string(), - author: "User Author".to_string(), + title: Some("Lipsum User Override".to_string()), + author: Some("User Author".to_string()), version: None, quarto_required: None, path: PathBuf::from("/user/quarto/lipsum"), @@ -457,7 +484,7 @@ contributes: // Org/name lookup: should find user (last match) let found = find_extension("quarto/lipsum", &extensions).unwrap(); - assert_eq!(found.title, "Lipsum User Override"); + assert_eq!(found.title.as_deref(), Some("Lipsum User Override")); } // === Built-in extension discovery tests === @@ -483,7 +510,7 @@ contributes: fs::create_dir_all(&input_dir).unwrap(); let input = input_dir.join("test.qmd"); - let extensions = discover_extensions(&input, None, Some(&builtin_dir), &runtime); + let (extensions, _diags) = discover_extensions(&input, None, Some(&builtin_dir), &runtime); assert_eq!(extensions.len(), 1); assert_eq!(extensions[0].id.name, "lipsum"); @@ -523,17 +550,17 @@ contributes: let runtime = make_runtime(); let input = project_dir.join("test.qmd"); - let extensions = discover_extensions(&input, None, Some(&builtin_dir), &runtime); + let (extensions, _diags) = discover_extensions(&input, None, Some(&builtin_dir), &runtime); // Both should be discovered assert_eq!(extensions.len(), 2); // Built-in first, user second - assert_eq!(extensions[0].title, "Lipsum Built-in"); - assert_eq!(extensions[1].title, "Lipsum User"); + assert_eq!(extensions[0].title.as_deref(), Some("Lipsum Built-in")); + assert_eq!(extensions[1].title.as_deref(), Some("Lipsum User")); // find_extension should return user (last match) let found = find_extension("lipsum", &extensions).unwrap(); - assert_eq!(found.title, "Lipsum User"); + assert_eq!(found.title.as_deref(), Some("Lipsum User")); } #[test] @@ -569,17 +596,17 @@ contributes: let runtime = make_runtime(); let input = project_dir.join("test.qmd"); - let extensions = discover_extensions(&input, None, Some(&builtin_dir), &runtime); + let (extensions, _diags) = discover_extensions(&input, None, Some(&builtin_dir), &runtime); assert_eq!(extensions.len(), 2); // find_extension with org/name should return user (last match) let found = find_extension("quarto/lipsum", &extensions).unwrap(); - assert_eq!(found.title, "Lipsum User Org"); + assert_eq!(found.title.as_deref(), Some("Lipsum User Org")); // find_extension with bare name should also return user let found = find_extension("lipsum", &extensions).unwrap(); - assert_eq!(found.title, "Lipsum User Org"); + assert_eq!(found.title.as_deref(), Some("Lipsum User Org")); } // === Format descriptor tests === diff --git a/crates/quarto-core/src/extension/read.rs b/crates/quarto-core/src/extension/read.rs index 0e19bb6ff..598cb6f30 100644 --- a/crates/quarto-core/src/extension/read.rs +++ b/crates/quarto-core/src/extension/read.rs @@ -75,37 +75,16 @@ pub fn read_extension_with_org( &mut diagnostics, ); - // Extract required fields - let title = config - .get("title") - .and_then(|v| v.as_str()) - .ok_or_else(|| { - crate::error::QuartoError::Other(format!( - "{}: missing required 'title' field", - extension_file.display() - )) - })? - .to_string(); - - let author = config - .get("author") - .and_then(|v| v.as_str()) - .ok_or_else(|| { - crate::error::QuartoError::Other(format!( - "{}: missing required 'author' field", - extension_file.display() - )) - })? - .to_string(); - - let version = config - .get("version") - .and_then(|v| v.as_str()) - .map(|s| s.to_string()); + // Optional metadata fields (Q1-compat: no named field is required — + // bd-8b0af414). `as_plain_text` rather than `as_str` so values that + // parse as PandocInlines still come through. + let title = config.get("title").and_then(|v| v.as_plain_text()); + let author = config.get("author").and_then(|v| v.as_plain_text()); + + let version = config.get("version").and_then(|v| v.as_plain_text()); let quarto_required = config .get("quarto-required") - .and_then(|v| v.as_str()) - .map(|s| s.to_string()); + .and_then(|v| v.as_plain_text()); // Extract contributes let contributes_cv = config.get("contributes").ok_or_else(|| { @@ -372,6 +351,31 @@ mod tests { file } + #[test] + fn test_read_q1_compat_manifest_without_title_author() { + // Q1 requires no named fields in _extension.yml; real extensions + // (julia-engine, marimo) omit title/author. Only `contributes` is + // structurally required (bd-8b0af414). + let tmp = TempDir::new().unwrap(); + let ext_dir = tmp.path().join("_extensions/bare-ext"); + let file = write_extension( + &ext_dir, + r#" +contributes: + shortcodes: + - bare.lua +"#, + ); + + let runtime = make_runtime(); + let ext = read_extension(&file, &runtime).unwrap(); + + assert_eq!(ext.id.name, "bare-ext"); + assert_eq!(ext.title, None); + assert_eq!(ext.author, None); + assert_eq!(ext.contributes.shortcodes.len(), 1); + } + #[test] fn test_read_minimal_extension() { let tmp = TempDir::new().unwrap(); @@ -392,8 +396,8 @@ contributes: assert_eq!(ext.id.name, "test-ext"); assert_eq!(ext.id.organization, None); - assert_eq!(ext.title, "Test Extension"); - assert_eq!(ext.author, "Test Author"); + assert_eq!(ext.title.as_deref(), Some("Test Extension")); + assert_eq!(ext.author.as_deref(), Some("Test Author")); assert!(ext.version.is_none()); assert_eq!(ext.contributes.shortcodes.len(), 1); assert_eq!(ext.contributes.shortcodes[0], ext_dir.join("hello.lua")); @@ -505,6 +509,8 @@ contributes: #[test] fn test_read_extension_missing_title() { + // Q1-compat intake (bd-8b0af414): missing title is NOT an error; + // it just loads with title: None. let tmp = TempDir::new().unwrap(); let ext_dir = tmp.path().join("_extensions/test-ext"); let file = write_extension( @@ -518,12 +524,9 @@ contributes: ); let runtime = make_runtime(); - let err = read_extension(&file, &runtime).unwrap_err(); - assert!( - err.to_string().contains("title"), - "Error should mention 'title': {}", - err - ); + let ext = read_extension(&file, &runtime).unwrap(); + assert_eq!(ext.title, None); + assert_eq!(ext.author.as_deref(), Some("Author")); } #[test] diff --git a/crates/quarto-core/src/extension/types.rs b/crates/quarto-core/src/extension/types.rs index 11bfc4c68..3d877afa4 100644 --- a/crates/quarto-core/src/extension/types.rs +++ b/crates/quarto-core/src/extension/types.rs @@ -51,11 +51,15 @@ impl fmt::Display for ExtensionId { } /// A parsed and resolved Quarto extension. +/// +/// Q1-compat intake: only `contributes` is structurally required in +/// `_extension.yml`; `title`/`author` are optional metadata (real Q1 +/// extensions omit them — bd-8b0af414). #[derive(Debug, Clone)] pub struct Extension { pub id: ExtensionId, - pub title: String, - pub author: String, + pub title: Option, + pub author: Option, pub version: Option, pub quarto_required: Option, /// Absolute path to the extension directory. diff --git a/crates/quarto-core/src/filter_resolve.rs b/crates/quarto-core/src/filter_resolve.rs index 361304c69..5323cf2e7 100644 --- a/crates/quarto-core/src/filter_resolve.rs +++ b/crates/quarto-core/src/filter_resolve.rs @@ -480,8 +480,8 @@ mod tests { fn make_extension(name: &str, filters: Vec) -> Extension { Extension { id: ExtensionId::new(name), - title: name.to_string(), - author: "Test".to_string(), + title: Some(name.to_string()), + author: Some("Test".to_string()), version: None, quarto_required: None, path: PathBuf::from(format!("/project/_extensions/{}", name)), diff --git a/crates/quarto-core/src/stage/context.rs b/crates/quarto-core/src/stage/context.rs index b291b10f2..a39a90cf0 100644 --- a/crates/quarto-core/src/stage/context.rs +++ b/crates/quarto-core/src/stage/context.rs @@ -225,7 +225,7 @@ impl StageContext { document: DocumentInfo, ) -> Result { let builtin_ext_path = builtin_extensions_path(runtime.as_ref()); - let extensions = crate::extension::discover_extensions( + let (extensions, extension_diagnostics) = crate::extension::discover_extensions( &document.input, if project.is_single_file { None @@ -245,7 +245,7 @@ impl StageContext { extensions, artifacts: ArtifactStore::new(), includes: PandocIncludes::default(), - diagnostics: Vec::new(), + diagnostics: extension_diagnostics, ref_type_registry: None, crossref_index: None, resource_report: crate::project_resources::DocumentResourceReport::new(), diff --git a/crates/quarto-core/src/stage/stages/metadata_merge.rs b/crates/quarto-core/src/stage/stages/metadata_merge.rs index 477cf30ea..3bba54fa2 100644 --- a/crates/quarto-core/src/stage/stages/metadata_merge.rs +++ b/crates/quarto-core/src/stage/stages/metadata_merge.rs @@ -1622,8 +1622,8 @@ mod tests { ) -> Extension { Extension { id: ExtensionId::new(name), - title: name.to_string(), - author: "Test".to_string(), + title: Some(name.to_string()), + author: Some("Test".to_string()), version: None, quarto_required: None, path: PathBuf::from("/extensions").join(name), diff --git a/crates/quarto-core/src/transforms/shortcode_resolve.rs b/crates/quarto-core/src/transforms/shortcode_resolve.rs index 68750ee02..643cbe804 100644 --- a/crates/quarto-core/src/transforms/shortcode_resolve.rs +++ b/crates/quarto-core/src/transforms/shortcode_resolve.rs @@ -409,6 +409,7 @@ impl ShortcodeResolveTransform { // named as the cause, and keep loading the rest. diagnostics.push( DiagnosticMessageBuilder::warning("Shortcode script error") + .with_code("Q-16-2") .problem(format!( "Failed to load shortcode script `{}` from extension `{}`: {}", script_path.display(), @@ -430,6 +431,7 @@ impl ShortcodeResolveTransform { // Unknown shortcode - create error with diagnostic let diagnostic = DiagnosticMessageBuilder::warning("Unknown shortcode") + .with_code("Q-16-3") .problem(format!("Shortcode `{}` is not recognized", shortcode.name)) .add_hint("Check the shortcode name for typos") .with_location(ctx.source_info.clone()) @@ -566,6 +568,7 @@ async fn dispatch_lua_shortcode( Some(result) => lua_result_to_shortcode_result(result, ctx.source_info), None => { let diagnostic = DiagnosticMessageBuilder::warning("Shortcode handler not found") + .with_code("Q-16-3") .problem(format!( "Lua handler for shortcode `{}` was not found", shortcode.name @@ -980,6 +983,7 @@ impl AstTransform for ShortcodeResolveTransform { if let Err(e) = engine.load_script(path).await { diagnostics.push( DiagnosticMessageBuilder::warning("Shortcode script error") + .with_code("Q-16-2") .problem(format!( "Failed to load shortcode script `{}`: {}", path.display(), @@ -1060,6 +1064,22 @@ impl AstTransform for ShortcodeResolveTransform { .build(), ), } + + // Surface handler-name shadowing as informational diagnostics + // (D4). Info level on purpose: overriding a built-in extension + // is a supported pattern, so this must not trip + // warnings-as-errors or the smoke suite's default assertion. + for event in engine.take_shadow_events() { + diagnostics.push( + DiagnosticMessageBuilder::info("Shortcode handler shadowed") + .with_code("Q-16-4") + .problem(format!( + "Shortcode handler `{}` from `{}` is overridden by `{}` (later registration wins)", + event.handler, event.previous_script, event.new_script + )) + .build(), + ); + } } // Add any diagnostics to the render context @@ -2187,8 +2207,8 @@ mod tests { fn make_extension(name: &str, shortcode_paths: Vec) -> Extension { Extension { id: ExtensionId::new(name), - title: name.to_string(), - author: String::new(), + title: Some(name.to_string()), + author: None, version: None, quarto_required: None, path: PathBuf::from("/extensions").join(name), diff --git a/crates/quarto-error-catalog/error_catalog.json b/crates/quarto-error-catalog/error_catalog.json index a43be4925..c0d804ad6 100644 --- a/crates/quarto-error-catalog/error_catalog.json +++ b/crates/quarto-error-catalog/error_catalog.json @@ -1062,5 +1062,33 @@ "message_template": "A crossref identifier (e.g. a `fig-`/`tbl-` label or an explicit `#id`) is defined more than once in the same document. Each crossref id must be unique within a document, so a reference like `@fig-foo` resolves to exactly one target. Give one of the targets a different label.", "docs_url": "https://quarto.org/docs/errors/crossref/Q-15-1", "since_version": "99.9.9" + }, + "Q-16-1": { + "subsystem": "extension", + "title": "Extension Not Loaded", + "message_template": "An `_extension.yml` manifest could not be read or parsed, so the extension it describes is not active for this render. Any shortcodes, filters, or formats it contributes will be missing (a shortcode from it will report Q-16-3 Unknown Shortcode at its use site). The message names the manifest file and the underlying cause; fix the manifest to restore the extension.", + "docs_url": "https://quarto.org/docs/errors/extension/Q-16-1", + "since_version": "99.9.9" + }, + "Q-16-2": { + "subsystem": "extension", + "title": "Shortcode Script Load Failure", + "message_template": "A Lua shortcode script (from an extension's `contributes.shortcodes` or the document's `shortcodes:` list) failed to load or execute. Handlers defined by that script are unavailable; shortcodes relying on them will report Q-16-3 Unknown Shortcode. The message names the script file (and extension, when applicable) and the Lua error.", + "docs_url": "https://quarto.org/docs/errors/extension/Q-16-2", + "since_version": "99.9.9" + }, + "Q-16-3": { + "subsystem": "extension", + "title": "Unknown Shortcode", + "message_template": "A shortcode invocation does not match any built-in handler, extension-contributed handler, or document-level `shortcodes:` script. The invocation renders as a visible `?name` marker. Check the shortcode name for typos, and check earlier Q-16-1/Q-16-2 warnings — a failed extension or script load surfaces here at the use site.", + "docs_url": "https://quarto.org/docs/errors/extension/Q-16-3", + "since_version": "99.9.9" + }, + "Q-16-4": { + "subsystem": "extension", + "title": "Shortcode Handler Shadowed", + "message_template": "Two loaded shortcode scripts define a handler with the same name; the later registration wins (document `shortcodes:` scripts load first, then extensions in discovery order, so the more local definition takes precedence). This is informational — overriding a built-in extension is a supported pattern — but if the shadowing is unintentional, rename or remove one of the handlers.", + "docs_url": "https://quarto.org/docs/errors/extension/Q-16-4", + "since_version": "99.9.9" } } diff --git a/crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/.quarto/render-manifest.json b/crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/.quarto/render-manifest.json new file mode 100644 index 000000000..70308abaf --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-args-kwargs/.quarto/render-manifest.json @@ -0,0 +1,7 @@ +{ + "version": 1, + "rendered_files": [ + "test.html" + ], + "resources": [] +} \ No newline at end of file diff --git a/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/.quarto/render-manifest.json b/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/.quarto/render-manifest.json new file mode 100644 index 000000000..70308abaf --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/.quarto/render-manifest.json @@ -0,0 +1,7 @@ +{ + "version": 1, + "rendered_files": [ + "test.html" + ], + "resources": [] +} \ No newline at end of file diff --git a/crates/quarto/tests/smoke-all/extensions/q1-compat-minimal-manifest/_extensions/minimal/_extension.yml b/crates/quarto/tests/smoke-all/extensions/q1-compat-minimal-manifest/_extensions/minimal/_extension.yml new file mode 100644 index 000000000..61595d31d --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/q1-compat-minimal-manifest/_extensions/minimal/_extension.yml @@ -0,0 +1,3 @@ +contributes: + shortcodes: + - minimal.lua diff --git a/crates/quarto/tests/smoke-all/extensions/q1-compat-minimal-manifest/_extensions/minimal/minimal.lua b/crates/quarto/tests/smoke-all/extensions/q1-compat-minimal-manifest/_extensions/minimal/minimal.lua new file mode 100644 index 000000000..3adf180c5 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/q1-compat-minimal-manifest/_extensions/minimal/minimal.lua @@ -0,0 +1,5 @@ +return { + minimal = function() + return "MINIMAL-MANIFEST-OK" + end +} diff --git a/crates/quarto/tests/smoke-all/extensions/q1-compat-minimal-manifest/test.qmd b/crates/quarto/tests/smoke-all/extensions/q1-compat-minimal-manifest/test.qmd new file mode 100644 index 000000000..7f1b00ba0 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/q1-compat-minimal-manifest/test.qmd @@ -0,0 +1,12 @@ +--- +title: Q1-compat manifest without title/author loads +format: html +_quarto: + tests: + html: + noErrors: true + ensureFileRegexMatches: + - ["MINIMAL-MANIFEST-OK"] +--- + +{{< minimal >}} From 7c3b2f103bb074f2b62fc52ba2a1408cb0e23a0f Mon Sep 17 00:00:00 2001 From: Carlos Scheidegger Date: Fri, 31 Jul 2026 17:06:57 -0500 Subject: [PATCH 4/8] Shortcode Lua engine: scoped require, error_output table args, no process::exit (bd-540a976a) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes Phase 2 of the shortcode-extensions port: - Script-dir-aware require (register_scoped_require): resolves module names against the script-dir stack top-down — sibling-relative from the requiring module, then extension-root-relative (Q1's require('modules/brand/brand') convention). Sandboxed module env, cached by resolved path (nil result caches as true, like Lua), nested requires supported, falls back to the native stdlib require when present (WASM has none). All file access via SystemRuntime so native and WASM share the code path. New contract-require fixture + pampa unit test. - quarto.shortcode.error_output now accepts a table as message_or_args (values concatenated with spaces), matching Q1's contract; Q1's own fixture calls error_output('shorty', args, 'inline'). The error_args case is now ported into contract-doc-shortcodes. - shortcode_to_span no longer kills the process on nested shortcodes in keyword args: they encode Q1-style (param span with data-key, no data-value, nested shortcode span as content); the impossible KeyValue-in-kwargs case degrades visibly instead of exiting. Full workspace suite green (10,817 tests). Plan: claude-notes/plans/2026-07-31-shortcode-extensions-port.md Co-Authored-By: Claude Fable 5 --- .../2026-07-31-shortcode-extensions-port.md | 23 ++-- crates/pampa/src/lua/quarto_api.rs | 121 ++++++++++++++++++ crates/pampa/src/lua/shortcode.rs | 90 ++++++++++++- crates/pampa/src/pandoc/shortcode.rs | 78 +++++++++-- .../contract-doc-shortcodes/shorty.lua | 4 +- .../contract-doc-shortcodes/test.qmd | 4 +- .../_extensions/requirer/_extension.yml | 3 + .../_extensions/requirer/modules/helper.lua | 1 + .../_extensions/requirer/requirer.lua | 6 + .../extensions/contract-require/test.qmd | 12 ++ 10 files changed, 321 insertions(+), 21 deletions(-) create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-require/_extensions/requirer/_extension.yml create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-require/_extensions/requirer/modules/helper.lua create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-require/_extensions/requirer/requirer.lua create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-require/test.qmd diff --git a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md index 12292cc93..0a340da1e 100644 --- a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md +++ b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md @@ -247,15 +247,20 @@ where the study only has static reads. (doc `shortcodes:` < extensions in discovery order < Rust built-ins); per-script load failures warn naming extension id + script path. Verified end-to-end on connect-docs (`tier` renders, 0 warnings). -- [ ] Tests: malformed `_extension.yml` (bad YAML, bad semver, empty - contributes) → coded, source-mapped diagnostics; minimal Q1 manifest - (no title/author) loads. -- [ ] Relax `read.rs` required fields per D3; add `Q-*` codes for manifest - errors (extension subsystem: decide `Q-5-*` vs new subsystem number). -- [ ] `discover_extensions` returns structured failures; `dispatch_shortcode`'s - unknown-name fallthrough distinguishes "no such extension" from - "extension found but failed to load" (closes bd-nzdm1wry). -- [ ] Shadowing diagnostic per D4 (closes the silent-`rfind` gap). +- [x] Tests: malformed `_extension.yml` → Q-16-1 diagnostic naming file + + cause; minimal Q1 manifest (no title/author) loads + (`q1-compat-minimal-manifest` fixture + unit tests). Commit 5d004a3c. + Note: bad-semver validation deliberately not added (don't gold-plate; + Q2 doesn't enforce quarto-required yet — follow-on if needed). +- [x] `read.rs` relaxed per D3; new `Q-16` extension subsystem (Q-16-1 + manifest, Q-16-2 script load, Q-16-3 unknown shortcode, Q-16-4 + shadowed handler). +- [x] `discover_extensions` returns `(Vec, Vec)`; + failures surface at discovery (Q-16-1) and the use site keeps its own + Q-16-3 — verified end-to-end via `q2 render` (bd-nzdm1wry). +- [x] Shadowing diagnostic per D4: engine records cross-script collisions, + drained as Q-16-4 *info* (does not trip noErrorsOrWarnings/--strict; + built-in override stays a supported pattern). Commit 5d004a3c. ### Phase 2 — Handler contract parity + `require` (gap rows 4, 5, 7, 13) diff --git a/crates/pampa/src/lua/quarto_api.rs b/crates/pampa/src/lua/quarto_api.rs index b02082efa..35286c703 100644 --- a/crates/pampa/src/lua/quarto_api.rs +++ b/crates/pampa/src/lua/quarto_api.rs @@ -184,6 +184,127 @@ pub fn init_script_dir_stack(lua: &Lua) -> Result<()> { Ok(()) } +/// Replace the global `require` with a script-dir-aware loader. +/// +/// Q1 extensions `require` sibling modules relative to the requiring +/// script's directory (TS Quarto patches `require` through its script-file +/// stack, `init.lua:259-305`). This loader: +/// +/// 1. resolves `name` against the top of the script-dir stack, trying +/// `/.lua`, `/ />.lua`, and +/// `//init.lua`; +/// 2. executes the module in a sandboxed environment (globals-inheriting, +/// like shortcode scripts) with the module's own directory pushed on the +/// script-dir stack, so nested `require`s resolve relative to the module; +/// 3. caches by resolved absolute path (a module evaluating to `nil` caches +/// as `true`, matching Lua's `require`); +/// 4. falls back to the original `require` (when the target has one — the +/// WASM stdlib has no `package` lib) for anything not found on disk. +/// +/// File access goes through [`SystemRuntime`], so the same code serves +/// native and WASM (VFS) targets. +pub fn register_scoped_require( + lua: &Lua, + runtime: Arc, +) -> Result<()> { + let globals = lua.globals(); + let original: Option = globals.get("require").ok(); + globals.set("_quarto_require_cache", lua.create_table()?)?; + + let require = lua.create_function(move |lua, name: String| { + let cache: Table = lua.globals().get("_quarto_require_cache")?; + + // Candidate paths, walking the script-dir stack top-down: the + // currently-executing module's dir first (sibling-relative + // requires), then the dirs of the scripts that required it + // (Q1-style extension-root-relative paths like + // `require("modules/brand/brand")` from a nested module). + let stack: Table = lua.globals().get("_quarto_script_dir_stack")?; + let mut dirs: Vec = Vec::new(); + for i in (1..=stack.raw_len()).rev() { + if let Ok(d) = stack.get::(i) + && !d.is_empty() + && !dirs.contains(&d) + { + dirs.push(d); + } + } + let mut candidates: Vec = Vec::new(); + for dir in &dirs { + let base = PathBuf::from(dir); + candidates.push(base.join(format!("{}.lua", name))); + if name.contains('.') { + candidates.push(base.join(format!("{}.lua", name.replace('.', "/")))); + } + candidates.push(base.join(&name).join("init.lua")); + } + + for candidate in &candidates { + let key = candidate.to_string_lossy().to_string(); + if let Ok(cached) = cache.get::(key.as_str()) + && !matches!(cached, Value::Nil) + { + return Ok(cached); + } + let Ok(bytes) = runtime.file_read(candidate) else { + continue; + }; + let source = String::from_utf8(bytes).map_err(mlua::Error::external)?; + + // Sandboxed module environment inheriting globals. + let env = lua.create_table()?; + let env_mt = lua.create_table()?; + env_mt.set("__index", lua.globals())?; + env.set_metatable(Some(env_mt))?; + + let module_dir = candidate + .parent() + .unwrap_or(Path::new("")) + .to_string_lossy() + .to_string(); + push_script_dir(lua, &module_dir)?; + let result = lua + .load(&source) + .set_name(key.clone()) + .set_environment(env) + .eval::(); + pop_script_dir(lua)?; + + let value = result?; + // A module that returns nothing caches as `true`, like Lua. + let value = if matches!(value, Value::Nil) { + Value::Boolean(true) + } else { + value + }; + cache.set(key.as_str(), value.clone())?; + return Ok(value); + } + + // Not found relative to the script dir: defer to the original + // require (native stdlib) when available. + if let Some(orig) = &original { + return orig.call::(name); + } + Err(mlua::Error::RuntimeError(format!( + "module '{}' not found (searched relative to the current script directory: {})", + name, + if candidates.is_empty() { + "no script directory on the stack".to_string() + } else { + candidates + .iter() + .map(|c| c.display().to_string()) + .collect::>() + .join(", ") + } + ))) + })?; + + globals.set("require", require)?; + Ok(()) +} + /// Push a directory onto the script-dir stack. pub fn push_script_dir(lua: &Lua, dir: &str) -> Result<()> { let stack: Table = lua.globals().get("_quarto_script_dir_stack")?; diff --git a/crates/pampa/src/lua/shortcode.rs b/crates/pampa/src/lua/shortcode.rs index fee1cd88e..ba1ab8712 100644 --- a/crates/pampa/src/lua/shortcode.rs +++ b/crates/pampa/src/lua/shortcode.rs @@ -112,6 +112,11 @@ impl LuaShortcodeEngine { // Register quarto.json, quarto.log, quarto.utils register_quarto_api(&lua).map_err(LuaShortcodeError::LuaError)?; + // Script-dir-aware `require` so extension scripts can load sibling + // modules (Q1 parity). + super::quarto_api::register_scoped_require(&lua, runtime.clone()) + .map_err(LuaShortcodeError::LuaError)?; + // Register quarto.doc namespace (is_format, add_html_dependency, etc.) super::quarto_doc::register_quarto_doc(&lua).map_err(LuaShortcodeError::LuaError)?; @@ -545,11 +550,35 @@ fn register_shortcode_api(lua: &Lua) -> Result<()> { .eval::()?, )?; - // quarto.shortcode.error_output(name, message, context) + // quarto.shortcode.error_output(name, message_or_args, context) + // + // TS Quarto compat: the second parameter may be a plain message string + // OR a table of argument values (concatenated with spaces) — Q1's own + // test fixture calls `error_output("shorty", args, "inline")`. shortcode_ns.set( "error_output", lua.create_function( - |lua, (name, message, context): (String, String, String)| -> Result { + |lua, (name, message_or_args, context): (String, Value, String)| -> Result { + let message = match &message_or_args { + Value::Table(t) => { + let mut parts: Vec = Vec::new(); + for item in t.clone().sequence_values::() { + let item = item?; + parts.push(match item { + Value::String(s) => s.to_str()?.to_string(), + other => lua + .globals() + .get::("pandoc")? + .get::
("utils")? + .get::("stringify")? + .call::(other)?, + }); + } + parts.join(" ") + } + Value::String(s) => s.to_str()?.to_string(), + other => format!("{:?}", other), + }; let err_text = format!("[Shortcode Error ({}): {}]", name, message); let make_strong_inline = |text: String| -> Inline { Inline::Strong(crate::pandoc::Strong { @@ -1021,6 +1050,63 @@ return { } } + #[tokio::test] + async fn test_require_resolves_relative_to_script_dir() { + // Q1 extensions `require` sibling modules relative to the requiring + // script's directory (TS Quarto patches `require` via the + // script-file stack). Covers plain names, dotted paths, nested + // requires, and caching. + let tmp = TempDir::new().unwrap(); + std::fs::create_dir_all(tmp.path().join("modules")).unwrap(); + write_script( + tmp.path(), + "helper.lua", + r#" +local m = {} +m.greeting = "FROM-HELPER" +return m +"#, + ); + write_script( + &tmp.path().join("modules"), + "deep.lua", + r#" +local helper = require("helper") +return { text = "DEEP+" .. helper.greeting } +"#, + ); + let script = write_script( + tmp.path(), + "main.lua", + r#" +local helper = require("helper") +local deep = require("modules.deep") +local helper2 = require("helper") +return { + req = function() + local cached = tostring(helper == helper2) + return helper.greeting .. ";" .. deep.text .. ";cached=" .. cached + end +} +"#, + ); + + let runtime = make_runtime(); + let mut engine = LuaShortcodeEngine::new("html", runtime).unwrap(); + engine.load_script(&script).await.unwrap(); + + let result = engine + .call("req", &make_empty_args(), ShortcodeCallContext::Inline) + .await + .unwrap(); + match result { + LuaShortcodeResult::Text(s) => { + assert_eq!(s, "FROM-HELPER;DEEP+FROM-HELPER;cached=true") + } + other => panic!("Expected Text, got {:?}", other), + } + } + #[tokio::test] async fn test_shadow_events_recorded_on_cross_script_override() { let tmp = TempDir::new().unwrap(); diff --git a/crates/pampa/src/pandoc/shortcode.rs b/crates/pampa/src/pandoc/shortcode.rs index df2cc51a4..8f380044e 100644 --- a/crates/pampa/src/pandoc/shortcode.rs +++ b/crates/pampa/src/pandoc/shortcode.rs @@ -29,6 +29,28 @@ fn shortcode_value_span(str: String) -> Inline { }) } +/// Key-value param whose value is itself a shortcode. Q1 convention +/// (`lpegshortcode.lua`'s `md_keyvalue_param`): the param span carries +/// `data-key` but NO `data-value`, and the nested shortcode span is the +/// span's content. +fn shortcode_key_recursive_value_span(key: String, value: Shortcode) -> Inline { + let mut attr_hash = LinkedHashMap::new(); + attr_hash.insert("data-raw".to_string(), key.clone()); + attr_hash.insert("data-key".to_string(), key); + attr_hash.insert("data-is-shortcode".to_string(), "1".to_string()); + + Inline::Span(Span { + attr: ( + String::new(), + vec!["quarto-shortcode__-param".to_string()], + attr_hash, + ), + content: vec![Inline::Span(shortcode_to_span(value))], + source_info: empty_source_info(), + attr_source: AttrSourceInfo::empty(), + }) +} + fn shortcode_key_value_span(key: String, value: String) -> Inline { let mut attr_hash = LinkedHashMap::new(); @@ -93,9 +115,8 @@ pub fn shortcode_to_span(shortcode: Shortcode) -> Span { }, )); } - ShortcodeArg::Shortcode(_) => { - eprintln!("PANIC - Quarto doesn't support nested shortcodes"); - std::process::exit(1); + ShortcodeArg::Shortcode(inner) => { + content.push(shortcode_key_recursive_value_span(key, inner)); } _ => { panic!("Unexpected ShortcodeArg type in shortcode: {:?}", value); @@ -124,13 +145,14 @@ pub fn shortcode_to_span(shortcode: Shortcode) -> Span { }, )); } - ShortcodeArg::Shortcode(_) => { - eprintln!("PANIC - Quarto doesn't support nested shortcodes in keyword args"); - std::process::exit(1); + ShortcodeArg::Shortcode(inner) => { + content.push(shortcode_key_recursive_value_span(key, inner)); } ShortcodeArg::KeyValue(_) => { - eprintln!("PANIC - KeyValue shouldn't appear in keyword_args HashMap"); - std::process::exit(1); + // Structurally impossible from the parser (kv values are + // scalars or shortcodes); degrade visibly instead of + // killing the process. + content.push(shortcode_key_value_span(key, String::new())); } } } @@ -175,6 +197,46 @@ mod tests { } } + #[test] + fn test_nested_shortcode_in_keyword_arg_does_not_crash() { + // Regression: this used to hit process::exit(1). Nested shortcode + // values in keyword args encode Q1-style: a param span with + // data-key but NO data-value, and the nested shortcode span as + // content ("data-key present with no data-value means the value is + // recursive" — lpegshortcode.lua's md_keyvalue_param). + let inner = Shortcode { + is_escaped: false, + name: "inner".to_string(), + positional_args: vec![], + keyword_args: LinkedHashMap::new(), + source_info: si(), + }; + let mut kwargs = LinkedHashMap::new(); + kwargs.insert("k".to_string(), ShortcodeArg::Shortcode(inner)); + let sc = Shortcode { + is_escaped: false, + name: "outer".to_string(), + positional_args: vec![], + keyword_args: kwargs, + source_info: si(), + }; + + let span = shortcode_to_span(sc); + + assert_eq!(span.content.len(), 2); + assert_eq!(get_span_data_key(&span.content[1]), Some("k")); + assert_eq!(get_span_data_value(&span.content[1]), None); + let Inline::Span(kv_span) = &span.content[1] else { + panic!("expected span"); + }; + assert_eq!(kv_span.content.len(), 1); + let Inline::Span(nested) = &kv_span.content[0] else { + panic!("expected nested shortcode span"); + }; + assert!(nested.attr.1.contains(&"quarto-shortcode__".to_string())); + assert_eq!(get_span_data_value(&nested.content[0]), Some("inner")); + } + #[test] fn test_shortcode_name_only() { // Test a shortcode with just a name, no arguments diff --git a/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/shorty.lua b/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/shorty.lua index 19082dc18..740f7f672 100644 --- a/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/shorty.lua +++ b/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/shorty.lua @@ -1,6 +1,8 @@ return { shorty = function(args) - if args[1] == "error" then + if args[1] == "error_args" then + return quarto.shortcode.error_output("shorty", args, "inline") + elseif args[1] == "error" then return quarto.shortcode.error_output("shorty", "error message", "inline") else return pandoc.Strong(args[1]) diff --git a/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/test.qmd index 7636794be..eea302f2c 100644 --- a/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/test.qmd +++ b/crates/quarto/tests/smoke-all/extensions/contract-doc-shortcodes/test.qmd @@ -7,9 +7,11 @@ _quarto: tests: html: ensureFileRegexMatches: - - ["strong>_bringit_", "Shortcode Error \\(shorty\\): error message"] + - ["strong>_bringit_", "Shortcode Error \\(shorty\\): error message", "Shortcode Error \\(shorty\\): error_args"] --- {{< shorty _bringit_ >}} {{< shorty error >}} + +{{< shorty error_args >}} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-require/_extensions/requirer/_extension.yml b/crates/quarto/tests/smoke-all/extensions/contract-require/_extensions/requirer/_extension.yml new file mode 100644 index 000000000..c8bef8f7f --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-require/_extensions/requirer/_extension.yml @@ -0,0 +1,3 @@ +contributes: + shortcodes: + - requirer.lua diff --git a/crates/quarto/tests/smoke-all/extensions/contract-require/_extensions/requirer/modules/helper.lua b/crates/quarto/tests/smoke-all/extensions/contract-require/_extensions/requirer/modules/helper.lua new file mode 100644 index 000000000..53b429847 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-require/_extensions/requirer/modules/helper.lua @@ -0,0 +1 @@ +return { decorate = function(s) return "<<" .. s .. ">>" end } diff --git a/crates/quarto/tests/smoke-all/extensions/contract-require/_extensions/requirer/requirer.lua b/crates/quarto/tests/smoke-all/extensions/contract-require/_extensions/requirer/requirer.lua new file mode 100644 index 000000000..ff5b4a31c --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-require/_extensions/requirer/requirer.lua @@ -0,0 +1,6 @@ +local helper = require("modules.helper") +return { + decorated = function(args) + return "REQ" .. helper.decorate(pandoc.utils.stringify(args[1])) + end +} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-require/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-require/test.qmd new file mode 100644 index 000000000..af54bf13c --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-require/test.qmd @@ -0,0 +1,12 @@ +--- +title: Extension script requires sibling module +format: html +_quarto: + tests: + html: + noErrors: true + ensureFileRegexMatches: + - ["REQ<<sib>>"] +--- + +{{< decorated sib >}} From 5af6fb188f31ed757cc4df8c4de7695b9e23a4fe Mon Sep 17 00:00:00 2001 From: Carlos Scheidegger Date: Fri, 31 Jul 2026 17:33:22 -0500 Subject: [PATCH 5/8] Built-in shortcodes: var, env, pagebreak (bd-540a976a) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 4 of the shortcode-extensions port — the three easy-tier built-ins Q1 has and Q2 lacked: - var (Rust VarShortcodeHandler): reads _variables.yml from the project root (project-scoped, Q1 parity), parsed with InterpretationContext:: DocumentMetadata so markdown values render as markdown (documented Q1 behavior). Dotted-path lookup via get_nested. Unknown variable → Q-16-5 warning + visible ?var marker. StageContext loads the file once per document and threads it through build_transform_pipeline; an existing but unparseable _variables.yml warns naming the file. - env (Rust EnvShortcodeHandler): positional name + optional fallback arg (Q1 1.5 #8316). Unset without fallback → Q-16-5 warning (stricter than silently emitting nothing; on wasm32 env reads always fail, so env is uniformly unset there). - pagebreak: new embedded built-in Lua extension (resources/extensions/quarto/pagebreak/), a near-verbatim port of Q1's handlePagebreak: per-format raw payloads (html/epub/latex/openxml/odt/ context/typst), pptx empty, form-feed fallback. - New Q-16-5 'Shortcode Value Not Found' catalog entry; also applied to the meta handler's unknown-key warning. Fixtures: contract-var (simple/nested/markdown-link values), contract-var- unknown (printsMessage WARN + ?var marker), contract-env (fallback), contract-pagebreak; Rust unit tests for env (set/fallback/unset) and var (nested lookup/unknown). Full workspace suite green (10,819). Plan: claude-notes/plans/2026-07-31-shortcode-extensions-port.md Co-Authored-By: Claude Fable 5 --- .../2026-07-31-shortcode-extensions-port.md | 20 +- crates/quarto-core/src/pipeline.rs | 58 +++- crates/quarto-core/src/stage/context.rs | 55 +++- .../src/stage/stages/ast_transforms.rs | 2 + .../src/transforms/shortcode_resolve.rs | 269 +++++++++++++++++- .../quarto-error-catalog/error_catalog.json | 7 + .../extensions/contract-env/test.qmd | 12 + .../extensions/contract-pagebreak/test.qmd | 16 ++ .../contract-var-unknown/_quarto.yml | 2 + .../contract-var-unknown/_variables.yml | 4 + .../extensions/contract-var-unknown/test.qmd | 15 + .../extensions/contract-var/_quarto.yml | 2 + .../extensions/contract-var/_variables.yml | 4 + .../extensions/contract-var/test.qmd | 16 ++ .../quarto/pagebreak/_extension.yml | 6 + .../extensions/quarto/pagebreak/pagebreak.lua | 38 +++ 16 files changed, 501 insertions(+), 25 deletions(-) create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-env/test.qmd create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-pagebreak/test.qmd create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-var-unknown/_quarto.yml create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-var-unknown/_variables.yml create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-var-unknown/test.qmd create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-var/_quarto.yml create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-var/_variables.yml create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-var/test.qmd create mode 100644 resources/extensions/quarto/pagebreak/_extension.yml create mode 100644 resources/extensions/quarto/pagebreak/pagebreak.lua diff --git a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md index 0a340da1e..e24c5733a 100644 --- a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md +++ b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md @@ -273,13 +273,19 @@ where the study only has static reads. and arrive stringified; `pandoc.utils.stringify` handles booleans/numbers (pandoc parity — also fixes video auto-stretch gate under the new native-boolean meta). -- [ ] Sandboxed, script-dir-relative `require` (native + WASM via - `SystemRuntime`), scoped to the extension's directory; test with a - sibling-module fixture. (This is the highest-risk item for real-world - extensions; likely a `package.preload`-style loader rather than exposing - the C `package` lib.) -- [ ] `quarto.shortcode.read_arg`/`error_output` parity vs `init.lua:1002-1032`. -- [ ] Fix `shortcode_to_span`'s `process::exit(1)` (nested kv arg) → diagnostic. +- [x] Script-dir-aware `require` (commit 7c3b2f10): resolves against the + script-dir stack top-down (sibling-relative, then extension-root- + relative à la Q1's `require("modules/brand/brand")`); sandboxed module + env; path-keyed cache; native-stdlib fallback; `SystemRuntime` I/O so + WASM shares the path. `contract-require` fixture + pampa unit test. + Note: registered in the *shortcode* engine; the filter engine should + get the same call — follow-on item in Phase 6. +- [x] `read_arg` verified matching Q1; `error_output` now accepts table + message_or_args (Q1 contract; error_args case ported into + contract-doc-shortcodes). Message text deviates from Q1's `?name:msg` + ([Shortcode Error (name): msg]) — accepted deviation. +- [x] `shortcode_to_span` `process::exit(1)` removed: nested kv values + encode Q1-style recursive param spans (commit 7c3b2f10). ### Phase 3 — DEFERRED (2026-07-31): `text` context — shortcodes in code, attributes, targets (gap row 6) diff --git a/crates/quarto-core/src/pipeline.rs b/crates/quarto-core/src/pipeline.rs index 18c64e05a..96d697f28 100644 --- a/crates/quarto-core/src/pipeline.rs +++ b/crates/quarto-core/src/pipeline.rs @@ -1176,6 +1176,7 @@ pub fn build_transform_pipeline( extensions: Vec, runtime: std::sync::Arc, target_format: String, + variables: Option, ) -> TransformPipeline { let mut pipeline: TransformPipeline = TransformPipeline::new(); @@ -1199,6 +1200,7 @@ pub fn build_transform_pipeline( extensions, runtime.clone(), lua_format, + variables, ))); pipeline.push(Box::new(MetadataNormalizeTransform::new())); // Date normalization (bd-gx9cic8z P4): resolves today/now/ @@ -1536,9 +1538,15 @@ pub fn build_q2_preview_transform_pipeline( extensions: Vec, runtime: std::sync::Arc, target_format: String, + variables: Option, ) -> TransformPipeline { - let mut pipeline = - build_transform_pipeline(shortcode_paths, extensions, runtime, target_format); + let mut pipeline = build_transform_pipeline( + shortcode_paths, + extensions, + runtime, + target_format, + variables, + ); pipeline.retain_excluding(Q2_PREVIEW_TRANSFORM_EXCLUDED); pipeline } @@ -2695,7 +2703,7 @@ mod tests { #[test] fn q2_preview_transform_excluded_names_exist_in_html_pipeline() { let runtime = make_test_runtime(); - let html = build_transform_pipeline(vec![], vec![], runtime, "html".to_string()); + let html = build_transform_pipeline(vec![], vec![], runtime, "html".to_string(), None); let html_names: Vec<&str> = html.iter().map(|t| t.name()).collect(); let unknown: Vec<&&str> = Q2_PREVIEW_TRANSFORM_EXCLUDED @@ -3121,8 +3129,13 @@ mod tests { #[test] fn q2_preview_pipeline_includes_link_rewrite() { let runtime = make_test_runtime(); - let pipeline = - build_q2_preview_transform_pipeline(vec![], vec![], runtime, "q2-preview".to_string()); + let pipeline = build_q2_preview_transform_pipeline( + vec![], + vec![], + runtime, + "q2-preview".to_string(), + None, + ); let names: Vec<&str> = pipeline.iter().map(|t| t.name()).collect(); assert!( names.contains(&"link-rewrite"), @@ -3139,8 +3152,13 @@ mod tests { #[test] fn q2_preview_pipeline_includes_chrome_transforms() { let runtime = make_test_runtime(); - let pipeline = - build_q2_preview_transform_pipeline(vec![], vec![], runtime, "q2-preview".to_string()); + let pipeline = build_q2_preview_transform_pipeline( + vec![], + vec![], + runtime, + "q2-preview".to_string(), + None, + ); let names: Vec<&str> = pipeline.iter().map(|t| t.name()).collect(); for required in [ "navbar-render", @@ -3186,7 +3204,7 @@ mod tests { #[test] fn html_pipeline_includes_code_block_decoration_transforms() { let runtime = make_test_runtime(); - let pipeline = build_transform_pipeline(vec![], vec![], runtime, "html".to_string()); + let pipeline = build_transform_pipeline(vec![], vec![], runtime, "html".to_string(), None); let names: Vec<&str> = pipeline.iter().map(|t| t.name()).collect(); let gen_pos = names.iter().position(|&n| n == "code-block-generate"); @@ -3248,7 +3266,8 @@ mod tests { // covers them automatically. for format in ["html", "revealjs"] { let runtime = make_test_runtime(); - let pipeline = build_transform_pipeline(vec![], vec![], runtime, format.to_string()); + let pipeline = + build_transform_pipeline(vec![], vec![], runtime, format.to_string(), None); let steps: Vec<(&str, TransformPhase)> = pipeline.iter().map(|t| (t.name(), t.phase())).collect(); @@ -3289,8 +3308,13 @@ mod tests { #[test] fn q2_preview_pipeline_includes_code_block_decoration_transforms() { let runtime = make_test_runtime(); - let pipeline = - build_q2_preview_transform_pipeline(vec![], vec![], runtime, "q2-preview".to_string()); + let pipeline = build_q2_preview_transform_pipeline( + vec![], + vec![], + runtime, + "q2-preview".to_string(), + None, + ); let names: Vec<&str> = pipeline.iter().map(|t| t.name()).collect(); for required in ["code-block-generate", "code-block-render"] { assert!( @@ -3309,7 +3333,8 @@ mod tests { fn mermaid_render_present_before_code_block_render() { for format in ["html", "revealjs"] { let runtime = make_test_runtime(); - let pipeline = build_transform_pipeline(vec![], vec![], runtime, format.to_string()); + let pipeline = + build_transform_pipeline(vec![], vec![], runtime, format.to_string(), None); let names: Vec<&str> = pipeline.iter().map(|t| t.name()).collect(); let mermaid_pos = names.iter().position(|&n| n == "mermaid-render"); @@ -3335,8 +3360,13 @@ mod tests { fn q2_preview_pipeline_excludes_mermaid_render() { for format in ["q2-preview", "q2-slides"] { let runtime = make_test_runtime(); - let pipeline = - build_q2_preview_transform_pipeline(vec![], vec![], runtime, format.to_string()); + let pipeline = build_q2_preview_transform_pipeline( + vec![], + vec![], + runtime, + format.to_string(), + None, + ); let names: Vec<&str> = pipeline.iter().map(|t| t.name()).collect(); assert!( !names.contains(&"mermaid-render"), diff --git a/crates/quarto-core/src/stage/context.rs b/crates/quarto-core/src/stage/context.rs index a39a90cf0..c72928d89 100644 --- a/crates/quarto-core/src/stage/context.rs +++ b/crates/quarto-core/src/stage/context.rs @@ -80,6 +80,12 @@ pub struct StageContext { /// Extensions discovered for this document pub extensions: Vec, + /// Project variables from `_variables.yml` (project root), parsed as + /// document metadata so markdown values render as markdown. `None` for + /// single-file renders (Q1 parity: `var` is project-scoped) or when the + /// file does not exist. + pub variables: Option, + // === Mutable state === /// Artifact store for dependencies and intermediates pub artifacts: ArtifactStore, @@ -225,7 +231,7 @@ impl StageContext { document: DocumentInfo, ) -> Result { let builtin_ext_path = builtin_extensions_path(runtime.as_ref()); - let (extensions, extension_diagnostics) = crate::extension::discover_extensions( + let (extensions, mut startup_diagnostics) = crate::extension::discover_extensions( &document.input, if project.is_single_file { None @@ -236,6 +242,9 @@ impl StageContext { runtime.as_ref(), ); + let variables = + load_project_variables(runtime.as_ref(), &project, &mut startup_diagnostics); + Ok(Self { runtime, format, @@ -243,9 +252,10 @@ impl StageContext { document, temp_dir: std::sync::OnceLock::new(), extensions, + variables, artifacts: ArtifactStore::new(), includes: PandocIncludes::default(), - diagnostics: extension_diagnostics, + diagnostics: startup_diagnostics, ref_type_registry: None, crossref_index: None, resource_report: crate::project_resources::DocumentResourceReport::new(), @@ -743,6 +753,47 @@ mod tests { /// - **Native**: extracts the embedded `ResourceBundle` to a temp dir. /// - **WASM**: returns the VFS path `/__quarto_resources__/extensions` /// if it exists (populated during WASM init). +/// Load `/_variables.yml` for the `var` shortcode. +/// +/// Project-scoped like Q1 (no variables in single-file mode). A missing +/// file is normal; a file that exists but fails to parse produces a +/// warning diagnostic naming the file. +fn load_project_variables( + runtime: &dyn SystemRuntime, + project: &crate::project::ProjectContext, + diagnostics: &mut Vec, +) -> Option { + if project.is_single_file { + return None; + } + let path = project.dir.join("_variables.yml"); + let content = runtime.file_read_string(&path).ok()?; + match quarto_yaml::parse_file(&content, &path.display().to_string()) { + Ok(yaml) => { + let mut collector = pampa::utils::diagnostic_collector::DiagnosticCollector::new(); + Some(pampa::pandoc::yaml_to_config_value( + yaml, + quarto_pandoc_types::InterpretationContext::DocumentMetadata, + &mut collector, + )) + } + Err(e) => { + diagnostics.push( + quarto_error_reporting::DiagnosticMessageBuilder::warning( + "Project variables not loaded", + ) + .problem(format!( + "`{}` could not be parsed: {}; `var` shortcodes will not resolve", + path.display(), + e + )) + .build(), + ); + None + } + } +} + fn builtin_extensions_path( _runtime: &dyn quarto_system_runtime::SystemRuntime, ) -> Option { diff --git a/crates/quarto-core/src/stage/stages/ast_transforms.rs b/crates/quarto-core/src/stage/stages/ast_transforms.rs index 077848e6c..052f6e706 100644 --- a/crates/quarto-core/src/stage/stages/ast_transforms.rs +++ b/crates/quarto-core/src/stage/stages/ast_transforms.rs @@ -141,12 +141,14 @@ impl PipelineStage for AstTransformsStage { ctx.extensions.clone(), ctx.runtime.clone(), ctx.format.target_format.clone(), + ctx.variables.clone(), ), _ => build_transform_pipeline( shortcode_paths, ctx.extensions.clone(), ctx.runtime.clone(), ctx.format.target_format.clone(), + ctx.variables.clone(), ), }; &jit_pipeline diff --git a/crates/quarto-core/src/transforms/shortcode_resolve.rs b/crates/quarto-core/src/transforms/shortcode_resolve.rs index 643cbe804..e5c816156 100644 --- a/crates/quarto-core/src/transforms/shortcode_resolve.rs +++ b/crates/quarto-core/src/transforms/shortcode_resolve.rs @@ -150,6 +150,7 @@ impl ShortcodeHandler for MetaShortcodeHandler { Some(value) => ShortcodeResult::Inlines(config_value_to_inlines(value)), None => { let diagnostic = DiagnosticMessageBuilder::warning("Unknown metadata key") + .with_code("Q-16-5") .problem(format!("Metadata key `{}` not found in document", key)) .add_hint("Check that the key exists in your YAML frontmatter") .with_location(ctx.source_info.clone()) @@ -163,6 +164,150 @@ impl ShortcodeHandler for MetaShortcodeHandler { } } +/// Convert a scalar positional argument to its string form. +fn positional_arg_to_string(arg: &ShortcodeArg) -> Option { + match arg { + ShortcodeArg::String(s) => Some(s.clone()), + ShortcodeArg::Number(n) => Some(n.to_string()), + ShortcodeArg::Boolean(b) => Some(b.to_string()), + _ => None, + } +} + +/// Built-in handler for `{{< env NAME >}}` / `{{< env NAME fallback >}}`. +/// +/// Reads a process environment variable; the optional second positional +/// argument is the value used when the variable is unset (Q1 1.5, #8316). +/// Unset with no fallback is a Q-16-5 warning (Q1 warned and emitted +/// `?env`). On wasm32 `std::env::var` always errs, so `env` behaves as +/// uniformly-unset there. +pub struct EnvShortcodeHandler; + +impl ShortcodeHandler for EnvShortcodeHandler { + fn name(&self) -> &str { + "env" + } + + fn resolve( + &self, + shortcode: &Shortcode, + ctx: &ShortcodeContext, + _resolution_ctx: ResolutionContext, + ) -> ShortcodeResult { + let Some(name) = shortcode + .positional_args + .first() + .and_then(positional_arg_to_string) + else { + let diagnostic = DiagnosticMessageBuilder::warning("Missing shortcode argument") + .problem("The `env` shortcode requires an environment variable name") + .add_hint("Use `{{< env NAME >}}` or `{{< env NAME fallback >}}`") + .with_location(ctx.source_info.clone()) + .build(); + return ShortcodeResult::Error(ShortcodeError { + key: "env".to_string(), + diagnostic, + }); + }; + + let value = std::env::var(&name).ok().or_else(|| { + shortcode + .positional_args + .get(1) + .and_then(positional_arg_to_string) + }); + + match value { + Some(text) => ShortcodeResult::Inlines(vec![Inline::Str(Str { + text, + source_info: ctx.source_info.clone(), + })]), + None => { + let diagnostic = DiagnosticMessageBuilder::warning("Environment variable not set") + .with_code("Q-16-5") + .problem(format!( + "Environment variable `{}` is not set and no fallback was given", + name + )) + .add_hint("Set the variable, or pass a fallback: `{{< env NAME fallback >}}`") + .with_location(ctx.source_info.clone()) + .build(); + ShortcodeResult::Error(ShortcodeError { + key: format!("env:{}", name), + diagnostic, + }) + } + } + } +} + +/// Built-in handler for `{{< var key >}}` — reads `_variables.yml` from the +/// project root (project-scoped, like Q1). Values are parsed as document +/// metadata, so markdown variable values render as markdown. +pub struct VarShortcodeHandler { + variables: Option, +} + +impl VarShortcodeHandler { + pub fn new(variables: Option) -> Self { + Self { variables } + } +} + +impl ShortcodeHandler for VarShortcodeHandler { + fn name(&self) -> &str { + "var" + } + + fn resolve( + &self, + shortcode: &Shortcode, + ctx: &ShortcodeContext, + _resolution_ctx: ResolutionContext, + ) -> ShortcodeResult { + let Some(key) = shortcode + .positional_args + .first() + .and_then(positional_arg_to_string) + else { + let diagnostic = DiagnosticMessageBuilder::warning("Missing shortcode argument") + .problem("The `var` shortcode requires a variable name") + .add_hint("Use `{{< var key >}}` where `key` is defined in `_variables.yml`") + .with_location(ctx.source_info.clone()) + .build(); + return ShortcodeResult::Error(ShortcodeError { + key: "var".to_string(), + diagnostic, + }); + }; + + match self + .variables + .as_ref() + .and_then(|vars| vars.get_nested(&key)) + { + Some(value) => ShortcodeResult::Inlines(config_value_to_inlines(value)), + None => { + let diagnostic = DiagnosticMessageBuilder::warning("Unknown variable") + .with_code("Q-16-5") + .problem(format!( + "Variable `{}` is not defined in `_variables.yml`", + key + )) + .add_hint( + "Define the variable in `_variables.yml` next to `_quarto.yml` (variables are project-scoped)", + ) + .with_location(ctx.source_info.clone()) + .build(); + ShortcodeResult::Error(ShortcodeError { + key: format!("var:{}", key), + diagnostic, + }) + } + } + } +} + /// Convert a ConfigValue to inline content. /// /// The synthesized `Inline::Str` instances reuse the input @@ -296,7 +441,7 @@ impl ShortcodeResolveTransform { /// Used in tests that don't need Lua support. pub fn new() -> Self { Self { - handlers: vec![Box::new(MetaShortcodeHandler)], + handlers: Self::builtin_handlers(None), lua_shortcode_paths: Vec::new(), extensions: Vec::new(), runtime: None, @@ -304,6 +449,16 @@ impl ShortcodeResolveTransform { } } + /// The built-in Rust handlers: `meta`, `env`, and `var` (fed by the + /// project's `_variables.yml`, when present). + fn builtin_handlers(variables: Option) -> Vec> { + vec![ + Box::new(MetaShortcodeHandler), + Box::new(EnvShortcodeHandler), + Box::new(VarShortcodeHandler::new(variables)), + ] + } + /// Create a shortcode resolve transform with Lua support. /// /// The `LuaShortcodeEngine` is NOT created here (it's `!Send + !Sync`). @@ -313,9 +468,10 @@ impl ShortcodeResolveTransform { extensions: Vec, runtime: Arc, target_format: String, + variables: Option, ) -> Self { Self { - handlers: vec![Box::new(MetaShortcodeHandler)], + handlers: Self::builtin_handlers(variables), lua_shortcode_paths, extensions, runtime: Some(runtime), @@ -1698,6 +1854,108 @@ mod tests { } } + #[test] + fn test_env_shortcode_handler_set_and_fallback() { + let handler = EnvShortcodeHandler; + let meta = ConfigValue::new_map(vec![], dummy_source_info()); + + // Set variable wins over fallback. + // SAFETY: test-local variable name; nextest runs each test in its + // own process, so no cross-test env races. + unsafe { std::env::set_var("QUARTO_TEST_ENV_SC", "env-value") }; + let shortcode = make_shortcode("env", vec!["QUARTO_TEST_ENV_SC", "fallback"]); + let ctx = ShortcodeContext { + metadata: &meta, + source_info: &shortcode.source_info, + }; + match handler.resolve(&shortcode, &ctx, ResolutionContext::Inline) { + ShortcodeResult::Inlines(inlines) => { + let Inline::Str(s) = &inlines[0] else { + panic!("expected Str"); + }; + assert_eq!(s.text, "env-value"); + } + other => panic!("Expected Inlines, got {:?}", std::mem::discriminant(&other)), + } + + // Unset variable falls back to the second positional arg (Q1 1.5). + let shortcode = make_shortcode("env", vec!["QUARTO_TEST_ENV_SC_UNSET", "fallback"]); + let ctx = ShortcodeContext { + metadata: &meta, + source_info: &shortcode.source_info, + }; + match handler.resolve(&shortcode, &ctx, ResolutionContext::Inline) { + ShortcodeResult::Inlines(inlines) => { + let Inline::Str(s) = &inlines[0] else { + panic!("expected Str"); + }; + assert_eq!(s.text, "fallback"); + } + _ => panic!("Expected Inlines"), + } + + // Unset without fallback: Q-16-5 error. + let shortcode = make_shortcode("env", vec!["QUARTO_TEST_ENV_SC_UNSET"]); + let ctx = ShortcodeContext { + metadata: &meta, + source_info: &shortcode.source_info, + }; + match handler.resolve(&shortcode, &ctx, ResolutionContext::Inline) { + ShortcodeResult::Error(err) => { + assert_eq!(err.key, "env:QUARTO_TEST_ENV_SC_UNSET"); + assert!(format!("{:?}", err.diagnostic).contains("Q-16-5")); + } + _ => panic!("Expected Error"), + } + } + + #[test] + fn test_var_shortcode_handler_lookup_and_unknown() { + let vars = ConfigValue::new_map( + vec![make_map_entry( + "nested", + ConfigValue::new_map( + vec![make_map_entry( + "key", + ConfigValue::new_string("nested-value", dummy_source_info()), + )], + dummy_source_info(), + ), + )], + dummy_source_info(), + ); + let handler = VarShortcodeHandler::new(Some(vars)); + let meta = ConfigValue::new_map(vec![], dummy_source_info()); + + let shortcode = make_shortcode("var", vec!["nested.key"]); + let ctx = ShortcodeContext { + metadata: &meta, + source_info: &shortcode.source_info, + }; + match handler.resolve(&shortcode, &ctx, ResolutionContext::Inline) { + ShortcodeResult::Inlines(inlines) => { + let Inline::Str(s) = &inlines[0] else { + panic!("expected Str"); + }; + assert_eq!(s.text, "nested-value"); + } + _ => panic!("Expected Inlines"), + } + + let shortcode = make_shortcode("var", vec!["missing"]); + let ctx = ShortcodeContext { + metadata: &meta, + source_info: &shortcode.source_info, + }; + match handler.resolve(&shortcode, &ctx, ResolutionContext::Inline) { + ShortcodeResult::Error(err) => { + assert_eq!(err.key, "var:missing"); + assert!(format!("{:?}", err.diagnostic).contains("Q-16-5")); + } + _ => panic!("Expected Error"), + } + } + #[test] fn test_meta_shortcode_handler_success() { let handler = MetaShortcodeHandler; @@ -2234,6 +2492,7 @@ mod tests { Vec::new(), runtime, "html".to_string(), + None, ); let mut ast = Pandoc { @@ -2282,6 +2541,7 @@ mod tests { vec![ext], runtime, "html".to_string(), + None, ); let mut ast = Pandoc { @@ -2328,6 +2588,7 @@ mod tests { Vec::new(), runtime, "html".to_string(), + None, ); let meta = ConfigValue::new_map( @@ -2374,6 +2635,7 @@ mod tests { Vec::new(), runtime, "html".to_string(), + None, ); let mut ast = Pandoc { @@ -2426,6 +2688,7 @@ mod tests { vec![ext], runtime, "html".to_string(), + None, ); // Shortcode alone in Para → block context @@ -2470,6 +2733,7 @@ mod tests { Vec::new(), runtime, "html".to_string(), + None, ); let mut ast = Pandoc { @@ -2524,6 +2788,7 @@ mod tests { Vec::new(), runtime, "html".to_string(), + None, ); let tok = token_si(); diff --git a/crates/quarto-error-catalog/error_catalog.json b/crates/quarto-error-catalog/error_catalog.json index c0d804ad6..b8defcf33 100644 --- a/crates/quarto-error-catalog/error_catalog.json +++ b/crates/quarto-error-catalog/error_catalog.json @@ -1090,5 +1090,12 @@ "message_template": "Two loaded shortcode scripts define a handler with the same name; the later registration wins (document `shortcodes:` scripts load first, then extensions in discovery order, so the more local definition takes precedence). This is informational — overriding a built-in extension is a supported pattern — but if the shadowing is unintentional, rename or remove one of the handlers.", "docs_url": "https://quarto.org/docs/errors/extension/Q-16-4", "since_version": "99.9.9" + }, + "Q-16-5": { + "subsystem": "extension", + "title": "Shortcode Value Not Found", + "message_template": "A value-lookup shortcode (`meta`, `var`, `env`) could not resolve its key: the metadata key is absent, the variable is not defined in `_variables.yml`, or the environment variable is unset and no fallback argument was given. The invocation renders as a visible `?name` marker. Check the key for typos, define it at the source the shortcode reads from, or (for `env`) pass a fallback: `{{< env NAME fallback >}}`.", + "docs_url": "https://quarto.org/docs/errors/extension/Q-16-5", + "since_version": "99.9.9" } } diff --git a/crates/quarto/tests/smoke-all/extensions/contract-env/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-env/test.qmd new file mode 100644 index 000000000..22e3f702b --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-env/test.qmd @@ -0,0 +1,12 @@ +--- +title: Env shortcode fallback +format: html +_quarto: + tests: + html: + noErrors: true + ensureFileRegexMatches: + - ["ENV-FALLBACK-VALUE"] +--- + +{{< env QUARTO_SMOKE_NO_SUCH_VAR ENV-FALLBACK-VALUE >}} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-pagebreak/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-pagebreak/test.qmd new file mode 100644 index 000000000..c3d1e38b3 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-pagebreak/test.qmd @@ -0,0 +1,16 @@ +--- +title: Pagebreak shortcode +format: html +_quarto: + tests: + html: + noErrors: true + ensureFileRegexMatches: + - ["page-break-after: always"] +--- + +Before. + +{{< pagebreak >}} + +After. diff --git a/crates/quarto/tests/smoke-all/extensions/contract-var-unknown/_quarto.yml b/crates/quarto/tests/smoke-all/extensions/contract-var-unknown/_quarto.yml new file mode 100644 index 000000000..b8bae5830 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-var-unknown/_quarto.yml @@ -0,0 +1,2 @@ +project: + type: default diff --git a/crates/quarto/tests/smoke-all/extensions/contract-var-unknown/_variables.yml b/crates/quarto/tests/smoke-all/extensions/contract-var-unknown/_variables.yml new file mode 100644 index 000000000..f552d56d2 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-var-unknown/_variables.yml @@ -0,0 +1,4 @@ +simple: simple-value +nested: + key: nested-value +linky: "[VarLink](https://example.com/vartarget)" diff --git a/crates/quarto/tests/smoke-all/extensions/contract-var-unknown/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-var-unknown/test.qmd new file mode 100644 index 000000000..8063c8ab1 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-var-unknown/test.qmd @@ -0,0 +1,15 @@ +--- +title: Unknown var warns with Q-16-5 +format: html +_quarto: + tests: + html: + noErrors: true + printsMessage: + level: WARN + regex: "Unknown variable" + ensureFileRegexMatches: + - ["\\?var"] +--- + +{{< var no-such-variable >}} diff --git a/crates/quarto/tests/smoke-all/extensions/contract-var/_quarto.yml b/crates/quarto/tests/smoke-all/extensions/contract-var/_quarto.yml new file mode 100644 index 000000000..b8bae5830 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-var/_quarto.yml @@ -0,0 +1,2 @@ +project: + type: default diff --git a/crates/quarto/tests/smoke-all/extensions/contract-var/_variables.yml b/crates/quarto/tests/smoke-all/extensions/contract-var/_variables.yml new file mode 100644 index 000000000..f552d56d2 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-var/_variables.yml @@ -0,0 +1,4 @@ +simple: simple-value +nested: + key: nested-value +linky: "[VarLink](https://example.com/vartarget)" diff --git a/crates/quarto/tests/smoke-all/extensions/contract-var/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-var/test.qmd new file mode 100644 index 000000000..f465fd4d2 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-var/test.qmd @@ -0,0 +1,16 @@ +--- +title: Var shortcode +format: html +_quarto: + tests: + html: + noErrors: true + ensureFileRegexMatches: + - ["simple-value", "nested-value", "href=\"https://example.com/vartarget\"", "VarLink"] +--- + +Simple: {{< var simple >}} + +Nested: {{< var nested.key >}} + +Markdown: {{< var linky >}} diff --git a/resources/extensions/quarto/pagebreak/_extension.yml b/resources/extensions/quarto/pagebreak/_extension.yml new file mode 100644 index 000000000..bcf6b9bc4 --- /dev/null +++ b/resources/extensions/quarto/pagebreak/_extension.yml @@ -0,0 +1,6 @@ +title: Pagebreak +author: Posit, PBC +organization: quarto +contributes: + shortcodes: + - pagebreak.lua diff --git a/resources/extensions/quarto/pagebreak/pagebreak.lua b/resources/extensions/quarto/pagebreak/pagebreak.lua new file mode 100644 index 000000000..c0f4d2bfa --- /dev/null +++ b/resources/extensions/quarto/pagebreak/pagebreak.lua @@ -0,0 +1,38 @@ +-- pagebreak.lua +-- Ported from Quarto 1's handlePagebreak +-- (quarto-cli src/resources/filters/quarto-pre/shortcodes-handlers.lua) + +local payloads = { + epub = '

', + html = '
', + latex = '\\newpage{}', + ooxml = '', + odt = '', + context = '\\page', + typst = '#pagebreak()' +} + +return { + pagebreak = function() + if FORMAT == 'docx' then + return pandoc.RawBlock('openxml', payloads.ooxml) + elseif FORMAT == 'pptx' then + return pandoc.Blocks({}) + elseif FORMAT:match('latex') or FORMAT == 'pdf' then + return pandoc.RawBlock('tex', payloads.latex) + elseif FORMAT:match('odt') then + return pandoc.RawBlock('opendocument', payloads.odt) + elseif FORMAT == 'typst' then + return pandoc.RawBlock('typst', payloads.typst) + elseif FORMAT:match('html.*') or FORMAT:match('revealjs') then + return pandoc.RawBlock('html', payloads.html) + elseif FORMAT:match('epub') then + return pandoc.RawBlock('html', payloads.epub) + elseif FORMAT:match('context') then + return pandoc.RawBlock('context', payloads.context) + else + -- fall back to a form feed character + return pandoc.Para(pandoc.Inlines({ pandoc.Str('\f') })) + end + end +} From 87644d047801ba11a5c3b592e2966318b7054efd Mon Sep 17 00:00:00 2001 From: Carlos Scheidegger Date: Fri, 31 Jul 2026 17:39:16 -0500 Subject: [PATCH 6/8] Unknown-shortcode policy: explicit passthrough + no silent writer drops (bd-540a976a) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 5 of the shortcode-extensions port: - New shortcode-passthrough metadata key: a list of shortcode names to pass through verbatim (renders the literal {{< ... >}} text, no warning). This replaces Q1's implicit pass-anything-unknown-through (a Hugo-interop behavior) with an explicit declaration; anything not declared keeps the source-mapped Q-16-3 warning. Composes with q2 render --strict for unknown-shortcode-as-error. Fixture: contract-passthrough. - HTML writer no longer silently drops a surviving Inline::Shortcode — it renders a visible ?name marker (a surviving node is a resolution leak; silent disappearance made it undiagnosable). Unit test added. Note: catalog codes Q-3-30/Q-3-42 remain unreferenced — writers have no diagnostics channel today; the visible marker covers discoverability. Left as-is rather than gold-plating a writer diagnostics channel. Full workspace suite green (10,821). Plan: claude-notes/plans/2026-07-31-shortcode-extensions-port.md Co-Authored-By: Claude Fable 5 --- .../2026-07-31-shortcode-extensions-port.md | 25 +++++----- crates/pampa/src/writers/html.rs | 48 ++++++++++++++++++- .../src/transforms/shortcode_resolve.rs | 22 +++++++++ .../extensions/contract-passthrough/test.qmd | 15 ++++++ 4 files changed, 98 insertions(+), 12 deletions(-) create mode 100644 crates/quarto/tests/smoke-all/extensions/contract-passthrough/test.qmd diff --git a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md index e24c5733a..530755902 100644 --- a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md +++ b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md @@ -310,17 +310,20 @@ below for the eventual follow-on strand: ### Phase 4 — Missing built-ins: `var`, `env`, `pagebreak` (gap row 8, easy tier) -- [ ] `var`: `_variables.yml` loading (project-scoped, values parsed as qmd - inlines), dotted lookup, unknown-var diagnostic (coded, source-mapped — - improvement over Q1's `?var:name`); `quarto.variables.get` Lua API. -- [ ] `env`: positional name + optional fallback arg (Q1 1.5 `#8316`); decide - unset-and-no-fallback behavior (Q1: `Null`; propose coded warning). -- [ ] `pagebreak`: per-format raw table (html/latex/typst/docx/odt/context/epub, - `\f` fallback) — implement as Rust handler or built-in Lua extension; - follow the existing built-in-extension pattern - (`claude-notes/plans/2026-04-01-builtin-extensions.md`). -- [ ] Each: implement in whichever tier (Rust handler vs embedded Lua ext) - matches its needs; document the choice in the strand. +All landed in commit 5af6fb18; full workspace green (10,819). + +- [x] `var`: Rust `VarShortcodeHandler`; `_variables.yml` loaded in + `StageContext::new` (project-scoped; single-file mode gets none — Q1 + parity), parsed as `DocumentMetadata` so markdown values render; + dotted lookup; unknown var → Q-16-5 + `?var` marker; unparseable + variables file warns naming the file. `quarto.variables.get` Lua API + **not** done — moved to Phase 6 (Lua API parity follow-on). +- [x] `env`: Rust `EnvShortcodeHandler`; fallback arg per Q1 1.5; unset + without fallback → Q-16-5 coded warning (decided: stricter than Q1's + Null). wasm32: env reads always fail → uniformly unset. +- [x] `pagebreak`: embedded built-in Lua extension + (`resources/extensions/quarto/pagebreak/`), near-verbatim Q1 port. + Tier choices documented in the commit message. ### Phase 5 — Unknown-shortcode policy + writer hardening (D1, gap row 9) diff --git a/crates/pampa/src/writers/html.rs b/crates/pampa/src/writers/html.rs index 632a42aea..4d4074d99 100644 --- a/crates/pampa/src/writers/html.rs +++ b/crates/pampa/src/writers/html.rs @@ -1055,8 +1055,19 @@ fn write_inline( } write!(ctx, "")?; } + // A shortcode that survives to the writer was never resolved — a + // resolution leak. Render the Q1-style visible ?name marker so the + // gap is diagnosable in the output; never drop it silently + // (bd-540a976a Phase 5). + Inline::Shortcode(sc) => { + write!( + ctx, + "?{}", + escape_html(&sc.name) + )?; + } // Quarto extensions - render as raw HTML or skip - Inline::Shortcode(_) | Inline::NoteReference(_) | Inline::Attr(_) => { + Inline::NoteReference(_) | Inline::Attr(_) => { // These should not appear in final output } Inline::Insert(ins) => { @@ -2022,6 +2033,41 @@ mod tests { SourceInfo::for_test() } + #[test] + fn test_unresolved_shortcode_renders_visible_marker() { + use crate::pandoc::ASTContext; + + // An Inline::Shortcode surviving to the writer is a resolution + // leak. It must render as a visible ?name marker — never be + // silently dropped (bd-540a976a, Phase 5). + let ctx = ASTContext::anonymous(); + let sc = quarto_pandoc_types::Shortcode { + is_escaped: false, + name: "mystery".to_string(), + positional_args: vec![], + keyword_args: hashlink::LinkedHashMap::new(), + source_info: dummy_source_info(), + }; + let para = Block::Paragraph(Paragraph { + content: vec![Inline::Shortcode(sc)], + source_info: dummy_source_info(), + }); + let pandoc = Pandoc { + meta: ConfigValue::default(), + blocks: vec![para], + }; + + let mut output = Vec::new(); + write(&pandoc, &ctx, &mut output).unwrap(); + let html = String::from_utf8(output).unwrap(); + + assert!( + html.contains("?mystery"), + "expected visible marker, got: {html}" + ); + assert!(html.contains("quarto-unresolved-shortcode")); + } + #[test] fn test_write_paragraph_without_source_tracking() { use crate::pandoc::ASTContext; diff --git a/crates/quarto-core/src/transforms/shortcode_resolve.rs b/crates/quarto-core/src/transforms/shortcode_resolve.rs index e5c816156..e72f1f975 100644 --- a/crates/quarto-core/src/transforms/shortcode_resolve.rs +++ b/crates/quarto-core/src/transforms/shortcode_resolve.rs @@ -585,6 +585,18 @@ impl ShortcodeResolveTransform { } } + // Declared foreign shortcodes (e.g. Hugo's) pass through verbatim. + // Unlike Q1 — which silently passed through anything unknown — the + // passthrough set is an explicit declaration in metadata: + // + // shortcode-passthrough: [ref, figure] + // + // (Plan D1: explicit declaration over inference; everything not + // declared gets a source-mapped Q-16-3 warning below.) + if passthrough_names(ctx.metadata).contains(&shortcode.name) { + return ShortcodeResult::Preserve; + } + // Unknown shortcode - create error with diagnostic let diagnostic = DiagnosticMessageBuilder::warning("Unknown shortcode") .with_code("Q-16-3") @@ -689,6 +701,16 @@ impl ShortcodeResolveTransform { } } +/// Shortcode names declared for verbatim passthrough via the +/// `shortcode-passthrough` metadata key. +fn passthrough_names(metadata: &ConfigValue) -> Vec { + metadata + .get("shortcode-passthrough") + .and_then(|v| v.as_array()) + .map(|items| items.iter().filter_map(|i| i.as_plain_text()).collect()) + .unwrap_or_default() +} + /// Does this shortcode carry any nested shortcode arguments (positional or /// keyword)? fn has_nested_shortcode_args(shortcode: &Shortcode) -> bool { diff --git a/crates/quarto/tests/smoke-all/extensions/contract-passthrough/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-passthrough/test.qmd new file mode 100644 index 000000000..15bf9ea19 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/contract-passthrough/test.qmd @@ -0,0 +1,15 @@ +--- +title: Declared foreign shortcodes pass through verbatim +format: html +shortcode-passthrough: + - hugoref +_quarto: + tests: + html: + noErrors: true + ensureFileRegexMatches: + - ["\\{\\{< hugoref foo >\\}\\}"] + - ["quarto-unresolved-shortcode"] +--- + +A foreign shortcode: {{< hugoref foo >}} From 6590485392990148dc5440f651631c8ed4133c52 Mon Sep 17 00:00:00 2001 From: Carlos Scheidegger Date: Fri, 31 Jul 2026 17:48:29 -0500 Subject: [PATCH 7/8] Q-16-6: warn when block shortcode output is dropped in inline position (bd-u145dg3y) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A shortcode used mid-sentence whose handler returns block-level output (e.g. video's raw HTML block) used to vanish into an empty flatten with no trace. Flattening that loses ALL output now emits a source-mapped Q-16-6 warning naming the shortcode, with the fix hint (own paragraph, blank lines around it). Partial flattens stay silent — that's Q1's documented blocks_to_inlines behavior. Also adds claude-notes/instructions/q1-feature-porting.md — the process guide for future Q1→Q2 feature ports, distilled from this port (three-way study, two divergence levers, corpus-first TDD, diagnostics idiom, repo gotchas), plus plan checklist updates for phases 4/5. Full workspace green; cargo xtask verify (all 14 steps incl. WASM + hub-client) passed; cargo xtask lint clean. Co-Authored-By: Claude Fable 5 --- .../instructions/q1-feature-porting.md | 184 ++++++++++++++++++ .../2026-07-31-shortcode-extensions-port.md | 46 +++-- .../src/transforms/shortcode_resolve.rs | 22 +++ .../quarto-error-catalog/error_catalog.json | 7 + .../block-shortcode-inline-warns/test.qmd | 13 ++ 5 files changed, 254 insertions(+), 18 deletions(-) create mode 100644 claude-notes/instructions/q1-feature-porting.md create mode 100644 crates/quarto/tests/smoke-all/extensions/block-shortcode-inline-warns/test.qmd diff --git a/claude-notes/instructions/q1-feature-porting.md b/claude-notes/instructions/q1-feature-porting.md new file mode 100644 index 000000000..e4e2fd2d2 --- /dev/null +++ b/claude-notes/instructions/q1-feature-porting.md @@ -0,0 +1,184 @@ +# Porting a Quarto 1 feature to Quarto 2 + +A process guide, distilled from the shortcode-extensions port +(bd-540a976a, `claude-notes/plans/2026-07-31-shortcode-extensions-port.md`). +Follow it when the task is "study whether Q1 feature X exists in Q2 and +port what's missing." + +## The two levers (read this first) + +Q2 ports aim for direct compatibility, *except* where Q2's structural +advantages let us do better. Every deliberate divergence must be justified +by one of exactly two levers: + +1. **Strictness with actionable diagnostics.** Q1 often guessed what the + user meant because it couldn't point at source locations. Q2 can: + every AST node carries `SourceInfo`. Added strictness is acceptable + iff the failure produces a source-mapped, `Q-*`-coded, actionable + message. (Example: unknown shortcodes warn with location instead of + silently passing through.) +2. **Explicit declaration over inference.** Where Q1 inferred (engine + detection, pass-through-anything-unknown), Q2 asks the user to declare + (`engine:` metadata, `shortcode-passthrough: [names]`). An inference + the user can't see can't be diagnosed; a declaration can be validated. + +Everything else should behave like Q1, *including its documented quirks*. +Undocumented Q1 bugs (e.g. the shadowed `local result` in +`shortcodes.lua`) are candidates to drop — but record the decision. + +## Phase A — Study (before writing any plan) + +Run **three parallel studies** and only then reconcile: + +- **Q1 documented contract** — `external-sources/quarto-web`. What users + were promised. Collect: syntax, config keys, authoring workflow, escape + hatches, documented restrictions, and the changelog entries (they + encode contract details the prose never mentions, e.g. "must not crash + on bad input" fixes). +- **Q1 implementation** — `external-sources/quarto-cli`. What actually + happens: real data shapes, dispatch order, precedence rules, error + behavior, and *where* in Q1's architecture the feature runs (TS + pre-engine? Lua filter? postprocessor?). Get `file:line` references. +- **Q2 current state** — this repo. **It is rarely zero.** Check, in + order: `claude-notes/plans/` + `claude-notes/designs/` (prior epics + often did phases of this already), the braid skein (`braid list`, grep + `.braid/snapshot.jsonl`), then the crates. Existing open strands often + reframe the work from "port a feature" to "close gaps". + +The interesting deltas: +- **doc vs Q1 impl** → undocumented features (decide: port or drop) and + under-documented contract points (the impl is the contract); +- **Q1 impl vs Q2** → the gap list; +- **Q1 tests** → `external-sources/quarto-cli/tests/` encodes the + de-facto contract better than the docs. Port fixtures, don't reread + prose. + +**Spot-check the studies against the tree before trusting them.** If +subagents did the reading, grep the load-bearing claims yourself (does +that handler exist? is that CLI a stub? is that enum variant reachable?). + +**Ask which Q1 mechanisms were workarounds for infrastructure Q2 +replaced.** Q1 had four shortcode parsers only because Pandoc's reader +was in the way; Q2 parses in-grammar and the whole problem class (and +its documented restrictions, e.g. grid tables) evaporates. Don't port +mechanism-by-mechanism; port the contract. + +## Phase B — Plan document + +Write `claude-notes/plans/YYYY-MM-DD--port.md` with: + +1. **The Q1 contract, condensed** — what a user relying on the feature + gets to assume. Cite `file:line` for each claim. +2. **Gap table** — one row per contract item: Q2 status (✅/⚠️/❌), + precise gap, fix sketch. +3. **Design decisions** — every divergence, tagged `[decided]` or + `[needs user sign-off]`, each justified by lever 1 or 2. Flag scope + exclusions (defer big adjacent features to their own strands). +4. **Phased checklist**, Phase 0 always being the compatibility corpus. +5. **Related strands** (link with `braid dep add ... --type related`) and + prior art in claude-notes. + +Create a braid epic referencing the plan. Iterate with the user; get +explicit sign-off on the decision points *before* implementing. Record +their answers in the plan (`[decided YYYY-MM-DD]`) and in a braid +comment — sessions end; the plan file is the memory. + +## Phase C — Compatibility corpus (Phase 0 of every port) + +Encode the contract as **many small fixtures** before fixing anything: + +- Port Q1's own test fixtures (`tests/docs//`, smoke tests) — + copy into the repo, never reference `external-sources/` from tests. +- Author contract fixtures: one per contract item, uniquely-grepable + output tokens (`DASH-OK`, `META[deep-val]`). +- Include at least one **real published extension/document** as an + acceptance fixture, plus a real-world project if one is at hand + (connect-docs caught the extension-id≠shortcode-name bug that all + synthetic fixtures initially missed — realistic naming matters; + synthetic fixtures tend to accidentally mirror implementation + assumptions). +- Run the corpus. Sort failures into: (a) TDD targets for a phase — + keep the fixture uncommitted (or `tests.run.skip` with a reason + naming the strand) until its fix lands; (b) accepted deviations — + assert Q2's actual behavior and record the deviation in the plan; + (c) deferred gaps — park with `tests.run.skip` + reason. + +Never commit a red fixture without a skip reason. Never enshrine +known-bad behavior as a passing assertion. + +## Phase D — Implement, one gap at a time + +Strict TDD loop per gap (non-negotiable, per CLAUDE.md): failing test → +verify the failure is the expected one → fix → test green → full +`cargo nextest run --workspace` (monorepo: your pampa change breaks +quarto-core consumers; the meta-structuring change here broke video's +auto-stretch gate, caught only by the workspace suite). + +Commit in checkpoint-sized units (one theme per commit), message +explaining Q1 parity rationale and naming the strand. Update the plan +checkboxes and braid comments as you go. + +### Diagnostics idiom + +- New failure class → new `Q-*` code in + `crates/quarto-error-catalog/error_catalog.json` (`since_version: + "99.9.9"` placeholder). Claim a subsystem number deliberately + (extensions = 16). +- Emit via `DiagnosticMessageBuilder::warning(...).with_code("Q-16-N") + .problem(...).add_hint(...).with_location(source_info)`. +- **Misattribution is the enemy**: a failure at load/discovery time must + be reported *there*, naming the failing artifact — never left to + surface later as a confusing failure attributed to the user's document + (bd-nzdm1wry pattern). +- Advisory notices that must not break `--strict` or the smoke suite's + default assertion → `DiagnosticMessageBuilder::info` (info does not + trip `noErrorsOrWarnings`). + +### End-to-end verification (per CLAUDE.md, non-negotiable) + +Tests passing ≠ done. Render a real fixture through `cargo run --bin q2 +-- render`, grep the actual output, and record invocation + observed +output in the plan/transcript. If a real-world project exhibits the bug, +verify against it after the fix. + +## Repo gotchas (learned the hard way; will bite again) + +- **`include_dir!` staleness**: adding files under `resources/` does not + trigger a rebuild; `touch` the Rust file containing the macro (e.g. + `crates/quarto-core/src/extension/mod.rs`) or the new resource is + silently absent from the test binary. +- **`metadata-as-str` failure class**: bare YAML strings in + document-metadata context are `ConfigValueKind::PandocInlines`; + `as_str()` silently returns `None`. Use `as_plain_text()`. The lint + only catches some shapes — check every metadata read by hand. +- **Smoke fixture assertions are parsed as markdown**: an + `ensureFileRegexMatches` pattern containing `` triggers + "HTML element converted to raw HTML" warnings that fail the fixture's + own default assertion. Write patterns without `<` (e.g. + `strong>TOKEN`). +- **`noErrorsOrWarnings` is on by default** in smoke fixtures; + `printsMessage` alone does not suppress it — pair with `noErrors: + true`. The captured message text is the diagnostic *title*, not the + problem body. +- **`nextest` runs each test in its own process** — a `process::exit` + in library code shows up as an opaque test failure; also means + `std::env::set_var` in tests is race-free. +- **Interior state must live with the resource it describes**: a + loaded/initialized flag on a transform outlives the per-document + engine; wrap them together (`LuaEngineState`) so a fresh engine can't + see a stale flag. +- **Precedence is part of the contract**: registration/override order + (doc-level < extension < built-in; later file wins) must be chosen + deliberately, tested, and diagnosed when shadowing occurs. + +## Phase E — Wrap-up + +- Full `cargo xtask verify` (not just `--skip-hub-build`) when pampa / + quarto-core / quarto-pandoc-types changed — the WASM leg compiles a + different target and the hub-client vitest sweep runs the same + smoke-all fixtures. +- `cargo xtask lint`. +- Plan checkboxes current; braid comment summarizing commits, remaining + gaps, and decisions still pending; file follow-on strands + (`--deps discovered-from:`) for anything deferred. +- Do not push without explicit user approval. diff --git a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md index 530755902..d18a1e883 100644 --- a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md +++ b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md @@ -223,15 +223,18 @@ where the study only has static reads. lookup), `contract-nested-arg`. Parked with `tests.run.skip`: `contract-escape-comment` (grammar gap, decision pending — recommend targeted Q-2-x diagnostic over porting the Hugo `/* */` form). -- [ ] Integration tests driving the real binary path (`render_document_to_file` - / `q2 render` on fixtures) — not `HtmlRenderConfig::default()` shortcuts. -- [ ] Baseline probes (tests that *document* current behavior, marked - known-gap): shortcode in code block / attribute / link target / image src; - shortcode in YAML metadata values (title etc.); shortcode in grid table; - extension with missing `title`; extension whose Lua `require`s a sibling. -- [ ] Pick 2–3 real published Q1 shortcode extensions (e.g. `quarto-ext/fontawesome`, - `shafayetShafee/bsicons` class) and add them as fixtures; record what - breaks. These are the acceptance tests for the whole plan. +- [x] Integration tests drive the real render path (smoke-all runner via + quarto-test), plus manual `q2 render` e2e checks recorded per phase. +- [x] Baseline probes resolved by implementation instead of enshrined: + missing-title manifests and sibling-`require` now *work* (fixtures + q1-compat-minimal-manifest, contract-require); text-position probes + (code block / attribute / link target / image src / grid table / + metadata values) deferred with Phase 3 — still unwritten, revisit when + Phase 3 opens. +- [~] Real published extensions: `quarto-tiers` (Posit, real-world) verified + end-to-end via connect-docs (badge spans render, 0 warnings). Adding + copied fixtures of fontawesome-class extensions remains open — good + first item for a follow-up session. - [ ] Real-world acceptance target: `external-sources/connect-docs/docs-quarto-2` (Posit Connect docs). Known failure today: `{{< tier … >}}` from `_extensions/quarto-tiers/` → "Unknown shortcode" (gap row 3). Copy the @@ -327,15 +330,22 @@ All landed in commit 5af6fb18; full workspace green (10,819). ### Phase 5 — Unknown-shortcode policy + writer hardening (D1, gap row 9) -- [ ] Implement the D1 policy as signed off: `Q-*` code, `.with_location()`, - visible marker; passthrough config for foreign shortcodes. -- [ ] HTML writer: surviving `Inline::Shortcode` is never silently dropped — - emit marker + diagnostic (relates to orphaned `Q-3-30`/`Q-3-42` catalog - entries; wire or retire them). -- [ ] Backfill `.with_code()` on the existing uncoded warnings in - `shortcode_resolve.rs` (`:376`, `:398`, `:431`, extract sites). -- [ ] bd-u145dg3y: block-level shortcode used inline → coded warning - (absorbed here from deferred Phase 3). +- [x] D1 landed (commit 87644d04): Q-16-3 source-mapped warning + visible + marker; new `shortcode-passthrough: [names]` metadata key for declared + foreign shortcodes (verbatim literal, no warning) — explicit + declaration replacing Q1's pass-anything-unknown. Composes with + `--strict`. Fixture: contract-passthrough. +- [x] HTML writer renders `?name` + for surviving `Inline::Shortcode` instead of dropping (unit test). + Q-3-30/Q-3-42 left unreferenced deliberately — writers have no + diagnostics channel; noted in commit message. +- [x] `.with_code()` backfilled across shortcode_resolve (Q-16-2/3/5, + commits 5d004a3c + 5af6fb18). + (absorbed from deferred Phase 3): DONE, commit 4f2d2b50 — Q-16-6 when + flattening loses all output; partial flattens stay silent (Q1 parity). +- [ ] Decision pending (user): Q1's `{{}}` comment-escape form — + recommendation is a targeted Q-2-x parse diagnostic, not the syntax + (fixture parked via tests.run.skip). ### Phase 6 — Deferred / follow-on strands (file, don't implement here) diff --git a/crates/quarto-core/src/transforms/shortcode_resolve.rs b/crates/quarto-core/src/transforms/shortcode_resolve.rs index e72f1f975..883b6fb50 100644 --- a/crates/quarto-core/src/transforms/shortcode_resolve.rs +++ b/crates/quarto-core/src/transforms/shortcode_resolve.rs @@ -1564,8 +1564,30 @@ fn resolve_inlines<'a>( } ShortcodeResult::Blocks(blocks) => { // Graceful degradation: flatten blocks to inlines + // (Q1's blocks_to_inlines behavior). When flattening + // loses ALL the output (e.g. video's raw HTML block + // inside a sentence), that silent vanishing is + // undiagnosable — warn with the position named + // (bd-u145dg3y). let replacement = flatten_blocks_to_inlines(&blocks, &shortcode_owned.source_info); + if replacement.is_empty() && !blocks.is_empty() { + diagnostics.push( + DiagnosticMessageBuilder::warning( + "Block shortcode output dropped in inline position", + ) + .with_code("Q-16-6") + .problem(format!( + "The `{}` shortcode produced block-level output that cannot be placed in inline position; it was dropped", + shortcode_owned.name + )) + .add_hint( + "Put the shortcode in its own paragraph, surrounded by blank lines", + ) + .with_location(shortcode_owned.source_info.clone()) + .build(), + ); + } let replacement_len = replacement.len(); inlines.splice(i..=i, replacement); i += replacement_len.max(1); diff --git a/crates/quarto-error-catalog/error_catalog.json b/crates/quarto-error-catalog/error_catalog.json index b8defcf33..1a5d2261a 100644 --- a/crates/quarto-error-catalog/error_catalog.json +++ b/crates/quarto-error-catalog/error_catalog.json @@ -1097,5 +1097,12 @@ "message_template": "A value-lookup shortcode (`meta`, `var`, `env`) could not resolve its key: the metadata key is absent, the variable is not defined in `_variables.yml`, or the environment variable is unset and no fallback argument was given. The invocation renders as a visible `?name` marker. Check the key for typos, define it at the source the shortcode reads from, or (for `env`) pass a fallback: `{{< env NAME fallback >}}`.", "docs_url": "https://quarto.org/docs/errors/extension/Q-16-5", "since_version": "99.9.9" + }, + "Q-16-6": { + "subsystem": "extension", + "title": "Block Shortcode In Inline Position", + "message_template": "A shortcode used inline (inside a sentence or other inline content) produced block-level output (for example a raw HTML block from the `video` shortcode), and none of it could be carried into the inline position — the output was dropped. Put the shortcode in its own paragraph, surrounded by blank lines.", + "docs_url": "https://quarto.org/docs/errors/extension/Q-16-6", + "since_version": "99.9.9" } } diff --git a/crates/quarto/tests/smoke-all/extensions/block-shortcode-inline-warns/test.qmd b/crates/quarto/tests/smoke-all/extensions/block-shortcode-inline-warns/test.qmd new file mode 100644 index 000000000..0b4432902 --- /dev/null +++ b/crates/quarto/tests/smoke-all/extensions/block-shortcode-inline-warns/test.qmd @@ -0,0 +1,13 @@ +--- +title: Block-producing shortcode in inline position warns +format: html +_quarto: + tests: + html: + noErrors: true + printsMessage: + level: WARN + regex: "inline position" +--- + +Some text {{< video https://www.youtube.com/watch?v=dQw4w9WgXcQ >}} more text. From aca33bb4d562628d5bfc27d1bd54eb4d9cebfc97 Mon Sep 17 00:00:00 2001 From: Carlos Scheidegger Date: Fri, 31 Jul 2026 18:44:46 -0500 Subject: [PATCH 8/8] Defer Hugo comment-escape decision to bd-kmo1pzc2 (bd-540a976a) Per review: how Q2 does Hugo interop is not yet designed, so the {{}} escape form (and the related paired-shortcode question) moves to its own question strand rather than complicating this port. Parked fixture's skip reason now points at bd-kmo1pzc2. Co-Authored-By: Claude Fable 5 --- claude-notes/plans/2026-07-31-shortcode-extensions-port.md | 2 +- .../tests/smoke-all/extensions/contract-escape-comment/test.qmd | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md index d18a1e883..0757a9e40 100644 --- a/claude-notes/plans/2026-07-31-shortcode-extensions-port.md +++ b/claude-notes/plans/2026-07-31-shortcode-extensions-port.md @@ -343,7 +343,7 @@ All landed in commit 5af6fb18; full workspace green (10,819). commits 5d004a3c + 5af6fb18). (absorbed from deferred Phase 3): DONE, commit 4f2d2b50 — Q-16-6 when flattening loses all output; partial flattens stay silent (Q1 parity). -- [ ] Decision pending (user): Q1's `{{}}` comment-escape form — +- [x] Decided 2026-07-31: deferred to strand bd-kmo1pzc2 (Hugo interop unclear; recommendation is a targeted Q-2-x parse diagnostic, not the syntax (fixture parked via tests.run.skip). diff --git a/crates/quarto/tests/smoke-all/extensions/contract-escape-comment/test.qmd b/crates/quarto/tests/smoke-all/extensions/contract-escape-comment/test.qmd index af2a4252b..d5862b5d3 100644 --- a/crates/quarto/tests/smoke-all/extensions/contract-escape-comment/test.qmd +++ b/crates/quarto/tests/smoke-all/extensions/contract-escape-comment/test.qmd @@ -4,7 +4,7 @@ format: html _quarto: tests: run: - skip: "Known gap: Q1's {{}} comment-escape form is not in the qmd grammar (parse error). Pending decision on bd-540a976a (shortcode-extensions port, Phase 5); unskip when the grammar supports it." + skip: "Known gap: Q1's {{}} comment-escape form is not in the qmd grammar (parse error). Deferred to bd-kmo1pzc2 (Hugo interop question); unskip if that strand adds the escape form." html: noErrors: true ensureFileRegexMatches: