fix: restore Fern fallback live reload

[andreatgretel]

📋 Summary

The unauthenticated Fern fallback served source files through symlinks outside Fern's watched temporary tree, so documentation edits did not trigger live reload. This change keeps the deterministic NVIDIA-style fallback while mirroring source changes into real files that Fern can watch.

🔗 Related Issue

N/A

🔄 Changes

• Materialize the fallback preview tree with real files instead of source-pointing symlinks.
• Poll source metadata every 250 ms and mirror additions, changes, deletions, component aliases, and transformed docs.yml updates.
• Stop the Fern child process if synchronization fails instead of serving stale content.
• Add six regression tests covering materialization, atomic replacement, additions, deletions, configuration updates, component aliases, and process cleanup.
• Document deterministic fallback live reload behavior.

🔍 Attention Areas

⚠️ Reviewers: Please pay special attention to the following:

• serve-local-docs-preview.py - incremental synchronization and child-process lifecycle behavior.

🧪 Testing

• .venv/bin/ruff check . passes.
• .venv/bin/ruff format --check . passes.
• .venv/bin/pytest fern/scripts/tests/test_serve_local_docs_preview.py passes with 6 tests.
• make check-fern-docs passes with 0 errors.
• Local MDX, TSX, and docs.yml edits trigger Fern reloads and render updated content.
• Full E2E suite (N/A - change is isolated to the local Fern preview wrapper).

✅ Checklist

• Follows commit message conventions.
• Commits are signed off (DCO) - the existing pushed commit has no DCO trailer.
• Architecture docs updated (N/A - no architecture changes).

[github-actions]

Fern preview: https://nvidia-preview-pr-788.docs.buildwithfern.com/nemo/datadesigner

Fern previews include the docs-website version archive with PR changes synced into latest. Notebook tutorials are rendered without execution outputs in previews.

[greptile-apps]

Greptile Summary

This PR replaces the symlink-based Fern fallback preview tree with a materialized copy of source files, then adds incremental polling to mirror edits into the preview root every 250 ms so Fern's file watcher triggers live reloads correctly.

• serve-local-docs-preview.py: Introduces snapshot_source, sync_preview_root, and a polling loop inside run_command; extends component-alias handling and docs-config regeneration to work incrementally; kills the child process on unexpected synchronization failures instead of silently serving stale content.
• test_serve_local_docs_preview.py: Adds six focused regression tests covering initial materialization (no symlinks), file additions/deletions, transient copy failures, invalid docs.yml retry, component alias updates/removals, and child-process lifecycle on sync errors.
• fern/README.md and plans/788/local-preview-live-reload.md: Update troubleshooting notes and record the design rationale.

Confidence Score: 5/5

Safe to merge; the change is isolated to the local Fern preview wrapper and does not affect production build paths.

The polling-and-copy approach is self-contained, the state rollback logic for transient failures is correct, and the child-process teardown on unexpected sync errors is well-guarded. The six regression tests exercise every substantive code path introduced by the change.

No files require special attention.

Important Files Changed

Filename	Overview
fern/scripts/serve-local-docs-preview.py	Core change: symlinks replaced with real file copies plus a 250 ms incremental sync loop; state rollback on transient failures and process teardown on unexpected errors are correctly implemented.
fern/scripts/tests/test_serve_local_docs_preview.py	Six new regression tests cover materialization, sync round-trips, transient errors, invalid docs.yml retry, alias lifecycle, and process cleanup; all assertions are precise and test behaviour rather than implementation details.
fern/README.md	Minor documentation update: troubleshooting row updated to note that the fallback supports live reload; no functional impact.
plans/788/local-preview-live-reload.md	Design plan document tracking PR #788; all tasks checked off; no code impact.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Main as main()
    participant BPR as build_preview_root()
    participant SS as snapshot_source()
    participant RC as run_command()
    participant Fern as Fern child process
    participant SPR as sync_preview_root()

    Main->>SS: snapshot source tree
    SS-->>BPR: SourceState (mtime, inode, size)
    BPR->>BPR: copy all source files + write_local_docs_config
    BPR->>BPR: materialize component aliases
    BPR-->>Main: initial SourceState
    Main->>RC: run_command(command, preview_root, root, state)
    RC->>Fern: "Popen(command, cwd=preview_root)"
    loop every 250 ms (TimeoutExpired)
        RC->>SPR: sync_preview_root(root, preview_root, state)
        SPR->>SS: snapshot source tree
        SS-->>SPR: current SourceState
        SPR->>SPR: "remove deleted paths & aliases"
        SPR->>SPR: copy changed paths (or regenerate docs.yml)
        SPR->>SPR: update changed aliases
        SPR->>SPR: roll back failed paths in state
        SPR-->>RC: updated SourceState
    end
    Fern-->>RC: exit code
    RC-->>Main: exit code
    Note over RC,Fern: On KeyboardInterrupt → SIGINT → wait
    Note over RC,Fern: On unexpected Exception → SIGINT → wait → re-raise

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Main as main()
    participant BPR as build_preview_root()
    participant SS as snapshot_source()
    participant RC as run_command()
    participant Fern as Fern child process
    participant SPR as sync_preview_root()

    Main->>SS: snapshot source tree
    SS-->>BPR: SourceState (mtime, inode, size)
    BPR->>BPR: copy all source files + write_local_docs_config
    BPR->>BPR: materialize component aliases
    BPR-->>Main: initial SourceState
    Main->>RC: run_command(command, preview_root, root, state)
    RC->>Fern: "Popen(command, cwd=preview_root)"
    loop every 250 ms (TimeoutExpired)
        RC->>SPR: sync_preview_root(root, preview_root, state)
        SPR->>SS: snapshot source tree
        SS-->>SPR: current SourceState
        SPR->>SPR: "remove deleted paths & aliases"
        SPR->>SPR: copy changed paths (or regenerate docs.yml)
        SPR->>SPR: update changed aliases
        SPR->>SPR: roll back failed paths in state
        SPR-->>RC: updated SourceState
    end
    Fern-->>RC: exit code
    RC-->>Main: exit code
    Note over RC,Fern: On KeyboardInterrupt → SIGINT → wait
    Note over RC,Fern: On unexpected Exception → SIGINT → wait → re-raise

Loading

_{Reviews (6): Last reviewed commit: "Merge branch 'main' into andreatgretel/f..." | Re-trigger Greptile}

[github-actions]

Nice work on this one, @andreatgretel — the incremental-sync approach is a clean way to get live reload back without a file-watching dependency.

Summary

This restores live reload for the unauthenticated Fern fallback by materializing the preview tree as real files (instead of source-pointing symlinks) and polling the source tree every 250 ms to mirror additions, edits, deletions, docs.yml transforms, and component aliases into Fern's watched temp dir. It also stops the Fern child cleanly if a sync fails. The implementation matches the stated intent in the PR description and the plan file.

Findings

Suggestions — Take it or leave it

fern/scripts/serve-local-docs-preview.py:194 — Broad except Exception tears down the preview on any transient sync error

• What: Inside the poll loop only FileNotFoundError is retried (via failed_paths). Any other exception from sync_preview_root propagates to the except Exception handler, which SIGINTs the child and re-raises, killing the whole preview server.
• Why: FileNotFoundError is the most common editor-atomic-save race, but not the only one — some editors and filesystems can briefly surface PermissionError or partial-read OSError mid-rename. Since this is a long-running local dev server, one transient blip would tear down the session and force a manual restart. The plan documents "stop the child … rather than continuing with a stale preview" as intentional, so this is a judgment call, not a bug.
• Suggestion: Consider widening the per-file retry in sync_preview_root from except FileNotFoundError to except OSError so ordinary editor races are retried on the next pass, and reserve the hard teardown for genuinely unexpected errors. If the current strictness is deliberate, a one-line comment on the except Exception block noting "any sync failure is fatal by design" would help future readers.

fern/scripts/tests/test_serve_local_docs_preview.py — Atomic-save retry path isn't exercised

• What: The failed_paths branch (a file present at snapshot time but gone during copy_preview_file, then popped from the returned state so it retries next pass) is the mechanism that makes editor atomic-saves robust, but no test drives it.
• Why: It's the trickiest bit of the sync logic and the main motivation cited in the plan; a regression here would silently drop files from the preview until the next unrelated edit.
• Suggestion: A small test that patches copy_preview_file to raise FileNotFoundError once for a given path and asserts (a) sync_preview_root doesn't raise and (b) the failed path is absent from the returned state (so it's re-attempted) would lock this in.

fern/scripts/serve-local-docs-preview.py:103 — Component alias basename collisions are nondeterministic

• What: component_aliases maps source.with_suffix("") → source. If two components share a basename across extensions (e.g. Foo.ts and Foo.tsx), both map to alias Foo and the dict keeps whichever is iterated last.
• Why: Very unlikely in practice, but the resulting alias would be arbitrary.
• Suggestion: Low priority — worth at most a comment noting the assumption that component basenames are unique within a directory.

What Looks Good

• The tests assert not path.is_symlink() and that every materialized path resolves inside preview_root — a nice guard against both the old symlink behavior regressing and any path-escape bug.
• Removal handling is thoughtful: remove_preview_file prunes now-empty parent directories, and processing removals before additions makes the file↔directory type-swap cases fall out correctly.
• Child-process lifecycle is handled well — KeyboardInterrupt forwarding is preserved and the failure path is covered by a dedicated test asserting the SIGINT.
• The plan file is unusually complete (context, design rationale for why serving the raw tree isn't acceptable, explicit tasks, and a closed-out open-questions section).

Structural Impact (graphify, 2.3s)

Risk: LOW (localized change)

• 2 Python files, 0 AST entities, 0/81 clusters

The structural analysis confirms a localized change contained to the fern/ preview tooling with no cross-package or import-direction impact, consistent with the correctness-focused review above.

Verdict

Ship it (with nits) — no blocking issues. The suggestions above (especially widening the transient-error retry and adding a test for the failed_paths branch) would make the sync loop more resilient, but none are merge blockers.

---

This review was generated by an AI assistant.

[nabinchha]

Thanks for putting this together, @andreatgretel — the focused polling approach keeps the fallback self-contained and restores the expected local workflow.

Summary

This PR replaces source-pointing symlinks with a materialized Fern preview tree and incrementally mirrors source additions, edits, deletions, transformed docs.yml changes, and component aliases while the child process runs. The implementation matches the stated intent, but one race in the retry bookkeeping can leave a permanently deleted source file stale in the preview.

Findings

Warnings — Worth addressing

fern/scripts/serve-local-docs-preview.py:178 — A copy race can make deleted files persist indefinitely

• What: When a file is present in current_state but disappears before copy_preview_file reads it, the path is added to failed_paths and then removed from the returned state. The old preview copy is left in place. If the source file remains deleted, the next poll compares absent source state with absent tracked state, so it never schedules that stale preview file (or component alias) for removal. I reproduced this by deleting a changed file from the mocked copy call and running a second sync: the source remained absent while the preview still contained the old content.
• Why: This breaks the stated mirror semantics for exactly the snapshot/copy race this branch is intended to tolerate. Fern can keep serving a deleted page or component until the preview process is restarted. The current deletion test removes the source before the snapshot, so it does not cover this path.
• Suggestion: For a failed path that existed in previous_state, carry its previous state into the returned state so the next poll either retries it if it reappears or detects/removes it if it stays absent; only omit failed paths that were genuinely new. Please add a regression test that makes copy_preview_file unlink an existing changed source and raise FileNotFoundError, then runs another sync and verifies the stale preview file is removed.

What Looks Good

• The change stays isolated to the local fallback path and preserves the authenticated Fern workflow.
• The materialization test explicitly proves that preview files are real files and resolve inside the temporary root, which directly guards the original regression.
• The standard add/update/delete, transformed-config, alias lifecycle, and child-process failure paths are covered with focused tests, and the README accurately describes the new behavior.

Verdict

Needs changes — preserve prior tracking for failed copies and add the race regression test so a permanent deletion cannot leave stale preview content behind.

---

This review was generated by an AI assistant.

[andreatgretel]

Fixed. Failed copies for paths that existed in the previous snapshot now retain their previous state. If the source remains deleted, the next poll detects the removal and deletes the stale preview file and any component alias. Genuinely new failed paths remain untracked so they retry as additions. I added a regression test that deletes a changed source during copying, runs another sync, and verifies the stale preview file is removed.

[nabinchha]

Thanks for putting this together, @andreatgretel — nice detective work tracing the broken reload back to the source-pointing symlinks.

Summary

This restores live reload for the unauthenticated Fern fallback by materializing the preview tree as real files inside Fern's watched temp dir and polling the source every 250 ms to mirror additions, changes, deletions, docs.yml transforms, and component aliases. The implementation matches the stated intent, and the regression tests cover the interesting paths well.

Findings

Warnings — Worth addressing

fern/scripts/serve-local-docs-preview.py:64-87,144-166 — A transient/invalid docs.yml during editing kills the whole preview server

• What: sync_preview_root only catches FileNotFoundError around the copy/transform. But write_local_docs_config calls yaml.safe_load(...) on the source docs.yml, which can raise yaml.YAMLError when a save is momentarily malformed, or return None for an empty/partially-written file — after which config.pop("global-theme", None) raises AttributeError. Neither is a FileNotFoundError, so it propagates to run_command's except Exception, which SIGINTs the child and re-raises, tearing down the whole preview process.
• Why: The headline feature of this PR is docs.yml live reload. In practice, developers frequently hit save while docs.yml is temporarily invalid (a typo, mid-edit indentation), expecting to fix it and save again. Native Fern would surface a parse error and keep watching; here the server dies and the dev has to restart make serve-fern-docs-.... That's a worse experience than before for the exact file this PR is trying to support.
• Suggestion: Treat a failed docs.yml transform the same way you treat a vanished file — skip it this pass and retry on the next one, so a transient bad state self-heals once the file is valid again. For example, wrap the docs.yml branch so yaml.YAMLError (and the None/AttributeError case) are folded into failed_paths alongside FileNotFoundError:

except (FileNotFoundError, yaml.YAMLError, AttributeError):
    failed_paths.add(relative_path)

Keeping the "fail loud and stop the child" behavior for genuinely unexpected errors is reasonable, but transient invalid config while editing shouldn't be fatal.

Suggestions — Take it or leave it

fern/scripts/serve-local-docs-preview.py:186-193 — cwd vs root naming reads a bit backwards

• What: run_command(command, cwd, root, source_state) calls sync_preview_root(root, cwd, source_state), where cwd is the preview root and root is the source. It's correct, but the parameter names invite a mix-up on the next edit.
• Why: Minor readability/maintenance nit — the mapping of cwd → preview_root isn't obvious at the call site.
• Suggestion: Consider renaming the run_command param cwd to preview_root (the child still runs with cwd=preview_root), so both functions speak the same vocabulary.

fern/scripts/serve-local-docs-preview.py:90-100 — Non-atomic saves can cause a brief delete→re-add churn

• What: If an editor saves in place (truncate + rewrite) rather than atomically (rename), a poll landing in that window can miss the file and momentarily remove it from the preview tree, then re-copy it on the next pass — a double reload. The failed_paths retry only covers copy failures, not the snapshot-based removal path.
• Why: Cosmetic flicker/extra reload, not a correctness issue, and atomic-save editors avoid it entirely. Noting it in case someone reports "my page blinked."
• Suggestion: Fine to leave as-is for a local-only tool; if it ever becomes annoying, a small "confirm-still-missing across two consecutive polls before deleting" guard would smooth it out.

What Looks Good

• The failed_paths retry strategy is a clean way to ride out editor atomic-save races without a file-watching dependency, and it's directly exercised by test_sync_preview_root_removes_file_deleted_during_copy.
• Good, behavior-focused test coverage — materialization (including the no-symlink / no-escape-the-temp-root assertions), edits, adds, deletes, docs.yml regeneration, alias updates, and child-process lifecycle are all covered.
• Including st_ino alongside mtime/size in the snapshot is a nice touch for catching atomic-rename replacements that could otherwise share mtime/size.
• The plan doc and README troubleshooting update keep the "why" discoverable for the next person.

Verdict

Needs changes — one thing to address before merge:

• Don't let a transient/invalid docs.yml tear down the preview server (sync_preview_root / write_local_docs_config). The two suggestions are optional.

Nice work overall — this is a well-scoped fix with solid tests.

---

This review was generated by an AI assistant.

[andreatgretel]

Addressed in 68655393:

• Invalid or empty docs.yml edits now retain the last valid preview config and retry on the next poll.
• Transient per-file OSErrors are retried.
• Component alias collisions are deterministic and handle source fallback.
• Renamed cwd to preview_root.
• Added regression coverage for these cases.

Validation: 11 focused tests passed, Ruff passed, and Fern check reported 0 errors.

I left the optional two-poll deletion debounce unchanged because it only avoids cosmetic reload churn while delaying legitimate deletions.

[nabinchha]

Follow-up pass on the latest changes, @andreatgretel — just a couple of things left.

Warnings — Worth addressing

fern/scripts/serve-local-docs-preview.py:92 — snapshot_source only tolerates FileNotFoundError

• is_file() / stat() can raise other OSError subclasses (PermissionError, too-many-symlink-levels, etc.). Those propagate out and, via run_command's broad except Exception (line 203), SIGINT the child and tear down the whole preview. Since the rest of the sync path is deliberately tolerant (failed_paths + retry-next-pass), one odd file under fern/ killing a live session is inconsistent. Suggest broadening to except OSError: so the file is skipped for that pass.

Commits are not DCO signed-off — the checklist flags this; if DCO is enforced in CI it'll block merge. git rebase --signoff before merge.

Suggestions — Take it or leave it

• :117 — preview_root / (target or source) relies on Path truthiness; source if target is None else target states the intent more clearly.
• :94 — full-tree rglob + stat every 250 ms is fine for the current tree and documented as intentional; just worth revisiting if generated notebooks grow large. No action needed.

Verdict

Needs changes — no happy-path correctness issues; tightening the snapshot_source exception scope and adding DCO sign-off are both quick and worth doing before merge.

---

This review was generated by an AI assistant.

[andreatgretel]

Addressed in 2e005dc0. snapshot_source now skips any OSError (with a regression test), and copy_preview_file spells out the None check instead of relying on Path truthiness.

[nabinchha]

Both findings from the prior review are resolved — snapshot_source now catches OSError (with a new test_snapshot_source_skips_unreadable_file regression test), and copy_preview_file uses the explicit source if target is None else target form. Ruff is clean and all 12 tests pass locally.

Approving on the code. Nice work on this one.

---

This review was generated by an AI assistant.