Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/jekyll-gh-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ jobs:
- name: Render book PDF
run: |
mkdir -p _pdf
node render-book.mjs _site-pdf/book.html -o "_pdf/twinBASIC Book.pdf" --outline-tags h1,h2,h3,h4 --additional-script ../perf/detach-pages.js
node ../book/render-book.mjs _site-pdf/book.html -o "_pdf/twinBASIC Book.pdf" --outline-tags h1,h2,h3,h4 --additional-script ../perf/detach-pages.js
working-directory: ./docs
- name: Upload Pages artifact
uses: actions/upload-pages-artifact@v5
Expand Down
10 changes: 4 additions & 6 deletions WIP.md
Original file line number Diff line number Diff line change
Expand Up @@ -432,17 +432,15 @@ Historical engineering notes from the Jekyll era --- the original build pipeline

## Build / preview

From `docs/`:

- `build.bat` — runs `node ..\builder\tbdocs.mjs --src .` which produces three trees in one pass: the online copy at `_site/`, a `file://`-browsable copy at `_site-offline/`, and the sparse pagedjs source at `_site-pdf/`. The offline pass adds ~700 ms and the PDF pass adds ~150 ms on top of the ~2 s online build. Toggle `also_build_offline` / `also_build_pdf` in `_config.yml` (or pass `--no-offline` / `--no-pdf`) to skip a sibling output.
- `serve.bat` — runs `tbdocs --serve`: initial build, then a long-lived process with watcher, debounced rebuilds, and SSE-driven browser auto-reload. Ctrl+C to stop.
- `build.bat` — runs `node builder\tbdocs.mjs --src docs` which produces three trees in one pass: the online copy at `_site/`, a `file://`-browsable copy at `_site-offline/`, and the sparse pagedjs source at `_site-pdf/`. The offline pass adds ~700 ms and the PDF pass adds ~150 ms on top of the ~2 s online build. Toggle `also_build_offline` / `also_build_pdf` in `_config.yml` (or pass `--no-offline` / `--no-pdf`) to skip a sibling output.
- `serve.bat` — runs `tbdocs --serve`: initial build, then a long-lived process with watcher, debounced rebuilds, and SSE-driven browser auto-reload. Writes to `docs/_serve/` (disjoint from `build.bat`'s `_site*/`) and skips the offline + PDF passes — so a one-off `build.bat` for the PDF or offline mirror doesn't disturb the live preview. Ctrl+C to stop.
- `check.bat` — link + integrity check (offline `scripts/check_links.mjs` against `_site/` and `_site-offline/`; the offline pass also runs `--forbid 'https://docs.twinbasic.com'` to catch surviving live-site links).
- `book.bat` — renders the PDF from `_site-pdf/book.html` via `node render-book.mjs` into `_pdf/book.pdf`. Run `build.bat` first to populate `_site-pdf/`.
- `book.bat` — renders the PDF from `docs\_site-pdf\book.html` via `node book\render-book.mjs` into `docs\_pdf\book.pdf`. Run `build.bat` first to populate `_site-pdf/`.


## Site integrity check

After a batch of changes, verify the site builds clean and all links resolve. From the `docs/` folder, run:
After a batch of changes, verify the site builds clean and all links resolve:

```sh
build.bat && check.bat
Expand Down
21 changes: 11 additions & 10 deletions docs/book.bat → book.bat
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
@echo off
rem PDF render only. Run build.bat (or `bundle exec jekyll build`) first
rem so _site-pdf\book.html and its dependencies exist; this script
rem assumes the Pdfify plugin has already populated _site-pdf\.
@pushd "%~dp0"
rem PDF render only. Run build.bat first so docs\_site-pdf\book.html and
rem its dependencies exist.
rem
rem render-book.mjs drives puppeteer + paged.js + pdf-lib directly so
rem we control pdf-lib's parseSpeed (the default yields the event loop
Expand All @@ -10,19 +10,20 @@ rem for no reason in Node -- see perf\README.md "Profiling pdf-lib's
rem load" for the full diagnosis). pagedjs-cli passed no options to
rem load/save and inherited that cost; we don't.
rem
rem --additional-script ..\perf\detach-pages.js injects a Paged.Handler
rem --additional-script perf\detach-pages.js injects a Paged.Handler
rem that hides each finalised page from Chromium's layout tree and
rem restores them all before page.pdf() runs. Drops total render from
rem ~104s to ~51s on the 1638-page book by eliminating the O(n^2)
rem getBoundingClientRect cost in paged.js's overflow walker.
if not exist _site-pdf\book.html (
echo _site-pdf\book.html not found. Run build.bat first.
if not exist docs\_site-pdf\book.html (
echo docs\_site-pdf\book.html not found. Run build.bat first.
exit /b 1
)
if not exist ..\node_modules\puppeteer\package.json (
if not exist node_modules\puppeteer\package.json (
echo Installing dependencies...
pushd .. && call npm install && popd
call npm install
if errorlevel 1 exit /b 1
)
if not exist _pdf mkdir _pdf
node render-book.mjs _site-pdf\book.html -o "_pdf\twinBASIC Book.pdf" --outline-tags h1,h2,h3,h4 --additional-script ..\perf\detach-pages.js
if not exist docs\_pdf mkdir docs\_pdf
node book\render-book.mjs docs\_site-pdf\book.html -o "docs\_pdf\twinBASIC Book.pdf" --outline-tags h1,h2,h3,h4 --additional-script perf\detach-pages.js
@popd
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion docs/render-book.mjs → book/render-book.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ import { PDFDocument } from 'pdf-lib';
// heap traffic drops ~13 % (123 MB -> 107 MB). PDFNumber is
// immutable so sharing is safe.
// measure-pass (Phase 1) -- no-allocate byte walker
// (docs/lib/measure-pass.mjs) that runs in front of
// (lib/measure-pass.mjs) that runs in front of
// PDFDocument.load on the raw Chrome PDF and counts dictSlots
// + arraySlots. The counts drive setExpectedDictSlots() on
// fast-dict-onebuf and setExpectedArraySlots() on
Expand Down
3 changes: 3 additions & 0 deletions build.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
@pushd "%~dp0"
node builder\tbdocs.mjs --src docs %*
@popd
53 changes: 33 additions & 20 deletions builder/PLAN-1.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,16 +58,23 @@ project's `_config.yml` `exclude:` list, and tbdocs-specific opt-outs.
| `*.bat` | Per `_config.yml exclude:` — `build.bat`, `serve.bat`, `check.bat`, `book.bat`, `profile-rbspy.bat`, `profile-rubyprof.bat`. |
| `redirects.json` | Per `_config.yml exclude:`. Note: doesn't currently exist in the tree, but the entry is there to be defensive against future regressions. |

**Excluded directories — tbdocs-specific:**
**Excluded patterns — tbdocs-specific:**

| Path | Why |
| Pattern | Why |
|---|---|
| `assets/css/` | Theme CSS / SCSS. Jekyll currently runs Liquid + Sass over `just-the-docs-combined.scss` and `just-the-docs-head-nav.css`, copies `print.css` / `rouge.css` verbatim. tbdocs takes the four compiled files from `builder/assets/css/` instead (per PLAN.md "Static Asset Extraction"). Phase 5 handles the copy. Skipping in Phase 1 prevents the source tree from shadowing the prebuilt artifacts. |
| `assets/js/` | Same story. The single `theme-switch.js` file in the source comes through pre-built from `builder/assets/js/` together with the just-the-docs runtime and the lunr vendor bundle. |
| `**/*.scss` | SCSS sources fed into [`scss.mjs`](scss.mjs) (the only entry point currently lives at `docs/assets/css/just-the-docs-combined.scss`, with partials under `docs/_sass/custom/`). The compiled output is emitted on `generatedAssets` and written by Phase 5's `writeGeneratedAssets`, so letting Phase 1 enumerate the source `.scss` would shadow the generator output. |
| `**/*.mmd` | Mermaid diagram sources fed into [`mermaid.mjs`](mermaid.mjs)'s preprocessor. The `.svg` siblings are kept --- content pages reference those. |

`assets/images/` is **not** excluded — it carries the mermaid SVG renders
(`assets/images/mmd/*.svg`) that content pages reference, and the
`favicon.png` should keep landing at the same URL.
`assets/css/` and `assets/js/` themselves are **not** excluded any more ---
the project-owned theme files now live there (`assets/css/print.css`,
`assets/css/just-the-docs-head-nav.css`, `assets/js/theme-switch.js`) and
ride the normal static-file copy pipeline into `_site/`. Vendored
just-the-docs JS lives under `builder/vendor/just-the-docs/assets/` and
is copied separately by Phase 5's `copyTheme`.

`assets/images/` is also not excluded --- it carries the rendered mermaid
SVGs (`assets/images/mmd/*.svg`) referenced by content pages, plus the
favicon and any content images.

### Assumption: the exclude list is complete

Expand Down Expand Up @@ -291,8 +298,8 @@ export async function discover(srcRoot) {
"**/_*/**", // …and at any depth (catches _Images).
"**/.git/**",
"**/node_modules/**",
"assets/css/**",
"assets/js/**",
"**/*.scss", // SCSS sources -- compiled by scss.mjs.
"**/*.mmd", // Mermaid sources -- compiled by mermaid.mjs.
// Top-level Jekyll/toolchain files:
"Gemfile",
"Gemfile.lock",
Expand Down Expand Up @@ -436,17 +443,23 @@ Per the user's choice. Reasons:
The orchestrator (`tbdocs.mjs`) is responsible for invoking `book.mjs`
with the source root so it can read `_data/book.yml` itself.

### D5. `assets/css/` and `assets/js/` are excluded

Per PLAN.md "Static Asset Extraction": the production CSS / JS lives
prebuilt in `builder/assets/`. Letting Phase 1 also enumerate the source
versions would force Phase 5 to disambiguate which copy wins — better to
make the source/prebuilt split a Phase 1 concern.

`assets/images/` is **not** excluded. It carries the mermaid SVG renders
(`assets/images/mmd/*.svg`) referenced by content pages, plus is where
new content images would naturally land. Phase 5 copies these via the
normal static-file pipeline.
### D5. Source-vs-output split is by extension, not by directory

`**/*.scss` and `**/*.mmd` are excluded because they're inputs to
[`scss.mjs`](scss.mjs) and [`mermaid.mjs`](mermaid.mjs) respectively ---
both run before Phase 1 and write either to `generatedAssets` (CSS) or
back under `srcRoot` (the `.svg` siblings the Mermaid preprocessor
emits). Letting Phase 1 enumerate the `.scss` sources would race the
generator output at write time; letting it enumerate the `.mmd` sources
would publish a non-deployable artifact.

The broader `assets/` tree is **not** excluded. Project-owned theme files
(`assets/css/print.css`, `assets/css/just-the-docs-head-nav.css`,
`assets/js/theme-switch.js`) live there and ride the normal static-file
copy pipeline; `assets/images/` carries content images plus the rendered
mermaid SVGs. The vendored just-the-docs runtime JS lives outside
`docs/` under `builder/vendor/just-the-docs/assets/` and Phase 5's
`copyTheme` carries it across separately.

**Risk:** if someone adds a new content asset under `assets/css/` or
`assets/js/` (an inline icon, a tiny utility script), Phase 1 will drop
Expand Down
59 changes: 39 additions & 20 deletions builder/PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,8 +113,14 @@ It reads from `docs/` and writes to `docs/_site/` / `docs/_site-offline/`
/ `docs/_site-pdf/` -- the same destinations Jekyll uses, so deployment
tooling (GitHub Pages serving from `/docs/`) stays unchanged.

Static assets (CSS, JS, SVGs) extracted once from the current Jekyll build live in
`builder/assets/` and get copied verbatim to `docs/_site/assets/` on each build.
Site assets are assembled from three sources: project content under `docs/assets/`
(SCSS entry point, hand-written CSS, project JS, content images, Mermaid diagrams);
vendored just-the-docs bits under `builder/vendor/just-the-docs/` (`_sass/` compiled
on every build by [`scss.mjs`](scss.mjs); `assets/js/just-the-docs.js` + `vendor/lunr.min.js`
copied verbatim by [`write.mjs`](write.mjs)); and generated-in-process artifacts
(`just-the-docs-combined.css` from `scss.mjs`, `tb-highlight.css` from
[`highlight-theme.mjs`](highlight-theme.mjs)). See the **Asset layout** section below
for the full breakdown.

## Dependencies

Expand Down Expand Up @@ -169,12 +175,12 @@ with no perf budget set yet.

Input: the `docs/` source tree. Excluded: all `_*` directories (catches
`_site/`, `_site-offline/`, `_site-pdf/`, `_data/`, `_includes/`, `_layouts/`,
`_sass/`, `_plugins/`, `_profile/`, and every `_Images/`), `assets/css/` and
`assets/js/` (theme assets, sourced from `builder/assets/` instead),
top-level Jekyll/toolchain files (`_config.yml`, `Gemfile`, `Gemfile.lock`,
`*.bat`), and `.jekyll-cache` / `.sass-cache` / `node_modules`. The builder
itself lives at `../builder/` (outside `docs/`) and isn't part of the
source tree.
`_sass/`, `_plugins/`, `_profile/`, and every `_Images/`), SCSS sources
(`**/*.scss`, compiled separately by [`scss.mjs`](scss.mjs)), Mermaid sources
(`**/*.mmd`, the `.svg` siblings are kept), top-level Jekyll/toolchain files
(`_config.yml`, `Gemfile`, `Gemfile.lock`, `*.bat`), and `.jekyll-cache` /
`.sass-cache` / `node_modules`. The builder itself lives at `../builder/`
(outside `docs/`) and isn't part of the source tree.

Output: `{ pages, staticFiles }`.

Expand Down Expand Up @@ -327,10 +333,17 @@ block from `page.navLevels` -- positional `:nth-child()` selectors.
### Phase 5: WRITE ONLINE (`write.mjs`)

- For each page: write destPath to `_site/`
- Copy theme assets: `builder/assets/` -> `docs/_site/assets/` (CSS, JS, sprites)
- Copy vendored theme JS: `builder/vendor/just-the-docs/assets/` ->
`docs/_site/assets/` (`just-the-docs.js` + `vendor/lunr.min.js`)
- Copy every entry in `staticFiles[]` (from Phase 1) to its `destRel` under
`_site/` -- content images, `favicon.png`, `CNAME`, `render-book.mjs`,
`lib/*.mjs`, `assets/images/mmd/*`
`_site/` -- project-owned theme files (`assets/css/print.css`,
`assets/css/just-the-docs-head-nav.css`, `assets/js/theme-switch.js`),
content images, `favicon.png`, `CNAME`, `render-book.mjs`, `lib/*.mjs`,
`assets/images/mmd/*.svg`
- After the parallel batch, write `generatedAssets[]` --- the SCSS-compiled
`just-the-docs-combined.css` and the highlight-theme `tb-highlight.css`.
CSS files in any of the three paths get a baseurl rewrite when the deployment
baseurl is non-empty

Pure filesystem I/O -- no transformation, no URL rewriting. Phase 6
adds `search-data.json`, redirects, sitemap, and robots.txt alongside
Expand Down Expand Up @@ -452,18 +465,24 @@ discipline of Phases 3-9 had deferred. All five PRs landed:
B6 (linkify) and B18 (streaming book.html write) were dropped.
Full spec: [PLAN-11.md](PLAN-11.md).

## Static Asset Extraction (One-Time Setup)
## Asset layout

Before the first build, extract from the current Jekyll output:
`_site/assets/` is assembled from three sources at build time --- nothing is
extracted out of a Jekyll build any more.

1. **CSS** -- `_site/assets/css/just-the-docs-combined.css` (compiled theme with custom
colors baked in), `just-the-docs-head-nav.css`, `print.css`, `rouge.css`
2. **JS** -- `_site/assets/js/just-the-docs.js`, `vendor/lunr.min.js`
3. **SVG sprites** -- the `<svg>` defs block from any rendered page
4. **Favicon** -- `favicon.png`
| Source on disk | What lives there | Phase that delivers it |
|---|---|---|
| `docs/assets/` | Project-owned content: the SCSS entry point (`css/just-the-docs-combined.scss`, excluded from copy, fed into Sass), project JS (`js/theme-switch.js`), hand-written stylesheets (`css/print.css`, `css/just-the-docs-head-nav.css`), Mermaid diagrams (`.mmd` sources excluded, `.svg` renders included), content images. | Discovered by Phase 1, copied by Phase 5's `copyStaticFiles`. |
| `builder/vendor/just-the-docs/` | Vendored from the just-the-docs gem (v0.10.1): `_sass/` (theme SCSS sources, fed into the compilation); `assets/js/just-the-docs.js` + `assets/js/vendor/lunr.min.js` (chrome runtime, copied verbatim, `just-the-docs.js` patched in tree). See [`builder/vendor/just-the-docs/README.md`](vendor/just-the-docs/README.md). | `_sass/` consumed by [`scss.mjs`](scss.mjs); `assets/` copied by Phase 5's `copyTheme`. |
| Generated in-process | `just-the-docs-combined.css` from [`scss.mjs`](scss.mjs); `tb-highlight.css` from [`highlight-theme.mjs`](highlight-theme.mjs). Neither is committed; both are rebuilt every run. | Pushed onto `generatedAssets`; written by Phase 5's `writeGeneratedAssets` after the parallel copy batch. |

These live in `builder/assets/` and get copied verbatim to `docs/_site/assets/` on each build.
If the custom color scheme ever changes, recompile once manually with `sass`.
CSS files in any of the three paths run through a baseurl rewrite
(`url("/path")` → `url("<baseurl>/path")`) when the deployment baseurl is
non-empty.

Bumping the just-the-docs gem version is a re-vendor of `_sass/` and
`assets/js/` from the new tag plus re-applying the `just-the-docs.js`
patches; procedure in the vendor README.

## What Doesn't Get Ported

Expand Down
Loading