From 8d7d8f5501b4e24015c3c7734af65f13dbc81d55 Mon Sep 17 00:00:00 2001 From: Miguel Angel Simon Sierra Date: Mon, 24 Aug 2026 20:16:45 -0400 Subject: [PATCH] =?UTF-8?q?feat(media-use):=20paint=20=E2=80=94=20compile?= =?UTF-8?q?=20images=20into=20brushstroke-code=20compositions?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a media-use capability that compiles an image (or one frame of a video) into a standalone HyperFrames composition whose generated code repaints the subject in brushstrokes. The artifact is the code: the stroke list ships as an editable data module (strokes.js), and the inlined runtime reveals it over the timeline. - scripts/paint.mjs: CLI entry. Decodes via ffmpeg rawvideo (images and video frames), compiles, emits, prints a JSON summary. Deterministic: same input + seed produces byte-identical stroke data. - scripts/lib/paint-compiler.mjs: pure compiler. Sobel flow field, circular smoothing, seeded k-means palette with saturation restore, layered jittered-grid placement coarse to fine, variance-gated detail layers. low/medium/high presets calibrated on The Great Wave off Kanagawa, Girl with a Pearl Earring, and The Scream. - scripts/lib/paint-emit.mjs: emits the composition. The painter runtime is inlined so static analysis sees the timeline registration; the reveal is seek-safe by construction (band canvases are pure functions of their band index; per-frame cost is O(strokes/bands)). - references/paint.md: capability doc (when to reach for it, invocation, output layout, mounting, editing the art). - SKILL.md: routing row. The emitted composition passes hyperframes check with zero findings. Tests: 6 node:test cases covering compile determinism, geometry bounds, the emit contract, emit determinism, and an end-to-end run against a real fixture image including a same-seed byte-equality check. --- skills-manifest.json | 4 +- skills/media-use/SKILL.md | 1 + skills/media-use/references/paint.md | 69 +++ .../media-use/scripts/lib/paint-compiler.mjs | 553 ++++++++++++++++++ skills/media-use/scripts/lib/paint-emit.mjs | 182 ++++++ skills/media-use/scripts/paint.mjs | 182 ++++++ skills/media-use/scripts/paint.test.mjs | 166 ++++++ 7 files changed, 1155 insertions(+), 2 deletions(-) create mode 100644 skills/media-use/references/paint.md create mode 100644 skills/media-use/scripts/lib/paint-compiler.mjs create mode 100644 skills/media-use/scripts/lib/paint-emit.mjs create mode 100644 skills/media-use/scripts/paint.mjs create mode 100644 skills/media-use/scripts/paint.test.mjs diff --git a/skills-manifest.json b/skills-manifest.json index e3860f309f..32c59f25c4 100644 --- a/skills-manifest.json +++ b/skills-manifest.json @@ -50,8 +50,8 @@ "files": 12 }, "media-use": { - "hash": "1b0ce647f5c7df95", - "files": 152 + "hash": "15f2c5759218773a", + "files": 157 }, "motion-graphics": { "hash": "69dc088b8e0d22fe", diff --git a/skills/media-use/SKILL.md b/skills/media-use/SKILL.md index 5655ccb439..7a27eb7fad 100644 --- a/skills/media-use/SKILL.md +++ b/skills/media-use/SKILL.md @@ -90,6 +90,7 @@ Rules that keep this a help, not nagware: **grounded, not generic** (no signal | color grading, LUTs, smart grade (`--for`), grade-compare | `references/grading.md` | | voiceover / TTS, music, SFX, captions, transcription (audio engine) | `references/audio.md` | | cut / reframe / transform existing media, exact error diffusion, HEVC | `references/operations.md` | +| paint an image (or video frame) as brushstroke code (image-to-code) | `references/paint.md` | | source-aware creative treatments, realtime effects, overlays, reveals | `references/media-treatments.md` | | install + auth, provider table, RAM ladders, `--local-only`, `--provider` | `references/setup-providers.md` | | remembered preferences + frozen recipes (user memory) | `references/memory.md` | diff --git a/skills/media-use/references/paint.md b/skills/media-use/references/paint.md new file mode 100644 index 0000000000..f9e9ea7790 --- /dev/null +++ b/skills/media-use/references/paint.md @@ -0,0 +1,69 @@ +# Paint — image-to-code compositions + +`paint` compiles an image (or one frame of a video) into a standalone +HyperFrames composition whose **generated code repaints the subject in +brushstrokes**. The artifact is the code: the stroke list ships as an editable +data module (`strokes.js`), and the runtime reveals it over the timeline. +Deterministic: same input + seed produces byte-identical output. + +## When to reach for it + +- A composition needs a painted, hand-marked look derived from a real image + (a portrait, a logo, a product shot, a frame of footage). +- The brief says "paint this", "make it look brush-painted", or asks for the + image to draw itself on screen. +- You want the image-to-code thesis artifact: code the user can edit. + +## Run it + +```bash +node /scripts/paint.mjs --input photo.jpg --out paint-photo [options] +``` + +| Option | Default | Meaning | +| ---------------------- | -------------- | ----------------------------------------- | +| `--input ` | required | source image or video (one frame sampled) | +| `--out ` | `paint-` | output folder | +| `--seed ` | `1337` | stroke randomness seed | +| `--width ` | `1100` | emitted canvas width | +| `--duration ` | `12` | reveal duration in seconds | +| `--detail ` | `medium` | `low` / `medium` / `high` stroke budget | +| `--video-position ` | `0` | frame sample time for video inputs | +| `--json` | off | machine-readable summary | + +Requires `ffmpeg`/`ffprobe` on PATH (the same decode path as the rest of +media-use's operations). + +## Output + +``` +paint-photo/ + index.html standalone composition (contract-correct root, clip, timeline) + strokes.js editable stroke data — window.__PAINT_STROKES = [...] +``` + +The painter runtime is inlined in `index.html`. It reveals strokes in array +order over `--duration` seconds and is **seek-safe by construction**: band +canvases are pure functions of their index, so any seek order lands on +identical pixels. `hyperframes check` passes on the emitted folder as-is. + +## Using the output + +- **As its own composition**: point `check` / `preview` / `render` at the + folder, or copy it into a project's `compositions/` and mount it. +- **Inside an existing composition**: mount via + `data-composition-src` (wrap the emitted root in `