feat(skills,studio,media-use): the review loop and user memory — sketch-first review, per-frame board comments, remembered defaults, recipes#2133
Conversation
…d, status-aware preview landing Per-frame comment boxes on the storyboard board batch into .hyperframes/frame-comments.json (a resubmit wins per frame; unconsumed comments on other frames are kept). Submitted-but-unconsumed comments stay visible — a toolbar banner plus a per-tile echo — until the agent consumes the file; the banner also says what to do next (reply anything in the agent chat). The board keeps itself current: GET /projects/:id/signature exposes the watcher-cached project signature, the storyboard payload carries the signature it was derived from, and the view polls at 2s (hidden tabs skipped, re-checked on visibility), refetching in place with no loading flash. Posters bake the signature into their URL so tiles fill in as sketches land and a poster that failed mid-write retries on the next version; the empty state upgrades itself when STORYBOARD.md appears, and its handoff prompt now points the agent at the review loop and uses the parser's real status vocabulary (outline, not planned). preview lands the browser on the storyboard view while the board is the review surface — any frame built, or pure planning (srcs declared, none on disk yet) — and on the timeline once the video is assembled.
…cess hyperframes-core/references/review-loop.md is the single source for the three-pass collaborative review: the plan proposed on a live board (§ 1), wireframe sketches marked built with one layout question (§ 2 — real words on plain blocks, run no CLI; a confirmed board is itself a valid deliverable when the user asked for a storyboard, not a video), the build dressing confirmed layouts (§ 3, worker or inline), and the final look (§ 4). Autonomous runs skip every gate and keep one question before render. The three narrative workflows' Steps 3/4/6 collapse to references plus their sketch stand-ins (captured-asset blocks for product-launch-video, plain code panels for pr-to-video); the confirmed-sketch handoff stays in each frame-worker prompt. general-video plans on a board for multi-scene narrative pieces in collaborative mode — its sketch pass is layout-before-animation with the user watching. The router treats "I want a storyboard" as a process request rather than a route, and closes exploratory intake by recommending a route plus how the run will review. The supporting contracts land next door: the comments channel (silent submit, one reply picks it up, check the file before the words) in brief-contract § 1; the sidecar schema and the built status rung in storyboard-format; the mode question asked first and alone in the three workflows' Step 0.
Two tiers of memory on media-use's existing two-tier storage split. Preferences (lightweight): confirmed brief answers — destination, aspect, language, mode, voice, style preset — recorded to the project's .media/preferences.json (committed, the team inherits it) and promoted to the personal ~/.media/preferences.json once the same value is confirmed in two different projects (a sightings ledger accumulates the cross-project evidence user-side, since project files can't see each other). prefs.mjs get/record; merge reads project-over-user; a changed value restarts its provenance. Recipes (heavyweight): one approved run frozen as a named, versioned bundle — frame.md, the storyboard skeleton (structure kept: durations, transitions, srcs, Video direction; statuses reset to outline; content blanked to per-frame fill-ins naming the beat's role), and the confirmed brief values. Named folders, not content hashes: re-freezing bumps version and archives <name>@v<N>; a freeze is already confirmed, so it promotes to the user tier immediately. recipe.mjs freeze/list/use, plus resolve --type recipe --entity <name> delegating like grade/lut. 16 new node --test cases; the media-use lib suite is 168/168.
brief-contract § 2 gains Remembered defaults: read the merged preferences before Round 2 and let a remembered value become the recommended option with a receipt naming its source project. Memory changes the default, never the question — every ask-marked field still gets asked, and what the request says this time beats what was picked last time. Record only what the user actually confirmed (a defaulted voice nobody chose is not an answer; a "go" that accepts the recommended defaults is). The first record announces itself once; after that the receipts carry the reminder. In autonomous mode a remembered value becomes the decided value, receipt included. The three narrative workflows read the remembered defaults before Round 2, record the confirmed answers at the Step 0 gate, record the chosen preset at the Step 2 gate (pr-to-video excepted — its preset is fixed), and fall back to the remembered voice when the request names none. general-video's discovery reads the same defaults. Recipes wire in at both ends: Step 0 checks for a matching recipe before the mode question — one question, plural-aware, and adopting one fills the brief, skips the design step, and drafts the storyboard from the frozen skeleton while every review gate still runs. The review loop's final look (§ 4) offers the freeze once after approval, and the confirmation teaches the recall phrase — the name is something the system reminds the user of, never something they must remember. The router recognizes a named recipe or "like last time" as a route.
miga-heygen
left a comment
miga-heygen
left a comment
There was a problem hiding this comment.
Review — feat(skills,studio,media-use): the review loop and user memory
Big feature, well-structured. Six interleaved pieces that compose into the three-pass review flow + user memory. I'll take them in the order they build on each other.
1. Project signature polling (studio-server + studio)
resolveProjectSignature extracted from preview.ts into projectSignature.ts and reused by preview, projects, and storyboard routes — good SSOT improvement, was inline before.
New /projects/:id/signature endpoint is a cheap poll target. useProjectSignaturePoll polls every 2s, skips hidden tabs, avoids concurrent requests, and no-ops when the baseline is undefined (initial load not yet done). Well-tested: signature mismatch fires, match stays quiet, hidden tabs skip, re-arms after caller refresh, cleanup on unmount.
The storyboard response now carries signature in both the absent and present branches — tested in storyboard.test.ts. The board uses it as posterVersion on FramePoster (cache-bust on the thumbnail URL via a ?sig= param).
2. Preview landing on storyboard view
studioLandingSearch determines the landing view from frame statuses: any built → board (sketch review in progress), all outline with declared srcs but no files → board (planning), otherwise timeline. Exported and tested. The deep-link URL puts ?view=storyboard before the hash — documented in preview-render.md.
3. Frame comments (studio + shared contract)
Clean separation between pure logic (frameComments.ts — all the data transforms, fully tested) and React state (useFrameComments.ts — the hook that wires drafts, submit, and pending-refresh to the file manager context).
The buildCommentsFile merge logic is right: a resubmit before the agent consumes the previous batch keeps unconsumed entries for frames the new batch doesn't mention, overwrites the ones it does, sorts by frame index. The passForFrames derivation (furthest status on the board determines the pass) is simple and correct.
The comment box per tile (StoryboardFrameTile.tsx) and the batch submit bar (CommentsSubmitBar in StoryboardLoaded.tsx) are clean UI additions. The useEffect on data.signature in StoryboardLoaded re-checks the comments file when the board refreshes — catches the moment the agent deletes it after consuming.
The contract is defined in storyboard-format.md § Frame comments and cross-referenced from brief-contract.md § 1 ("the comments channel"). Good.
4. Preferences — two-tier remembered defaults (media-use)
prefs-store.mjs: atomic writes (tmp + rename), tolerant reads (corrupt = empty), two-tier with a clean promotion rule (PROMOTE_AT = 2 distinct projects). style_preset keyed per workflow via preferenceKeyFor. User-tier sightings ledger keeps cross-project evidence in the only place that can see it (the user file). Best-effort user-tier writes — a read-only home directory never fails a brief.
Test coverage is solid: record + merge, no one-project promotion, two-project promotion, idempotency, changed-value provenance reset, project-overrides-user, per-workflow keys, unknown key rejection, corrupt file handling.
The consumption contract (brief-contract § 2, "Remembered defaults") is clear: a remembered value changes the recommended default with a receipt — never skips a question, and explicit request content always wins. Each workflow's Step 0 references this section.
5. Recipes — frozen video bundles (media-use)
recipe-store.mjs: skeletonizeStoryboard is the interesting piece — parses markdown, keeps structure (durations, transitions, src paths, Video direction block, style frontmatter), blanks content (message, audience, narration, per-frame prose → one <fill in> placeholder per frame naming the beat's role), resets all statuses to outline. Well-tested against a realistic storyboard fixture.
freezeRecipe: merges preferences into recipe.json, copies frame.md, generates the skeleton, appends a manifest record, promotes to user-tier immediately (a freeze is already confirmed). Re-freezing bumps version and archives the old folder as <name>@v<N>. useRecipe: imports from user tier if the project doesn't have it, copies frame.md over, returns brief values + skeleton path. listRecipes: two-tier merged (project wins), skips @v archives.
Also wired into resolve.mjs as --type recipe — consistent with the existing resolve pattern for other asset types.
6. Skill updates (faceless-explainer, pr-to-video, product-launch-video, general-video, hyperframes)
The same changes replicated across all storyboard-planning workflows:
- Recipe check before Round 1
- Round 1 sent alone (explicit "wait before Round 2" wording)
- Remembered defaults before Round 2 questions
- Sketch pass at Step 4 with workflow-specific stand-ins
- Worker context includes confirmed-sketch flag + keep-layout rule in frame-worker.md
- Gates include sketch board confirmation (collaborative)
- Voice fallback to remembered voice before pipeline default
- Review loop reference at plan presentation and final look
This is intentional replication — each workflow SKILL.md is a standalone document loaded independently. The shared process lives in review-loop.md; each skill wires its steps to specific sections. Correct pattern for the skills architecture.
SSOT check
resolveProjectSignatureunified — previously inline inpreview.ts, now shared across three routes. Good.- Frame comments contract: defined once in
storyboard-format.md, implemented inframeComments.ts, cross-referenced frombrief-contract.md. Single source. - Preferences consumption rule: defined once in
brief-contract.md§ 2, referenced by every workflow. Single source. - Review loop process: defined once in
review-loop.md, wired into each workflow at the appropriate steps. Single source. - Skill replication: intentional — standalone docs, not shared code.
CI note
Skills: manifest in sync is failing — the manifest hash/count may need a refresh after the skill file changes.
Verdict
LGTM. Clean architecture, good test coverage, well-separated concerns. The CI manifest check needs a fix before merge. Looking forward to the additional tests Miao mentioned.
— Review by Miga
jrusso1020
left a comment
jrusso1020
left a comment
There was a problem hiding this comment.
HF-specialist review — COMMENT (external author; stamp = James's call)
Audited (in addition to Miga's general coverage): the new shared-contract file (review-loop.md) end-to-end and its integration with brief-contract.md + storyboard-format.md; the Studio-side signature-polling design (useProjectSignaturePoll.ts + server-side projectSignature.ts); the frame-comments file protocol (frameComments.ts); the preferences store's two-tier promotion logic (prefs-store.mjs); and skeletonizeStoryboard's structure-keep/content-blank behavior (recipe-store.mjs).
Trusting: Miga's coverage on the 6-piece composition (project-signature polling, preview landing, frame comments, prefs two-tier, recipes, skill updates). Not duplicating; adds below focus on the HF architectural angle.
Design fit — shared-contract pattern
review-loop.md (+41 lines, new) follows the exact shape established by brief-contract.md and story-spine.md (HF#2058): shared rules file + workflow SKILLs bind to it via markdown pointers + explicit cross-references between siblings. Same "invariants centralized, executable script local" split. Load-bearing cross-refs from review-loop.md §§ 1-4 back into brief-contract.md § 1 (interaction mode / comments channel) and storyboard-format.md (frame statuses, comments file) are all present and land correctly on the referenced targets. Fits cleanly.
brief-contract.md gets two additions — "The comments channel" (§ 1) and "Remembered defaults" (§ 2) — that widen its scope from interaction to interaction + memory. Reasonable expansion; the file already handled ordering + gates + fields, so preferences memory as a field-level attribute belongs here.
Studio storyboard board — signature polling design
useProjectSignaturePoll.ts:29-67: 2s interval, skips when document.hidden, skips when previous request inFlight, visibility change triggers immediate re-check, no baseline (signatureRef.current === undefined) → no compare (safe), any fetch failure → skip tick (fails open). Solid shape. Comparison baseline is the signature the caller's data was loaded with, so refetch naturally re-arms.
Server-side projectSignature.ts computes a stable content-aware hash: text files ≤ 2MB use content, everything else uses mtime + size. Cache keyed by (mtime + size + relative-path) fingerprint avoids re-reading unchanged files — good. Excluded dirs list at projectSignature.ts:18-30 covers the standard cases (.git, node_modules, outputs, renders, .next, .cache, dist, build, coverage, .hyperframes, .vite).
Concern (nit-level, not blocking): STUDIO_SIGNATURE_MANIFEST_PATHS at projectSignature.ts:32-35 is a hand-maintained allowlist of .hyperframes/* files that DO count toward the signature (studio-manual-edits.json, studio-motion.json). The .hyperframes/ dir itself is otherwise excluded. This means a future Studio subsystem writing to .hyperframes/newthing.json will silently NOT trigger poll refreshes until someone remembers to add it here. Two mitigations to consider (either fine, both fine): (a) invert to a denylist that skips .hyperframes/frame-comments.json (outbound feedback, doesn't need to trigger self-refresh) and includes everything else under .hyperframes/; or (b) add a test that pins this list against the actual Studio-server write surface so a mismatch fails CI.
Concern (nit-level): collectProjectSignatureFiles at projectSignature.ts:65-99 recurses without a depth or file-count cap. Excluded-dirs handles the common problem cases, but a non-standard build directory (someone's .next renamed to custom-build) would churn the 2s poll traversing files. The cache short-circuits on unchanged fingerprints so steady state is cheap, but the first-hit or after-a-refactor cost is uncapped. Consider a soft cap or a file-count guard. Not blocking.
Frame comments file — the review's structured feedback channel
frameComments.ts:104-115 buildCommentsFile correctly handles the "resubmit before agent consumed" case: keeps entries for frames the new batch doesn't mention, overwrites the ones it does. Right invariant — a partial resubmit doesn't lose feedback on other frames.
parseCommentsFile:79-97 is lenient — malformed JSON, missing fields, or non-array comments returns null → treated as "no file". Aligned with the storyboard parser's lenient pattern (storyboard-format.md "The parser is lenient"). Good consistency across the read-side contract.
The cross-repo contract comment at frameComments.ts:6-10 explicitly pins the file shape to skills/hyperframes-core/references/storyboard-format.md § Frame comments. Nice discipline — keeps the shape from drifting between the Studio writer and the SKILL.md documented contract. A regression test that reads both the TS interface and the SKILL.md section and asserts field parity would tighten this further; not blocking.
skeletonizeStoryboard — structure/content split
recipe-store.mjs:123-141. What's kept: frame count, headings, duration, transition_in, src, style-ish frontmatter (format, arc). What's blanked: frontmatter content keys (message, audience, mode), per-frame content keys (voiceover / vo / voice_over / narration / scene / description / summary / caption / asset_candidates), every frame's status → outline, per-frame prose → placeholder <fill in: this video's content for the "..." beat — keep the layout role, replace the words.>.
Verified correct: mode correctly drops (interaction mode is per-run content, not reusable structure) per brief-contract.md § 1. arc correctly kept (reusable narrative structure) per story-spine.md. Case-insensitive detection via .toLowerCase() in skeletonFrameLine:96-101 matches how storyboard-format.md defines these keys.
Consistency observation: FRAME_CONTENT_KEYS at recipe-store.mjs:37-49 includes asset_candidates which isn't documented in storyboard-format.md's per-frame table. If that's an emerging convention worth documenting, add it to the storyboard-format table so both sides stay in lockstep. Not blocking — the skeletonizer treating unknown-but-content-shaped keys as content is the safer default.
Concern (nit-level): recipe archive accumulation. freezeRecipe:174-181 renames the old folder to <name>@v<N> before writing the new one, and scanRecipesDir:229-238 filters out @v entries from listings. No garbage collection — long-lived projects that re-freeze the same name repeatedly accumulate @v1, @v2, @v3, ... indefinitely. Not a bug, but worth a follow-up (either a hard cap like keep-last-3 or an explicit recipe.mjs gc command).
Preferences store — two-tier promotion
prefs-store.mjs:130-146 recordUserSighting: accumulates project sightings under sightings[fullKey][value] (per-value), promotes to preferences[fullKey] when seenIn.length >= PROMOTE_AT (2). Atomic writes via writePrefsFile:70-76 (tmp + rename) — good. Best-effort user-tier write in recordPreference:167-173 so a read-only home directory never fails a brief.
Concern (subtle, nit-level): sightings[fullKey][value] is keyed on basename(resolve(projectDir)) at prefs-store.mjs:161. Two distinct repos with the same basename (git clone in two different parents, e.g. ~/work/foo and ~/personal/foo) both writing the same value would deduplicate to ONE sighting via the dedupe([...]) on line 134, delaying promotion. Conversely, two separate projects with different basenames but the same identity (a directory rename) would double-count. The unaffected common case is the one that matters here (most people don't clone the same repo twice), so this is a low-frequency concern. Fix if it comes up: salt with a project-hash slice.
The keepProvenance logic in recordProjectTier:113-124 is nice: a changed value starts provenance over (old confirmations vouched for the old value), so a mid-flight preference change doesn't inherit stale trust. Right shape.
Manifest CI failure
Skills: manifest in sync fails at the "Verify skills-manifest.json matches skills/" step. Confirmed via the failing job log + the diff shape: hyperframes-core shows +1 file (14 → 15 — the new review-loop.md) and multiple hash rows updated for touched skills. This is a regen-only fix — bun scripts/gen-skills-manifest.ts (or whatever the local generator command is) will produce a manifest that matches. Miga's read is correct.
Verdict
COMMENT — the review-loop / preferences / recipes design fits HF's shared-contract pattern cleanly (same shape as brief-contract.md + story-spine.md); the Studio signature polling is thoughtful (visibility-aware, in-flight-guarded, fails-open); skeletonizeStoryboard's structure/content split is coherent with brief-contract.md § 2's memory semantics; frame-comments file protocol matches its SKILL.md documented contract. Concerns found are all nit-level (allowlist maintenance risk on signature manifest paths, no file-count cap in signature traversal, no GC on @v archives, project-basename collision in sightings ledger). Manifest CI failure is regen-only. Stamp = @jrusso1020's call (external author WaterrrForever; requester Miao is allowlisted but not a trusted-stamper, per delegation rule).
Given Miao is explicitly holding on merge for more tests to land, none of these nits need to block the eventual merge — noting them for the next follow-up.
— Rames Jusso
3 participants
What
The review loop — collaborative runs of the narrative workflows review in three passes of rising fidelity: text plan → wireframe sketch board → animated build, each gated on the user, with a per-frame comment channel on the Studio board that batches into a sidecar file the agent picks up on its next reply. Autonomous mode is untouched: no board, no sketch pass, one preview-or-render question before render.
User memory — two tiers on media-use's existing storage split. Preferences: confirmed brief answers (destination, aspect, language, mode, voice, style preset) become next run's recommended defaults with receipts naming their source. Recipes: one approved run frozen as a named, versioned bundle (design spec + storyboard skeleton + confirmed values); the next "make another weekly-promo" fills the brief, skips the design step, and drafts from the skeleton — while every review gate still runs.
Changes — skills (the flow)
hyperframes-core/references/review-loop.md(new) — the single source for the three passes: the plan proposed on a live board (§ 1), wireframe sketches markedbuiltwith one layout question (§ 2 — real words on plain blocks, run no CLI; a confirmed board is itself a valid deliverable when the user asked for a storyboard, not a video), the build dressing confirmed layouts (§ 3, worker or inline), the final look plus a once-only recipe-freeze offer whose confirmation teaches the recall phrase (§ 4).brief-contract.md§ 2, Remembered defaults — read the merged preferences before Round 2; a remembered value becomes the recommended option with a receipt ("1:1 — you confirmed this in world-cup-explained"). The comments channel (silent submit, one reply picks it up, check the file before the words) lands in § 1; the sidecar schema and thebuiltstatus rung instoryboard-format.md.Changes — studio + server (the surfaces)
Submit comments (N)writing.hyperframes/frame-comments.json(a resubmit wins per frame; unconsumed comments on other frames are kept); submitted-but-unconsumed comments stay visible — toolbar banner + per-tile echo — until the agent consumes the file.GET /projects/:id/signature(cheap: the CLI adapter's watcher-invalidated cache), the storyboard payload carries the signature it was derived from, and the view polls at 2 s (hidden tabs skipped), refetching in place with no loading flash. Posters bake the signature into their URL so tiles fill in as sketches land; the empty state upgrades itself when STORYBOARD.md appears, and its handoff prompt points the agent at the review loop.previewlands where review is — the storyboard view while any frame isbuiltor the board is pure planning; the timeline once assembled.Changes — media-use (the memory engine)
scripts/lib/prefs-store.mjs+scripts/prefs.mjs— project.media/preferences.json(committed, the team inherits it) + personal~/.media/preferences.json; a value earns the personal tier by being confirmed in two different projects (a user-side sightings ledger accumulates the cross-project evidence), so a one-off choice never pollutes the global defaults.scripts/lib/recipe-store.mjs+scripts/recipe.mjs— named versioned folders, not content hashes (a recipe is an evolving bundle):freezeskeletonizes the storyboard (structure kept — durations, transitions, srcs, Video direction; statuses reset; content blanked to per-frame fill-ins naming each beat's role),listmerges tiers,useimports and copies the frame spec in.resolve --type recipe --entity <name>delegates like grade/lut.The trust model
Tests / verification
node --test(+9 preferences, +7 recipes: merge precedence, promotion at two projects, provenance reset on value change, skeletonization, version bump + archive, tier import).preview --type recipefreeze→fresh-project use round-trips through the user tier.