Skip to content

feat(persona): add Pocket-native parity POC - #3

Draft
doodlewind wants to merge 2 commits into
mainfrom
agent/persona-parity-poc
Draft

feat(persona): add Pocket-native parity POC#3
doodlewind wants to merge 2 commits into
mainfrom
agent/persona-parity-poc

Conversation

@doodlewind

@doodlewind doodlewind commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • keep the Persona product slice in pocket-character: native host, QuickJS policy, catalog handling, bridge/MCP compatibility, visual acceptance, and fail-closed benchmark tooling
  • refresh against Persona bb7ef2455b23aee685f68c9e83a185347d257964 (0.1.0-beta.0), including its packaged avatar, idle clip, and 17 modular speaking clips
  • match the current speaking lifecycle: ping-pong gestures, no immediate repeats, randomized dwell/fade timing, audio-zero pause, 0.7 s resume hold, 0.9 s talk-to-idle hold, 0.35 s body transitions, and one-shot return to the underlying voice role
  • make Pocket's production default 30 fps / 2048px textures and report both absolute CPU and CPU per delivered frame
  • fix the visible VRMA humanoid-basis error that twisted wrists/hands and ankles/feet, then add deterministic idle and real modular-speaking screenshot receipts

One-line acceptance

bun run accept:persona -- --cycles 1
bun run accept:pocket -- --cycles 1

Omit -- --cycles 1 to leave either target running for interactive inspection. Both commands validate and stage the same controlled VRM/VRMA fixture and drive idle → speaking/lip-sync → listening → greeting. Visual acceptance must inspect the whole character, especially shoulder/wrist/hand and hip/knee/ankle/foot axes.

Sequential packaged-asset measurements:

bun run bench:persona
bun run bench:persona:speaking
bun run bench:persona:controlled

The benchmark runs Persona and Pocket sequentially, requires the requested voice state at settle and every sample, validates Pocket animation/amplitude, and fails if Pocket's render frame count does not advance.

Repository boundary

All Persona-specific behavior remains in this repository. PocketJS only receives two generic, independently reviewable capabilities:

This branch pins vendor/pocketjs to #207 head 0341db00a92b9c8a040288a25bd690af7e55fe61. #207 is stacked on #204 and can be retargeted to main after #204 lands. Repin this submodule to the final merge SHA(s) before merging this PR.

Measured result

MacBook Pro Mac15,8, Apple M3 Max, macOS 26.5.2, 120 Hz display. Each final lane ran sequentially with a 30-second settle and nine 5-second process-tree samples using Persona's clean packaged catalog at bb7ef245….

Lane Observed fps, Persona / Pocket Median CPU, Persona / Pocket Pocket CPU delta CPU per delivered frame Pocket summed RSS delta
120 Hz idle diagnostic 120.013 / 111.077 21.196% / 9.002% 57.5% less 54.1% less 89.8% less
30 fps production idle 120.021 / 28.358 17.603% / 4.600% 73.9% less 10.6% worse 92.4% less
30 fps sustained speaking 120.011 / 29.062 17.999% / 3.000% 83.3% less 31.2% less 88.6% less

The old 89.5% idle claim is invalid: that run visibly mis-retargeted raw local VRMA rotations, so it did not render the same pose. These are post-fix results paired with actual live Persona screenshots and Pocket idle plus modular-speaking screenshots. The idle absolute CPU win is largely a shipping-configuration win; its per-frame cost is still 10.6% worse. Speaking and the near-refresh-matched lane retain per-frame CPU advantages.

Validation

  • bun run typecheck
  • bun test tests/persona-bench.test.ts — 5 tests, 13 assertions
  • cargo test -p pocket-persona — 23 passed
  • cargo clippy -p pocket-persona --all-targets -- -D warnings
  • cargo check --workspace --all-targets
  • PocketJS pocket-vrm: 19 unit + 6 fixture tests passed; Clippy and targeted rustfmt check passed
  • bun run accept:persona -- --cycles 1
  • bun run accept:pocket -- --cycles 1
  • final sustained-speaking report completed with status: "ok"
  • single-sample frame-receipt smoke completed with status: "ok" and advanced Pocket renderFrameCount from 23 to 54 during its sampling window
  • inspected an 8-frame actual --headless-speaking sequence plus idle/reference captures; hands and feet remain fully framed with natural joint axes

Notes

  • Persona media and generated screenshots/reports remain ignored and are not committed.
  • Persona's source is MIT; its packaged media keeps the upstream terms documented in public/assets/LICENSES.md.
  • This is a feature-parity/performance POC, not a claim of pixel-identical renderer output or full Settings/import persistence parity.

@doodlewind

Copy link
Copy Markdown
Contributor Author

Final visual and performance conclusion

This follow-up replaces the earlier CPU headline with the post-retarget, screenshot-validated result.

Visual validity

  • The old 89.5% idle CPU reduction is invalid. Pocket was interpreting raw local VRMA rotations as normalized humanoid rotations, which visibly twisted the wrists/hands and ankles/feet.
  • With the generic normalized-humanoid retarget fix in fix(vrm): normalize VRMA humanoid rotations pocketjs#207, the packaged idle capture and an actual eight-frame modular-speaking sequence keep the shoulder-to-hand and hip-to-foot axes natural and fully in frame.
  • This establishes comparable pose semantics; it does not claim frame-identical choreography or pixel-identical rendering.

Final measurements

All lanes used Persona bb7ef245…, a 30-second settle, nine 5-second samples, sequential target execution, activity receipts, and advancing frame receipts.

Lane Persona / Pocket fps Persona / Pocket median CPU Pocket absolute CPU delta Pocket CPU per delivered frame
120 Hz idle diagnostic 120.013 / 111.077 21.196% / 9.002% 57.5% less 54.1% less
30 fps production idle 120.021 / 28.358 17.603% / 4.600% 73.9% less 10.6% worse
30 fps sustained speaking 120.011 / 29.062 17.999% / 3.000% 83.3% less 31.2% less

CPU percentages are percentages of one logical core: 100% means one saturated core.

Stock Persona at full refresh

Persona's default React Three Fiber Canvas has no explicit frame cap, so it follows display-driven requestAnimationFrame. On this 120 Hz M3 Max system, two independent final settled-idle runs measured 17.603% and 21.196% median CPU, with 21.996% and 22.559% p95. A practical default-open, hands-off expectation on this machine is therefore roughly 18–21% of one logical core at ~120 fps, with p95 around 22–23%. The sustained-speaking run measured 17.999% median / 18.442% p95.

Startup model decoding, moving or clicking the widget, and shader warm-up can briefly read much higher. A 60 Hz display should generally cost less, but the relationship is not guaranteed to be linear and was not inferred as a 60 Hz result here.

Interpretation

The 30 fps idle result is primarily a user-facing shipping-configuration saving: Pocket delivers about one quarter as many frames and is still 10.6% worse per delivered idle frame. The modular-speaking path and near-refresh-matched lane retain genuine per-frame CPU advantages. Persona-specific policy remains in this repository; PocketJS only gains the generic window-visibility hook (#204) and generic VRMA retarget fix (#207).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant