Video Game Testing Workflow (vgtw)

A game-agnostic continuous audit-loop framework: detect → diagnose → fix → re-measure, on a single deterministic spine, with a structured, diffable verdict a loop controller (human or agent) can act on each turn. Built for web / WebGL games (Playwright + CDP + WebGL + ws-netcode patterns).

Extracted from the Thirteen Days audit loop — its first and reference adapter. The generic core lives here; a game keeps only its concrete GameAdapter impl + the dims that bind to it.

The idea

Each quality concern — lag, errors, boot, render, feel, co-op desync, regression replay, mechanic fuzzing, aesthetic conformance — is a dim: a small measure + diagnose unit that returns a DimRun. The spine runs one dim, diffs its metrics against a stored baseline, derives a fix-routing tier, and writes one flat verdict JSON (latest.json + history/). The loop reads the verdict, applies one fix in the indicated layer, re-gates, and re-runs — until a stop condition.

vgtw-audit --dim=<name> [--dims-dir=./dims --reports-dir=./.vgtw/reports --secs=15 --<key>=<val> ...]
vgtw-board                                          # aggregate every dim's verdict into one browsable HTML
vgtw-audit --dim=<name> --promote                   # baseline.candidate.json → the committed baseline.json

Verify (the framework's own gate)

npm test          # tsc --noEmit  →  example dim runs green  →  example health board renders

That one command is the CI gate — the proof the generic core works standalone, with no game and no GitHub setup. It runs anywhere (locally, any CI). To also gate every push/PR on GitHub Actions, drop ci.example.yml in as .github/workflows/ci.yml (it runs the same three checks).

Write a dim

import type { DimContext, DimRun } from 'vgtw/core/types.ts';
export const meta = { name: 'mydim', summary: 'what it measures' };
export async function run(ctx: DimContext): Promise<DimRun> {
  const metric = { thing_ms: 12, errors: 0 };
  return {
    metric, target: { thing_ms: 16, errors: 0 }, lowerIsBetter: ['thing_ms', 'errors'],
    bottleneck: null,                              // or { class, layer, calibration_serial, suggested_tunable }
    verdict: 'thing within budget',
    next: 'target_met',                            // keep_fixing | target_met | confirm_on_device | blocked_escalate | tried_exhausted
    honest_limit: 'what this number does / does NOT prove (e.g. "relative GPU — confirm on device")',
  };
}

adapters/example/dims/smoke.ts is a runnable reference; npm run example runs it green.

The GameAdapter seam

A game implements as much of GameAdapter (core/types.ts) as its enabled dims need — the spine never touches it; dims do.

• name — stable id.
• serve() — launch/serve the game (dev server / preview / in-process server).
• drive(handle) / metrics() — reach a testable state + the generic metric schema.
• net — netcode bindings (spin server + clients, the authoritative-state getter, lag injection).
• assets / aestheticRubric — for the aesthetic / visual dims.

The honesty boundary (load-bearing)

A verdict is only as honest as its honest_limit. Headless software-GL is meaningless for absolute GPU/perf — those dims emit next: confirm_on_device, never a hard fail. Ground truth (real regardless of GL): a frame was submitted, uncaught errors/rejections, sim-space/netcode determinism, and bundle bytes. The spine routes a fix that touches the calibration-locked core to Tier-S/ calibration (re-green the determinism gate, or escalate) so a measurement can never silently rewrite the thing it measures against.

Layout

• core/types.ts — the verdict schema, the dim contract, the GameAdapter interface.
• core/audit.ts — the spine (run a dim, diff baseline, derive tier, write the verdict).
• core/healthboard.ts — aggregate every dim's verdict + trend into one HTML.
• adapters/example/ — the reference adapter + dim that npm test runs green.

A consuming game depends on this repo (git dep → npm package) and points the spine at its own --dims-dir; vgtw stays the generic core.

Status

0.1.0 — extracted from Thirteen Days. The spine, verdict schema, baseline/diff, tier routing, and the health board are stable; the dim engines (replay/regression, coop-desync, netlag, errboot, render, feel, …) generalize out of their game adapters over time.

© 2026 Liberty Hill Studios LLC. All rights reserved. Private.