Skip to content

FE-1263: Sweep scenario parameters with a navigator that computes the selection - #9093

Merged
kube merged 22 commits into
mainfrom
cf/fe-1263-experiment-parameter-ranges
Sep 3, 2026
Merged

FE-1263: Sweep scenario parameters with a navigator that computes the selection#9093
kube merged 22 commits into
mainfrom
cf/fe-1263-experiment-parameter-ranges

Conversation

@kube

@kube kube commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Important

Experimental
Behind the Parameter sweeps feature flag.

Summary

Before this PR, experiments take one fixed value per scenario parameter. FE-1263 prototype swept parameters with a bespoke lazy-grid scheduler outside the shared experiment runtime.

Experiments can now sweep scenario parameters. A Sweep toggle per numeric parameter defines evenly spaced values. Ranges form a grid capped at 200 combinations, and the results drawer gains a pinned navigator over it. Only the selected combination computes: runs accumulate in escalating batches that stream into the charts. Moving any control cancels the in-flight batch and restarts on the new combination. Visited combinations keep their results. Prototype's scheduler is replaced by batches through the backend registry.

Links

Changes

UI

  • Create drawer gains a Sweep toggle per numeric parameter

    Toggle reveals Min/Max/Values inputs.
    Combination counter warns above 50 and blocks above 200.
    "Runs" is relabelled "Runs per combination" when sweeping.

  • Section gains a sticky-band slot in the results drawer

    Band content stays pinned with the title while the body scrolls.
    Sweep's Parameters strip lives there: a segmented control per short axis, a stepper per long one, and a refinement line reading "61 of 100 runs — refining while you stay here".

Sweep session and grid

  • New react/experiments/sweep-session.ts runs the refinement loop

    One combination is live at a time.
    Batch b covering runs [from, target) seeds itself with deriveRunSeed(seed, from), so every combination samples the same common-random-numbers sequence and a cancelled rung re-runs identically.

  • Finished batches fold into a per-combination cache

    In-flight frames merge with the cache for display and are discarded on cancellation.
    Session is backend-agnostic and consumes an injected instantiateBatch.

  • react/experiments/parameter-grid.ts is ported from the prototype

    Range expansion with float-artifact stripping and integer/ratio validation, the cartesian grid with its 200-combination cap, and the run ladder of 8, 25, 100, 500, 1000 up to the per-combination budget.
    Prototype's cell levelling and cross-cell marginalization stay behind, since compute follows the selection.

  • Scalar frames merge through the run-aggregate monoid they carry

    Exact rather than a weighted mean.

  • Provider runs a sweep's first batch through the plain backend-selection walk

    Same walk as a plain experiment.
    selectExperimentBackend now returns the chosen backend.
    Later batches re-assess that backend per request.

  • GPU backend regenerates its shader for each combination's parameter values

    Scenario recompiles per batch because swept values change the compiled initial state.

  • New non-terminal status idle means the selected combination is saturated

    Moving a control resumes running.

Test coverage

  • parameter-grid.test.ts, 21 tests:

    Range expansion, validation, the grid, the ladder, exact frame merging.

  • sweep-session.test.ts, 8 tests:

    Ladder climbing with batch folding, restart on selection change with in-flight discard, mid-ladder resume on revisit, error stop, cell reads for other consumers.

  • provider.test.tsx, 4 new tests:

    Fixed/range splitting, per-parameter range errors, the combination cap, undeclared-parameter inputs ignored.

  • Parameter sweep story, verified interactively:

    Navigator renders over a streaming distribution chart. Selection click, sticky band under scroll.

How to test

  • Open Petrinaut preview on Vercel
  • Viewport controls > Settings > Simulation > Parameter sweeps
  • Load example, select scenario with numeric parameters
  • Simulate > Experiments > Create
  • Flip Sweep on one or two parameters, Run
  • Move controls in the drawer's Parameters strip
  • Expect charts to re-stream

@kube kube self-assigned this Jul 24, 2026
@vercel

vercel Bot commented Jul 24, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hash Ready Ready Preview Sep 2, 2026 7:54pm UTC
petrinaut Ready Ready Preview Sep 2, 2026 7:54pm UTC
petrinaut-docs Ready Ready Preview Sep 2, 2026 7:54pm UTC
1 Skipped Deployment
Project Deployment Actions Updated
hashdotdesign-tokens Ignored Ignored Preview Sep 2, 2026 7:54pm UTC

Request Review

@github-actions github-actions Bot added area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > frontend Owned by the @frontend team area/apps > hash.design Affects the `hash.design` design site (app) labels Jul 24, 2026
@vercel
vercel Bot temporarily deployed to Preview – petrinaut July 24, 2026 01:47 Inactive
@kube kube changed the title FE-1263: Add parameter ranges to Petrinaut experiments with a sweep-results navigator FE-1263: Add lazily-computed parameter sweeps to Petrinaut experiments Jul 24, 2026
@vercel
vercel Bot temporarily deployed to Preview – petrinaut July 24, 2026 02:26 Inactive
kube added a commit that referenced this pull request Aug 27, 2026
Ranges, axes, the cartesian grid with its 200-combination cap, and the
run ladder port unchanged from PR #9093's parameter-grid module. The
lazy-grid pieces stay behind: the rework refines only the selected
combination, so cell-levelling and cross-cell marginalization go.
@kube
kube changed the base branch from main to cf/fe-1429-run-trial-replicates-through-the-shared-experiment-backend August 27, 2026 01:59
@kube
kube force-pushed the cf/fe-1263-experiment-parameter-ranges branch from ad7c211 to bfa165e Compare August 27, 2026 01:59
@github-actions github-actions Bot added area/deps Relates to third-party dependencies (area) type/eng > backend Owned by the @backend team labels Aug 27, 2026
@kube kube changed the title FE-1263: Add lazily-computed parameter sweeps to Petrinaut experiments FE-1263: Sweep scenario parameters with a navigator that computes the selection Aug 27, 2026
Comment thread libs/@hashintel/petrinaut-core/src/webgpu/emit-wgsl.ts Fixed
kube added a commit that referenced this pull request Aug 27, 2026
Ranges, axes, the cartesian grid with its 200-combination cap, and the
run ladder port unchanged from PR #9093's parameter-grid module. The
lazy-grid pieces stay behind: the rework refines only the selected
combination, so cell-levelling and cross-cell marginalization go.
@kube
kube force-pushed the cf/fe-1263-experiment-parameter-ranges branch from bfa165e to bdcb114 Compare August 27, 2026 16:40
@github-actions github-actions Bot removed the area/deps Relates to third-party dependencies (area) label Aug 27, 2026
@kube
kube force-pushed the cf/fe-1263-experiment-parameter-ranges branch from bdcb114 to 367970a Compare August 27, 2026 18:24
kube added 13 commits September 2, 2026 21:31
The worker body moves to attachMonteCarloWorker(runtime), so Web Workers,
Node worker_threads, and an in-process loopback all run the same protocol.
Experiments accept explicit per-run configs (sliced per shard), and each
run's final metric values come back as runResults — the per-seed objectives
optimization replicates need.
Each trial becomes one Monte Carlo experiment: explicit run seeds keep
replicate 0 on the base seed, worker_threads carry the shards under Node,
and the in-process worker covers source runs and tests. Replicates no
longer fail fast — they run in parallel and validate together.
The experiment stops sniffing navigator and defaults to one shard. The
editor's provider passes getDefaultMonteCarloShardCount(); the CLI gains
--threads, defaulting to one per core minus one, where --threads 1 spawns
no workers and simulates on the calling thread.
Ranges, axes, the cartesian grid with its 200-combination cap, and the
run ladder port unchanged from PR #9093's parameter-grid module. The
lazy-grid pieces stay behind: the rework refines only the selected
combination, so cell-levelling and cross-cell marginalization go.
createSweepSession computes one combination in ladder-climbing batches,
restarts on the combination the navigator moves to, and folds finished
batches into a per-combination cache so revisits resume mid-ladder.
Batch seeds derive from the run range, so every combination samples the
same seed sequence (common random numbers) and cancelled rungs repeat
exactly. Scalar frame merging becomes exact through the run-aggregate
monoid the frames now carry.
Scenario parameters gain a Sweep toggle in the create drawer (min, max,
value count), capped at 200 combinations. A sweep's first batch runs the
same GPU-or-CPU backend selection walk as a plain experiment — the walk
now returns the chosen backend, and later batches re-assess it directly.
Sweeps report idle, a non-terminal rest state, when their selected
combination saturates.
Sections gain a sticky-band slot: content pinned with the title while
the section's body scrolls. A sweep's drawer uses it for the navigator —
a segmented control per short axis, a stepper per long one, and a
refinement line showing the ladder climb — so changing a parameter while
watching the charts below is one motion.
The user guide gains a Parameter sweeps section and the Idle status; the
architecture docs gain a parameter-sweeps page with the refinement-loop
diagram, attached to react.experiments.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit e79367f. Configure here.

Comment thread libs/@hashintel/petrinaut/src/react/experiments/sweep-session.ts
@kube
kube added this pull request to the merge queue Sep 3, 2026
Base automatically changed from cf/fe-1429-run-trial-replicates-through-the-shared-experiment-backend to main September 3, 2026 12:06
@kube
kube requested a review from a team as a code owner September 3, 2026 12:06
@github-actions github-actions Bot added the area/deps Relates to third-party dependencies (area) label Sep 3, 2026
Merged via the queue into main with commit 6e11ffa Sep 3, 2026
215 of 248 checks passed
@kube
kube deleted the cf/fe-1263-experiment-parameter-ranges branch September 3, 2026 12:07
@hash-release hash-release Bot mentioned this pull request Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps > hash.design Affects the `hash.design` design site (app) area/deps Relates to third-party dependencies (area) area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

4 participants