FE-1408: Run a trial's seeded simulations in parallel in the Petrinaut CLI, aggregated by mean - #9223
Draft
kube wants to merge 1 commit into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
…t CLI
Optimization trials now run execution.seedsPerTrial seeded simulations on
a worker-thread pool (dist/simulation-worker.js, pool capped by
PETRINAUT_CLI_MAX_PARALLEL_SIMULATIONS or min(cores-1, 4), in-process
sequential fallback at 1), aggregated into the trial objective by mean.
Seeds derive deterministically from execution.seed - replicate 0 keeps
the base seed, later replicates reuse the Monte Carlo derivation - and
every trial reuses the same list, so Optuna compares configurations
under common random numbers. The evaluate response stays
{objective}-compatible and reports per-seed replicates; describe reports
seedsPerTrial. The deployed optimizer image needs FE-1412 (--allow-worker
and its bootstrap/timeout handling) before manifests with
seedsPerTrial > 1 can run there.
kube
force-pushed
the
cf/fe-1408-petrinaut-cli-run-a-trials-seeded-simulations-in-parallel
branch
from
August 16, 2026 17:10
797689e to
ec38102
Compare
kube
changed the base branch from
main
to
cf/fe-1411-petrinaut-cli-make-the-protocol-handler-async-safe
August 16, 2026 17:12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🌟 What is the purpose of this PR?
Optimization trials currently run one simulation with the fixed
execution.seed, so a stochastic net is scored on a single replicate. This PR makes the Petrinaut CLI run a trial'sexecution.seedsPerTrialseeded simulations in parallel on a worker-thread pool and aggregate the per-seed objectives by mean, keeping the{objective}response shape so the Python service needs no logic changes. Simulator side of FE-1273; aggregation-method selection (FE-1277) and Optuna-side handling of per-seed values (FE-1281) are out of scope.Stack: FE-1410 (contract) → FE-1411 (async-safe protocol) → this PR → arch-docs manual → FE-1270 (Python bindings) → FE-1412 (Python client seeded-trials support).
🔗 Related links
🔍 What does this change?
@hashintel/petrinaut-clionly:optimization.evaluateruns the trial's seeds and returns their mean objective plus areplicates: [{ seed, objective }]array;optimization.describereportsstudy.seedsPerTrial. Seeds derive once per study: replicate 0 keepsexecution.seed(M=1 stays bit-identical to today, and one editor run still reproduces a trial replicate), replicate i uses the exported Monte CarloderiveRunSeed(seed, i). Every trial reuses the same list → common random numbers across Optuna steps.dist/simulation-worker.js, a second vite entry): each worker compiles the model once at bootstrap, before the readiness line. Pool size ismin(seedsPerTrial, cores − 1, 4), overridable viaPETRINAUT_CLI_MAX_PARALLEL_SIMULATIONS; an effective size of 1 uses an in-process sequential runner (no worker threads at all — the default path, sinceseedsPerTrialdefaults to 1).Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
@hashintel/petrinaut-cliis private; the@hashintel/petrinaut-corechangeset landed with FE-1410)📜 Does this require a change to the docs?
The changes in this PR:
OPTIMIZATION_INTEGRATION.mdand the CLI README document the field, seed derivation, response shape, and pool controls. (The upcoming arch-docs PR folds these files into a Usage Manual.) The in-app user guide is untouched: the UI never setsseedsPerTrialyet, so in-app behaviour is unchanged until FE-1273's UI half lands.🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
turbo.json's have been updated to reflect thispetrinaut-cli'stest:unitnow depends on its ownbuild, so the built-CLI smoke test always runs against a fresh bundle in CI.seedsPerTrial > 1: its Node permission flags lack--allow-workerand its 240 s response timeout is unscaled. Such runs fail loudly at evaluate/bootstrap; nothing sends such manifests today.🐾 Next steps
🛡 What tests cover this?
optimization.test.ts— mean aggregation, per-seedreplicates, identical seed list across trials (CRN), non-finite replicate rejection,deriveTrialSeedsrange/stability/uniqueness.simulation-pool.test.ts— pool scheduling and ordering, per-run failure vs. worker crash, readiness failure, dispose.transports.test.ts— end-to-end multi-seed evaluate over stdio,seedsPerTrialin describe.built-cli.test.ts— spawns the real bundled CLI with two worker threads and checks the full describe/evaluate exchange.❓ How to test this?
turbo run test:unit --filter @hashintel/petrinaut-cli"seedsPerTrial": 4toexecutioninlibs/@hashintel/petrinaut-cli/examples/supply-chain-profit-optimization.json,yarn workspace @hashintel/petrinaut-cli build, thennode libs/@hashintel/petrinaut-cli/dist/cli.js serve --optimization <manifest> --stdioand send anoptimization.evaluaterequest.objectiveplus fourreplicates, and that repeating the request returns identical values.🤖 Generated with Claude Code