Skip to content

workbench: make the tx-generator a workload - #6648

Draft
Russoul wants to merge 8 commits into
masterfrom
russoul/tx-generator-workload
Draft

workbench: make the tx-generator a workload#6648
Russoul wants to merge 8 commits into
masterfrom
russoul/tx-generator-workload

Conversation

@Russoul

@Russoul Russoul commented Aug 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Description

The tx-generator was a first-class service of the workbench, side by side with the nodes, the tracer and the healthcheck. It is now just an entry of the profile's workloads list, named tx-generator, which removes the generator-specific code paths from every backend (start-generator, allocate-run-directory-generator, stop-all-generator, download-{logs,zstd,config}-generator, generator-service.json, the withGenerator supervisord program, ...).

Two generalizations of the workloads machinery make this possible:

  • phase (before-nodes | setup | load) replaces the before_nodes boolean. Workloads of the new setup phase run to completion after the nodes are up and before any load workload is started; the scenario fails the run if a setup workload exits non-zero.
  • placement (producers | explorer) plus a single entrypoint replaces the entrypoints pair. Voting's pre_generator entrypoint, which used to be textually inlined into the generator's start.sh, is now a voting-setup workload of the setup phase, placed next to the tx-generator.

On the Nomad backends, placement is resolved by a new workload-target-nodes subcommand, which replaces the hardcoded "every producer" assumption in log fetching, stopping and waiting.

Profile-side, the typed Generator record is kept internally (derived properties need it) and translated to/from the tx-generator workload entry at the JSON boundary. All 132 golden profiles are regenerated accordingly; the profile name set is unchanged.

Backwards compatibility: locli and the run directory keep reading legacy runs — the generator's configuration is looked up as a top-level generator property first and as the tx-generator workload afterwards — so no historical run needs a meta.json migration.

Follow-up commits, all found while reviewing and test-running this:

  • tx_fee of the block memory budget profiles. An overlay's "generator" key can no longer reach the generator through the profile's JSON encoding, so plutus-volt-memx15-nomadperf, plutus-volt-memx2-nomadperf, plutuscall-loop-memx2 and the two calibrate-blockmem-*-fill playground profiles silently lost their calibrated fee and would have rejected every transaction with FeeTooSmallUTxO. The fee moves to the txFeeOverwrite primitive and overlay now rejects a "generator" key.
  • startsecs is dropped for setup phase workloads, which are expected to run to completion; the other phases keep it as the immediate-crash detector.
  • The setup phase waited for the load phase's proposals. workflow_generator waited for the total proposals_count, which includes the withdrawal proposals the producers create in the load phase — unreachable now that the phases no longer overlap. It waits for its own constitution instead.
  • Two leftover references to the old generator/ directory.

Three more, all pre-existing breakage of the voting workload that this branch merely surfaced (identical on master, where voting profiles are equally stuck):

  • cardano-cli conway transaction txid prints JSON now, so query tx-mempool tx-exists never resolved and the fund splitting looped forever.
  • The proposals' anchor URLs pointed at the CIPs repo's master branch, whose content drifted away from the recorded hashes; they are pinned to a commit (abbreviated — the ledger caps an anchor URL at 128 bytes).
  • The constitution proposal's --constitution-hash was the hash of a dummy local text file, never of the IPFS document --constitution-url points at.

Checklist

  • Commit sequence broadly makes sense and commits have useful messages
  • New tests are added if needed and existing tests are updated. These may include:
    • golden tests
    • property tests
    • roundtrip tests
    • integration tests
      See Running tests for more details
  • Any changes are noted in the CHANGELOG.md for affected package
  • The version bounds in .cabal files are updated
  • CI passes. See note on CI. The following CI checks are required:
    • Code is linted with hlint. See .github/workflows/check-hlint.yml to get the hlint version
    • Code is formatted with stylish-haskell. See .github/workflows/stylish-haskell.yml to get the stylish-haskell version
    • Code builds on Linux, MacOS and Windows for ghc-9.6 and ghc-9.12
  • Self-reviewed the diff

Russoul added 8 commits August 11, 2026 17:34
@Russoul
workbench: make the tx-generator a workload
97259b9 
The tx-generator was a first-class service of the workbench, side by side
with the nodes, the tracer and the healthcheck. It is now just an entry of
the profile's "workloads" list, named "tx-generator", which removes the
generator-specific code paths from every backend.

Two generalizations of the workloads machinery make this possible:

- "phase" ("before-nodes" | "setup" | "load") replaces the "before_nodes"
  boolean. Workloads of the new "setup" phase run to completion after the
  nodes are up and before any load workload is started.

- "placement" ("producers" | "explorer") plus a single "entrypoint"
  replaces the "entrypoints" pair. Voting's "pre_generator" entrypoint,
  which used to be inlined into the generator's "start.sh", is now a
  "voting-setup" workload of the "setup" phase.

The typed `Generator` record is kept internally, derived profile
properties need it, and translated to/from the "tx-generator" workload at
the JSON boundary.

`locli` and the run directory keep reading legacy runs: the generator's
configuration is looked up as a top-level "generator" property first and
as the "tx-generator" workload afterwards, so no run needs a "meta.json"
migration.
@Russoul
workbench: fix two leftover references to the generator's directory
d8c734d 
- `runner.nix` dumped `generator/{stdout,stderr}` when `wb start` failed,
  which is now `workloads/tx-generator/{stdout,stderr}`. The `|| true`
  made it print nothing precisely when the tx-generator's log was needed.

- `utils/diff.sh` special-cased a "generator-service.json" that the
  profile output no longer contains.
@Russoul
cardano-profile: keep the calibrated tx_fee of the memory budget pr…
922c3f4 
…ofiles

`P.overlay` merges its JSON object with the profile's JSON encoding, where
the generator is not a top-level property anymore but a "tx-generator"
entry of the "workloads" array. The "generator" key of
`calibrateLoopBlockMemx15` / `calibrateLoopBlockMemx2` was therefore
dropped when decoding the merged JSON back into a `Profile`, silently
reverting `tx_fee` to the base 1360000 of `V.plutusTypeLoop` in
"plutus-volt-memx15-nomadperf", "plutus-volt-memx2-nomadperf",
"plutuscall-loop-memx2" and the two "calibrate-blockmem-*-fill" playground
profiles. Those fees are the calibrated minimums the nodes' mempool
rejections asked for, so the profiles would reject every transaction with
"FeeTooSmallUTxO".

The fee moves out of the overlays and into the `txFeeOverwrite` primitive,
next to the block memory budget primitives it belongs with, and `overlay`
now refuses an object with a "generator" key so this can't silently happen
again.
@Russoul
workbench: don't require a workload to stay running for 5 seconds
ea944a7 
`startsecs` is what makes `supervisorctl start` fail when a program dies
immediately, which is how a crashing node or tx-generator is caught at
start with its logs dumped. A "setup" phase workload, though, is expected
to run to completion, and a short one would be reported as a failed start.
The scenario waits for those workloads and fails the run using their
"exit_code" file instead, so they don't need it.
@Russoul
workbench: fix the voting workload's transaction id extraction
b06fa96 
`cardano-cli conway transaction txid` prints a JSON object with a
"txhash" property, not a bare transaction id. The whole object ended up
in the `tx_id` variables, so `query tx-mempool tx-exists` rejected it
("Failed to deserialise as TxId", once a second) and the "wait until the
transaction is not in the mempool anymore" loop of `funds_submit_retry`
never finished: a voting run stalls in its fund splitting phase until the
scenario's force-termination. The same value is also fed to `jq` as
`--argjson`, which rejected it too.

Not related to the tx-generator becoming a workload: these three call
sites are the same on master, where voting profiles are equally stuck.
@Russoul
workbench: the voting setup phase waits only for its own proposals
33b072e 
The voting flow is now a "voting-setup" workload of the "setup" phase and
a "voting" workload of the "load" phase, and the latter is not started
until the former exits. But `workflow_generator` still waited for
`proposals_count` proposals: the total, which includes the withdrawal
proposals that the producers create in the "load" phase. The setup phase
could therefore never finish, it waited for proposals that only its
successor creates. Before this commit's parent the two overlapped, so the
total was the right thing to wait for.

Wait for the proposals it creates itself instead, the constitution(s) from
genesis. `workflow_producer_deployed` already waits for the full count
after creating its own proposal and before voting.
@Russoul
workbench: pin the voting proposals' anchor URLs to a CIPs commit
c015f8b 
`cardano-cli` downloads a proposal's "--anchor-url" to check it against
"--anchor-data-hash" when building the transaction. Both URLs pointed at
the "master" branch of "github.com/cardano-foundation/CIPs", so when the
CIP-0100 common schema changed upstream every voting run started failing
to create its constitution proposal:

  Error: Hash of the file is not valid. Hashes do not match!
  Expected: "9d99fbca260b2d77e6d3012204e1a8658f872637ae94cdb1d8a53f4369400aa9"
    Actual: "c407dda548dbbbfb4dc89b5a980f75fe0b7b6721d33d8f151ea3e711bede3cda"

Point both at a commit instead, and update the CIP-0100 hash to the one
of the file at that commit. The CIP-0108 example is unchanged there. The
commit hash is abbreviated because the ledger rejects an anchor URL of
more than 128 bytes ("Text exceeds 128 bytes", 135 with the full hash).
@Russoul
workbench: hash the constitution proposal's own document
81d2bb6 
`cardano-cli` downloads "--constitution-url" when building the proposal's
transaction and refuses to build if the document does not hash to
"--constitution-hash":

  Error: Hash of the file is not valid. Url: https://ipfs.io/ipfs/Qmdo2J...
  Expected: "860f614bdfc26c070f53c7287460288e38d0569c54c43ebd7c90c5ad79b2e0ef"
    Actual: "a77245f63bc7504c6ce34383633729692388dc1823723b0ee9825743a87a6a6d"

The hash was the one of a dummy text file created next to it ("My
Constitution: free mate and asado"), which never was the content of the
IPFS document the proposal points to. Hash that document instead, with
`hash anchor-data --url`, and drop the dummy file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Russoul