Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions bench/tx-generator/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,29 @@
# ChangeLog

## 2.17 -- Jun 2026

* **New remote submission endpoint** — send transactions to a remote endpoint
instead of the local socket / Node-to-Node protocols, behind a generic,
backend-agnostic transport interface.
* Turn it on with the optional `submissionEndpointProtocol` and
`submissionEndpointURI` keys, which must be set together, e.g.
`"submissionEndpointProtocol": "Ogmios", "submissionEndpointURI": "ws://127.0.0.1:1337"`.
* The only endpoint type currently supported is `Ogmios`: each transaction is
sent over a WebSocket as a JSON-RPC 2.0 `submitTransaction` call.
* It reroutes **every** phase, but only tx submission: genesis fund import, UTxO splitting, and benchmarking.
* It is a **functional transport, not a benchmark**: no TPS pacing, no metrics.
* So it requires `debugMode: true`. The compiler rejects an endpoint config on its own.
* **A rejected transaction fails the whole run** (the process exits non-zero):
* Setup phases stop at the first rejection.
* The benchmark phase finishes the stream, then fails if anything was rejected.
* Rejections are traced through the normal pipeline, including the detail the endpoint reports.

* **Fix: clean exit for non-benchmark runs.** Scripts that never start the
Node-to-Node benchmark machinery now exit cleanly instead of crashing at
shutdown with "AsyncBenchmarkControl absent".
* Affects `debugMode: true` runs and low-level `json` scripts with no
`Benchmark` submit phase.

## 2.16 -- Apr 2026

* Added a `--testnet-config-dir` flag to `tx-generator json_highlevel` that auto-discovers connection settings config (socket path, signing key, node config, target nodes) from a `cardano-testnet` output directory.
Expand Down
26 changes: 26 additions & 0 deletions bench/tx-generator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@ cabal run tx-generator -- json_highlevel config.json \
| `targetNodes` | List of nodes to submit transactions to (Node-to-Node protocol) | Yes |
| `nodeConfigFile` | Path to node configuration file | Yes |
| `sigKey` | Path to signing key with sufficient funds (genesis key) | Yes |
| `submissionEndpointProtocol` | Optional submission endpoint kind (`Ogmios`); set together with `submissionEndpointURI` | No |
| `submissionEndpointURI` | Optional submission endpoint URI (e.g. `ws://host:port`); set together with `submissionEndpointProtocol` | No |

### Transaction Settings

Expand Down Expand Up @@ -111,6 +113,30 @@ The high-level JSON configuration is automatically compiled into a multi-phase s

**Important**: All phases use the **local node socket** for setup (phases 1-3), and only phase 4 connects to **target nodes** via Node-to-Node protocol for actual benchmarking.

### Submitting through a remote endpoint

Setting a submission endpoint (`submissionEndpointProtocol` together with `submissionEndpointURI`) reroutes the submission of **every** phase — genesis fund import, UTxO splitting, and the final phase — through that endpoint instead of the local socket / Node-to-Node protocols.

This is a **functional submission transport, not a benchmarking one**:

- Transactions are submitted strictly one per round trip, so throughput is bounded by the latency to the endpoint; `tps` pacing and `targetNodes` are not used.
- No benchmark metrics or submission summary are produced; per-transaction rejections (including the failure detail reported by the endpoint) and a final sent/failed count go to the trace output.
- Rejected transactions make the run fail: setup phases (genesis import, splitting) abort at the first rejection, and the final phase exits non-zero if any transaction was rejected. Exit codes can be trusted in scripts.
- Because no real benchmark runs, the config compiler requires `debugMode: true` alongside the endpoint and rejects the config otherwise.

Note that the **local node socket and config file are still required**: protocol-parameter and era queries as well as protocol startup keep using the local node; only transaction submission goes through the endpoint.

#### Ogmios

The only endpoint type currently supported is `Ogmios`. Set:

```json
"submissionEndpointProtocol": "Ogmios",
"submissionEndpointURI": "ws://127.0.0.1:1337"
```

Transactions are sent to the [Ogmios](https://ogmios.dev) endpoint as JSON-RPC 2.0 `submitTransaction` calls over a WebSocket (only plain `ws://` is supported).

## Low-Level JSON Script (Advanced)

For fine-grained control, use low-level JSON scripts. Example in `test/script.json`:
Expand Down
48 changes: 40 additions & 8 deletions bench/tx-generator/src/Cardano/Benchmarking/Compiler.hs
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,9 @@ importGenesisFunds = do
wallet <- newWallet "genesis_wallet"
era <- askNixOption _nix_era
txParams <- askNixOption txGenTxParams
setupMode <- getSetupSubmitMode
cmd1 (ReadSigningKey keyNameGenesisInputFund) _nix_sigKey
emit $ Submit era LocalSocket txParams $ SecureGenesis wallet keyNameGenesisInputFund keyNameTxGenFunds
emit $ Submit era setupMode txParams $ SecureGenesis wallet keyNameGenesisInputFund keyNameTxGenFunds
delay
logMsg "Importing Genesis Fund. Done."
return wallet
Expand All @@ -102,7 +103,8 @@ addCollaterals src = do
(PayToAddr keyNameCollaterals collateralWallet)
(PayToAddr keyNameTxGenFunds src)
[ safeCollateral ]
emit $ Submit era LocalSocket txParams generator
setupMode <- getSetupSubmitMode
emit $ Submit era setupMode txParams generator
logMsg "Create collaterals. Done."
return $ Just collateralWallet

Expand All @@ -129,7 +131,8 @@ splittingPhase srcWallet = do
let generator = case split of
SplitWithChange lovelace count -> Split src payMode (PayToAddr keyNameTxGenFunds src) $ replicate count lovelace
FullSplits txCount -> Take txCount $ Cycle $ SplitN src payMode maxOutputsPerTx
emit $ Submit era LocalSocket txParams generator
setupMode <- getSetupSubmitMode
emit $ Submit era setupMode txParams generator
delay
logMsg "Splitting step: Done"

Expand Down Expand Up @@ -195,16 +198,27 @@ benchmarkingPhase wallet collateralWallet = do
inputs <- askNixOption _nix_inputs_per_tx
outputs <- askNixOption _nix_outputs_per_tx
txParams <- askNixOption txGenTxParams
endpoint <- resolveSubmissionEndpoint
doneWallet <- newWallet "done_wallet"
-- A submission endpoint is a functional submission transport, not a
-- benchmarking one: it ignores tps and targetNodes and produces no
-- submission metrics, so a config that asks for a real benchmark must fail
-- fast here instead of running unpaced and unmeasured.
submitMode <- case (endpoint, debugMode) of
(Just (eType, uri), True) -> pure $ SubmitToEndpoint eType uri
Comment thread
palas marked this conversation as resolved.
(Just _, False) -> throwCompileError $ SomeCompilerError
"submissionEndpointURI is a functional submission transport: it ignores \
\tps and targetNodes and produces no benchmark metrics. Set debugMode: \
\true to acknowledge this, or remove it to run a real benchmark."
(Nothing, True) -> pure LocalSocket
(Nothing, False) -> pure $ Benchmark targetNodes tps txCount
let
payMode = PayToAddr keyNameBenchmarkDone doneWallet
submitMode = if debugMode
then LocalSocket
else Benchmark targetNodes tps txCount
generator = Take txCount $ Cycle $ NtoM wallet payMode inputs outputs (Just $ txParamAddTxSize txParams) collateralWallet
emit $ Submit era submitMode txParams generator
unless debugMode $ do
emit WaitBenchmark
case submitMode of
Benchmark {} -> emit WaitBenchmark
_ -> return ()
return doneWallet

data Fees = Fees {
Expand Down Expand Up @@ -246,6 +260,24 @@ cmd1 cmd arg = emit . cmd =<< askNixOption arg
askNixOption :: (NixServiceOptions -> v) -> Compiler v
askNixOption = asks

getSetupSubmitMode :: Compiler SubmitMode
getSetupSubmitMode =
maybe LocalSocket (uncurry SubmitToEndpoint) <$> resolveSubmissionEndpoint

-- | Resolve the configured submission endpoint, requiring its type and URI to
-- be set together (or both omitted). The URI itself is already parsed: decoding
-- the config only accepts a well-formed absolute URI.
resolveSubmissionEndpoint :: Compiler (Maybe (SubmissionEndpointProtocol, EndpointUri))
resolveSubmissionEndpoint = do
mType <- askNixOption _nix_submissionEndpointProtocol
mUri <- askNixOption _nix_submissionEndpointURI
case (mType, mUri) of
(Nothing, Nothing) -> pure Nothing
(Just t, Just u) -> pure $ Just (t, u)
_ -> throwCompileError $ SomeCompilerError
"submissionEndpointProtocol and submissionEndpointURI must be set together \
\(or both omitted)."

delay :: Compiler ()
delay = cmd1 Delay _nix_init_cooldown

Expand Down
33 changes: 12 additions & 21 deletions bench/tx-generator/src/Cardano/Benchmarking/Script.hs
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,9 @@ import Cardano.Benchmarking.LogTypes
import Cardano.Benchmarking.Script.Action
import Cardano.Benchmarking.Script.Aeson (parseScriptFileAeson)
import Cardano.Benchmarking.Script.Core (setProtocolParameters)
import qualified Cardano.Benchmarking.Script.Env as Env (ActionM, Env (..), Error (TxGenError),
import qualified Cardano.Benchmarking.Script.Env as Env (ActionM, Env (..), Error,
getEnvThreads, runActionMEnv, traceError)
import Cardano.Benchmarking.Script.Types
import qualified Cardano.TxGenerator.Types as Types (TxGenError (..))

import Prelude

Expand All @@ -26,46 +25,38 @@ import Control.Concurrent.STM.TVar as STM (readTVar)
import Control.Monad
import Control.Monad.IO.Class
import Control.Monad.STM as STM (atomically)
import Control.Monad.Trans.Except as Except (throwE)
import qualified Data.List as List (unwords)
import System.Mem (performGC)

type Script = [Action]

runScript :: Env.Env -> Script -> EnvConsts -> IO (Either Env.Error (), AsyncBenchmarkControl)
-- | Run a benchmarking script. The second component of the result carries
-- the 'AsyncBenchmarkControl' of the submission threads when the script
-- started any: submit modes other than 'Benchmark' (e.g. 'LocalSocket',
-- 'Ogmios') never create one, in which case it is 'Nothing'.
runScript :: Env.Env -> Script -> EnvConsts -> IO (Either Env.Error (), Maybe AsyncBenchmarkControl)
runScript env script constants@EnvConsts { .. } = do
result <- go
performGC
threadDelay $ 150 * 1_000
return result
where
go :: IO (Either Env.Error (), AsyncBenchmarkControl)
go :: IO (Either Env.Error (), Maybe AsyncBenchmarkControl)
go = Env.runActionMEnv env execScript constants >>= \case
(Right abc, env', ()) -> do
(Right abcMaybe, env', ()) -> do
cleanup env' shutDownLogging
pure (Right (), abc)
pure (Right (), abcMaybe)
(Left err, env', ()) -> do
cleanup env' (Env.traceError (show err) >> shutDownLogging)
abcMaybe <- STM.atomically $ STM.readTVar envThreads
case abcMaybe of
Just abc -> pure (Left err, abc)
Nothing -> error $ List.unwords
[ "Cardano.Benchmarking.Script.runScript:"
, "AsyncBenchmarkControl uninitialized" ]
pure (Left err, abcMaybe)
where
cleanup :: Env.Env -> Env.ActionM () -> IO ()
cleanup env' acts = void $ Env.runActionMEnv env' acts constants
execScript :: Env.ActionM AsyncBenchmarkControl
execScript :: Env.ActionM (Maybe AsyncBenchmarkControl)
execScript = do
setProtocolParameters QueryLocalNode
forM_ script action
abcMaybe <- Env.getEnvThreads
case abcMaybe of
Nothing -> throwE $ Env.TxGenError $ Types.TxGenError $
List.unwords
[ "Cardano.Benchmarking.Script.runScript:"
, "AsyncBenchmarkControl absent from map in execScript" ]
Just abc -> pure abc
Env.getEnvThreads

shutDownLogging :: Env.ActionM ()
shutDownLogging = do
Expand Down
7 changes: 7 additions & 0 deletions bench/tx-generator/src/Cardano/Benchmarking/Script/Core.hs
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ import Cardano.Benchmarking.LogTypes as Core (AsyncBenchmarkControl (.
import Cardano.Benchmarking.OuroborosImports as Core (LocalSubmitTx, SigningKeyFile,
makeLocalConnectInfo, protocolToCodecConfig)
import Cardano.Benchmarking.Script.Aeson (prettyPrintOrdered, readProtocolParametersFile)
import qualified Cardano.Benchmarking.Script.Ogmios as OgmiosBackend
import qualified Cardano.Benchmarking.Script.Submission as Submission
import Cardano.Benchmarking.Script.Env hiding (Error (TxGenError))
import qualified Cardano.Benchmarking.Script.Env as Env (Error (TxGenError))
import Cardano.Benchmarking.Script.Types
Expand Down Expand Up @@ -244,6 +246,11 @@ submitInEra submitMode generator txParams era = do
NodeToNode _ -> error "NodeToNode deprecated: ToDo: remove"
Benchmark nodes tpsRate txCount -> benchmarkTxStream txStream nodes tpsRate txCount era
LocalSocket -> submitAll (void . localSubmitTx . Utils.mkTxInModeCardano) txStream
SubmitToEndpoint endpointType (EndpointUri uri) -> case endpointType of
Ogmios -> Submission.runSubmitTransport
(Submission.onRejectionFor generator)
(OgmiosBackend.withOgmiosTransport uri)
txStream
DumpToFile filePath -> liftIO $ Streaming.writeFile filePath $ Streaming.map showTx txStream
DiscardTX -> liftIO $ Streaming.mapM_ forceTx txStream
where
Expand Down
Loading
Loading