keeper: reserve submit journal before the venue call#572
Draft
mfw78 wants to merge 91 commits into
Draft
Conversation
* feat(sdk): add the keeper conditional-source seam, retry ledger, and run Introduce the world-neutral half of the poll loop in nexum_sdk::keeper: a ConditionalSource trait (one watch in, one outcome out, generic over the host, no venue-transport pre-abstraction), a Tick carrying the dispatch instant, and the RetryAction (try-next-block / backoff / drop) whose effects the Retrier runs through the gate and watch-set stores. RetryAction moves out of shepherd-sdk; the CoW crate re-exports it so no module rewires. shepherd-sdk keeps classify_api_error as the CoW errorType table but returns the keeper action, and encodes two fixes in the one classification path: classify_poll_error maps an unrecognised revert selector with a structured payload to DontTryAgain instead of re-polling every block forever, and DuplicatedOrder (both spellings) classifies as already-submitted - the run records the submitted: receipt and keeps the watch rather than dropping every future tranche. classify_submit_error widens the table to the whole CowApiError surface and gives Backoff its producer: a rate-limit fault with server guidance gates the watch on the epoch clock. cow::run composes the loop: watch-set scan -> gate check -> source poll -> PollOutcome dispatch, driving submission through the existing CowApiHost seam with the journal as the idempotency guard and the retry ledger as the failure dispatch. Acceptance tests run against the composed shepherd-sdk-test MockHost as integration tests; no module is rewired yet. * docs(sdk): disambiguate the run intra-doc link * fix(cow): build the classify_poll_error test RpcError data as Bytes RpcError.data is now Bytes; the test helper takes raw Vec<u8> and wraps it (Vec -> Bytes is O(1)). * fix(cow): keep the sweep alive when a post-submit journal write faults journal.record after a successful submit_order was propagating a store fault out of submit_ready, aborting the whole watch sweep with the receipt unwritten - the next tick then re-polled and re-posted the same order. Log the failure and carry on instead; the already-submitted arm keeps the re-post idempotent. Both post-submit writes are covered: the Ok arm and the already-submitted (DuplicatedOrder) arm.
* feat(sdk): log run diagnostics through the tracing facade The run logged through the LoggingHost seam, which the guest tracing capture cannot observe - module tests proving behaviour identity for keeper ports assert on tracing events. Route the submit-path diagnostics through the tracing macros with the wording the legacy twap poll loop established, and give ConditionalSource a defaulted label so shared log lines attribute the strategy that produced them. * refactor(twap-monitor): port the poll loop onto the keeper composition Reduce strategy.rs to decode and evaluate: log decoding persists watches through the keeper watch set, and the getTradeableOrderWithSignature evaluation moves behind ConditionalSource. The hand-rolled gate keys, watch-update lifecycle, submitted: markers, and submit-retry dispatch are deleted in favour of cow::run over the keeper stores and retry ledger, still on the legacy CowApiHost seam. The MockHost dispatch tests are untouched - the behaviour-identity proof for the port.
* feat(sdk-test): namespace the mock local store and share its limits Rework MockLocalStore to mirror the runtime store's shape: namespaced views over one shared row map, so identical key strings in sibling namespaces never collide, plus store-wide entry and byte limits that span namespaces the way one database file does. Fault injection stays per-view. * feat(sdk-test): add the programmable MockVenue on the cow-api seam Script per-call venue behaviour where MockCowApi replays one response: a strictly-draining queue of submit outcomes with a configurable steady-state fallback, per-route response sequences whose final entry sticks (a terminal order status persists across re-polls), and venue fault injection that overrides both without consuming them. MockHost grows a defaulted venue type parameter so the composed host carries either mock on the same CowApiHost seam. Acceptance tests drive the keeper run across multi-tick retry, backoff, and outage scenarios, and module-shaped strategy code against status sequences.
* fix(sdk): drop the watch when the OrderCreation constructor rejects the order A constructor rejection (zero from, validTo beyond the client-side one-year horizon) is deterministic for the polled payload: skipping with the watch intact re-polled and re-warned on every block forever. Route it through the retry ledger as a drop, matching the pre-keeper net effect where the orderbook rejected the shipped body (e.g. ExcessiveValidTo) and the classifier dropped the watch. The unknown-enum-marker skip keeps its pinned keep-the-watch behaviour. Also log the keeper-level DontTryAgain removal so a permanent drop is observable even for sources that do not log their own outcomes. * fix(stop-loss): record the submitted receipt on a duplicate rejection classify_api_error maps DuplicatedOrder to TryNextBlock on the premise that the caller records the submitted: receipt - true for the run, but stop-loss consumed the classification without the compensating write, so a duplicate rejection re-POSTed every block until validTo. Mirror run::submit_ready: treat already-submitted as success wearing an error status, write the marker the dedup guard reads, and idle. * refactor(sdk): hoist the watch key to a free fn and shed the cross-layer dev cycle WatchSet::key never used the host parameter, forcing a meaningless turbofish at every call site (one test existed solely to prove two instantiations agree). keeper::watch_key is the free canon; WatchSet::key stays as a thin delegate for discoverability. The keeper acceptance tests only touch the local-store seam, so they now run against nexum-sdk-test's MockHost (the same MockLocalStore type), shrinking nexum-sdk's dev cycle from the cross-layer shepherd-sdk-test pair - which dragged the whole CoW domain layer into the world-neutral crate's dev graph - to the within-layer pair its own mock crate documents. * feat(twap-monitor): carry the revert cause on the permanent poll-drop path A revert that classifies to DontTryAgain destroyed the watch with only an Info outcome label; the selector and node message needed to triage a wrongly-dropped watch were discarded. Warn with both before returning the outcome, and pin the log lines in the drop test. The test-only watch_key wrapper now reuses the keeper free fn. * docs(sdk): correct stale order.rs rustdoc OrderCreation::from_signed_order_data was renamed to OrderCreation::new in cowprotocol 0.2; and only the unknown-marker case of order_uid_hex returning None also stops the submit path downstream - an unsupported chain id does not, so say so instead of claiming both do. * chore(backtest): retire the dead app_data resolution scripting The epic removed resolve_app_data and the orderbook-mock's GET /api/v1/app_data route, but the replay harness still programmed that route against a strategy that never requests it. Drop the dead programming (and the now-unused shepherd-sdk dependency) and refresh the module doc to the hash-only submission flow. The RejectedExpected classification is kept for historical report comparability until the harness is reworked around the observer-only strategy (follow-up). * docs: surface the keeper and run in the overview; record the cow-rs patch-channel move The overview's SDK table predated the epic's headline surface: add the nexum-sdk keeper row and the shepherd-sdk cow::run row. Amend ADR-0004 with the move of the [patch.crates-io] channel from bleu/cow-rs to nullislabs/cow-rs and what the pinned rev carries. * style: rustfmt the sweep additions * docs: reword PR-number references out of module comments Numeric issue/PR references in .rs files go stale across repository moves; describe the change instead of pointing at the review.
* feat(sdk): scaffold nexum-macros and ship #[module] attribute Add the `nexum-macros` proc-macro crate. `#[nexum_sdk::module]` generates the per-cdylib glue every module hand-wrote: the `wit_bindgen::generate!` call for the blanket `shepherd:cow/shepherd` world, the host adapter via `bind_host_via_wit_bindgen!`, the `Guest` implementation whose `on-event` dispatches to the named handlers (`init`, `on_block`, `on_chain_logs`, `on_tick`, `on_message`, with undefined handlers ignored), and `export!`. The WIT directory is located by walking up from the consuming crate's manifest, so the emitted paths need no per-module tuning. Re-export it as `nexum_sdk::module` and port the example, balance-tracker and price-alert modules onto it; each strategy MockHost suite passes unchanged. * refactor(http-probe): port onto #[nexum::module] Complete the acceptance list: drop the hand-written wit-bindgen glue, host adapter, and Guest/export block in favour of the attribute, leaving only the init and on-block handlers. Strategy MockHost suite unchanged.
Doc 05 was stamped future-direction and described the superseded macro/TypedState/Signer/HostTransport vision. Replace it with the shipped module-author persona (nexum-sdk + shepherd-sdk + the landed #[nexum::module] macro) and the planned venue-adapter persona (nexum-venue-sdk / per-venue-crate), clearly labelled as design intent tracked by a separate epic. Drop the deferred-features table and cross-link ADR-0009 and sdk.md.
* fix(macros): reject typo'd handlers and malformed impls at expansion A method named on_blocks (or any other on_-prefixed typo) previously compiled as an ordinary helper while its event silently no-opped; the only signal was a dead_code warning that vanishes on pub items. Reserve the on_ prefix for the recognised handler set and error on anything else. Also reject trait impls, generic impls, and impls with no recognised handlers - all previously slipped past the self-type guard and failed later with incidental, unhygienic errors (or compiled into a do-nothing module). Document the wit-bindgen direct-dependency and prelude- shadowing invariants inherited from the generated glue. * docs(migration): rewrite SDK section 7 around the shipped surface Section 7 still described the superseded Signer/Messaging/RemoteStore/ provider() vision - APIs that never shipped - and a #[shepherd::module] macro that does not exist, while doc 05 endorsed it as the authoritative 0.1 -> 0.2 SDK table. Replace it with the real story: 0.1 had no SDK crate, 0.2 ships nexum-sdk/shepherd-sdk host traits, typed errors, and the single #[nexum_sdk::module] attribute. Point doc 05's cross-link text at what the section now says, and fix doc 05 details flagged by the sweep: the CowApiHost parenthetical (submit_order, cow_api_request), the crate trees (proptests.rs in both SDK crates), the testing-story claim (nexum-runtime's feature-gated test_utils::TestRuntime is an in-tree component-level harness), and the first #[nexum::module] mention now names the real nexum_sdk path. * docs(sdk): document the #[nexum_sdk::module] attribute in sdk.md sdk.md is doc 05's designated day-to-day reference but still presented the pre-macro bind_host_via_wit_bindgen! flow as the authoring pattern. Add an authoring section covering the attribute and its handler set, mention it in the intro, and include nexum-macros in the rustdoc command.
feat(wit): author the nexum:value-flow types package Add the egress-neutral value-flow vocabulary at 0.1.0: the settlement domain (evm-chain or offchain), the asset variant (native-token, erc20, erc721, erc1155, service, offchain), the service and offchain descriptors, and the big-endian asset-amount. The package is dependency-free so it can outlive any interface built on it. Every identifier is vetted against WIT keywords, in-flight component-model proposals, and the nine binding-target reserved-word lists, preferring a two-word kebab id wherever a single word is a keyword anywhere: native-token over native (a Java modifier), offchain over external (a Dart keyword), value-flow over value (value-imports is circling that word). A wasmtime bindgen smoke, compiled under test in the host crate through a throwaway world, names every generated type and field by its plain Rust spelling so a keyword collision surfaces as an escaped identifier and fails the build rather than a downstream binding.
feat(wit): author the nexum:intent package with the pool interface Add the venue-neutral intent ontology at 0.1.0 over the value-flow vocabulary: the intent-header (gives/wants/valid-until/settlement/ authorisation), the auth-scheme variant (eip712, eip1271, presign, offchain-sig, unsigned), the intent-status lifecycle with a structured fail-reason, the opaque receipt alias, and a self-contained venue-error (deliberately not embedding the host fault type, so the package's freeze cadence is not pinned to nexum:host versioning). The pool interface routes submit/status/cancel by a venue string with the body as opaque bytes. submit returns a submit-outcome variant from day one: accepted(receipt), or requires-signing(unsigned-tx) for venues whose settlement is a host-signed on-chain call. The unsigned-tx record only describes the call (chain, to, value, input); the host routes it through the guard's host-signed class and fills gas and fees, so adapters still structurally cannot move value. Identifier hygiene follows the value-flow rule (internal-error rather than internal, a Swift declaration keyword). A wasmtime bindgen smoke, compiled under test in the host crate through a throwaway world that imports the pool interface, names every generated type, case, and field by its plain Rust spelling and pins the three pool function signatures with a dummy host impl.
* feat(runtime): introduce the venue-adapter component kind Add the second component kind alongside the event-module. A venue adapter speaks one venue's protocol over scoped transport only and exports the intent adapter face. - WIT: a nexum:intent/adapter interface (derive-header, submit, status, cancel) and a nexum:adapter/venue-adapter world that imports scoped chain and messaging, exports init plus the adapter face, and withholds the core-only primitives. - Bindings: a second bindgen path binding VenueAdapter beside EventModule, reusing the shared nexum:host interfaces via with. - Manifest: a module-kind discriminator defaulting to event-module, plus an adapter capability registry that recognises only the scoped transport. - Engine config: an [[adapters]] table carrying path, manifest, a per-adapter HTTP allowlist, and messaging-topic scopes. - Supervisor: adapters instantiate into supervised stores against a dedicated scoped linker, reusing the store, fuel, and restart machinery; the kind discriminator gates the load path. No routing yet. - Messaging: per-store content-topic scope enforced ahead of the deferred backend. * docs(runtime): keep the venue-adapter world token on one line The bindgen module rustdoc split the `nexum:adapter/venue-adapter` code span across a line break after the slash, so rustdoc rendered it with a stray space. Rewrap so the token stays intact.
) Implement the strategy-facing pool import as a host binding linked into every module linker, dispatching to a shared router that owns the installed adapters. - Bindings: a pool-host bindgen world importing nexum:intent/pool, reusing the intent and value-flow types from the venue-adapter bindings via `with`, so the outcome and error the router hands back to a module are the very types an adapter's submit produced. - Router: resolve a venue id to its adapter, serialise invocation per adapter behind an async mutex (the wasmtime Store is not Sync), sequence derive-header, a no-op guard interposition seam, then submit. Status and cancel pass through without the header, guard, or quota. - Quota: a per-caller sliding-window submission budget, with adapter decode failures charged to the caller so a module feeding garbage bodies exhausts its own budget rather than the adapter's fuel. - Supervisor: adapters instantiate first and install into the router; modules then boot carrying the shared handle, rebuilt identically on restart. An init-failed adapter is loaded but not routable. - Manifest: register nexum:intent/pool as a declarable module capability. - Config: a [limits.quota] section resolving the per-caller budget.
* feat(sdk): ship nexum-venue-sdk over the venue-adapter world Introduce the venue-author persona crate: the VenueAdapter trait and export macro over an in-crate bindgen of the adapter world, the borsh IntentBody derive whose outer version enum fails unknown tags typedly, the typed intent client core over the byte-level pool seam, and typed wrappers over the scoped chain, messaging, and wasi:http imports. The derive expansion lands in nexum-macros and re-exports from the SDK; borsh enters the workspace dependency table. * fix(venue-sdk): wrap the wit chain-error data into Bytes The SDK RpcError.data is Bytes; the wit-bindgen chain-error carries Vec<u8>, so wrap it (Vec -> Bytes is O(1)) when reprojecting.
* feat(sdk): capability-select the wit-bindgen host adapter Split bind_host_via_wit_bindgen! into a types-only base plus one block per capability, selected through a caps list; the zero-argument form keeps emitting the full chain, local_store, logging set for modules compiled against a blanket world. Narrow read_latest_answer to ChainHost + LoggingHost, the two capabilities it exercises, so modules whose worlds omit local-store can still call it. * feat(sdk): emit a per-module world from declared capabilities Teach #[nexum_sdk::module] to read the crate's module.toml and synthesize an inline WIT world whose imports are exactly the [capabilities].required and optional declarations, replacing the blanket shepherd:cow/shepherd world every module compiled against. The built component's imports now equal its declarations by construction, so the runtime's capability check no longer leans on the toolchain eliding unused imports; an undeclared capability has no bindings at all, and an unknown capability name is a compile-time error. The emitted host adapter is capability-selected to match, and a rebuild anchor on module.toml recompiles the module when the manifest changes. Narrow price-alert's strategy bound to ChainHost + LoggingHost: its world imports only its declared chain and logging capabilities, so the full Host supertrait (which adds local-store) is unimplementable there by design. * test(runtime): pin the example component's imports to its declarations The per-module world acceptance: compile the built example component and assert its capability-bearing imports resolve to exactly the manifest's declared set, with no extension interface leaking in. Skips gracefully when the wasm fixture is not built, like the other e2e tests. * docs: record the retirement of import-elision for macro-built modules Macro-built components import what they declare by construction, so capability enforcement is a backstop for them; the elision dependency ADR-0009 flagged now applies only to hand-rolled modules still compiled against the supertype world.
* feat(runtime): add the intent-status case to the host event variant The host event variant gains intent-status(intent-status-update): a venue id, the receipt, and the nexum:intent status vocabulary, so a subscriber sees where an intent is in its life without knowing how the host learnt it. nexum:host now depends on nexum:intent, so every bindgen over the host package carries the intent and value-flow packages on its resolve path, in dependency order; the runtime generates the intent types once and the adapter and pool bindgens remap onto them with 'with'. The module macro recognizes an on_intent_status handler and the example module ships one. * feat(runtime): poll venue statuses and fan intent-status events to subscribers The pool router puts every accepted receipt under watch and, on a configurable cadence ([limits.status_poll] interval_ms), polls each adapter's status export, reporting only transitions: the first poll reports the current status, repeats are deduplicated, a terminal status prunes the watch, and a receipt the venue disowns is dropped. A dedicated event-loop stream fans each transition to modules declaring [[subscription]] kind = "intent-status", optionally filtered by venue; polling only runs when there is at least one subscriber and one installed adapter.
Extend nexum-macros with the venue-adapter counterpart to #[module]: read the crate's module.toml, synthesize a per-component venue-adapter world exporting init plus nexum:intent/adapter and importing exactly the declared scoped transport, then emit the per-cdylib wit-bindgen call, the Guest export glue wiring the adapter's associated functions, and export!. An adapter built this way imports what its manifest declares and nothing else, retiring the toolchain-elision dependency on the venue side rather than as follow-on work. A capability outside the venue-permitted set (chain, messaging, http) is rejected at expansion, so an adapter structurally cannot reach host key material or persistent state. Ship echo-venue as the reference adapter and pin its built component's capability-bearing imports to its single declared capability.
Venue adapters need a way to prove their wire behaviour against the venue's published contract, in a form a non-Rust author can consume: codec vectors and header goldens ship as JSON files (bytes as lowercase hex) rather than Rust-only test code. CodecVectors publishes IntentBody wire bytes and holds a codec to them: round-trip vectors must decode and re-encode byte-exactly, and the failure vectors pin the typed empty, unknown-version, and malformed error contract. HeaderGoldens pairs published bodies with the header a conforming derive-header projects, spelled in serde mirror types whose case names match the WIT. MockTransport composes chain, messaging, and outbound HTTP mocks behind the SDK seams, including the messaging_topics scoping refusal. A reference venue (schema, derivation, and checked-in fixture files pinned by regeneration tests) documents both formats and proves the borsh primitive encodings for authors porting the codec.
feat(examples): pair echo-venue with an echo-client and prove the round-trip Add echo-client, the strategy half of the echo pair: it submits an opaque body through nexum:intent/pool on every block and logs the intent-status transitions the router fans back. Wire both real components through an integration test that proves the intent core end to end, module -> host router -> echo adapter submit and status back, which is the first-train acceptance path. Settle the echo venue instantly (status now reports settled) so the round-trip reaches a terminal transition, and hold its header derivation to the nexum-venue-test kit so echo-venue doubles as the conformance target alongside its tutorial role. Register echo-client in the workspace, the build recipes, and CI.
* docs(wit): pin canonical amount encoding and reserve native-token(offchain) Mandate minimal-length big-endian for asset-amount and require decoders to compare by value, so the same amount has one canonical wire form across the binding targets. Document native-token(offchain) as representable-but-invalid: an off-chain settlement domain has no gas token, so consumers must reject the pairing rather than interpret it. * fix(macros): correct venue manifest error attribution and accumulate venue packages The shared missing-[capabilities] error named #[nexum_sdk::module] even on the venue path; make it macro-neutral so a venue crate is not told to use the wrong attribute. Mirror synthesize's package-accumulation loop in synthesize_venue so a future venue capability carrying an extra WIT package reaches the resolve path (all venue capabilities are packageless today, so the base set is unchanged). * docs: normalise doc comments to British -ise spelling Align the authorise/authorisation prose in the intent WIT and the conformance kit with the British spelling used elsewhere and with the authorisation identifier itself. Doc comments only; no identifier, wire format, or API name changes. * docs(runtime): document the pool submit charge asymmetry State the deliberate rule that a forwarded submission is charged once the guard admits it regardless of the venue outcome, while a derive-stage non-decode venue error is left uncharged so the caller may retry.
* feat(cow-venue): scaffold default body slice with borsh IntentBody codec Stage the CoW venue as a crate of feature slices. The default `body` slice carries the venue-neutral order and composable intent body types and the borsh IntentBody codec over the outer per-venue version enum, so a future adapter component or a strategy module can link the bodies and codec without the host-side CoW machinery. The slice links only the venue SDK (for the IntentBody derive) and borsh; `--no-default-features` drops it to an empty crate. shepherd-sdk gains a re-export shim under `cow` so the module ports can move off the legacy path without a breaking rename. * style(cow-venue): use -ize spelling and drop test reach into borsh __private
* feat(cow-venue): drive cow retry classification from a shipped data table Ship the CoW orderbook errorType retry policy as data in crates/cow-venue/data/classification.toml, readable and editable by non-Rust authors, and add a `client` feature slice that embeds it, exposes a typed CowClient bound to the CoW venue, and a table-driven classification API returning the keeper RetryAction. Replace the hand-coded classify_api_error/is_retriable in the cow error surface with the table lookup, preserving the already-submitted and drop-on-unrecognized behaviour. A throttle errorType now maps to Backoff, so every RetryAction arm is reachable from the table alone, guarded by a data-vs-code parity test and an untyped-parse test that proves any TOML reader consumes the same file. * perf(cow-venue): generate the classification table at build time shepherd-sdk unconditionally enables the cow-venue client slice, and the wasm modules (twap-monitor, ethflow-watcher) reach classify_api_error through shepherd_sdk::cow::run, so the runtime LazyLock TOML parse linked the full toml/serde/winnow stack into every guest: the twap-monitor wasm grew ~237 KiB (350 KiB -> 586 KiB). Move the parse to build.rs. It reads data/classification.toml, validates the table invariants, and emits a static lookup table; the runtime slice carries only generated rows and no TOML parser. serde/toml/thiserror become build- and dev-only. The parse and invariants live in a shared classification_data module that build.rs and the parity test both link, so the data-vs-code parity test now re-parses the shipped file and checks the generated table against it. The guest returns to 351 KiB.
* fix(cow): use canonical InvalidEip1271Signature errorType spelling The permanent_kinds_yield_drop test asserted on "InvalidErc1271Signature", which is not a real CoW orderbook errorType (the OpenAPI OrderPostError enum and cowprotocol's OrderbookApiErrorType both spell it InvalidEip1271Signature, matching data/classification.toml). The phantom spelling passed only via the unlisted-default Drop, so it never exercised the real table row. Use the canonical spelling so the test covers the actual entry. * docs(cow-venue): ratify deliberate divergence from cowprotocol retry_hint shepherd-sdk already depends on cowprotocol, whose ApiError::retry_hint() classifies the same orderbook errorTypes and disagrees with this table on several (e.g. it retries/backs off InvalidEip1271Signature, InsufficientBalance, InsufficientAllowance and InvalidAppData where this table drops, and backs off TooManyLimitOrders for an hour rather than 30s). The table is deliberately not delegated to it: it is shepherd's own, more conservative policy, kept as data of record so a non-Rust author owns it and the guest client slice stays free of the upstream error module. Record that this divergence is intentional so a future maintainer revisits the policy here rather than swapping the source of truth. (Also fixes the -ise spelling in the same comment.) * style(cow-venue): adopt British -ise spelling to match house convention The crate used American -ize (normalizes/normalization, unrecognized) while the adjacent domain code (shepherd-sdk cow/order.rs normalised, composable.rs unrecognised, run.rs) and the repo at large use British -ise. Align the prose so the sibling modules read consistently. * fix(cow-venue): allow GenAction variants unused by shipped data GenAction's variants are constructed only by the build-generated table, so a classification.toml that carries no row of a given action leaves that variant unconstructed and trips the -D warnings build. Since which actions appear is a property of the data, not the code, mark the enum allow(dead_code) so a data-only edit cannot break the workspace lint gate.
…ve 1, ADR-0013) (#334)
Tip-level residual after the chassis to keeper rename was folded down through the train. Carries the doc-comment prose the fold could not move mechanically (the private-keeper caveat, keeper-run wording) and the current-state design docs brought in line with the Verdict poll seam (PollOutcome to Verdict, decode_revert to LegacyRevertAdapter).
…351) * docs: multi-chain deployment patterns — Mainnet/Gnosis/Arbitrum/Base (#124) New docs/deployment/multi-chain.md covering the verbatim M5 grant deliverable: per-chain [chains.<id>] + env-var wiring, CoW orderbook URL slugs per network, per-chain contract addresses (CREATE2-stable vs EthFlow per-network caveat), the [[subscription]] duplication pattern with twap-monitor and ethflow-watcher examples, event topic reference, and resource sizing guidance. * docs(multi-chain): correct the EthFlow address claim and scope statements Review catches: - The EthFlow section claimed the address is per-network and told operators to hunt for a different mainnet address. The current production deployment is address-identical on every supported chain (cowprotocol's ETH_FLOW_PRODUCTION, from networks.prod.json) - the old text would have steered a mainnet port toward the legacy v1.0.0 deployment and missed every OrderPlacement event. The real caveat is narrower and now stated as such: sameness is not a CREATE2 guarantee, legacy per-version deployments exist, verify on version bumps. - "every chain the orderbook supports" was false (the Chain enum also carries BNB, Polygon, Avalanche, Linea, Plasma); the table is scoped to the chains this repo's modules target. - Dropped the unsupported "Gnosis volume roughly equal to Mainnet" claim; require_ws violations are an ERROR log, not a warning; the RPC recommendation names a rate requirement instead of a specific provider's free tier. --------- Co-authored-by: Luiz Gustavo Abou Hatem de Liz <luizgustavoahsc@gmail.com>
…ata (#352) * docs: stale-reference sweep — engine rename, Dockerfile, ADR-0007 errata (#123, #108, #62) - deployment.md: replace the "planned for M5" Dockerfile sketch with a pointer to the real Dockerfile + docker.md; update nexum-runtime log directive examples - qa-signoff.md: nexum-engine → nexum-runtime in crate-dependency row - adr/0007: add errata noting the nexum-engine → nexum-runtime rename and the [patch.crates-io] retirement now that cowprotocol 0.1.0 is on crates.io * docs: fix the sweep's own stale claims - cowprotocol override, production.md 3.1 Review catches on the stale-reference sweep itself: - ADR-0007 errata and sdk.md Versioning both claimed cowprotocol is a clean 0.1.0 registry dependency "with no git override". The branch's own Cargo.toml pins 0.2.0 WITH an active [patch.crates-io] override to nullislabs/cow-rs (pending the hash-only OrderCreationAppData constructor release). Both now describe that state and point at the patch-block comment as the source of truth. - production.md 3.1 still shipped the "interim" hand-rolled Dockerfile recipe issue #123 flagged; replaced with a pointer to the real root Dockerfile and docs/deployment/docker.md. - 02-modules-events-packaging.md: un-garble the parenthetical the earlier rename made self-contradictory ("was module.toml in earlier drafts" -> "was nexum.toml"). --------- Co-authored-by: Luiz Gustavo Abou Hatem de Liz <luizgustavoahsc@gmail.com>
* test(event-loop): add ordering and graceful-shutdown tests (#56, #58) Four new tests covering the two open issues: Issue #56 — event ordering guarantees: - `open_block_streams_opens_one_task_per_chain`: structural check that N chains → N independent reconnect tasks, documenting the per-chain task isolation that prevents stream-A starvation from blocking stream-B events. - `open_chain_log_streams_opens_one_task_per_subscription`: same structural check for chain-log subscriptions. - `run_delivers_block_and_chain_log_events_without_starvation`: integration test over a zero-module supervisor that pre-queues one block and one chain-log event, runs the loop, and verifies it drains without hanging — the `biased` select delivers both event kinds in the same session. - `harness_delivers_block_and_chain_log_events_without_starvation`: E2E test over the real example module (skipped when wasm not built) that pushes a block and a chain-log and waits for both dispatch log lines. Issue #58 — graceful shutdown completion: - `run_drains_reconnect_tasks_cleanly_on_shutdown`: verifies `run()` awaits `tasks.shutdown()` before returning — reconnect tasks observe a closed channel (ReceiverGone) rather than an abort, proving clean teardown. - `harness_shutdown_after_push_completes_cleanly`: E2E test that calls shutdown immediately after a block push; `wait()` returning Ok proves no partial dispatch corrupts module state. * test(event-loop): make the #56/#58 tests falsifiable Review follow-ups - three of the six tests could not fail for the property they claimed to verify: - run() now returns its (blocks, chain_logs) dispatch tally (the same numbers the shutdown log line reports). The no-starvation test asserts both queued events were drained instead of merely observing that run() terminates on the shutdown timer - a broken or reordered select arm now leaves a count at 0 and fails. - New direct test: a reconnect task parked on a dropped receiver exits with TaskExit::ReceiverGone on its own, joined on the bare handle. The previous test claimed to verify this through TaskSet::shutdown, which aborts every handle before joining, so ReceiverGone was never exercised; its doc comment now states the abort-then-join contract it actually proves. - harness_shutdown_after_push_completes_cleanly raced shutdown against dispatch and accepted either outcome. Replaced with harness_shutdown_preserves_completed_dispatch: prove the dispatch completed, tear down, then re-read the log record after teardown. - The harness no-starvation test now queues both event kinds before awaiting either, so the biased select genuinely arbitrates two ready streams instead of replaying the two pre-existing sequential tests. - New harness_delivers_blocks_in_push_order: blocks 7,8,9 pushed back-to-back must surface in the module's log records in that order - issue #56's ordering guarantee, previously untested. - run()-level tests wrapped in tokio::time::timeout so a shutdown-path hang fails in 10 s instead of stalling the suite to the CI job limit. * test(event-loop): adapt test call sites to the ChainLogSub struct The train base moved under the branch again: open_chain_log_streams now takes Vec<ChainLogSub> (resume cursors + max_lookback) instead of (module, chain, filter) tuples, and the rebase left the two test call sites building tuples. Construct ChainLogSub with no cursor and no lookback - these tests exercise stream/task plumbing, not resume. --------- Co-authored-by: Luiz Gustavo Abou Hatem de Liz <luizgustavoahsc@gmail.com>
#354) * feat(chain): cap JSON-RPC response size before lowering into the guest (#154) Adds a configurable `[limits.chain] response_max_bytes` knob (default 1 MiB) that is enforced host-side in `chain::request()` before the response string is copied into the guest heap. Oversized responses are rejected with an `InvalidInput` fault, a `WARN` log, and a dedicated `shepherd_chain_response_capped_total` metric. * fix(#154): bound batch aggregates, harden config, test the real wiring Review follow-ups on the chain response cap: - request_batch: the per-entry cap bounds each body, but the aggregate Vec<RpcResult> lowered into guest memory was unbounded - ~64 entries just under the cap is ~64 MiB, exactly the guest-heap saturation issue #154 exists to prevent, reachable via the block-range chunking the issue itself recommends. A running total now converts entries past the cap into typed invalid-input results. - config: renamed [limits.chain].response_max_bytes to response_body_max_bytes (u64) for exact symmetry with the [limits.http] sibling; a degenerate 0 saturates to 1, matching the logs/poison zero handling. - new harness test drives the cap through the real path - config -> ModuleLimits -> HostState -> chain::request - with the price-alert module: an over-cap oracle response surfaces to the guest as the typed fault and never reaches classify. The existing unit tests only covered the free function, so a deleted and_then would have passed the suite. - TOML parse tests for [limits.chain] (absent -> 1 MiB default, override, zero saturation), mirroring the HTTP section's tests. - engine.example.toml documents the new section; warn message dash matches house style. --------- Co-authored-by: Luiz Gustavo Abou Hatem de Liz <luizgustavoahsc@gmail.com>
# Conflicts: # crates/nexum-runtime/src/builder.rs # crates/nexum-runtime/src/supervisor.rs # crates/shepherd-backtest/src/replay.rs
Fold the M0 runtime completion (#424) into the intent/venue train: nexum-tasks executor, WASI capability gate, dispatch rate limit, per-module state quota and graceful drain. Reconciles the shared event-loop, supervisor, engine-config and manifest surfaces where both lines grew, and repoints the slow-host fixture at the multi-package wit path.
`IntentStatusUpdate::encode` now leads with its own version tag and `decode` refuses an unknown one, mirroring the inner `StatusBody` codec. The envelope tag frames venue/receipt/status, the body tag frames the status payload, and the two version independently. `EnvelopeError` becomes the same typed empty/unknown-version/malformed enum as `DecodeError`. Skew tests cover a future envelope tag and the pre-tag framing at both the codec and the SDK event seam, and a golden vector pins the whole v1 envelope framing. Closes #548
…Type enum (#464) Prune the phantom PriceExceedsMarketPrice row, record the table (not cowprotocol RetryHint) as the classification source of truth in the data header, and pin the reconciliation with parity tests: every row must name a real upstream errorType, and the divergence from retry_hint() must be exactly the ratified set.
…ng (#465) docs: call out the grant-M2 contract-mods divergence in milestone reporting
The submitted: journal now keys on the deterministic intent-id (the generic sweep's venue-and-body submission key over the encoded CowIntentBody), derived pre-submit without assembling OrderCreation. SubmitOutcome's accepted receipt is fixed as the canonical 56-byte order UID (OrderUid), and the CowIntent schema gains the Signed kind a conditional-order keeper emits. A regression test covers resubmit after restart with a single orderbook POST.
cow: flip the keeper run onto the typed venue client
…469) * twap: re-point the monitor onto pool submit through the cow adapter The module flips from the shepherd:cow world onto #[videre_sdk::keeper]: the manifest declares the client capability and body version 1, the keeper run submits through the typed CowClient over the module's own videre:venue/client import, and the direct cow-api import and legacy cow client bridge drop out. Status transitions the registry polls back arrive on a cow intent-status subscription. Behaviour identity is proven at the VenueTransport seam: the dispatch tests script submit outcomes against the mock host and assert the same journal, gate, and retry effects as the legacy bridge, including the throttle hint surviving as an epoch backoff and the appData digest riding the body verbatim. The bundle boot proof moves to the videre platform suite (twap against the installed cow adapter); the cow-api boot-order invariant re-pins on ethflow-watcher. Engine configs that boot twap install the bundled adapter. * cow: match the adapter manifest chain to each engine config's run The adapter fixes its orderbook at init from its manifest chain, so a Sepolia run wired to the mainnet manifest submits to the wrong orderbook. Add per-chain manifest variants (sepolia, load-mock) and point every twap-wired engine config at the one matching the chain it indexes.
Delete shepherd-cow-host, the shepherd:cow legacy worlds (cow-api, cow-ext, shepherd; cow-events stays as the event-ABI package of record), the shepherd-sdk cow surface, and the shepherd-sdk-test mocks. The shepherd binary composes the core lattice with the videre platform alone. The keeper sweep moves to composable-cow behind the sweep slice; stop-loss migrates onto #[videre_sdk::keeper] and pool submit through the typed CowClient, keyed on the venue-and-body intent-id. twap gates topic-0 on the sol decoder pinned against cow-events; the WIT layering gate moves to cow-venue. ADR-0005/0006 marked superseded.
twap: index the v2 ConditionalOrderRemoved event and drop the watch Subscribe the twap-monitor to the ComposableCoW v2 ConditionalOrderRemoved topic-0 and pin it in the shepherd:cow/cow-events package of record. The created and removed subscription streams merge in arrival order, not chain order, so each watch row carries the create log's (block, log-index) stamp and a removal drops the watch (with its gates) only when it postdates the stamp: a stale removal for a re-registered order is ignored, and an unprovable ordering keeps the watch for the poll path's self-healing drop.
…nature (#473) A first-time user's Safe wiring and ComposableCoW create() land in one block, so the orderbook rejects the first submission against its own head and the keeper dropped a valid watch. The classification table gains a drop-on-repeat action, the retry ledger records the refused block and gates one next-block retry, and a denied refusal re-enters the table by its errorType prefix so the grace survives the coarse venue-error collapse. A repeat on a later block still drops.
docs: rewrite the platform docs as the shipped-venue source of truth Doc 05 documents both personas as shipped: the videre-sdk surface, the venue and keeper macros, the typed client, and an author-a-venue walkthrough on the echo pair, with the CoW crates as the production instance. Doc 08 names venue adapters as the domain-extension mechanism, replaces the Layer-3 world-include pattern with the videre:venue contract and the registry route, marks shepherd:cow as the retired legacy read path (cow-events stays as the event-ABI package of record), and squares the WIT layout, server table, and SDK layering with the tree. One inbound sdk.md anchor follows the doc 05 heading.
feat(nexum-runtime): reject colliding extension claims at boot One boot-time uniqueness pass fails fast on a service namespace, subscription kind, or manifest section two wired extensions both claim, each of which silently dedupes downstream.
fix(nexum-runtime): spell out CoreRuntime::components logs builder param
docs(videre-host): note admission-check boundaries Document the opt-in precondition on admit_worker/admit_provider (a manifest omitting [venue] is admitted unconditionally) and the post-instantiation, pre-init ordering of the registry export-divergence check.
* test(videre-host): pin unknown-venue on a registry-less client boot * docs(videre-host): correct the registry-less wrapper variance note
* feat(nexum-runtime): give PresetBuilder an escape hatch to the component seams Add PresetBuilder::with_components, mapping the preset's ComponentsBuilder to the builders to open so an embedder overrides one seam (chain, store, ext, or logs) while the preset's extensions and add-ons carry through. The new PresetComponentsBuilder stage gathers those before consuming the preset and launches otherwise as PresetBuilder::launch. * test(nexum-runtime): load the real manifest in the preset with_components e2e The manifest path derived from wasm.ancestors().nth(3) resolved to <root>/target, so the explicit manifest pointed at a nonexistent file and the loader silently fell back to an anonymous module. Derive the repo root once and reuse it for both the wasm fixture and the manifest.
* docs(videre-host): note install mutex discipline * refactor(videre-host): gate install to pub(crate) Install is boot-time only; drop it off the shared public handle so a post-boot clone cannot register with no compiler signal. Out-of-crate tests reach a mock adapter through the test-utils install_for_test seam.
Contributor
Author
|
Held, do not merge. A multi-lens architecture study found this approach ships a correctness regression, and #538 is being reframed. The reserve-before-submit guard is checked with It is worse on the CoW path this fix targets: the skip happens before the venue call, so the adapter's self-healing The root cause is that a |
mfw78
marked this pull request as draft
July 24, 2026 01:35
This was referenced Jul 24, 2026
mfw78
added a commit
that referenced
this pull request
Jul 25, 2026
Rewire submit_ready onto Journal::guard: the six inline journal.mark/reserve/commit/release calls that carry the #572 reserve-before-effect ordering fold into one guard call, with the venue outcome mapped to a Disposition (Accepted commits, requires-signing and known refusals release, an unknown outcome retains for reconcile). Behaviour is identical: all 23 composable-cow run tests including w1/w2/w3 and anti_572 pass unchanged. The remaining code is the CoW-specific outcome handling (refusal clear, retry ledger, tracing) that no primitive absorbs; the value is that the durable-effect ordering is no longer inlined, so a second venue's keeper reuses it rather than re-deriving it. Part of #609. Stacks on the Journal::guard combinator (#629).
mfw78
added a commit
that referenced
this pull request
Jul 25, 2026
Rewire submit_ready onto Journal::guard: the six inline journal.mark/reserve/commit/release calls that carry the #572 reserve-before-effect ordering fold into one guard call, with the venue outcome mapped to a Disposition (Accepted commits, requires-signing and known refusals release, an unknown outcome retains for reconcile). Behaviour is identical: all 23 composable-cow run tests including w1/w2/w3 and anti_572 pass unchanged. The remaining code is the CoW-specific outcome handling (refusal clear, retry ledger, tracing) that no primitive absorbs; the value is that the durable-effect ordering is no longer inlined, so a second venue's keeper reuses it rather than re-deriving it. Part of #609. Stacks on the Journal::guard combinator (#629).
mfw78
added a commit
that referenced
this pull request
Jul 26, 2026
Rewire submit_ready onto Journal::guard: the six inline journal.mark/reserve/commit/release calls that carry the #572 reserve-before-effect ordering fold into one guard call, with the venue outcome mapped to a Disposition (Accepted commits, requires-signing and known refusals release, an unknown outcome retains for reconcile). Behaviour is identical: all 23 composable-cow run tests including w1/w2/w3 and anti_572 pass unchanged. The remaining code is the CoW-specific outcome handling (refusal clear, retry ledger, tracing) that no primitive absorbs; the value is that the durable-effect ordering is no longer inlined, so a second venue's keeper reuses it rather than re-deriving it. Part of #609. Stacks on the Journal::guard combinator (#629).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
Journal::reserveandJournal::release(L1nexum-sdk) alongside the existingrecord/contains, using a distinct non-empty pre-submit marker thatreleasedrops but never clobbers a committed receipt. Both dedup sites - the L2videre-sdkkeeper sweep and the L3composable-cowsweep - now reserve the key before the venue call, commit onAccepted(best-effort, no longer sweep-aborting on a write fault), and release on every non-accepting outcome (RequiresSigning, a venue fault, an encode fault, or an unmatched non-exhaustiveClientErrorarm).Why
The old
contains->submit->recordshape left a window where an accepted submit whose subsequentrecordwrite faulted was indistinguishable from a never-submitted watch: the next sweep sawcontains == falseand resubmitted, sending a duplicate order to the venue. Reserving the key before the venue call closes that window - the reservation, not the commit, is what a re-post checks - so a post-accept record fault or a trap still leaves a durable marker and the accepted body is never resubmitted. A retryable submit fault releases the reservation so the retry path is preserved.Testing
New unit tests cover a faulted commit write after acceptance (reservation survives, no re-post) and a retryable submit fault (reservation releases, retry still reaches the venue).
AI Assistance
Implemented with Claude Code.
Closes #538