From 75d6cf9dbc709498f41e8b7296377a61b552eec4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Sat, 23 May 2026 16:03:10 +0000 Subject: [PATCH 01/14] feat: add design document for removing `Spawn` trait and refactoring of futures handling in AimDB --- docs/design/028-M13-remove-spawn-trait.md | 828 ++++++++++++++++++++++ 1 file changed, 828 insertions(+) create mode 100644 docs/design/028-M13-remove-spawn-trait.md diff --git a/docs/design/028-M13-remove-spawn-trait.md b/docs/design/028-M13-remove-spawn-trait.md new file mode 100644 index 0000000..7f53254 --- /dev/null +++ b/docs/design/028-M13-remove-spawn-trait.md @@ -0,0 +1,828 @@ +# Remove `Spawn` Trait β€” `build()` Collects, `run()` Drives + +**Version:** 0.1 (draft) +**Status:** πŸ“ Design +**Issue:** [#88](https://github.com/aimdb-dev/aimdb/issues/88) +**Last Updated:** May 23, 2026 +**Milestone:** M13 β€” Architectural clean-up + +--- + +## Table of Contents + +- [Summary](#summary) +- [Motivation](#motivation) +- [Current Architecture β€” Audit](#current-architecture--audit) + - [Where `Spawn` propagates today](#where-spawn-propagates-today) + - [Embassy workaround](#embassy-workaround) + - [WASM workaround](#wasm-workaround) + - [The key audit finding](#the-key-audit-finding) +- [Proposed Design](#proposed-design) + - [New public API](#new-public-api) + - [Internal mechanics](#internal-mechanics) + - [Connector futures](#connector-futures) + - [Remote supervisor](#remote-supervisor) +- [Type-System Changes](#type-system-changes) + - [Bound removals β€” summary table](#bound-removals--summary-table) + - [unsafe impl Send/Sync analysis](#unsafe-impl-sendsync-analysis) + - [R phantom-data after removal](#r-phantom-data-after-removal) +- [Platform-Specific Concerns](#platform-specific-concerns) + - [Tokio (std)](#tokio-std) + - [Embassy (no_std + alloc)](#embassy-nostd--alloc) + - [WASM (wasm32-unknown-unknown)](#wasm-wasm32-unknown-unknown) +- [Shutdown / Cancellation](#shutdown--cancellation) +- [Connector API impact](#connector-api-impact) +- [Breaking Changes](#breaking-changes) +- [Implementation Plan](#implementation-plan) +- [Alternatives Considered](#alternatives-considered) +- [Open Questions](#open-questions) + +--- + +## Summary + +Remove `Spawn` from the `Runtime` bundle trait and from every `R: Spawn` bound +in `aimdb-core`. All futures that would previously be individually spawned +inside `build()` are instead **collected** into a `Vec`. A new +`AimDb::run()` method drives them via `FuturesUnordered`, blocking until +shutdown. + +The core invariant that makes this safe: **all `spawn()` calls in AimDB happen +inside `build()`.** No task is ever spawned after initialisation completes +(with one narrow exception β€” the remote-access supervisor, discussed below). + +--- + +## Motivation + +### Problem 1 β€” Spawn permeates the type system + +`Spawn` is currently a supertrait of `Runtime`: + +```rust +pub trait Runtime: RuntimeAdapter + TimeOps + Logger + Spawn {} +``` + +Because `AimDb` requires `R: Spawn`, and `AimDb` is embedded in +`TypedRecord`, `Producer`, `Consumer`, and +`RuntimeContext`, a `Spawn` bound propagates to every layer. Adding a new +runtime adapter requires implementing `Spawn` even if the adapter never needs +to spawn dynamically. + +### Problem 2 β€” Embassy task pool workaround + +Embassy's task system requires statically-typed futures. To implement `Spawn`, +`aimdb-embassy-adapter` heap-allocates every future, type-erases it into a +`BoxedFuture`, and feeds it into a compile-time-fixed task pool via: + +```rust +unsafe { Pin::new_unchecked(boxed_future) } +``` + +The pool size is selected with a feature flag (`embassy-task-pool-8/16/32`). +Choosing the wrong size panics at runtime. The `unsafe` block has no audit +trail. + +### Problem 3 β€” unsafe impl Send/Sync + +`EmbassyAdapter` wraps an `Option`. Because `Spawner` is `!Send`, +`EmbassyAdapter` is `!Send` without an `unsafe impl`. The adapter adds it to +satisfy `F: Send + 'static` demanded by the `Spawn` trait: + +```rust +// SAFETY: Embassy executor handles spawner synchronization internally. +unsafe impl Send for EmbassyAdapter {} +unsafe impl Sync for EmbassyAdapter {} +``` + +`WasmAdapter` has the same pattern for WASM's single-threaded executor. +`Producer` and `Consumer` also carry `unsafe impl Send/Sync` +because they hold `Arc>` which transitively requires `R: Send + Sync`. + +--- + +## Current Architecture β€” Audit + +### Where `Spawn` propagates today + +| Location | Bound | Role | +|---|---|---| +| `aimdb-executor/src/lib.rs` | `Runtime: … + Spawn` | bundle supertrait | +| `aimdb-core/src/builder.rs` | `AimDb`, `AimDbBuilder` | primary database types | +| `aimdb-core/src/builder.rs` | `AimDbInner::get_typed_record_by_key/id` | lookup helpers | +| `aimdb-core/src/typed_record.rs` | `TypedRecord` | per-record storage | +| `aimdb-core/src/typed_record.rs` | `RecordSpawner::spawn_all_tasks` | spawn orchestrator | +| `aimdb-core/src/typed_record.rs` | `AnyRecordExt::as_typed` | downcast helper | +| `aimdb-core/src/typed_record.rs` | `spawn_producer_service`, `spawn_consumer_tasks`, `spawn_transform_task` | direct spawning | +| `aimdb-core/src/typed_api.rs` | `Producer`, `Consumer` | typed handles | +| `aimdb-core/src/transform/mod.rs` | `TransformDescriptor` | deferred transform | +| `aimdb-core/src/database.rs` | `Database` | high-level wrapper | +| `aimdb-core/src/remote/supervisor.rs` | `R: Spawn` | supervisor spawning | +| `aimdb-core/src/remote/handler.rs` | `R: Spawn` (Γ—14 bounds) | handler dispatch | + +### Embassy workaround + +``` +EmbassyAdapter::spawn(future: F) + β”‚ + β”œβ”€ Box::new(future) // heap-allocate + β”œβ”€ type-erase β†’ BoxedFuture + └─ TASK_POOL.spawn( // fixed-size array + unsafe { Pin::new_unchecked(boxed) } + ) +``` + +`TASK_POOL_SIZE` is selected at compile time via feature flags. If more tasks +are spawned than the pool size, the `spawn()` call panics. Choosing a pool +that is too large wastes RAM on constrained targets. + +### WASM workaround + +```rust +impl Spawn for WasmAdapter { + fn spawn(&self, future: F) -> ExecutorResult<()> + where F: Future + Send + 'static + { + wasm_bindgen_futures::spawn_local(future); + Ok(()) + } +} +// Required because Spawn demands F: Send, but WASM is single-threaded: +unsafe impl Send for WasmAdapter {} +unsafe impl Sync for WasmAdapter {} +``` + +The `unsafe` exists solely to satisfy the `Spawn` trait's `F: Send` bound, +which is vacuously satisfied on single-threaded WASM. + +### The key audit finding + +All `spawn()` call sites in AimDB are inside `build()`: + +| Call site | File | One future per… | +|---|---|---| +| `spawn_producer_service` | `typed_record.rs` | `.source()` | +| `spawn_consumer_tasks` | `typed_record.rs` | `.tap()` | +| `spawn_transform_task` | `typed_record.rs` | `.transform()` / `.transform_join()` | +| on_start tasks | `builder.rs` | `.on_start()` | +| Connector tasks | `builder.rs` (via `ConnectorBuilder::build`) | per-connector | +| Remote supervisor | `builder.rs` β†’ `supervisor.rs` | one per `with_remote_access()` | + +The total future count is **fully determined** by the database configuration at +the point `build()` is called. This is exactly the use case `FuturesUnordered` +is designed for. + +> **Exception β€” per-connection handlers:** The remote-access supervisor spawns +> **per-connection** handlers at runtime using bare `tokio::spawn`. These are +> already outside the `Spawn` trait abstraction and are unaffected by this +> change. See [Remote supervisor](#remote-supervisor) below. +> +> **Exception β€” connector infrastructure tasks:** Each Tokio connector also +> spawns one permanent infrastructure task via bare `tokio::spawn` β€” entirely +> separate from the outbound-publisher tasks that go through `runtime.spawn()`: +> +> | Connector | Infrastructure task | Current call | +> |---|---|---| +> | MQTT (Tokio) | `rumqttc` EventLoop poll loop | `spawn_event_loop()` β†’ `tokio::spawn` | +> | KNX (Tokio) | UDP connection + reconnect loop | `spawn_connection_task()` β†’ `tokio::spawn` | +> | WebSocket | Axum server (`axum::serve`) | `start_server()` β†’ `tokio::spawn` | +> +> Unlike the per-connection handlers, these infrastructure tasks **must be +> converted** to returned futures as part of Step 6. If left unchanged, +> connectors continue to call `tokio::spawn` directly and their infrastructure +> futures are invisible to `AimDbRunner`. + +--- + +## Proposed Design + +### New public API + +```rust +// Build β€” returns a (handle, runner) pair +let (db, runner) = AimDbBuilder::new() + .runtime(adapter) + .configure::("sensor.temp", |reg| { ... }) + .build() + .await?; + +// db is a plain Clone-able handle; clone freely before starting the runner +let handle = db.clone(); + +// Run β€” drives all futures collected during build(), blocks until shutdown +runner.run().await; +``` + +`build()` returns `(AimDb, AimDbRunner)`. `AimDb` is an ordinary +clone-able handle (same as today). `AimDbRunner` is a non-`Clone` struct that +owned the collected futures; it has no `Arc` or `Mutex` wrapping. + +`AimDbRunner::run()` takes `self` by value, consuming the futures vec. + +### Internal mechanics + +#### `build()` β€” collection phase + +Replace every `runtime.spawn(future)` call with: + +```rust +futures.push(future); +``` + +where `futures: Vec + Send + 'static>>>`. + +At the end of `build()`, the vec is wrapped in `AimDbRunner` and returned +alongside the `AimDb` handle: + +```rust +/// Non-Clone runner returned by build(). +/// Owns the complete set of futures that drive the database. +pub struct AimDbRunner { + futures: Vec>, +} + +// AimDb itself is unchanged β€” no futures field, no Mutex. +pub struct AimDb { + inner: Arc, + runtime: Arc, + // ... profiling, etc. β€” exactly as today +} +``` + +#### `run()` β€” driving phase + +```rust +use futures::stream::{FuturesUnordered, StreamExt}; + +impl AimDbRunner { + pub async fn run(self) { + if self.futures.is_empty() { + return; // nothing to drive + } + + let mut set = FuturesUnordered::new(); + for f in self.futures { + set.push(f); + } + + // Drive all futures to completion (normally: forever, until shutdown) + while set.next().await.is_some() {} + } +} +``` + +`FuturesUnordered` polls each future cooperatively. Tasks that finish early +(e.g. a one-shot `on_start`) are dropped; tasks that run indefinitely (producer +services, consumer loops) keep the set alive. + +### Connector futures + +`ConnectorBuilder::build()` currently spawns its own tasks as a side effect. +With this change, connectors must **return** their futures instead of spawning +them. The `ConnectorBuilder` trait becomes: + +```rust +pub trait ConnectorBuilder { + async fn build( + self: Box, + db: &AimDb, + ) -> DbResult>>; +} +``` + +`AimDbBuilder::build()` collects the returned futures and adds them to the +accumulator. This is a **breaking change** to the `ConnectorBuilder` trait +(connector authors must update `build()`). + +#### Two layers of futures per connector + +Each connector currently has two distinct categories of futures, both of which +must be returned: + +1. **Outbound publisher futures** β€” one per `link_to()` route, currently + spawned via `runtime.spawn()` inside `spawn_outbound_publishers()`. These + subscribe to a typed record and publish serialised values to the external + system. + +2. **Infrastructure futures** β€” one per connector instance, currently spawned + via bare `tokio::spawn()` inside internal helpers (see audit table above). + Converting these is covered in Step 6b. + +#### The dropped `Arc` object + +The current `ConnectorBuilder::build()` returns `Arc`, but +`builder.rs` already discards this value immediately: +`let _connector = builder.build(&db).await?;`. The `Connector::publish()` +method β€” used for direct programmatic publishing β€” is already inaccessible +through the `AimDbBuilder` public API. Changing the return type to +`Vec` causes no behavioural regression here. + +Connector implementations keep their data alive via `Arc` clones captured +inside the returned futures (e.g. `Arc` in MQTT, +`mpsc::Sender` in KNX) β€” not via the connector object itself. + +#### KNX channel ownership ordering + +The KNX connector creates an `mpsc::channel` inside `spawn_connection_task()`: +the receiver is captured by the connection task, and the sender is cloned into +each outbound publisher task. In the new model the channel must be created +before either set of futures is constructed: + +```rust +// 1. Create channel first +let (cmd_tx, cmd_rx) = mpsc::channel(16); + +// 2. Connection future captures cmd_rx +let connection_future: BoxFuture<'static, ()> = Box::pin(async move { + // UDP connection + reconnect loop, reads from cmd_rx +}); + +// 3. Outbound publisher futures each clone cmd_tx +let publisher_futures: Vec> = routes + .into_iter() + .map(|route| { + let tx = cmd_tx.clone(); + Box::pin(async move { /* subscribe β†’ serialize β†’ tx.send() */ }) as BoxFuture<'static, ()> + }) + .collect(); + +// 4. All futures returned together +Ok(std::iter::once(connection_future).chain(publisher_futures).collect()) +``` + +This ordering is already implicit in the current code and must be preserved +explicitly when refactoring. + +### Remote supervisor + +`supervisor::spawn_supervisor()` currently calls `runtime.spawn(supervisor_loop)`. +In the new model it returns the supervisor future instead: + +```rust +pub fn build_supervisor( + db: Arc>, + config: AimxConfig, +) -> DbResult> +``` + +The supervisor loop internally calls `tokio::spawn` for per-connection handlers +β€” this is a Tokio-specific behaviour that already bypasses the `Spawn` trait +and is **unchanged** by this refactor. Per-connection spawning happens inside a +future that is driven by `FuturesUnordered`, which is fine because +`FuturesUnordered` is poll-based: the supervisor future yields control between +accepted connections. + +--- + +## Type-System Changes + +### Bound removals β€” summary table + +| Type / fn | Before | After | +|---|---|---| +| `Runtime` supertrait | `RuntimeAdapter + TimeOps + Logger + Spawn` | `RuntimeAdapter + TimeOps + Logger` | +| `AimDb` | `R: Spawn + 'static` | `R: RuntimeAdapter + 'static` | +| `AimDbBuilder` | `R: Spawn + 'static` | `R: RuntimeAdapter + 'static` | +| `TypedRecord` | `R: Spawn + 'static` | `R: 'static` | +| `Producer` | `R: Spawn + 'static` | `R: 'static` | +| `Consumer` | `R: Spawn + 'static` | `R: 'static` | +| `TransformDescriptor` | `R: Spawn + 'static` | `R: 'static` | +| `RecordSpawner::spawn_all_tasks` | `R: Spawn + 'static` | removed (see below) | +| `AnyRecordExt::as_typed` | `R: Spawn + 'static` | `R: 'static` | +| `Database` | `A: Spawn + 'static` | `A: RuntimeAdapter + 'static` | +| `remote/handler.rs` (Γ—14) | `R: Spawn + 'static` | `R: RuntimeAdapter + 'static` | +| `remote/supervisor.rs` | `R: Spawn + 'static` | `R: RuntimeAdapter + 'static` | + +### `RecordSpawner` β†’ `RecordFutureCollector` + +`RecordSpawner::spawn_all_tasks()` currently calls `runtime.spawn()` for +each task. It is renamed and refactored to **collect** futures instead: + +```rust +pub struct RecordFutureCollector { _phantom: PhantomData } + +impl RecordFutureCollector { + pub fn collect_all_futures( + record: &dyn AnyRecord, + db: &Arc>, + record_key: &str, + ) -> DbResult>> { + let mut futures = Vec::new(); + // ... downcast, collect producer, transform, consumer futures ... + Ok(futures) + } +} +``` + +Similarly, `TypedRecord::spawn_producer_service` β†’ `collect_producer_future`, +`spawn_consumer_tasks` β†’ `collect_consumer_futures`, +`spawn_transform_task` β†’ `collect_transform_future`. + +### `unsafe impl Send/Sync` analysis + +**`Producer` and `Consumer`:** + +Currently marked `unsafe impl Send/Sync` because `R: Spawn` does not guarantee +`R: Send + Sync` on all platforms (notably Embassy and WASM). After this +change, `R: RuntimeAdapter` (which is `Send + Sync + 'static`), and +`Arc>` is `Send + Sync` if `R: Send + Sync`. Since `RuntimeAdapter` +already requires `Send + Sync + 'static`, `Producer` and `Consumer` +will auto-derive `Send + Sync` without `unsafe`. **Remove the unsafe impls.** + +**`EmbassyAdapter`:** + +`EmbassyAdapter` currently holds `Option`. `Spawner` is `!Send`. +With `Spawn` removed, the `Spawner` field is no longer needed (it was only +used in `impl Spawn for EmbassyAdapter`). **Remove the `spawner` field.** + +After removing `spawner`, `EmbassyAdapter` becomes a simple struct holding +only an `Option<&'static Stack<'static>>` (the network stack, gated by +`embassy-net-support`). `&'static Stack<'static>` is `Send + Sync`, so +`EmbassyAdapter` auto-derives `Send + Sync`. **Remove the `unsafe impl`.** + +The `new_with_spawner()` constructor is removed (breaking change β€” see +[Breaking Changes](#breaking-changes)). + +**`WasmAdapter`:** + +`WasmAdapter` is already a ZST (`#[derive(Clone, Copy, Debug)]` with no +fields). With `Spawn` removed it no longer needs the `unsafe impl`. Since it is +a ZST it auto-derives `Send + Sync`. **Remove the `unsafe impl`.** + +**Embassy connector futures:** + +Embassy connector implementations (e.g. the Embassy variant of +`aimdb-mqtt-connector`) currently wrap their outbound publisher futures in +`SendFutureWrapper` to satisfy the `F: Send + 'static` bound imposed by +`impl Spawn for EmbassyAdapter`. With `Spawn` removed, `runtime.spawn()` calls +disappear β€” but `Vec>` (where +`BoxFuture = Pin>`) still requires `Send`. +Embassy futures are `!Send`. Embassy connector implementations must therefore +still wrap their returned futures in `SendFutureWrapper` before pushing them +to the Vec. The `unsafe` burden shifts from the adapter to the connectors β€” +the total number of `unsafe impl` blocks decreases, but is not reduced to zero. + +### R phantom-data after removal + +`Producer` and `Consumer` currently hold `Arc>` and bind +`R` for type inference at call sites (`producer.produce(val).await`). After the +bound change, `R` is still carried but is now only constrained to `R: 'static`. +The `_phantom: PhantomData` field remains as-is. + +Whether to collapse `R` entirely (making `Producer`) is a larger question +deferred to a follow-up refactor β€” it would change all public API signatures. + +--- + +## Platform-Specific Concerns + +### Tokio (std) + +`FuturesUnordered` from `futures-util` is the natural choice. Tokio users call: + +```rust +tokio::select! { + _ = runner.run() => {}, + _ = shutdown_signal() => {}, +} +``` + +No adapter changes beyond removing `impl Spawn for TokioAdapter`. + +### Embassy (no_std + alloc) + +`FuturesUnordered` from `futures-util` compiles in `no_std + alloc` mode +(requires `futures-util` with `default-features = false, features = ["alloc"]`). + +Embassy main becomes: + +```rust +#[embassy_executor::main] +async fn main(_spawner: Spawner) { + let adapter = EmbassyAdapter::new().unwrap(); + let db = AimDbBuilder::new() + .runtime(Arc::new(adapter)) + .configure::(...) + .build() + .await + .unwrap(); + + runner.run().await; // drives FuturesUnordered inside the Embassy main task +} +``` + +Embassy's cooperative scheduler handles the `FuturesUnordered::next().await` +poll loop the same way it handles any `await` point. This replaces the task +pool entirely. + +**Memory profile:** Each boxed future occupies one heap allocation. The total +number of futures is bounded by the database configuration, same as today. +Heap usage is therefore unchanged. + +**Pool size feature flags** (`embassy-task-pool-8/16/32`) are **deleted**. + +### WASM (wasm32-unknown-unknown) + +WASM's single-threaded runtime presents no new concerns because +`FuturesUnordered` works on single-threaded executors. `runner.run()` is +awaited inside the WASM main function or a `wasm_bindgen_futures::spawn_local` +root. + +--- + +## Shutdown / Cancellation + +`FuturesUnordered::next()` returns `None` only when all futures have completed. +Producer services and consumer loops are infinite by design; `run()` therefore +blocks for the application's lifetime. + +Cancellation strategies: +1. **Tokio `CancellationToken`** β€” wrap `db.run()` in `select!` with a + cancellation token. Producer/consumer tasks should check the token in their + loop body. +2. **CTRL-C / signal handler** β€” same `select!` pattern with + `tokio::signal::ctrl_c()`. +3. **Embassy** β€” cooperative via task cancellation or a shared `AtomicBool` + stop flag. + +The design does not prescribe a shutdown mechanism; that is the application's +responsibility. A future enhancement could add a `db.shutdown()` method that +drops the futures set. + +--- + +## Connector API impact + +Connectors that implement `ConnectorBuilder` must be updated: + +**Before:** +```rust +async fn build(self: Box, db: &AimDb) -> DbResult> { + // spawns tasks internally via runtime.spawn() + runtime.spawn(my_task_future); + Ok(Box::new(MyConnector { ... })) +} +``` + +**After:** +```rust +async fn build( + self: Box, + db: &AimDb, +) -> DbResult>> { + // returns futures for AimDbBuilder to collect + Ok(vec![Box::pin(my_task_future)]) +} +``` + +Affected connectors: `aimdb-mqtt-connector` (Tokio and Embassy), `aimdb-knx-connector`, +`aimdb-websocket-connector`, and aimdb-pro call sites. + +--- + +## Breaking Changes + +| Area | Change | +|---|---| +| Public API | `db.run().await` must be called after `build()` β€” tasks do not start until `run()` | +| `Runtime` supertrait | `Spawn` removed β€” custom adapters no longer need to implement it | +| `EmbassyAdapter` | `new_with_spawner(spawner)` constructor removed | +| `EmbassyAdapter` feature flags | `embassy-task-pool-8/16/32` removed | +| `ConnectorBuilder` trait | `build()` now returns `Vec` instead of `Box` | +| `spawn_fns` in `AimDbBuilder` | internal β€” no public API change, but internal structure changes | + +--- + +## Implementation Plan + +Listed in dependency order. Each step should pass `make check` before the next +begins. + +### Step 1 β€” Executor layer + +**File:** `aimdb-executor/src/lib.rs` + +- Remove `Spawn` from the `Runtime` supertrait. +- Delete the `Spawn` trait entirely from `aimdb-executor`. +- Remove the `SpawnFailed` variant from `ExecutorError`. +- Add `futures-util` dependency with `default-features = false, features = ["alloc"]`. + +**Test:** `cargo check --all-features` + `cargo check --target thumbv7em-none-eabihf`. + +--- + +### Step 2 β€” Core: loosen bounds in typed_api and typed_record + +**Files:** `aimdb-core/src/typed_api.rs`, `aimdb-core/src/typed_record.rs`, +`aimdb-core/src/transform/mod.rs` + +- Remove `R: Spawn` bound from `Producer`, `Consumer`, `RecordRegistrar`. +- Remove `unsafe impl Send/Sync` from `Producer` and `Consumer` β€” + verify auto-derivation works. +- Remove `R: Spawn` bound from `TypedRecord` and `TransformDescriptor`. +- Remove `R: Spawn` bound from `AnyRecordExt::as_typed`. + +--- + +### Step 3 β€” Core: refactor spawn β†’ collect + +**File:** `aimdb-core/src/typed_record.rs` + +Rename and refactor: +- `RecordSpawner` β†’ `RecordFutureCollector` +- `spawn_all_tasks` β†’ `collect_all_futures` (returns `Vec`) +- `spawn_producer_service` β†’ `collect_producer_future` (returns `Option`) +- `spawn_consumer_tasks` β†’ `collect_consumer_futures` (returns `Vec`) +- `spawn_transform_task` β†’ `collect_transform_future` (returns `Option`) + +Each method constructs the future and returns it rather than calling +`runtime.spawn()`. + +--- + +### Step 4 β€” Core: `build()` accumulates, `run()` drives + +**File:** `aimdb-core/src/builder.rs` + +- Add `AimDbRunner` struct with a `futures: Vec>` field. +- Change `build()` return type from `DbResult>` to + `DbResult<(AimDb, AimDbRunner)>`. +- In `build()`, replace each `runtime.spawn(f)` with `futures_acc.push(f)`. +- At end of `build()`, wrap the accumulated vec in `AimDbRunner` and return + it alongside the `AimDb` handle. +- Implement `AimDbRunner::run(self)` using `FuturesUnordered`. +- `AimDb` itself gains no new fields β€” it remains a plain clone-able handle. +- Remove `R: Spawn` bound from `AimDb`, `AimDbBuilder`, + `AimDbInner::get_typed_record_by_key/id`. +- Collapse the `std`/`no_std` `on_start` bifurcation: unify + `StartFnType` and `NoStdStartFnType` into a single alias + `type StartFnType = Box) -> BoxFuture<'static, ()>>;` + (drop `+ Send` from the `FnOnce` β€” the closure is called once during + `build()`, not moved across threads; `Send` is already required by the + `BoxFuture` output type). Remove the `#[cfg(feature = "std")]` split. + +--- + +### Step 5 β€” Core: remote access bounds + +**Files:** `aimdb-core/src/remote/supervisor.rs`, +`aimdb-core/src/remote/handler.rs` + +- Rename `spawn_supervisor` β†’ `build_supervisor_future`; return + `DbResult>` instead of spawning. +- Remove `R: Spawn` bounds from all 14 handler function signatures; replace + with `R: RuntimeAdapter`. +- `build()` in `builder.rs` calls `build_supervisor_future()` and pushes the + returned future to the accumulator. + +--- + +### Step 6 β€” Connector API + +**Files:** `aimdb-core/src/connector.rs`, all connector crates + +#### 6a β€” Update trait signature and builder + +- Update `ConnectorBuilder::build()` return type to + `DbResult>>`. +- Remove `R: Spawn` bound from the `ConnectorBuilder` trait definition. +- Update `builder.rs` to extend the accumulator with the returned Vec. + +#### 6b β€” Convert infrastructure `tokio::spawn` calls + +Each Tokio connector has one permanent infrastructure future currently spawned +via bare `tokio::spawn`. Convert each to construct and return a `BoxFuture` +instead: + +- **MQTT:** `spawn_event_loop()` β†’ `build_event_loop_future()` β€” return the + `rumqttc` EventLoop poll loop as a `BoxFuture` instead of calling + `tokio::spawn`. +- **KNX:** `spawn_connection_task()` β†’ `build_connection_future()` β€” return + `(BoxFuture<'static, ()>, mpsc::Sender)`. The `Sender` is then + cloned into outbound publisher futures (see [KNX channel ownership + ordering](#knx-channel-ownership-ordering) in the Connector futures section). +- **WebSocket:** `start_server()` β†’ `build_server_future()` β€” return the + `axum::serve(...)` loop as a `BoxFuture`. Per-connection tasks that Axum + spawns internally remain as `tokio::spawn` β€” same category as the remote + supervisor's per-connection handlers. + +#### 6c β€” Replace outbound publisher spawn with collect + +Replace `spawn_outbound_publishers()` with `collect_outbound_futures()` on each +connector implementation, returning `Vec` instead of calling +`runtime.spawn()` per route. + +#### 6d β€” Embassy connector `SendFutureWrapper` + +Embassy connector implementations must wrap returned futures in +`SendFutureWrapper` before pushing them to the Vec (Embassy futures are `!Send` +but `BoxFuture<'static, ()>` requires `Send`). This is the same pattern already +used on the Embassy path β€” the wrapping moves from the spawn call site to the +collection point. + +--- + +### Step 7 β€” Adapter: Tokio + +**File:** `aimdb-tokio-adapter/src/runtime.rs` + +- Remove `impl Spawn for TokioAdapter`. + +--- + +### Step 8 β€” Adapter: Embassy + +**File:** `aimdb-embassy-adapter/src/runtime.rs` + +- Remove `impl Spawn for EmbassyAdapter`. +- Remove `spawner: Option` field. +- Remove `new_with_spawner()` constructor. +- Remove `generic_task_runner`, `BoxedFuture`, task pool declarations. +- Remove `embassy-task-pool-8/16/32` Cargo features. +- Remove `unsafe impl Send for EmbassyAdapter` and `unsafe impl Sync for EmbassyAdapter`. +- Remove `build.rs` logic that configures task pool size (if applicable). + +--- + +### Step 9 β€” Adapter: WASM + +**File:** `aimdb-wasm-adapter/src/runtime.rs` + +- Remove `impl Spawn for WasmAdapter`. +- Remove `unsafe impl Send for WasmAdapter` and `unsafe impl Sync for WasmAdapter`. + +--- + +### Step 10 β€” Examples and aimdb-pro call sites + +Update all examples to add `db.run().await` after `build()`. Update aimdb-pro +demo binaries, connectors, and any internal tooling that calls `build()`. + +--- + +### Step 11 β€” Changelog and API docs + +- Add entry to `CHANGELOG.md` documenting breaking changes. +- Update `README.md` quickstart snippet. +- Update doc comments on `build()` and `run()`. + +--- + +## Alternatives Considered + +### A β€” Keep `Spawn` on `Runtime`, drive with a shared `JoinSet` + +`build()` could return a `JoinSet<()>` that callers await. This keeps the +`Spawn` trait but adds a handle-based shutdown mechanism. **Rejected** because +`JoinSet` is Tokio-specific and doesn't help Embassy or WASM. + +### B β€” Static dispatch via `collect()` + join macro + +Embassy already uses `join_array` / `select_array`. A macro could call the +right primitive per platform. **Rejected** because it requires platform-specific +`run()` implementations and adds conditional compilation complexity. + +### C β€” Remove `R` from `Producer` and `Consumer` now + +Collapsing `R` out of the typed handles would simplify the API further. +**Deferred** β€” it is a larger change to all connector and user-facing APIs. +Can be done in a follow-up milestone once this foundation is in place. + +### D β€” Dual API: keep `spawn()` as an escape hatch + +Expose `db.spawn(future)` for callers who legitimately need post-build task +spawning (e.g. a custom connector). **Deferred** β€” no current use case inside +the AimDB codebase requires it. Can be re-introduced if a concrete need arises, +backed by an explicit `Spawn` impl on the adapter. + +--- + +## Decisions + +All design questions raised during review have been resolved: + +1. **`build()` return type** β†’ **Tuple `(AimDb, AimDbRunner)`.** + Consistent with Rust channel idioms (`mpsc::channel`, `oneshot::channel`). + No extra type to import; destructuring at every call site is idiomatic. + +2. **`Spawn` trait retention** β†’ **Delete entirely.** + This is already a semver-breaking release; no deprecation cycle is needed. + Any downstream crate that implemented `Spawn` must update for the other + breaking changes regardless. + +3. **`ExecutorError::SpawnFailed` variant** β†’ **Remove.** + With `Spawn` deleted, `SpawnFailed` is unreachable. Remove it in the same + commit as the trait deletion (Step 1). + +4. **`Database` promotion** β†’ **Defer.** + Drop the `A: Spawn` bound as part of this milestone but keep `Database` + as a convenience wrapper. Renaming the primary user-facing type is a larger + mechanical change with no functional benefit at this stage. + +5. **Connector `build()` return type** β†’ **`Vec` only.** + `builder.rs` already discards `Arc` today (`let _connector + = ...`). No behavioural regression. Introspection handles can be + reconsidered in a future milestone when a concrete use case exists. + +6. **`on_start` std/no_std bifurcation** β†’ **Collapse.** + Unify into a single `StartFnType` alias (see Step 4 for details). From 4707a2fe50ab8fc5b4a69f09b27105f740d47e0a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Sat, 23 May 2026 20:38:00 +0000 Subject: [PATCH 02/14] Refactor AimDB to remove `Spawn` trait and collect futures in `build()` - Updated `WebSocketConnectorImpl` to collect outbound publisher futures instead of spawning them directly. - Refactored `ServerState` to build the WebSocket Axum server future and return it as a `BoxFuture`. - Revised design documentation to reflect the removal of the `Spawn` trait and the new approach for handling futures. - Modified examples to adapt to the new `build()` and `run()` pattern, ensuring proper initialization and concurrent execution of tasks. - Removed unnecessary `Spawn` trait bounds from various components and updated related code generation. - Cleaned up example projects to align with the new architecture, including adjustments to how the Embassy adapter is initialized and how the database runner is executed. --- Cargo.lock | 1 + Cargo.toml | 1 + aimdb-codegen/src/rust.rs | 12 +- aimdb-core/src/builder.rs | 328 ++++++++-------- aimdb-core/src/connector.rs | 28 +- aimdb-core/src/database.rs | 48 +-- aimdb-core/src/error.rs | 11 - aimdb-core/src/lib.rs | 2 +- aimdb-core/src/remote/handler.rs | 28 +- aimdb-core/src/remote/supervisor.rs | 33 +- aimdb-core/src/transform/join.rs | 97 +++-- aimdb-core/src/transform/mod.rs | 20 +- aimdb-core/src/transform/single.rs | 25 +- aimdb-core/src/typed_api.rs | 70 ++-- aimdb-core/src/typed_record.rs | 180 ++++----- aimdb-embassy-adapter/Cargo.toml | 6 - aimdb-embassy-adapter/src/lib.rs | 18 - aimdb-embassy-adapter/src/runtime.rs | 186 ++------- aimdb-executor/Cargo.toml | 3 + aimdb-executor/src/join.rs | 4 +- aimdb-executor/src/lib.rs | 27 +- aimdb-knx-connector/src/embassy_client.rs | 84 ++-- aimdb-knx-connector/src/tokio_client.rs | 197 +++------- aimdb-mqtt-connector/src/embassy_client.rs | 202 ++++------ aimdb-mqtt-connector/src/tokio_client.rs | 96 ++--- aimdb-persistence/src/builder_ext.rs | 6 +- aimdb-persistence/src/ext.rs | 6 +- aimdb-persistence/src/query_ext.rs | 6 +- aimdb-persistence/tests/query_skip_invalid.rs | 6 +- aimdb-sync/src/handle.rs | 15 +- aimdb-tokio-adapter/src/connector.rs | 221 ----------- aimdb-tokio-adapter/src/lib.rs | 1 - aimdb-tokio-adapter/src/runtime.rs | 20 +- .../tests/drain_integration_tests.rs | 12 +- .../tests/multi_instance_tests.rs | 24 +- aimdb-tokio-adapter/tests/stage_profiling.rs | 6 +- .../tests/transform_join_integration_tests.rs | 6 +- aimdb-wasm-adapter/src/bindings.rs | 9 +- aimdb-wasm-adapter/src/lib.rs | 2 +- aimdb-wasm-adapter/src/runtime.rs | 57 +-- aimdb-websocket-connector/src/builder.rs | 30 +- .../src/client/builder.rs | 31 +- .../src/client/connector.rs | 272 +++++++------ aimdb-websocket-connector/src/connector.rs | 43 +- aimdb-websocket-connector/src/server.rs | 18 +- docs/design/028-M13-remove-spawn-trait.md | 369 +++++++++++++++--- .../embassy-knx-connector-demo/src/main.rs | 22 +- .../embassy-mqtt-connector-demo/Cargo.toml | 1 - .../embassy-mqtt-connector-demo/src/main.rs | 22 +- examples/remote-access-demo/src/server.rs | 3 +- .../weather-station-alpha/src/main.rs | 3 +- .../weather-station-beta/src/main.rs | 3 +- .../weather-station-gamma/Cargo.toml | 1 - .../weather-station-gamma/src/main.rs | 22 +- 54 files changed, 1276 insertions(+), 1668 deletions(-) delete mode 100644 aimdb-tokio-adapter/src/connector.rs diff --git a/Cargo.lock b/Cargo.lock index 8e1e8a3..acd2c25 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -140,6 +140,7 @@ name = "aimdb-executor" version = "0.2.0" dependencies = [ "embassy-executor", + "futures-util", "thiserror 2.0.17", "tokio", ] diff --git a/Cargo.toml b/Cargo.toml index d55c419..88a54d7 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -75,6 +75,7 @@ tracing = { version = "0.1", default-features = false } # Async utilities futures = "0.3" +futures-util = { version = "0.3", default-features = false, features = ["alloc"] } # CLI (for aimdb-cli) clap = { version = "4.0", features = ["derive"] } diff --git a/aimdb-codegen/src/rust.rs b/aimdb-codegen/src/rust.rs index eb876a8..eaf4754 100644 --- a/aimdb-codegen/src/rust.rs +++ b/aimdb-codegen/src/rust.rs @@ -557,7 +557,7 @@ fn emit_imports(state: &ArchitectureState) -> TokenStream { use aimdb_core::builder::AimDbBuilder; use aimdb_core::RecordKey; use aimdb_data_contracts::{#(#contract_traits),*}; - use aimdb_executor::Spawn; + use aimdb_executor::RuntimeAdapter; use serde::{Deserialize, Serialize}; } } @@ -731,7 +731,7 @@ fn emit_configure_schema(state: &ArchitectureState) -> TokenStream { /// addresses. Producers, consumers, serializers, and deserializers contain /// business logic and must be provided by application code β€” they are not /// generated here. - pub fn configure_schema(builder: &mut AimDbBuilder) { + pub fn configure_schema(builder: &mut AimDbBuilder) { #(#record_blocks)* } } @@ -1250,7 +1250,7 @@ pub fn generate_hub_schema_rs(state: &ArchitectureState) -> String { let file_tokens = quote! { use aimdb_core::buffer::BufferCfg; use aimdb_core::builder::AimDbBuilder; - use aimdb_executor::Spawn; + use aimdb_executor::RuntimeAdapter; use #common_crate::*; #configure_fn @@ -1833,8 +1833,8 @@ url = "mqtt://ota/cmd/{variant}" "Missing RecordKey import:\n{out}" ); assert!( - out.contains("use aimdb_executor::Spawn;"), - "Missing Spawn import:\n{out}" + out.contains("use aimdb_executor::RuntimeAdapter;"), + "Missing RuntimeAdapter import:\n{out}" ); assert!( out.contains("use serde::{Deserialize, Serialize};"), @@ -1938,7 +1938,7 @@ url = "mqtt://ota/cmd/{variant}" let out = generated(); assert!( out.contains( - "pub fn configure_schema(builder: &mut AimDbBuilder)" + "pub fn configure_schema(builder: &mut AimDbBuilder)" ), "Missing configure_schema function:\n{out}" ); diff --git a/aimdb-core/src/builder.rs b/aimdb-core/src/builder.rs index cffbbab..8e48cac 100644 --- a/aimdb-core/src/builder.rs +++ b/aimdb-core/src/builder.rs @@ -24,30 +24,26 @@ use std::{boxed::Box, sync::Arc}; use crate::extensions::Extensions; use crate::graph::DependencyGraph; +/// Shorthand for a heap-pinned, `Send`, `'static` future β€” the unit of work +/// the `AimDbRunner` drives. +pub type BoxFuture = core::pin::Pin + Send + 'static>>; + /// Type-erased on_start function stored in `AimDbBuilder::start_fns`. /// /// Defined once here so `on_start()` (which stores) and `build()` (which /// downcasts) share the *exact same* type and a silent type mismatch cannot -/// cause a runtime panic. -#[cfg(feature = "std")] -type StartFnType = Box< - dyn FnOnce( - Arc, - ) - -> core::pin::Pin + Send + 'static>> - + Send, ->; -#[cfg(not(feature = "std"))] -type NoStdStartFnType = Box< - dyn FnOnce( - Arc, - ) - -> core::pin::Pin + Send + 'static>> - + Send, ->; +/// cause a runtime panic. Single alias regardless of `std`/`no_std`. +type StartFnType = Box) -> BoxFuture + Send>; + +/// Type-erased per-record future collector stored in `AimDbBuilder::spawn_fns`. +/// +/// At `build()` time each is invoked in topological order; the returned +/// `Vec` is appended to the runner's accumulator. +type SpawnFnType = + Box, &Arc>, RecordId) -> DbResult> + Send>; use crate::record_id::{RecordId, RecordKey, StringKey}; use crate::typed_api::{RecordRegistrar, RecordT}; -use crate::typed_record::{AnyRecord, AnyRecordExt, TypedRecord}; +use crate::typed_record::{AnyRecord, AnyRecordExt, RecordFutureCollector, TypedRecord}; use crate::{DbError, DbResult}; /// Type alias for outbound route tuples returned by `collect_outbound_routes` @@ -182,7 +178,7 @@ impl AimDbInner { ) -> DbResult<&TypedRecord> where T: Send + 'static + Debug + Clone, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { let key_str = key.as_ref(); @@ -207,7 +203,7 @@ impl AimDbInner { pub fn get_typed_record_by_id(&self, id: RecordId) -> DbResult<&TypedRecord> where T: Send + 'static + Debug + Clone, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { use crate::typed_record::AnyRecordExt; @@ -393,7 +389,7 @@ impl AimDbBuilder { /// are not parameterized by the runtime type. pub fn runtime(self, rt: Arc) -> AimDbBuilder where - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { AimDbBuilder { records: self.records, @@ -411,7 +407,7 @@ impl AimDbBuilder { impl AimDbBuilder where - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { /// Returns a shared reference to the extension storage. /// @@ -452,10 +448,7 @@ where // Type-erase so the field can be shared with the `NoRuntime` builder struct. // Uses the module-level `StartFnType` alias β€” must stay in sync with // the downcast in `build()`. - #[cfg(feature = "std")] let boxed: StartFnType = Box::new(move |runtime| Box::pin(f(runtime))); - #[cfg(not(feature = "std"))] - let boxed: NoStdStartFnType = Box::new(move |runtime| Box::pin(f(runtime))); self.start_fns.push(Box::new(boxed)); self } @@ -597,27 +590,22 @@ where if is_new_record { let spawn_key = record_key; - #[allow(clippy::type_complexity)] - let spawn_fn: Box< - dyn FnOnce(&Arc, &Arc>, RecordId) -> DbResult<()> + Send, - > = Box::new(move |runtime: &Arc, db: &Arc>, id: RecordId| { - // Use RecordSpawner to spawn tasks for this record type - use crate::typed_record::RecordSpawner; - - let typed_record = db.inner().get_typed_record_by_id::(id)?; - // Get the record key from the database to enable key-based producer/consumer - #[cfg(feature = "alloc")] - let key = db - .inner() - .key_for(id) - .map(|k| k.as_str().to_string()) - .unwrap_or_else(|| alloc::format!("__record_{}", id.index())); - #[cfg(not(feature = "alloc"))] - let key = ""; - #[cfg(feature = "alloc")] - let key = key.as_str(); - RecordSpawner::::spawn_all_tasks(typed_record, runtime, db, key) - }); + let spawn_fn: SpawnFnType = + Box::new(move |runtime: &Arc, db: &Arc>, id: RecordId| { + let typed_record = db.inner().get_typed_record_by_id::(id)?; + // Resolve the record's key for key-based Producer/Consumer construction. + let key = db + .inner() + .key_for(id) + .map(|k| k.as_str().to_string()) + .unwrap_or_else(|| alloc::format!("__record_{}", id.index())); + RecordFutureCollector::::collect_all_futures( + typed_record, + runtime, + db, + key.as_str(), + ) + }); // Store the spawn function (type-erased in Box) self.spawn_fns.push((spawn_key, Box::new(spawn_fn))); @@ -650,16 +638,17 @@ where self.configure::(key, |reg| T::register(reg, cfg)) } - /// Runs the database indefinitely (never returns) + /// Builds the database and drives every collected future to completion. /// - /// This method builds the database, spawns all producer and consumer tasks, and then - /// parks the current task indefinitely. This is the primary way to run AimDB services. + /// Convenience wrapper for the common case: call `build()`, then immediately + /// `runner.run().await`. The database handle is dropped on exit. /// - /// All logic runs in background tasks via producers, consumers, and connectors. The - /// application continues until interrupted (e.g., Ctrl+C). + /// For programmatic access to the handle (manual subscriptions, holding the + /// `AimDb` for other uses), prefer `build()` directly. /// /// # Returns - /// `DbResult<()>` - Ok when database starts successfully, then parks forever + /// `DbResult<()>` β€” Ok once the database starts; the call then blocks until + /// every future the runner is driving has completed (typically forever). /// /// # Example /// @@ -683,66 +672,55 @@ where #[cfg(feature = "tracing")] tracing::info!("Building database and spawning background tasks..."); - let _db = self.build().await?; + let (_db, runner) = self.build().await?; #[cfg(feature = "tracing")] - tracing::info!("Database running, background tasks active. Press Ctrl+C to stop."); + tracing::info!("Database running, runner driving collected futures."); - // Park indefinitely - the background tasks will continue running - // The database handle is kept alive here to prevent dropping it - core::future::pending::<()>().await; + runner.run().await; Ok(()) } - /// Builds the database and returns the handle (async) + /// Builds the database and returns a `(handle, runner)` pair. /// - /// Use this when you need programmatic access to the database handle for - /// manual subscriptions or production. For typical services, use `.run().await` instead. + /// `build()` collects every future the database needs to drive (producer + /// services, consumer taps, transforms with fan-in forwarders, connectors, + /// remote-access supervisor, `on_start` tasks) into the [`AimDbRunner`] + /// returned alongside the clone-able [`AimDb`] handle. **No background + /// work runs until `runner.run().await` is awaited.** /// - /// **Automatic Task Spawning:** This method spawns all producer services and - /// `.tap()` observer tasks that were registered during configuration. + /// Use this when you need programmatic access to the handle for manual + /// subscriptions or production. For typical services, use `.run().await` + /// which calls `build()` and `runner.run()` for you. /// - /// **Connector Setup:** Connectors must be created manually before calling `build()`: + /// **Connector Setup:** Connectors are constructed during `build()` from + /// the `ConnectorBuilder`s previously registered via `.with_connector()`. + /// Their driving futures are appended to the runner. /// - /// ```rust,ignore - /// use aimdb_mqtt_connector::{MqttConnector, router::RouterBuilder}; + /// # Returns + /// `DbResult<(AimDb, AimDbRunner)>` β€” the handle (cloneable) and the + /// non-`Clone` runner that owns the collected futures. + /// + /// # Example /// - /// // Configure records with connector links - /// let builder = AimDbBuilder::new() + /// ```rust,ignore + /// let (db, runner) = AimDbBuilder::new() /// .runtime(runtime) - /// .configure::("temp", |reg| { - /// reg.link_from("mqtt://commands/temp") - /// .with_buffer(BufferCfg::SingleLatest) - /// .with_remote_access(); - /// }); - /// - /// // Create MQTT connector with router - /// let router = RouterBuilder::new() - /// .route("commands/temp", /* deserializer */) - /// .build(); - /// let connector = MqttConnector::new("mqtt://localhost:1883", router).await?; - /// - /// // Register connector and build - /// let db = builder - /// .with_connector("mqtt", Arc::new(connector)) + /// .configure::("temp", |reg| { /* … */ }) + /// .with_connector(mqtt_builder) /// .build().await?; - /// ``` /// - /// # Returns - /// `DbResult>` - The database instance - #[cfg_attr(not(feature = "std"), allow(unused_mut))] - pub async fn build(self) -> DbResult> + /// let handle = db.clone(); // clone freely before runner.run() + /// runner.run().await; // drives everything to completion + /// ``` + pub async fn build(self) -> DbResult<(AimDb, AimDbRunner)> where R: crate::RuntimeForProfiling, { - use crate::DbError; - // Validate all records for (key, _, record) in &self.records { record.validate().map_err(|_msg| { - // Suppress unused warning for key in no_std - let _ = &key; #[cfg(feature = "std")] { DbError::RuntimeError { @@ -751,6 +729,8 @@ where } #[cfg(not(feature = "std"))] { + // Suppress unused warning for key in no_std + let _ = &key; DbError::RuntimeError { _message: () } } })?; @@ -853,32 +833,28 @@ where profiling_clock, }); + // Accumulator for every future the runner will drive. + let mut futures_acc: Vec = Vec::new(); + #[cfg(feature = "tracing")] - tracing::info!( - "Spawning producer services and tap observers for {} records", - self.spawn_fns.len() - ); + tracing::info!("Collecting futures for {} records", self.spawn_fns.len()); // Build a lookup map from spawn_fns for topological ordering let mut spawn_fn_map: HashMap> = self.spawn_fns.into_iter().collect(); - // Execute spawn functions in topological order - // This ensures transforms are spawned after their input records + // Execute collectors in topological order β€” transforms collect after their inputs. for key_str in inner.dependency_graph.topo_order() { - // Find the StringKey that matches this key string let key = match inner.by_key.keys().find(|k| k.as_str() == key_str) { Some(k) => *k, - None => continue, // Key not in spawn_fns, skip + None => continue, }; - // Take the spawn function (if any) for this key let spawn_fn_any = match spawn_fn_map.remove(&key) { Some(f) => f, - None => continue, // No spawn function for this key + None => continue, }; - // Resolve key to RecordId let id = inner.resolve(&key).ok_or({ #[cfg(feature = "std")] { @@ -892,27 +868,22 @@ where } })?; - // Downcast from Box back to the concrete spawn function type - type SpawnFnType = - Box, &Arc>, RecordId) -> DbResult<()> + Send>; - let spawn_fn = spawn_fn_any .downcast::>() .expect("spawn function type mismatch"); - // Execute the spawn function - (*spawn_fn)(&runtime, &db, id)?; + futures_acc.extend((*spawn_fn)(&runtime, &db, id)?); } #[cfg(feature = "tracing")] - tracing::info!("Automatic spawning complete"); + tracing::info!("Record future collection complete"); - // Spawn remote access supervisor if configured (std only) + // Collect the remote-access supervisor future, if configured (std only). #[cfg(feature = "std")] if let Some(remote_cfg) = self.remote_config { #[cfg(feature = "tracing")] tracing::info!( - "Spawning remote access supervisor on socket: {}", + "Building remote access supervisor for socket: {}", remote_cfg.socket_path.display() ); @@ -923,77 +894,54 @@ where #[cfg(feature = "tracing")] tracing::debug!("Marking record '{}' as writable", key_str); - // Mark the record as writable (type-erased call) inner.storages[id.index()].set_writable_erased(true); } } - // Spawn the remote supervisor task - crate::remote::supervisor::spawn_supervisor(db.clone(), runtime.clone(), remote_cfg)?; + let supervisor_future = + crate::remote::supervisor::build_supervisor_future(db.clone(), remote_cfg)?; + futures_acc.push(supervisor_future); #[cfg(feature = "tracing")] - tracing::info!("Remote access supervisor spawned successfully"); + tracing::info!("Remote access supervisor future collected"); } - // Build connectors from builders (after database is fully constructed) - // This allows connectors to use collect_inbound_routes() which creates - // producers tied to this specific database instance + // Collect connector futures. After issue #88 connector builders return + // a `Vec` instead of an `Arc` (which previously + // was discarded anyway β€” see design doc 028 Β§"The dropped Arc object"). for builder in self.connector_builders { #[cfg(feature = "tracing")] - let scheme = { - #[cfg(feature = "std")] - { - builder.scheme().to_string() - } - #[cfg(not(feature = "std"))] - { - alloc::string::String::from(builder.scheme()) - } - }; + let scheme = builder.scheme().to_string(); #[cfg(feature = "tracing")] tracing::debug!("Building connector for scheme: {}", scheme); - // Build the connector (this spawns tasks as a side effect) - let _connector = builder.build(&db).await?; + let connector_futures = builder.build(&db).await?; + futures_acc.extend(connector_futures); #[cfg(feature = "tracing")] - tracing::info!("Connector built and spawned successfully: {}", scheme); + tracing::info!("Connector '{}' contributed {} future(s)", scheme, "n"); } - // Spawn on_start tasks (registered by external crates like aimdb-persistence) + // Collect on_start futures (registered by external crates like aimdb-persistence). if !self.start_fns.is_empty() { #[cfg(feature = "tracing")] - tracing::debug!("Spawning {} on_start task(s)", self.start_fns.len()); + tracing::debug!("Collecting {} on_start future(s)", self.start_fns.len()); - #[cfg(feature = "std")] for (idx, start_fn_any) in self.start_fns.into_iter().enumerate() { let start_fn = start_fn_any .downcast::>() .unwrap_or_else(|_| { panic!("on_start fn[{idx}] type mismatch β€” this is a bug in aimdb-core") }); - let future = (*start_fn)(runtime.clone()); - runtime.spawn(future).map_err(DbError::from)?; - } - - #[cfg(not(feature = "std"))] - for (idx, start_fn_any) in self.start_fns.into_iter().enumerate() { - let start_fn = start_fn_any - .downcast::>() - .unwrap_or_else(|_| { - panic!("on_start fn[{idx}] type mismatch β€” this is a bug in aimdb-core") - }); - let future = (*start_fn)(runtime.clone()); - runtime.spawn(future).map_err(DbError::from)?; + futures_acc.push((*start_fn)(runtime.clone())); } } - // Unwrap the Arc to return the owned AimDb - // This is safe because we just created it and hold the only reference + // Unwrap the Arc to return the owned AimDb (safe: we just created it and hold the only ref). let db_owned = Arc::try_unwrap(db).unwrap_or_else(|arc| (*arc).clone()); - Ok(db_owned) + Ok((db_owned, AimDbRunner::new(futures_acc))) } } @@ -1022,7 +970,7 @@ impl Default for AimDbBuilder { /// .register_record::(&TemperatureConfig) /// .build()?; /// ``` -pub struct AimDb { +pub struct AimDb { /// Internal state inner: Arc, @@ -1034,7 +982,7 @@ pub struct AimDb { profiling_clock: crate::profiling::Clock, } -impl Clone for AimDb { +impl Clone for AimDb { fn clone(&self) -> Self { Self { inner: self.inner.clone(), @@ -1045,7 +993,56 @@ impl Clone for AimDb { } } -impl AimDb { +// ============================================================================ +// AimDbRunner β€” drives every future the builder collected +// ============================================================================ + +/// Non-`Clone` runner returned alongside [`AimDb`] from +/// [`AimDbBuilder::build()`]. +/// +/// Owns the complete set of futures that drive the database (producer +/// services, consumer taps, transforms with fan-in forwarders, connectors, +/// remote-access supervisor, `on_start` tasks). `run()` consumes `self`, +/// drives every future cooperatively via a `FuturesUnordered`, and returns +/// only when **every** future has completed β€” typically never, since +/// producer/consumer loops run for the application's lifetime. +/// +/// Cancellation is the application's responsibility β€” wrap `runner.run()` +/// in `tokio::select!` with a shutdown signal, or use a `CancellationToken` +/// inside each registered service. +pub struct AimDbRunner { + futures: Vec, +} + +impl AimDbRunner { + pub(crate) fn new(futures: Vec) -> Self { + Self { futures } + } + + /// Returns the number of futures the runner will drive. Useful for tests + /// and diagnostic logging. + pub fn future_count(&self) -> usize { + self.futures.len() + } + + /// Drives every collected future to completion. + /// + /// Returns when all futures complete (normally: never, while producer and + /// consumer loops are alive). Drop the runner β€” or wrap this call in a + /// `select!` with a shutdown signal β€” to cancel. + pub async fn run(self) { + use futures_util::stream::{FuturesUnordered, StreamExt}; + + if self.futures.is_empty() { + return; + } + + let mut set: FuturesUnordered = self.futures.into_iter().collect(); + while set.next().await.is_some() {} + } +} + +impl AimDb { /// Internal accessor for the inner state /// /// Used by adapter crates and internal spawning logic. @@ -1084,23 +1081,6 @@ impl AimDb { b.run().await } - /// Spawns a task using the database's runtime adapter - /// - /// This method provides direct access to the runtime's spawn capability. - /// - /// # Arguments - /// * `future` - The future to spawn - /// - /// # Returns - /// `DbResult<()>` - Ok if the task was spawned successfully - pub fn spawn_task(&self, future: F) -> DbResult<()> - where - F: core::future::Future + Send + 'static, - { - self.runtime.spawn(future).map_err(DbError::from)?; - Ok(()) - } - /// Produces a value to a specific record by key /// /// Uses O(1) key-based lookup to find the correct record. @@ -1403,10 +1383,10 @@ impl AimDb { let type_id = self.inner.types[id.index()]; let key = self.inner.keys[id.index()]; let record_metadata = record.collect_metadata(type_id, key, id); - let runtime = self.runtime.clone(); - // Spawn consumer task that forwards JSON values from buffer to channel - let spawn_result = runtime.spawn(async move { + // Bridge state (design 028 Β§"Remote supervisor"): drop into `tokio::spawn` + // directly. The nested-`FuturesUnordered` rewrite is the AimX follow-up. + tokio::spawn(async move { #[cfg(feature = "tracing")] tracing::debug!( "Subscription consumer task started for {}", @@ -1468,8 +1448,6 @@ impl AimDb { tracing::debug!("Subscription consumer task terminated"); }); - spawn_result.map_err(DbError::from)?; - Ok((value_rx, cancel_tx)) } diff --git a/aimdb-core/src/connector.rs b/aimdb-core/src/connector.rs index 9cc93fe..b9e4287 100644 --- a/aimdb-core/src/connector.rs +++ b/aimdb-core/src/connector.rs @@ -43,7 +43,7 @@ use alloc::{ #[cfg(feature = "std")] use alloc::format; -use crate::{builder::AimDb, transport::Connector, DbResult}; +use crate::{builder::AimDb, DbResult}; /// Error that can occur during serialization /// @@ -930,7 +930,7 @@ fn parse_connector_url(url: &str) -> DbResult { /// /// impl ConnectorBuilder for MqttConnectorBuilder /// where -/// R: aimdb_executor::Spawn + 'static, +/// R: aimdb_executor::RuntimeAdapter + 'static, /// { /// fn build<'a>( /// &'a self, @@ -951,26 +951,32 @@ fn parse_connector_url(url: &str) -> DbResult { /// ``` pub trait ConnectorBuilder: Send + Sync where - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { - /// Build the connector using the database + /// Build the connector and return its driving futures. /// - /// This method is called during `AimDbBuilder::build()` after the database - /// has been constructed. The builder can use the database to: - /// - Collect inbound routes via `db.collect_inbound_routes()` - /// - Access database configuration - /// - Register subscriptions + /// Called during `AimDbBuilder::build()` after the database has been + /// constructed. The returned futures (infrastructure loops + per-route + /// publishers) are appended to the builder's accumulator and driven by + /// `AimDbRunner::run()`. /// /// # Arguments /// * `db` - The constructed database instance /// /// # Returns - /// An `Arc` that will be registered with the database + /// All `BoxFuture`s the connector needs to operate. Empty if the connector + /// has no work to drive. #[allow(clippy::type_complexity)] fn build<'a>( &'a self, db: &'a AimDb, - ) -> Pin>> + Send + 'a>>; + ) -> Pin< + Box< + dyn Future + Send + 'static>>>>> + + Send + + 'a, + >, + >; /// The URL scheme this connector handles /// diff --git a/aimdb-core/src/database.rs b/aimdb-core/src/database.rs index 33a3442..ea54054 100644 --- a/aimdb-core/src/database.rs +++ b/aimdb-core/src/database.rs @@ -4,8 +4,7 @@ //! in-memory storage with type-safe records and data synchronization across //! MCU β†’ edge β†’ cloud environments. -use crate::{AimDb, DbError, DbResult, RuntimeAdapter, RuntimeContext}; -use aimdb_executor::Spawn; +use crate::{AimDb, DbResult, RuntimeAdapter, RuntimeContext}; use core::fmt::Debug; #[cfg(not(feature = "std"))] @@ -24,12 +23,12 @@ use std::{boxed::Box, sync::Arc}; /// /// This is a thin wrapper around `AimDb` that adds adapter-specific functionality. /// Most users should use `AimDbBuilder` directly to create databases. -pub struct Database { +pub struct Database { adapter: A, aimdb: AimDb, } -impl Database { +impl Database { /// Internal accessor for the AimDb instance /// /// Used by adapter crates. Should not be used by application code. @@ -139,44 +138,3 @@ impl Database { RuntimeContext::from_arc(Arc::new(self.adapter.clone())) } } - -// Spawn implementation for databases with spawn-capable adapters -impl Database -where - A: RuntimeAdapter + Spawn, -{ - /// Spawns a service on the database's runtime - /// - /// # Example - /// ```rust,ignore - /// # use aimdb_core::Database; - /// # use aimdb_executor::{Runtime, Spawn}; - /// # #[cfg(feature = "tokio-runtime")] - /// # { - /// async fn my_service(ctx: aimdb_core::RuntimeContext) -> aimdb_core::DbResult<()> { - /// // Service implementation - /// Ok(()) - /// } - /// - /// # async fn example(db: Database) -> aimdb_core::DbResult<()> { - /// let ctx = db.context(); - /// db.spawn(async move { - /// if let Err(e) = my_service(ctx).await { - /// eprintln!("Service error: {:?}", e); - /// } - /// })?; - /// # Ok(()) - /// # } - /// # } - /// ``` - pub fn spawn(&self, future: F) -> DbResult<()> - where - F: core::future::Future + Send + 'static, - { - #[cfg(feature = "tracing")] - tracing::debug!("Spawning service on database runtime"); - - self.adapter.spawn(future).map_err(DbError::from)?; - Ok(()) - } -} diff --git a/aimdb-core/src/error.rs b/aimdb-core/src/error.rs index 0f504d5..dcfaa9b 100644 --- a/aimdb-core/src/error.rs +++ b/aimdb-core/src/error.rs @@ -723,17 +723,6 @@ impl From for DbError { use aimdb_executor::ExecutorError; match err { - ExecutorError::SpawnFailed { message } => { - #[cfg(feature = "std")] - { - DbError::RuntimeError { message } - } - #[cfg(not(feature = "std"))] - { - let _ = message; // Avoid unused warnings - DbError::RuntimeError { _message: () } - } - } ExecutorError::RuntimeUnavailable { message } => { #[cfg(feature = "std")] { diff --git a/aimdb-core/src/lib.rs b/aimdb-core/src/lib.rs index 919606a..b415fa1 100644 --- a/aimdb-core/src/lib.rs +++ b/aimdb-core/src/lib.rs @@ -67,7 +67,7 @@ pub use extensions::Extensions; // Runtime trait re-exports from aimdb-executor // These traits define the platform-specific execution, timing, and logging capabilities pub use aimdb_executor::{ - ExecutorError, ExecutorResult, Logger, Runtime, RuntimeAdapter, RuntimeInfo, Spawn, TimeOps, + ExecutorError, ExecutorResult, Logger, Runtime, RuntimeAdapter, RuntimeInfo, TimeOps, }; // Database implementation exports diff --git a/aimdb-core/src/remote/handler.rs b/aimdb-core/src/remote/handler.rs index d5db291..c52e591 100644 --- a/aimdb-core/src/remote/handler.rs +++ b/aimdb-core/src/remote/handler.rs @@ -157,7 +157,7 @@ pub async fn handle_connection( stream: UnixStream, ) -> DbResult<()> where - R: crate::RuntimeAdapter + crate::Spawn + 'static, + R: crate::RuntimeAdapter + 'static, { #[cfg(feature = "tracing")] tracing::info!("New remote access connection established"); @@ -369,7 +369,7 @@ async fn perform_handshake( db: &Arc>, ) -> DbResult> where - R: crate::RuntimeAdapter + crate::Spawn + 'static, + R: crate::RuntimeAdapter + 'static, { let (reader, mut writer) = stream.into_split(); let mut reader = BufReader::new(reader); @@ -555,7 +555,7 @@ async fn handle_request( request: Request, ) -> Response where - R: crate::RuntimeAdapter + crate::Spawn + 'static, + R: crate::RuntimeAdapter + 'static, { #[cfg(feature = "tracing")] tracing::debug!( @@ -614,7 +614,7 @@ async fn handle_record_list( request_id: u64, ) -> Response where - R: crate::RuntimeAdapter + crate::Spawn + 'static, + R: crate::RuntimeAdapter + 'static, { #[cfg(feature = "tracing")] tracing::debug!("Listing records"); @@ -639,7 +639,7 @@ async fn handle_profiling_reset( request_id: u64, ) -> Response where - R: crate::RuntimeAdapter + crate::Spawn + 'static, + R: crate::RuntimeAdapter + 'static, { if matches!( config.security_policy, @@ -670,7 +670,7 @@ async fn handle_buffer_metrics_reset( request_id: u64, ) -> Response where - R: crate::RuntimeAdapter + crate::Spawn + 'static, + R: crate::RuntimeAdapter + 'static, { if matches!( config.security_policy, @@ -715,7 +715,7 @@ async fn handle_record_get( params: Option, ) -> Response where - R: crate::RuntimeAdapter + crate::Spawn + 'static, + R: crate::RuntimeAdapter + 'static, { // Extract record name from params let record_name = match params { @@ -796,7 +796,7 @@ async fn handle_record_set( params: Option, ) -> Response where - R: crate::RuntimeAdapter + crate::Spawn + 'static, + R: crate::RuntimeAdapter + 'static, { use crate::remote::SecurityPolicy; @@ -951,7 +951,7 @@ async fn handle_record_subscribe( params: Option, ) -> Response where - R: crate::RuntimeAdapter + crate::Spawn + 'static, + R: crate::RuntimeAdapter + 'static, { // Extract record name from params let record_name = match params { @@ -1231,7 +1231,7 @@ async fn handle_record_drain( params: Option, ) -> Response where - R: crate::RuntimeAdapter + crate::Spawn + 'static, + R: crate::RuntimeAdapter + 'static, { // Extract record name from params let record_name = match params { @@ -1403,7 +1403,7 @@ async fn handle_record_query( params: Option, ) -> Response where - R: crate::RuntimeAdapter + crate::Spawn + 'static, + R: crate::RuntimeAdapter + 'static, { // Extract the query handler from Extensions. let handler = match db.extensions().get::() { @@ -1473,7 +1473,7 @@ where #[cfg(feature = "std")] async fn handle_graph_nodes(db: &Arc>, request_id: u64) -> Response where - R: crate::RuntimeAdapter + crate::Spawn + 'static, + R: crate::RuntimeAdapter + 'static, { #[cfg(feature = "tracing")] tracing::debug!("Getting dependency graph nodes"); @@ -1504,7 +1504,7 @@ where #[cfg(feature = "std")] async fn handle_graph_edges(db: &Arc>, request_id: u64) -> Response where - R: crate::RuntimeAdapter + crate::Spawn + 'static, + R: crate::RuntimeAdapter + 'static, { #[cfg(feature = "tracing")] tracing::debug!("Getting dependency graph edges"); @@ -1536,7 +1536,7 @@ where #[cfg(feature = "std")] async fn handle_graph_topo_order(db: &Arc>, request_id: u64) -> Response where - R: crate::RuntimeAdapter + crate::Spawn + 'static, + R: crate::RuntimeAdapter + 'static, { #[cfg(feature = "tracing")] tracing::debug!("Getting topological order"); diff --git a/aimdb-core/src/remote/supervisor.rs b/aimdb-core/src/remote/supervisor.rs index cfcd4bc..ea5ab86 100644 --- a/aimdb-core/src/remote/supervisor.rs +++ b/aimdb-core/src/remote/supervisor.rs @@ -3,6 +3,7 @@ //! Manages the Unix domain socket server and spawns connection handlers for //! remote clients connecting via the AimX protocol. +use crate::builder::BoxFuture; use crate::remote::AimxConfig; use crate::{AimDb, DbError, DbResult}; @@ -15,31 +16,30 @@ use std::os::unix::fs::PermissionsExt; #[cfg(feature = "std")] use tokio::net::UnixListener; -/// Spawns the remote access supervisor task +/// Builds the remote access supervisor future. /// -/// This function spawns a background task that: -/// 1. Binds to the Unix domain socket -/// 2. Sets appropriate file permissions -/// 3. Accepts incoming connections -/// 4. Spawns a ConnectionHandler for each client +/// Synchronously: binds the Unix domain socket and sets file permissions +/// (so binding errors surface from `build()` rather than at task-start time). +/// +/// The returned `BoxFuture` is appended to the `AimDbRunner` accumulator; +/// when driven, it accepts incoming connections in a loop and uses +/// `tokio::spawn` to dispatch a per-connection handler (bridge state per +/// design 028 Β§"Remote supervisor" β€” future work in the AimX portability +/// follow-up replaces this with a nested `FuturesUnordered`). /// /// # Arguments /// * `db` - Database instance (for introspection and subscriptions) -/// * `runtime` - Runtime adapter (for spawning tasks) /// * `config` - Remote access configuration /// -/// # Returns -/// `DbResult<()>` - Ok if supervisor spawned successfully -/// /// # Errors /// Returns error if: /// - Socket path already exists and cannot be removed /// - Socket binding fails /// - Permission setting fails #[cfg(feature = "std")] -pub fn spawn_supervisor(db: Arc>, runtime: Arc, config: AimxConfig) -> DbResult<()> +pub fn build_supervisor_future(db: Arc>, config: AimxConfig) -> DbResult where - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { #[cfg(feature = "tracing")] tracing::info!( @@ -104,8 +104,10 @@ where #[cfg(feature = "tracing")] tracing::info!("Socket permissions set to {:o}", permissions); - // Spawn supervisor task using runtime adapter - let _ = runtime.spawn(async move { + // The accept loop is the future the runner drives. Per-connection handlers + // still use `tokio::spawn` (AimX is `#[cfg(feature = "std")]`-gated β€” see + // design doc 028 Β§"Out of Scope" / AimX follow-up). + let supervisor_future: BoxFuture = Box::pin(async move { #[cfg(feature = "tracing")] tracing::info!("Remote access supervisor task started"); @@ -118,7 +120,6 @@ where let db_clone = db.clone(); let config_clone = config.clone(); - // Spawn connection handler for this client tokio::spawn(async move { #[cfg(feature = "tracing")] tracing::debug!("Connection handler spawned for client"); @@ -147,5 +148,5 @@ where } }); - Ok(()) + Ok(supervisor_future) } diff --git a/aimdb-core/src/transform/join.rs b/aimdb-core/src/transform/join.rs index 1ea6bd1..12a3e39 100644 --- a/aimdb-core/src/transform/join.rs +++ b/aimdb-core/src/transform/join.rs @@ -11,7 +11,7 @@ use alloc::{ use aimdb_executor::{ExecutorResult, JoinFanInRuntime, JoinQueue, JoinReceiver, JoinSender}; -use crate::transform::TransformDescriptor; +use crate::transform::{CollectedTransform, TransformDescriptor}; use crate::typed_record::BoxFuture; // ============================================================================ @@ -240,8 +240,8 @@ where JoinPipeline { spawn_factory: Box::new(move |_| TransformDescriptor { input_keys: input_keys_for_descriptor, - spawn_fn: Box::new(move |producer, db, ctx| { - Box::pin(run_join_transform(db, inputs, producer, handler, ctx)) + build_fn: Box::new(move |producer, db, runtime| { + build_join_collected(db, inputs, producer, handler, runtime) }), }), } @@ -269,44 +269,35 @@ where } // ============================================================================ -// Join Transform Task Runner +// Join Transform Build (forwarders + handler future, both collected at build time) // ============================================================================ #[cfg(feature = "alloc")] #[allow(unused_variables)] -async fn run_join_transform( +fn build_join_collected( db: Arc>, inputs: Vec<(String, JoinInputFactory)>, producer: crate::Producer, handler: F, - runtime_ctx: Arc, -) where + runtime: Arc, +) -> CollectedTransform +where O: Send + Sync + Clone + Debug + 'static, R: JoinFanInRuntime + 'static, F: FnOnce(JoinEventRx, crate::Producer) -> Fut + Send + 'static, Fut: core::future::Future + Send + 'static, { let output_key = producer.key().to_string(); - let input_keys: Vec = inputs.iter().map(|(k, _)| k.clone()).collect(); #[cfg(feature = "tracing")] - tracing::info!( - "πŸ”„ Join transform started: {:?} β†’ '{}'", - input_keys, - output_key - ); - - let runtime: &R = match runtime_ctx.downcast_ref::() { - Some(r) => r, - None => { - #[cfg(feature = "tracing")] - tracing::error!( - "πŸ”„ Join transform '{}' FATAL: runtime context downcast failed", - output_key - ); - return; - } - }; + { + let input_keys: Vec = inputs.iter().map(|(k, _)| k.clone()).collect(); + tracing::info!( + "πŸ”„ Join transform building: {:?} β†’ '{}'", + input_keys, + output_key + ); + } let queue = match runtime.create_join_queue::() { Ok(q) => q, @@ -316,37 +307,43 @@ async fn run_join_transform( "πŸ”„ Join transform '{}' FATAL: failed to create join queue", output_key ); - return; + // Empty collected transform β€” caller still receives a valid descriptor. + return CollectedTransform { + task_future: Box::pin(async {}), + fanin_futures: Vec::new(), + }; } }; let (tx, rx) = queue.split(); - for (index, (_key, factory)) in inputs.into_iter().enumerate() { - let sender = tx.clone(); - let db = db.clone(); - - let forwarder_future = factory(db, index, sender); - if let Err(_e) = runtime.spawn(forwarder_future) { - #[cfg(feature = "tracing")] - tracing::error!( - "πŸ”„ Join transform '{}' FATAL: failed to spawn forwarder for input index {}", - output_key, - index - ); - return; - } - } - + // Build all forwarder futures eagerly; they will be driven by AimDbRunner. + let fanin_futures: Vec> = inputs + .into_iter() + .enumerate() + .map(|(index, (_key, factory))| { + let sender = tx.clone(); + factory(db.clone(), index, sender) + }) + .collect(); + + // Drop our local sender so the receiver closes once all forwarders exit. drop(tx); - #[cfg(feature = "tracing")] - tracing::debug!( - "βœ… Join transform '{}' all forwarders spawned, handing receiver to user task", - output_key - ); + let task_future: BoxFuture<'static, ()> = Box::pin(async move { + #[cfg(feature = "tracing")] + tracing::debug!( + "βœ… Join transform '{}' handing receiver to user task", + output_key + ); - handler(JoinEventRx::new(rx), producer).await; + handler(JoinEventRx::new(rx), producer).await; - #[cfg(feature = "tracing")] - tracing::warn!("πŸ”„ Join transform '{}' user task exited", output_key); + #[cfg(feature = "tracing")] + tracing::warn!("πŸ”„ Join transform '{}' user task exited", output_key); + }); + + CollectedTransform { + task_future, + fanin_futures, + } } diff --git a/aimdb-core/src/transform/mod.rs b/aimdb-core/src/transform/mod.rs index c7e5665..b85115c 100644 --- a/aimdb-core/src/transform/mod.rs +++ b/aimdb-core/src/transform/mod.rs @@ -10,7 +10,6 @@ //! - Single-input: `.transform()` with `TransformBuilder` //! - Multi-input: `.transform_join()` with `JoinBuilder` -use core::any::Any; use core::fmt::Debug; use alloc::{boxed::Box, string::String, sync::Arc, vec::Vec}; @@ -30,19 +29,24 @@ pub use join::{JoinBuilder, JoinEventRx, JoinPipeline, JoinTrigger}; // TransformDescriptor β€” stored per output record in TypedRecord // ============================================================================ -pub(crate) struct TransformDescriptor +/// Futures contributed by a transform at build-time collection. +/// +/// `task_future` is the transform's main event loop; `fanin_futures` is any +/// per-input forwarder needed for multi-input joins (empty for single-input). +pub(crate) struct CollectedTransform { + pub task_future: BoxFuture<'static, ()>, + pub fanin_futures: Vec>, +} + +pub(crate) struct TransformDescriptor where T: Send + 'static + Debug + Clone, { pub input_keys: Vec, #[allow(clippy::type_complexity)] - pub spawn_fn: Box< - dyn FnOnce( - crate::Producer, - Arc>, - Arc, - ) -> BoxFuture<'static, ()> + pub build_fn: Box< + dyn FnOnce(crate::Producer, Arc>, Arc) -> CollectedTransform + Send, >, } diff --git a/aimdb-core/src/transform/single.rs b/aimdb-core/src/transform/single.rs index c06fd5f..9436d99 100644 --- a/aimdb-core/src/transform/single.rs +++ b/aimdb-core/src/transform/single.rs @@ -8,7 +8,7 @@ use alloc::{ vec, }; -use crate::transform::TransformDescriptor; +use crate::transform::{CollectedTransform, TransformDescriptor}; // ============================================================================ // TransformBuilder β†’ TransformPipeline @@ -18,7 +18,7 @@ use crate::transform::TransformDescriptor; /// /// Created by `RecordRegistrar::transform_raw()`. Use `.map()` for stateless /// transforms or `.with_state()` for stateful transforms. -pub struct TransformBuilder { +pub struct TransformBuilder { input_key: String, _phantom: PhantomData<(I, O, R)>, } @@ -27,7 +27,7 @@ impl TransformBuilder where I: Send + Sync + Clone + Debug + 'static, O: Send + Sync + Clone + Debug + 'static, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { pub(crate) fn new(input_key: String) -> Self { Self { @@ -65,7 +65,7 @@ where } /// Intermediate builder for stateful single-input transforms. -pub struct StatefulTransformBuilder { +pub struct StatefulTransformBuilder { input_key: String, initial_state: S, _phantom: PhantomData<(I, O, R)>, @@ -76,7 +76,7 @@ where I: Send + Sync + Clone + Debug + 'static, O: Send + Sync + Clone + Debug + 'static, S: Send + Sync + 'static, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { /// Called for each input value. Receives mutable state, returns optional output. pub fn on_value(self, f: F) -> TransformPipeline @@ -98,7 +98,7 @@ where pub struct TransformPipeline< I, O: Send + Sync + Clone + Debug + 'static, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, > { pub(crate) input_key: String, pub(crate) spawn_factory: Box TransformDescriptor + Send + Sync>, @@ -109,7 +109,7 @@ impl TransformPipeline where I: Send + Sync + Clone + Debug + 'static, O: Send + Sync + Clone + Debug + 'static, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { pub(crate) fn into_descriptor(self) -> TransformDescriptor { (self.spawn_factory)(self.input_key) @@ -125,21 +125,22 @@ where I: Send + Sync + Clone + Debug + 'static, O: Send + Sync + Clone + Debug + 'static, S: Send + Sync + 'static, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { let input_key_clone = input_key.clone(); let input_keys = vec![input_key]; TransformDescriptor { input_keys, - spawn_fn: Box::new(move |producer, db, _ctx| { - Box::pin(run_single_transform::( + build_fn: Box::new(move |producer, db, _runtime| CollectedTransform { + task_future: Box::pin(run_single_transform::( db, input_key_clone, producer, initial_state, transform_fn, - )) + )), + fanin_futures: alloc::vec::Vec::new(), }), } } @@ -159,7 +160,7 @@ pub(crate) async fn run_single_transform( I: Send + Sync + Clone + Debug + 'static, O: Send + Sync + Clone + Debug + 'static, S: Send + 'static, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { let output_key = producer.key().to_string(); diff --git a/aimdb-core/src/typed_api.rs b/aimdb-core/src/typed_api.rs index 2d0fda9..3e70f05 100644 --- a/aimdb-core/src/typed_api.rs +++ b/aimdb-core/src/typed_api.rs @@ -88,7 +88,7 @@ use crate::{AimDb, DbResult}; /// - **Clear Intent**: Function signature shows what it produces /// - **Decoupling**: No access to other record types /// - **Security**: Cannot misuse database for unintended operations -pub struct Producer { +pub struct Producer { /// Reference to the database db: Arc>, /// Record key for key-based routing (required - all records have keys) @@ -96,14 +96,15 @@ pub struct Producer { /// Stage profiling state (set by the spawn machinery for `.source()` stages). #[cfg(feature = "profiling")] profiling: Option>, - /// Phantom data to bind the type parameter T - _phantom: PhantomData, + // `fn() -> T` carries T without forcing Producer's Send/Sync to depend on T. + // T is only a type-system marker here β€” it is never stored or referenced. + _phantom: PhantomData T>, } impl Producer where T: Send + 'static + Debug + Clone, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { /// Create a new producer bound to a specific record key pub(crate) fn new(db: Arc>, key: String) -> Self { @@ -150,7 +151,7 @@ where impl Clone for Producer where - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { fn clone(&self) -> Self { Self { @@ -163,14 +164,11 @@ where } } -unsafe impl Send for Producer {} -unsafe impl Sync for Producer {} - // Implement ProducerTrait for type-erased routing impl crate::connector::ProducerTrait for Producer where T: Send + 'static + Debug + Clone, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { fn produce_any<'a>( &'a self, @@ -215,7 +213,7 @@ where /// - **Decoupling**: No access to other record types /// - **Security**: Cannot misuse database for unintended operations #[derive(Clone)] -pub struct Consumer { +pub struct Consumer { /// Reference to the database db: Arc>, /// Record key for key-based routing (required - all records have keys) @@ -223,14 +221,14 @@ pub struct Consumer { /// Stage profiling state (set by the spawn machinery for `.tap()` / `.link()`). #[cfg(feature = "profiling")] profiling: Option<(Arc, crate::profiling::Clock)>, - /// Phantom data to bind the type parameter T - _phantom: PhantomData, + // See Producer: `fn() -> T` keeps Send/Sync independent of T. + _phantom: PhantomData T>, } impl Consumer where T: Send + Sync + 'static + Debug + Clone, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { /// Create a new consumer bound to a specific record key pub(crate) fn new(db: Arc>, key: String) -> Self { @@ -275,9 +273,6 @@ where } } -unsafe impl Send for Consumer {} -unsafe impl Sync for Consumer {} - // ============================================================================ // Type-erased Consumer Trait Implementation // ============================================================================ @@ -309,7 +304,7 @@ impl crate::connector::AnyReader for TypedAnyReader crate::connector::ConsumerTrait for Consumer where T: Send + Sync + 'static + Debug + Clone, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { fn subscribe_any<'a>( &'a self, @@ -352,7 +347,7 @@ pub enum StageKind { pub struct RecordRegistrar< 'a, T: Send + Sync + 'static + Debug + Clone, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, > { /// The typed record being configured pub(crate) rec: &'a mut TypedRecord, @@ -372,7 +367,7 @@ pub struct RecordRegistrar< impl<'a, T, R> RecordRegistrar<'a, T, R> where T: Send + Sync + 'static + Debug + Clone, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { /// Returns a reference to the builder's extension storage. /// @@ -666,7 +661,7 @@ where pub struct OutboundConnectorBuilder< 'a, T: Send + Sync + 'static + Debug + Clone, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, > { registrar: &'a mut RecordRegistrar<'a, T, R>, url: String, @@ -679,7 +674,7 @@ pub struct OutboundConnectorBuilder< impl<'a, T, R> OutboundConnectorBuilder<'a, T, R> where T: Send + Sync + 'static + Debug + Clone, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { /// Adds a configuration option to the connector pub fn with_config(mut self, key: &str, value: &str) -> Self { @@ -902,7 +897,7 @@ type TypedDeserializerFn = Arc Result + Send + Sy pub struct InboundConnectorBuilder< 'a, T: Send + Sync + 'static + Debug + Clone, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, > { registrar: &'a mut RecordRegistrar<'a, T, R>, url: String, @@ -915,7 +910,7 @@ pub struct InboundConnectorBuilder< impl<'a, T, R> InboundConnectorBuilder<'a, T, R> where T: Send + Sync + 'static + Debug + Clone, - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { /// Adds a configuration option to the connector pub fn with_config(mut self, key: &str, value: &str) -> Self { @@ -1129,7 +1124,7 @@ where /// /// Records implementing this trait register their producer and consumer /// functions, encapsulating behavior with their type. -pub trait RecordT: +pub trait RecordT: Send + Sync + 'static + Debug + Clone { /// Configuration type for this record @@ -1180,7 +1175,7 @@ mod tests { // Test infrastructure for InboundConnectorBuilder deserializer tests // ==================================================================== - /// Minimal mock runtime implementing Spawn (and Runtime for context tests) + /// Minimal mock runtime for context tests struct MockRuntime; impl aimdb_executor::RuntimeAdapter for MockRuntime { @@ -1189,16 +1184,6 @@ mod tests { } } - impl aimdb_executor::Spawn for MockRuntime { - type SpawnToken = (); - fn spawn(&self, _future: F) -> aimdb_executor::ExecutorResult - where - F: Future + Send + 'static, - { - Ok(()) - } - } - impl aimdb_executor::TimeOps for MockRuntime { type Instant = u64; type Duration = u64; @@ -1254,8 +1239,14 @@ mod tests { fn build<'a>( &'a self, _db: &'a crate::AimDb, - ) -> Pin>> + Send + 'a>> - { + ) -> Pin< + Box< + dyn Future< + Output = DbResult + Send + 'static>>>>, + > + Send + + 'a, + >, + > { unimplemented!("not needed for deserializer tests") } fn scheme(&self) -> &str { @@ -1602,7 +1593,10 @@ mod tests { { crate::transform::TransformDescriptor:: { input_keys: vec![], - spawn_fn: Box::new(|_p, _db, _ctx| Box::pin(async {})), + build_fn: Box::new(|_p, _db, _ctx| crate::transform::CollectedTransform { + task_future: Box::pin(async {}), + fanin_futures: vec![], + }), } } diff --git a/aimdb-core/src/typed_record.rs b/aimdb-core/src/typed_record.rs index 7f2bf8f..7613c4e 100644 --- a/aimdb-core/src/typed_record.rs +++ b/aimdb-core/src/typed_record.rs @@ -422,7 +422,7 @@ pub trait AnyRecordExt { /// /// # Returns /// `Some(&TypedRecord)` if types match, `None` otherwise - fn as_typed( + fn as_typed( &self, ) -> Option<&TypedRecord>; @@ -434,53 +434,57 @@ pub trait AnyRecordExt { /// /// # Returns /// `Some(&mut TypedRecord)` if types match, `None` otherwise - fn as_typed_mut( + fn as_typed_mut< + T: Send + 'static + Debug + Clone, + R: aimdb_executor::RuntimeAdapter + 'static, + >( &mut self, ) -> Option<&mut TypedRecord>; } impl AnyRecordExt for Box { - fn as_typed( + fn as_typed( &self, ) -> Option<&TypedRecord> { self.as_any().downcast_ref::>() } - fn as_typed_mut( + fn as_typed_mut< + T: Send + 'static + Debug + Clone, + R: aimdb_executor::RuntimeAdapter + 'static, + >( &mut self, ) -> Option<&mut TypedRecord> { self.as_any_mut().downcast_mut::>() } } -/// Helper for spawning tasks for a specific record type +/// Helper for collecting all build-time futures for a specific record type. /// -/// Captures record type `T` via PhantomData to spawn tasks without making AnyRecord non-object-safe. -pub struct RecordSpawner { +/// Captures record type `T` via PhantomData so `AnyRecord` stays object-safe. +/// Returns every future the record needs (producer, transform, consumers) so +/// `AimDbBuilder::build()` can hand them to the `AimDbRunner`. +pub struct RecordFutureCollector { _phantom: core::marker::PhantomData, } -impl RecordSpawner +impl RecordFutureCollector where T: Send + Sync + 'static + Debug + Clone, { - /// Spawns all tasks (producer, consumers, and inbound connectors) for a record + /// Collects all futures (producer, transform, consumers) for a record. /// - /// Downcasts type-erased AnyRecord to TypedRecord and spawns tasks. - /// - /// # Arguments - /// * `record` - The type-erased record to spawn tasks for - /// * `runtime` - The runtime adapter for spawning tasks - /// * `db` - The database instance - /// * `record_key` - The record's key for key-based producer/consumer creation - pub fn spawn_all_tasks( + /// Downcasts the type-erased `AnyRecord` to `TypedRecord` then returns + /// every future the record contributes to the runner. Source/transform are + /// mutually exclusive β€” at most one will appear. + pub fn collect_all_futures( record: &dyn AnyRecord, runtime: &Arc, db: &Arc>, record_key: &str, - ) -> crate::DbResult<()> + ) -> crate::DbResult>> where - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { use crate::DbError; @@ -499,30 +503,33 @@ where } })?; - // Spawn producer service if present (using key-based producer) - // Note: source and transform are mutually exclusive β€” only one will be present + let mut futures = Vec::new(); + if typed_record.has_producer_service() { - typed_record.spawn_producer_service(runtime, db, record_key)?; + if let Some(f) = typed_record.collect_producer_future(runtime, db, record_key)? { + futures.push(f); + } } - // Spawn transform task if present (mutually exclusive with source) if typed_record.has_transform() { - typed_record.spawn_transform_task(runtime, db, record_key)?; + futures.extend(typed_record.collect_transform_futures(runtime, db, record_key)?); } - // Spawn consumer tasks if present (using key-based consumer) if typed_record.consumer_count() > 0 { - typed_record.spawn_consumer_tasks(runtime, db, record_key)?; + futures.extend(typed_record.collect_consumer_futures(runtime, db, record_key)?); } - Ok(()) + Ok(futures) } } /// Typed record storage with producer/consumer functions /// /// Stores type-safe producer and consumer functions with optional buffering for async dispatch. -pub struct TypedRecord { +pub struct TypedRecord< + T: Send + 'static + Debug + Clone, + R: aimdb_executor::RuntimeAdapter + 'static, +> { /// Optional producer service - a task that generates data /// This will be auto-spawned during build() if present /// Stored as FnOnce that takes (Producer, RuntimeContext) and returns a Future @@ -600,7 +607,9 @@ pub struct TypedRecord>>, } -impl TypedRecord { +impl + TypedRecord +{ /// Creates a new empty typed record /// /// Call `.with_remote_access()` to enable JSON (std only). @@ -874,23 +883,22 @@ impl Type } } - /// Spawns the transform task for this record. + /// Collects the transform task future and any fan-in forwarder futures. /// - /// Takes the transform descriptor (can only spawn once) and spawns the - /// reactive task that subscribes to input records and produces to this record. - pub fn spawn_transform_task( + /// Single-input transforms return one future; join transforms return the + /// trigger-loop future plus one forwarder future per input. Returns an + /// empty `Vec` if no transform is registered. + pub fn collect_transform_futures( &self, runtime: &Arc, db: &Arc>, record_key: &str, - ) -> crate::DbResult<()> + ) -> crate::DbResult>> where - R: aimdb_executor::Spawn, + R: aimdb_executor::RuntimeAdapter, T: Sync, { - use crate::DbError; - - // Take the transform descriptor (can only spawn once) + // Take the transform descriptor (can only collect once) let descriptor = { #[cfg(feature = "std")] { @@ -905,27 +913,23 @@ impl Type if let Some(desc) = descriptor { #[cfg(feature = "tracing")] tracing::info!( - "πŸ”„ Spawning transform task for '{}' (inputs: {:?})", + "πŸ”„ Collecting transform futures for '{}' (inputs: {:?})", record_key, desc.input_keys ); // Create Producer bound to the specific record key let producer = crate::typed_api::Producer::new(db.clone(), record_key.to_string()); - let ctx: Arc = runtime.clone(); - // Call the spawn function (mirrors spawn_producer_service exactly) - let task_future = (desc.spawn_fn)(producer, db.clone(), ctx); - runtime.spawn(task_future).map_err(DbError::from)?; + let collected = (desc.build_fn)(producer, db.clone(), runtime.clone()); - #[cfg(feature = "tracing")] - tracing::info!( - "βœ… Successfully spawned transform task for record '{}'", - record_key - ); + let mut out = Vec::with_capacity(1 + collected.fanin_futures.len()); + out.push(collected.task_future); + out.extend(collected.fanin_futures); + Ok(out) + } else { + Ok(Vec::new()) } - - Ok(()) } /// Sets the buffer for this record @@ -1148,25 +1152,24 @@ impl Type } } - /// Spawns all registered consumer tasks + /// Collects all registered consumer (`.tap()`) futures. /// - /// Spawns `.tap()` consumers as background tasks. Called automatically by `Database::new()`. - /// Consumes registered consumers (FnOnce), can only be called once. - pub fn spawn_consumer_tasks( + /// Consumes registered consumers (FnOnce) β€” can only be called once. The returned + /// futures are pushed into the build-time accumulator by `RecordFutureCollector` + /// and driven by `AimDbRunner::run()`. + pub fn collect_consumer_futures( &self, runtime: &Arc, db: &Arc>, record_key: &str, - ) -> crate::DbResult<()> + ) -> crate::DbResult>> where - R: aimdb_executor::Spawn, + R: aimdb_executor::RuntimeAdapter, T: Sync, { - use crate::DbError; - #[cfg(feature = "tracing")] tracing::debug!( - "Spawning {} consumer tasks for record type {}", + "Collecting {} consumer futures for record type {}", self.consumer_count(), core::any::type_name::() ); @@ -1184,15 +1187,13 @@ impl Type } }; - #[cfg(feature = "tracing")] - let count = consumers.len(); - // Invariant: taps are pushed to `profiling` in the same order consumers are // added (both happen in `RecordRegistrar::tap_raw`), so index `i` lines up. #[cfg(feature = "profiling")] debug_assert_eq!(self.profiling.tap_count(), consumers.len()); - // Spawn each consumer as a background task + let mut futures = Vec::with_capacity(consumers.len()); + #[cfg_attr(not(feature = "profiling"), allow(unused_variables))] for (i, consumer_fn) in consumers.into_iter().enumerate() { // Create a Consumer handle bound to the specific record key @@ -1204,40 +1205,27 @@ impl Type consumer.set_profiling(entry.metrics.clone(), db.profiling_clock().clone()); } - // Get runtime context as Arc by cloning the Arc + // Type-erase the runtime for the consumer signature. let runtime_any: Arc = runtime.clone(); - // Spawn the consumer task with runtime context - let task_future = consumer_fn(consumer, runtime_any); - runtime.spawn(task_future).map_err(DbError::from)?; + futures.push(consumer_fn(consumer, runtime_any)); } - #[cfg(feature = "tracing")] - tracing::info!( - "Successfully spawned {} consumer tasks for record type {}", - count, - core::any::type_name::() - ); - - Ok(()) + Ok(futures) } - /// Spawns consumer tasks from a type-erased runtime - /// - /// Helper for automatic spawning system via database's spawn_task method. - pub fn spawn_producer_service( + /// Collects the producer-service future, if one is registered. + pub fn collect_producer_future( &self, runtime: &Arc, db: &Arc>, record_key: &str, - ) -> crate::DbResult<()> + ) -> crate::DbResult>> where - R: aimdb_executor::Spawn, + R: aimdb_executor::RuntimeAdapter, T: Sync, { - use crate::DbError; - - // Take the producer service (can only spawn once) + // Take the producer service (can only collect once) let service = { #[cfg(feature = "std")] { @@ -1252,7 +1240,7 @@ impl Type if let Some(service_fn) = service { #[cfg(feature = "tracing")] tracing::debug!( - "Spawning producer service for record type {}", + "Collecting producer service future for record type {}", core::any::type_name::() ); @@ -1267,20 +1255,10 @@ impl Type let ctx: Arc = runtime.clone(); - // Call the service function to get the future - let task_future = service_fn(producer, ctx); - - // Spawn the producer service task using the typed runtime - runtime.spawn(task_future).map_err(DbError::from)?; - - #[cfg(feature = "tracing")] - tracing::info!( - "Successfully spawned producer service for record type {}", - core::any::type_name::() - ); + Ok(Some(service_fn(producer, ctx))) + } else { + Ok(None) } - - Ok(()) } /// Produces a value by pushing to the buffer @@ -1382,7 +1360,7 @@ impl Type } } -impl Default +impl Default for TypedRecord { fn default() -> Self { @@ -1394,8 +1372,8 @@ impl Defa // This enables safe concurrent access to records from multiple connector tasks // in the bidirectional routing system. The Sync bound propagates from AnyRecord // trait and ensures thread-safe sharing of record values. -impl AnyRecord - for TypedRecord +impl + AnyRecord for TypedRecord { fn validate(&self) -> Result<(), &'static str> { // Producer service is optional - some records are driven by external events diff --git a/aimdb-embassy-adapter/Cargo.toml b/aimdb-embassy-adapter/Cargo.toml index a1dd771..eb58616 100644 --- a/aimdb-embassy-adapter/Cargo.toml +++ b/aimdb-embassy-adapter/Cargo.toml @@ -20,12 +20,6 @@ alloc = [] embassy-runtime = ["embassy-executor", "embassy-time", "embassy-sync"] embassy-net-support = ["embassy-net"] # Network stack support for connectors -# Task pool size configuration for dynamic spawning -# Only one should be enabled at a time (default is 8) -embassy-task-pool-8 = [] # Default: 8 concurrent dynamic tasks -embassy-task-pool-16 = [] # Medium: 16 concurrent dynamic tasks -embassy-task-pool-32 = [] # Large: 32 concurrent dynamic tasks - # Observability features (no_std compatible) tracing = ["aimdb-core/tracing", "dep:tracing"] diff --git a/aimdb-embassy-adapter/src/lib.rs b/aimdb-embassy-adapter/src/lib.rs index 39580ec..5dbcbee 100644 --- a/aimdb-embassy-adapter/src/lib.rs +++ b/aimdb-embassy-adapter/src/lib.rs @@ -12,24 +12,6 @@ //! - **No-std Compatible**: Designed for resource-constrained embedded systems //! - **Configurable Task Pool**: Adjustable pool size for dynamic task spawning //! -//! ## Task Pool Configuration -//! -//! The Embassy adapter uses a static task pool for dynamic spawning. By default, -//! it supports up to 8 concurrent dynamic tasks. You can increase this limit -//! by enabling one of the task pool feature flags: -//! -//! - `embassy-task-pool-8` (default): 8 concurrent dynamic tasks -//! - `embassy-task-pool-16`: 16 concurrent dynamic tasks -//! - `embassy-task-pool-32`: 32 concurrent dynamic tasks -//! -//! Only enable one task pool feature at a time. If you exceed the pool size, -//! you'll get a `ExecutorError::SpawnFailed` error. -//! -//! Example in `Cargo.toml`: -//! ```toml -//! [dependencies] -//! aimdb-embassy-adapter = { version = "0.1", features = ["embassy-runtime", "embassy-task-pool-16"] } -//! ``` //! //! # Architecture //! diff --git a/aimdb-embassy-adapter/src/runtime.rs b/aimdb-embassy-adapter/src/runtime.rs index 90921b9..900445c 100644 --- a/aimdb-embassy-adapter/src/runtime.rs +++ b/aimdb-embassy-adapter/src/runtime.rs @@ -7,10 +7,7 @@ use aimdb_core::{DbError, DbResult}; use aimdb_executor::{ExecutorResult, RuntimeAdapter}; #[cfg(feature = "tracing")] -use tracing::{debug, warn}; - -#[cfg(feature = "embassy-runtime")] -use embassy_executor::Spawner; +use tracing::debug; #[cfg(feature = "embassy-net-support")] use embassy_net::Stack; @@ -39,14 +36,19 @@ pub trait EmbassyNetwork { fn network_stack(&self) -> &'static Stack<'static>; } -/// Embassy runtime adapter for async task execution in embedded systems +/// Embassy runtime adapter for async task execution in embedded systems. +/// +/// When the `embassy-net-support` feature is enabled it holds a `&'static Stack<'static>` +/// for connector use; otherwise it is effectively a unit type. All futures are +/// driven by the `AimDbRunner` returned from `AimDbBuilder::build()`, which is +/// awaited inside the Embassy main task. /// -/// This adapter provides AimDB's runtime interface for Embassy-based embedded -/// applications. It can either work standalone or store an Embassy spawner -/// for integrated task management. +/// # Safety /// -/// When the `embassy-net-support` feature is enabled, it can also store a -/// reference to the network stack for connector use. +/// `embassy_net::Stack` contains a `RefCell` and is `!Sync`. Embassy executors +/// run cooperatively on a single core with no preemption or thread migration, +/// so `unsafe impl Send/Sync` is sound here. This is the only `unsafe` left in +/// the adapter after issue #88 removed the `Spawn` impl and task pool. /// /// # Example /// ```rust,no_run @@ -57,123 +59,67 @@ pub trait EmbassyNetwork { /// /// # async fn example() -> aimdb_core::DbResult<()> { /// let adapter = EmbassyAdapter::new()?; -/// -/// let result = adapter.spawn_task(async { -/// Ok::<_, aimdb_core::DbError>(42) -/// }).await?; /// # Ok(()) /// # } /// # } /// ``` #[derive(Clone)] pub struct EmbassyAdapter { - #[cfg(feature = "embassy-runtime")] - spawner: Option, #[cfg(feature = "embassy-net-support")] network: Option<&'static Stack<'static>>, - #[cfg(not(any(feature = "embassy-runtime", feature = "embassy-net-support")))] + #[cfg(not(feature = "embassy-net-support"))] _phantom: core::marker::PhantomData<()>, } -// SAFETY: EmbassyAdapter only contains an Option and Spawner is thread-safe. -// Embassy executor handles spawner synchronization internally. +// SAFETY: Embassy executors run cooperatively on a single core. The +// `embassy_net::Stack` (when present) is `!Sync` only because of its internal +// `RefCell`; in a single-threaded executor no concurrent access is possible. unsafe impl Send for EmbassyAdapter {} unsafe impl Sync for EmbassyAdapter {} impl core::fmt::Debug for EmbassyAdapter { fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { - let mut debug_struct = f.debug_struct("EmbassyAdapter"); - - #[cfg(feature = "embassy-runtime")] - debug_struct.field("spawner", &self.spawner.is_some()); - + let mut s = f.debug_struct("EmbassyAdapter"); #[cfg(feature = "embassy-net-support")] - debug_struct.field("network", &self.network.is_some()); - - #[cfg(not(any(feature = "embassy-runtime", feature = "embassy-net-support")))] - debug_struct.field("_phantom", &"no features enabled"); - - debug_struct.finish() + s.field("network", &self.network.is_some()); + #[cfg(not(feature = "embassy-net-support"))] + s.field("_phantom", &"no features enabled"); + s.finish() } } impl EmbassyAdapter { - /// Creates a new EmbassyAdapter without a spawner or network - /// - /// This creates a stateless adapter suitable for basic task execution. + /// Creates a new EmbassyAdapter without a network stack. /// /// # Returns - /// `Ok(EmbassyAdapter)` - Always succeeds + /// `Ok(EmbassyAdapter)` β€” always succeeds. pub fn new() -> ExecutorResult { #[cfg(feature = "tracing")] - debug!("Creating EmbassyAdapter (no spawner, no network)"); + debug!("Creating EmbassyAdapter (no network)"); Ok(Self { - #[cfg(feature = "embassy-runtime")] - spawner: None, #[cfg(feature = "embassy-net-support")] network: None, - #[cfg(not(any(feature = "embassy-runtime", feature = "embassy-net-support")))] + #[cfg(not(feature = "embassy-net-support"))] _phantom: core::marker::PhantomData, }) } - /// Creates a new EmbassyAdapter returning DbResult for backward compatibility - /// - /// This method provides compatibility with existing code that expects DbResult. + /// Creates a new EmbassyAdapter returning `DbResult` for backward compatibility. pub fn new_db_result() -> DbResult { Self::new().map_err(DbError::from) } - /// Creates a new EmbassyAdapter with an Embassy spawner - /// - /// This creates an adapter that can use the Embassy spawner for - /// advanced task management operations. - /// - /// # Arguments - /// * `spawner` - The Embassy spawner to use for task management - /// - /// # Returns - /// An EmbassyAdapter configured with the provided spawner - #[cfg(feature = "embassy-runtime")] - pub fn new_with_spawner(spawner: Spawner) -> Self { - #[cfg(feature = "tracing")] - debug!("Creating EmbassyAdapter with spawner"); - - Self { - spawner: Some(spawner), - #[cfg(feature = "embassy-net-support")] - network: None, - } - } - - /// Creates a new EmbassyAdapter with spawner and network stack - /// - /// This creates an adapter with full capabilities for task execution - /// and network operations. - /// - /// # Arguments - /// * `spawner` - The Embassy spawner to use for task management - /// * `network` - Static reference to the network stack - /// - /// # Returns - /// An EmbassyAdapter configured with spawner and network - #[cfg(all(feature = "embassy-runtime", feature = "embassy-net-support"))] - pub fn new_with_network(spawner: Spawner, network: &'static Stack<'static>) -> Self { + /// Creates a new EmbassyAdapter with a network stack + #[cfg(feature = "embassy-net-support")] + pub fn new_with_network(network: &'static Stack<'static>) -> Self { #[cfg(feature = "tracing")] - debug!("Creating EmbassyAdapter with spawner and network"); + debug!("Creating EmbassyAdapter with network"); Self { - spawner: Some(spawner), network: Some(network), } } - - /// Gets a reference to the stored spawner, if any - #[cfg(feature = "embassy-runtime")] - pub fn spawner(&self) -> Option<&Spawner> { - self.spawner.as_ref() - } } impl Default for EmbassyAdapter { @@ -190,76 +136,6 @@ impl RuntimeAdapter for EmbassyAdapter { } } -// Type-erased future wrapper for dynamic spawning -#[cfg(feature = "embassy-runtime")] -type BoxedFuture = alloc::boxed::Box + Send + 'static>; - -// Configurable task pool size for dynamic spawning -// Users can select the pool size via feature flags: -// - embassy-task-pool-8 (default): 8 concurrent tasks -// - embassy-task-pool-16: 16 concurrent tasks -// - embassy-task-pool-32: 32 concurrent tasks -#[cfg(all(feature = "embassy-runtime", feature = "embassy-task-pool-32"))] -const TASK_POOL_SIZE: usize = 32; - -#[cfg(all( - feature = "embassy-runtime", - feature = "embassy-task-pool-16", - not(feature = "embassy-task-pool-32") -))] -const TASK_POOL_SIZE: usize = 16; - -#[cfg(all( - feature = "embassy-runtime", - not(feature = "embassy-task-pool-16"), - not(feature = "embassy-task-pool-32") -))] -const TASK_POOL_SIZE: usize = 8; - -// Generic task runner for dynamic spawning in Embassy -// Pool size is configured at compile time via feature flags -#[cfg(feature = "embassy-runtime")] -#[embassy_executor::task(pool_size = TASK_POOL_SIZE)] -async fn generic_task_runner(mut future: BoxedFuture) { - use core::pin::Pin; - // Pin the boxed future so it can be awaited - let pinned = unsafe { Pin::new_unchecked(&mut *future) }; - pinned.await; -} - -// Implement Spawn trait for Embassy with dynamic spawning support -#[cfg(feature = "embassy-runtime")] -impl aimdb_executor::Spawn for EmbassyAdapter { - type SpawnToken = (); // Embassy doesn't return a handle from static spawn - - fn spawn(&self, future: F) -> ExecutorResult - where - F: core::future::Future + Send + 'static, - { - // Box the future for type erasure - let boxed_future: BoxedFuture = alloc::boxed::Box::new(future); - - // Get the spawner - let spawner = - self.spawner - .as_ref() - .ok_or(aimdb_executor::ExecutorError::RuntimeUnavailable { - message: "No spawner available - use EmbassyAdapter::new_with_spawner()", - })?; - - // Spawn using our generic task pool - match generic_task_runner(boxed_future) { - Ok(spawn_token) => { - spawner.spawn(spawn_token); - Ok(()) - } - Err(_) => Err(aimdb_executor::ExecutorError::SpawnFailed { - message: "Task pool exhausted - enable a larger task pool feature (embassy-task-pool-16 or embassy-task-pool-32)", - }), - } - } -} - // Implement TimeOps trait for time operations #[cfg(feature = "embassy-time")] impl aimdb_executor::TimeOps for EmbassyAdapter { @@ -324,7 +200,7 @@ impl aimdb_executor::Logger for EmbassyAdapter { } } -// Runtime trait is auto-implemented when RuntimeAdapter + TimeOps + Logger + Spawn are all implemented +// Runtime trait is auto-implemented when RuntimeAdapter + TimeOps + Logger are all implemented // Implement EmbassyNetwork trait for accessing network stack #[cfg(feature = "embassy-net-support")] diff --git a/aimdb-executor/Cargo.toml b/aimdb-executor/Cargo.toml index 88b0c2b..66d1ab2 100644 --- a/aimdb-executor/Cargo.toml +++ b/aimdb-executor/Cargo.toml @@ -28,6 +28,9 @@ thiserror = { workspace = true, optional = true } tokio = { workspace = true, optional = true, default-features = false } embassy-executor = { workspace = true, optional = true } +# FuturesUnordered (no_std + alloc compatible) β€” used by AimDbRunner +futures-util = { workspace = true } + # Core async types (always available) # Note: We use core::future::Future which is available in no_std diff --git a/aimdb-executor/src/join.rs b/aimdb-executor/src/join.rs index a9cc2bb..4386ce8 100644 --- a/aimdb-executor/src/join.rs +++ b/aimdb-executor/src/join.rs @@ -1,12 +1,12 @@ use core::future::Future; -use crate::{ExecutorResult, Spawn}; +use crate::{ExecutorResult, RuntimeAdapter}; /// Runtime capability for creating join fan-in queues. /// /// Implemented by each runtime adapter. Queue capacity is an internal /// constant chosen per adapter (Tokio: 64, Embassy: 8, WASM: 64). -pub trait JoinFanInRuntime: Spawn { +pub trait JoinFanInRuntime: RuntimeAdapter { type JoinQueue: JoinQueue; fn create_join_queue(&self) -> ExecutorResult>; diff --git a/aimdb-executor/src/lib.rs b/aimdb-executor/src/lib.rs index bbb22ba..992978b 100644 --- a/aimdb-executor/src/lib.rs +++ b/aimdb-executor/src/lib.rs @@ -7,7 +7,7 @@ //! # Design Philosophy //! //! - **Runtime Agnostic**: No concrete runtime dependencies -//! - **Simple Trait Structure**: 4 focused traits covering all runtime needs +//! - **Simple Trait Structure**: 3 focused traits covering all runtime needs //! - **Platform Flexible**: Works across std and no_std environments //! - **Zero Dependencies**: Pure trait definitions with minimal coupling //! @@ -16,7 +16,10 @@ //! 1. **`RuntimeAdapter`** - Platform identity and metadata //! 2. **`TimeOps`** - Time operations (now, sleep, duration helpers) //! 3. **`Logger`** - Structured logging (info, debug, warn, error) -//! 4. **`Spawn`** - Task spawning with platform-specific tokens +//! +//! Task execution is driven by the `AimDbRunner` returned from +//! `AimDbBuilder::build()`, which collects every future the database needs +//! into a single `FuturesUnordered` β€” no per-adapter `Spawn` impl required. #![cfg_attr(not(feature = "std"), no_std)] @@ -34,14 +37,6 @@ pub type ExecutorResult = Result; #[derive(Debug)] #[cfg_attr(feature = "std", derive(thiserror::Error))] pub enum ExecutorError { - #[cfg_attr(feature = "std", error("Spawn failed: {message}"))] - SpawnFailed { - #[cfg(feature = "std")] - message: String, - #[cfg(not(feature = "std"))] - message: &'static str, - }, - #[cfg_attr(feature = "std", error("Runtime unavailable: {message}"))] RuntimeUnavailable { #[cfg(feature = "std")] @@ -105,20 +100,12 @@ pub trait Logger: RuntimeAdapter { fn error(&self, message: &str); } -/// Task spawning trait - adapter-specific implementation -pub trait Spawn: RuntimeAdapter { - type SpawnToken: Send + 'static; - fn spawn(&self, future: F) -> ExecutorResult - where - F: Future + Send + 'static; -} - // ============================================================================ // Convenience Trait Bundle // ============================================================================ /// Complete runtime trait bundle -pub trait Runtime: RuntimeAdapter + TimeOps + Logger + Spawn { +pub trait Runtime: RuntimeAdapter + TimeOps + Logger { fn runtime_info(&self) -> RuntimeInfo where Self: Sized, @@ -130,7 +117,7 @@ pub trait Runtime: RuntimeAdapter + TimeOps + Logger + Spawn { } // Auto-implement Runtime for any type with all traits -impl Runtime for T where T: RuntimeAdapter + TimeOps + Logger + Spawn {} +impl Runtime for T where T: RuntimeAdapter + TimeOps + Logger {} #[derive(Debug, Clone)] pub struct RuntimeInfo { diff --git a/aimdb-knx-connector/src/embassy_client.rs b/aimdb-knx-connector/src/embassy_client.rs index 6396eed..49d3e85 100644 --- a/aimdb-knx-connector/src/embassy_client.rs +++ b/aimdb-knx-connector/src/embassy_client.rs @@ -97,21 +97,17 @@ impl KnxConnectorBuilder { } } +type BoxFuture = Pin + Send + 'static>>; + /// Implement ConnectorBuilder trait for Embassy runtime with network stack access impl ConnectorBuilder for KnxConnectorBuilder where - R: aimdb_executor::Spawn + aimdb_embassy_adapter::EmbassyNetwork + 'static, + R: aimdb_executor::RuntimeAdapter + aimdb_embassy_adapter::EmbassyNetwork + 'static, { fn build<'a>( &'a self, db: &'a aimdb_core::builder::AimDb, - ) -> Pin< - Box< - dyn Future>> - + Send - + 'a, - >, - > { + ) -> Pin>> + Send + 'a>> { // Wrap in SendFutureWrapper since Embassy types aren't Send but we're single-threaded Box::pin(SendFutureWrapper(async move { // Collect inbound routes from database @@ -132,23 +128,32 @@ where router.resource_ids().len() ); - // Build the actual connector + // Build connection future and channel let runtime_ctx = db.runtime_any(); - let connector = KnxConnectorImpl::build_internal( + let (command_channel, connection_future) = KnxConnectorImpl::build_internal( self.gateway_url.as_str(), router, db.runtime(), - Some(runtime_ctx), + Some(runtime_ctx.clone()), ) .await .map_err(|_e| { #[cfg(feature = "defmt")] defmt::error!("Failed to build KNX connector"); - aimdb_core::DbError::RuntimeError { _message: () } + #[cfg(feature = "std")] + { + aimdb_core::DbError::RuntimeError { + message: format!("Failed to build KNX connector: {}", _e), + } + } + #[cfg(not(feature = "std"))] + { + aimdb_core::DbError::RuntimeError { _message: () } + } })?; - // Collect and spawn outbound publishers + // Collect outbound publisher futures let outbound_routes = db.collect_outbound_routes("knx"); #[cfg(feature = "defmt")] @@ -157,9 +162,15 @@ where outbound_routes.len() ); - connector.spawn_outbound_publishers(db, outbound_routes)?; + let mut futures: Vec = Vec::with_capacity(1 + outbound_routes.len()); + futures.push(connection_future); + futures.extend(KnxConnectorImpl::collect_outbound_futures( + command_channel, + runtime_ctx, + outbound_routes, + )); - Ok(Arc::new(connector) as Arc) + Ok(futures) })) } @@ -264,9 +275,15 @@ impl KnxConnectorImpl { router: Router, runtime: &R, runtime_ctx: Option>, - ) -> Result + ) -> Result< + ( + &'static Channel, + BoxFuture, + ), + &'static str, + > where - R: aimdb_executor::Spawn + aimdb_embassy_adapter::EmbassyNetwork + 'static, + R: aimdb_executor::RuntimeAdapter + aimdb_embassy_adapter::EmbassyNetwork + 'static, { // Parse the gateway URL let connector_url = ConnectorUrl::parse(gateway_url).map_err(|_| "Invalid KNX URL")?; @@ -290,8 +307,8 @@ impl KnxConnectorImpl { // Initialize command channel let command_channel = get_command_channel(); - // Spawn KNX connection background task - let knx_task_future = SendFutureWrapper(async move { + // Build KNX connection future (returned instead of spawned) + let knx_task_future: BoxFuture = Box::pin(SendFutureWrapper(async move { #[cfg(feature = "defmt")] defmt::trace!("KNX background task starting for {}:{}", gateway_ip, port); @@ -308,16 +325,12 @@ impl KnxConnectorImpl { ) .await; } - }); - - runtime - .spawn(Box::pin(knx_task_future)) - .map_err(|_| "Failed to spawn KNX connection task")?; + })); #[cfg(feature = "defmt")] defmt::trace!("KNX connector initialized"); - Ok(Self { command_channel }) + Ok((command_channel, knx_task_future)) } /// Background task that maintains KNX connection and receives telegrams @@ -987,25 +1000,20 @@ impl KnxConnectorImpl { /// /// If a topic provider is configured, it will be called for each value to /// dynamically determine the KNX group address. Otherwise, the static default is used. - fn spawn_outbound_publishers( - &self, - db: &aimdb_core::builder::AimDb, + fn collect_outbound_futures( + command_channel: &'static Channel, + runtime_ctx: Arc, outbound_routes: Vec, - ) -> aimdb_core::DbResult<()> - where - R: aimdb_executor::Spawn + 'static, - { - let runtime = db.runtime(); - let runtime_ctx: Arc = db.runtime_any(); + ) -> Vec { + let mut futures = Vec::with_capacity(outbound_routes.len()); for (default_group_addr_str, consumer, serializer, _config, topic_provider) in outbound_routes { - let command_channel = self.command_channel; let default_group_addr_clone = default_group_addr_str.clone(); let runtime_ctx = runtime_ctx.clone(); - runtime.spawn(Box::pin(SendFutureWrapper(async move { + futures.push(Box::pin(SendFutureWrapper(async move { // Parse default group address using knx-pico's type-safe parser let default_group_addr = match default_group_addr_clone.parse::() { Ok(addr) => Some(addr), @@ -1126,10 +1134,10 @@ impl KnxConnectorImpl { "KNX outbound publisher stopped for: {}", default_group_addr_clone.as_str() ); - })))?; + })) as BoxFuture); } - Ok(()) + futures } } diff --git a/aimdb-knx-connector/src/tokio_client.rs b/aimdb-knx-connector/src/tokio_client.rs index b8e3588..e51397f 100644 --- a/aimdb-knx-connector/src/tokio_client.rs +++ b/aimdb-knx-connector/src/tokio_client.rs @@ -81,17 +81,13 @@ impl KnxConnectorBuilder { } } -impl ConnectorBuilder for KnxConnectorBuilder { +type BoxFuture = Pin + Send + 'static>>; + +impl ConnectorBuilder for KnxConnectorBuilder { fn build<'a>( &'a self, db: &'a aimdb_core::builder::AimDb, - ) -> Pin< - Box< - dyn Future>> - + Send - + 'a, - >, - > { + ) -> Pin>> + Send + 'a>> { Box::pin(async move { // Collect inbound routes from database let inbound_routes = db.collect_inbound_routes("knx"); @@ -111,9 +107,14 @@ impl ConnectorBuilder for KnxConnectorBui router.resource_ids().len() ); - // Build the actual connector + // Build the command channel + connection-task future. + // + // Channel ownership ordering (design 028 Β§"KNX channel ownership"): + // 1. mpsc::channel created here. + // 2. Receiver captured by `connection_future`. + // 3. Sender cloned into each outbound publisher future below. let runtime_ctx = db.runtime_any(); - let connector = + let (command_tx, connection_future) = KnxConnectorImpl::build_internal(&self.gateway_url, router, Some(runtime_ctx)) .await .map_err(|e| { @@ -129,7 +130,6 @@ impl ConnectorBuilder for KnxConnectorBui } })?; - // Collect and spawn outbound publishers let outbound_routes = db.collect_outbound_routes("knx"); #[cfg(feature = "tracing")] @@ -138,9 +138,16 @@ impl ConnectorBuilder for KnxConnectorBui outbound_routes.len() ); - connector.spawn_outbound_publishers(db, outbound_routes)?; + let runtime_ctx: Arc = db.runtime_any(); + let mut futures: Vec = Vec::with_capacity(1 + outbound_routes.len()); + futures.push(connection_future); + futures.extend(KnxConnectorImpl::collect_outbound_futures( + command_tx, + runtime_ctx, + outbound_routes, + )); - Ok(Arc::new(connector) as Arc) + Ok(futures) }) } @@ -149,21 +156,18 @@ impl ConnectorBuilder for KnxConnectorBui } } -/// Internal KNX connector implementation +/// Build-time helper aggregating KNX construction logic. /// -/// This is the actual connector created after collecting routes from the database. -pub struct KnxConnectorImpl { - router: Arc, - /// Command sender for outbound publishing - command_tx: mpsc::Sender, -} +/// `KnxConnectorBuilder::build()` produces a `Vec` containing one +/// connection-task future plus one publisher future per outbound route. There +/// is no longer a long-lived `KnxConnectorImpl` value β€” the previous +/// `Connector::publish` direct-publish path was unreachable through the +/// public API (`AimDbBuilder` discarded the `Arc`). +pub struct KnxConnectorImpl; impl KnxConnectorImpl { - /// Create a new KNX connector with pre-configured router (internal) - /// - /// Creates a connection to the KNX/IP gateway and monitors telegrams - /// for all group addresses defined in the router. The connection task - /// is spawned automatically with reconnection logic. + /// Builds the KNX connection-task future and returns it along with the + /// command sender for use by outbound publishers. /// /// # Arguments /// * `gateway_url` - Gateway URL (knx://host:port) @@ -172,7 +176,7 @@ impl KnxConnectorImpl { gateway_url: &str, router: Router, runtime_ctx: Option>, - ) -> Result { + ) -> Result<(mpsc::Sender, BoxFuture), String> { // Parse the gateway URL let mut url = gateway_url.to_string(); @@ -196,61 +200,40 @@ impl KnxConnectorImpl { let router_arc = Arc::new(router); - // Spawn background connection task with reconnection - let command_tx = spawn_connection_task( - gateway_ip.clone(), + // 1. Create command channel; receiver goes to the connection future, + // sender is returned for the publisher futures to clone. + let (command_tx, command_rx) = mpsc::channel::(32); + + // 2. Build the connection-task future (captures the receiver). + let connection_future = build_connection_future( + gateway_ip, gateway_port, - router_arc.clone(), + router_arc, runtime_ctx, + command_rx, ); - Ok(Self { - router: router_arc, - command_tx, - }) + Ok((command_tx, connection_future)) } - /// Get list of all group addresses this connector monitors + /// Collects outbound publisher futures for all configured routes (internal). /// - /// Returns the unique group addresses from the router configuration. - /// Useful for debugging and monitoring. - pub fn group_addresses(&self) -> Vec> { - self.router.resource_ids() - } - - /// Get the number of routes configured in this connector - /// - /// Each route represents a (group_address, type) mapping. - /// Multiple routes can exist for the same address if different types subscribe to it. - pub fn route_count(&self) -> usize { - self.router.route_count() - } - - /// Spawns outbound publisher tasks for all configured routes (internal) - /// - /// Called automatically during build() to start publishing data from AimDB to KNX. - /// Each route spawns an independent task that subscribes to the record - /// and publishes to the KNX gateway via the command queue. - /// - /// If a topic provider is configured, it will be called for each value to - /// dynamically determine the KNX group address. Otherwise, the static default is used. - fn spawn_outbound_publishers( - &self, - db: &aimdb_core::builder::AimDb, + /// Each route's future subscribes to its typed record, serializes values, and + /// sends them as `KnxCommand::GroupWrite` to the connection task via + /// `command_tx`. Returned futures are appended to the runner's accumulator. + fn collect_outbound_futures( + command_tx: mpsc::Sender, + runtime_ctx: Arc, routes: Vec, - ) -> aimdb_core::DbResult<()> - where - R: aimdb_executor::Spawn + 'static, - { - let runtime = db.runtime(); - let runtime_ctx: Arc = db.runtime_any(); + ) -> Vec { + let mut futures: Vec = Vec::with_capacity(routes.len()); for (default_group_addr_str, consumer, serializer, _config, topic_provider) in routes { - let command_tx = self.command_tx.clone(); + let command_tx = command_tx.clone(); let default_group_addr_clone = default_group_addr_str.clone(); let runtime_ctx = runtime_ctx.clone(); - runtime.spawn(async move { + futures.push(Box::pin(async move { // Parse default group address using knx-pico's type-safe parser let default_group_addr = match default_group_addr_clone.parse::() { Ok(addr) => Some(addr), @@ -368,69 +351,14 @@ impl KnxConnectorImpl { "KNX outbound publisher stopped for: {}", default_group_addr_clone ); - })?; + })); } - Ok(()) + futures } } -// Implement the connector trait from aimdb-core -impl aimdb_core::transport::Connector for KnxConnectorImpl { - fn publish( - &self, - destination: &str, - _config: &aimdb_core::transport::ConnectorConfig, - payload: &[u8], - ) -> Pin> + Send + '_>> - { - use aimdb_core::transport::PublishError; - - // Destination is the group address (from ConnectorUrl::resource_id()) - let group_addr_str = destination.to_string(); - let payload_owned = payload.to_vec(); - let command_tx = self.command_tx.clone(); - - Box::pin(async move { - // Parse group address using knx-pico's type-safe parser - let group_addr = group_addr_str - .parse::() - .map_err(|_| PublishError::InvalidDestination)?; - - // Create response channel for error reporting - let (response_tx, response_rx) = tokio::sync::oneshot::channel(); - - // Send command to connection task - let cmd = KnxCommand::GroupWrite { - group_addr, - data: payload_owned, - response: Some(response_tx), - }; - - command_tx - .send(cmd) - .await - .map_err(|_| PublishError::ConnectionFailed)?; - - // Wait for response from connection task - response_rx - .await - .map_err(|_| PublishError::ConnectionFailed)? - .map_err(|_e| { - #[cfg(feature = "tracing")] - tracing::error!("KNX publish failed: {}", _e); - - PublishError::ConnectionFailed - })?; - - #[cfg(feature = "tracing")] - tracing::debug!("Published to group address: {}", group_addr_str); - Ok(()) - }) - } -} - -/// Spawn the KNX connection task in the background with reconnection logic +/// Builds the KNX connection-task future with reconnection logic. /// /// The connection task handles: /// - KNXnet/IP connection establishment @@ -444,18 +372,15 @@ impl aimdb_core::transport::Connector for KnxConnectorImpl { /// * `gateway_port` - Gateway port (typically 3671) /// * `router` - Router for dispatching telegrams to producers /// * `runtime_ctx` - Optional type-erased runtime for context-aware deserializers -/// -/// # Returns -/// * Command sender for publishing outbound telegrams -fn spawn_connection_task( +/// * `command_rx` - Receiver half of the outbound command channel +fn build_connection_future( gateway_ip: String, gateway_port: u16, router: Arc, runtime_ctx: Option>, -) -> mpsc::Sender { - let (command_tx, mut command_rx) = mpsc::channel(32); // Queue size: 32 - - tokio::spawn(async move { + mut command_rx: mpsc::Receiver, +) -> BoxFuture { + Box::pin(async move { #[cfg(feature = "tracing")] tracing::info!( "KNX connection task started for {}:{}", @@ -486,9 +411,7 @@ fn spawn_connection_task( // Wait before reconnecting tokio::time::sleep(Duration::from_secs(5)).await; } - }); - - command_tx + }) } /// Build CONNECTIONSTATE_REQUEST for heartbeat using knx-pico diff --git a/aimdb-mqtt-connector/src/embassy_client.rs b/aimdb-mqtt-connector/src/embassy_client.rs index 0533b11..886190c 100644 --- a/aimdb-mqtt-connector/src/embassy_client.rs +++ b/aimdb-mqtt-connector/src/embassy_client.rs @@ -42,7 +42,6 @@ extern crate alloc; use aimdb_core::connector::ConnectorUrl; use aimdb_core::router::{Router, RouterBuilder}; -use aimdb_core::transport::PublishError; use aimdb_core::ConnectorBuilder; use alloc::boxed::Box; use alloc::format; @@ -267,24 +266,21 @@ impl MqttConnectorBuilder { } } -/// Implement ConnectorBuilder trait for Embassy runtime with network stack access +type EmbassyBoxFuture = Pin + Send + 'static>>; + +/// Implement ConnectorBuilder trait for Embassy runtime with network stack access. /// -/// This implementation requires the runtime to provide the `EmbassyNetwork` trait -/// so the connector can access the network stack for creating TCP connections. +/// Requires the runtime to provide the `EmbassyNetwork` trait so the connector +/// can access the network stack for creating TCP connections. impl ConnectorBuilder for MqttConnectorBuilder where - R: aimdb_executor::Spawn + aimdb_embassy_adapter::EmbassyNetwork + 'static, + R: aimdb_executor::RuntimeAdapter + aimdb_embassy_adapter::EmbassyNetwork + 'static, { fn build<'a>( &'a self, db: &'a aimdb_core::builder::AimDb, - ) -> Pin< - Box< - dyn Future>> - + Send - + 'a, - >, - > { + ) -> Pin>> + Send + 'a>> + { // Wrap in SendFutureWrapper since Embassy types aren't Send but we're single-threaded Box::pin(SendFutureWrapper(async move { // Collect inbound routes from database @@ -302,9 +298,9 @@ where #[cfg(feature = "defmt")] defmt::info!("MQTT router has {} topics", router.resource_ids().len()); - // Build the actual connector + // Build the action sender + infrastructure futures (mqtt manager + event router). let runtime_ctx = db.runtime_any(); - let connector = MqttConnectorImpl::build_internal( + let (action_sender, infra_futures) = MqttConnectorImpl::build_internal( &self.broker_url, &self.client_id, router, @@ -316,10 +312,19 @@ where #[cfg(feature = "defmt")] defmt::error!("Failed to build MQTT connector"); - aimdb_core::DbError::RuntimeError { _message: () } + #[cfg(feature = "std")] + { + aimdb_core::DbError::RuntimeError { + message: format!("Failed to build MQTT connector: {}", _e), + } + } + #[cfg(not(feature = "std"))] + { + aimdb_core::DbError::RuntimeError { _message: () } + } })?; - // Collect and spawn outbound publishers + // Collect outbound publisher futures. let outbound_routes = db.collect_outbound_routes("mqtt"); #[cfg(feature = "defmt")] @@ -328,9 +333,18 @@ where outbound_routes.len() ); - connector.spawn_outbound_publishers(db, outbound_routes)?; + let runtime_ctx: Arc = db.runtime_any(); + let outbound_futures = MqttConnectorImpl::collect_outbound_futures( + action_sender, + runtime_ctx, + outbound_routes, + ); - Ok(Arc::new(connector) as Arc) + let mut all: Vec = + Vec::with_capacity(infra_futures.len() + outbound_futures.len()); + all.extend(infra_futures); + all.extend(outbound_futures); + Ok(all) })) } @@ -339,32 +353,18 @@ where } } -/// Internal MQTT connector implementation for Embassy -/// -/// This is the actual connector created after collecting routes from the database. -/// It manages the MQTT connection and routing of incoming messages to producers. -pub struct MqttConnectorImpl { - #[allow(dead_code)] // Stored for future use (stats, topic lists, etc.) - router: Arc, - action_sender: Sender<'static, NoopRawMutex, AimdbMqttAction, CHANNEL_SIZE>, -} - -// SAFETY: MqttConnectorImpl is safe to use in Embassy's single-threaded environment. -// The Sender is not Sync, but Embassy tasks run on a single thread. -unsafe impl Send for MqttConnectorImpl {} -unsafe impl Sync for MqttConnectorImpl {} +/// Build-time helper aggregating Embassy MQTT construction logic. +pub struct MqttConnectorImpl; impl MqttConnectorImpl { - /// Create a new MQTT connector with pre-configured router (internal) - /// - /// Creates a connection to the MQTT broker and subscribes to all topics - /// defined in the router. The background task is spawned automatically. + /// Build the MQTT manager + event router futures and return the action + /// channel sender for use by outbound publishers. /// /// # Arguments /// * `broker_url` - Broker URL (mqtt://host:port) /// * `client_id` - MQTT client identifier /// * `router` - Pre-configured router with all routes - /// * `runtime` - Embassy runtime adapter for spawning and network access + /// * `runtime` - Embassy runtime adapter for network access /// * `runtime_ctx` - Optional type-erased runtime for context-aware deserializers async fn build_internal( broker_url: &str, @@ -372,9 +372,15 @@ impl MqttConnectorImpl { router: Router, runtime: &R, runtime_ctx: Option>, - ) -> Result + ) -> Result< + ( + Sender<'static, NoopRawMutex, AimdbMqttAction, CHANNEL_SIZE>, + Vec, + ), + &'static str, + > where - R: aimdb_executor::Spawn + aimdb_embassy_adapter::EmbassyNetwork + 'static, + R: aimdb_executor::RuntimeAdapter + aimdb_embassy_adapter::EmbassyNetwork + 'static, { // Parse the broker URL let mut url = broker_url.to_string(); @@ -443,8 +449,8 @@ impl MqttConnectorImpl { // Get network stack for background task let network = runtime.network_stack(); - // Spawn MQTT manager background task - let mqtt_task_future = SendFutureWrapper(async move { + // Build the MQTT manager future (returned to the runner β€” design 028 Β§"Connector futures"). + let mqtt_task_future: EmbassyBoxFuture = Box::pin(SendFutureWrapper(async move { #[cfg(feature = "defmt")] defmt::info!("MQTT background task starting"); @@ -466,18 +472,10 @@ impl MqttConnectorImpl { ) .await; } - }); - - // Spawn the task - runtime - .spawn(mqtt_task_future) - .map_err(|_| "Failed to spawn MQTT manager task")?; + })); - #[cfg(feature = "defmt")] - defmt::info!("MQTT manager task spawned successfully"); - - // Spawn event monitoring task for inbound message routing - let event_router_future = SendFutureWrapper(async move { + // Build the event router future for inbound message routing. + let event_router_future: EmbassyBoxFuture = Box::pin(SendFutureWrapper(async move { #[cfg(feature = "defmt")] defmt::info!("MQTT event router task starting"); @@ -512,17 +510,11 @@ impl MqttConnectorImpl { } } } - }); - - // Spawn the event router task - runtime - .spawn(event_router_future) - .map_err(|_| "Failed to spawn MQTT event router task")?; + })); - #[cfg(feature = "defmt")] - defmt::info!("MQTT event router task spawned successfully"); - - // Subscribe to all topics from the router + // Subscribe to all topics from the router. The subscribe actions are + // queued in the action channel; the MQTT manager future (not yet + // driven) will drain them once `AimDbRunner::run()` polls it. for topic in &topics { #[cfg(feature = "defmt")] defmt::info!("Subscribing to MQTT topic: {}", &**topic); @@ -538,37 +530,27 @@ impl MqttConnectorImpl { #[cfg(feature = "defmt")] defmt::info!("Queued subscriptions for {} topics", topics.len()); - // Return the connector with action sender - Ok(Self { - router: router_arc, + let _ = router_arc; // router_arc lives inside event_router_future via clone + + Ok(( action_sender, - }) + alloc::vec![mqtt_task_future, event_router_future], + )) } - /// Spawns outbound publisher tasks for all configured routes (internal) - /// - /// Called automatically during build() to start publishing data from AimDB to MQTT. - /// Each route spawns an independent task that subscribes to the record - /// and publishes to the MQTT broker. + /// Collects outbound publisher futures for all configured routes (internal). /// - /// This method uses the ConsumerTrait + factory pattern to subscribe to - /// typed records without knowing the concrete type T at compile time. - /// - /// If a topic provider is configured, it will be called for each value to - /// dynamically determine the MQTT topic. Otherwise, the static default topic is used. - fn spawn_outbound_publishers( - &self, - db: &aimdb_core::builder::AimDb, + /// Each future subscribes to a typed record, serializes values, and sends + /// them to the MQTT action channel. Returned futures are appended to the + /// `AimDbRunner` accumulator. + fn collect_outbound_futures( + action_sender: Sender<'static, NoopRawMutex, AimdbMqttAction, CHANNEL_SIZE>, + runtime_ctx: Arc, routes: Vec, - ) -> aimdb_core::DbResult<()> - where - R: aimdb_executor::Spawn + 'static, - { - let runtime = db.runtime(); - let runtime_ctx: Arc = db.runtime_any(); + ) -> Vec { + let mut futures: Vec = Vec::with_capacity(routes.len()); for (default_topic, consumer, serializer, config, topic_provider) in routes { - let action_sender = self.action_sender; let default_topic_clone = default_topic.clone(); let runtime_ctx = runtime_ctx.clone(); @@ -592,7 +574,7 @@ impl MqttConnectorImpl { } } - runtime.spawn(SendFutureWrapper(async move { + futures.push(Box::pin(SendFutureWrapper(async move { // Subscribe to typed values (type-erased) let mut reader = match consumer.subscribe_any().await { Ok(r) => r, @@ -663,52 +645,10 @@ impl MqttConnectorImpl { "MQTT outbound publisher stopped for topic: {}", default_topic_clone.as_str() ); - }))?; + }))); } - Ok(()) - } -} - -/// Implement the Connector trait for MqttConnectorImpl -impl aimdb_core::transport::Connector for MqttConnectorImpl { - fn publish( - &self, - destination: &str, - config: &aimdb_core::transport::ConnectorConfig, - payload: &[u8], - ) -> Pin> + Send + '_>> { - // Destination is the MQTT topic - let topic = destination.to_string(); - let payload_owned = payload.to_vec(); - let qos = Self::map_qos(config.qos); - let retain = config.retain; - let action_sender = self.action_sender; - - Box::pin(SendFutureWrapper(async move { - #[cfg(feature = "defmt")] - defmt::debug!( - "Publishing to MQTT topic '{}', {} bytes", - topic.as_str(), - payload_owned.len() - ); - - // Create publish action - let action = AimdbMqttAction::Publish { - topic, - payload: payload_owned, - qos, - retain, - }; - - // Send via channel (this will queue the publish) - action_sender.send(action).await; - - #[cfg(feature = "defmt")] - defmt::debug!("MQTT publish queued successfully"); - - Ok(()) - })) + futures } } diff --git a/aimdb-mqtt-connector/src/tokio_client.rs b/aimdb-mqtt-connector/src/tokio_client.rs index f5009cd..42863f2 100644 --- a/aimdb-mqtt-connector/src/tokio_client.rs +++ b/aimdb-mqtt-connector/src/tokio_client.rs @@ -89,17 +89,13 @@ impl MqttConnectorBuilder { } } -impl ConnectorBuilder for MqttConnectorBuilder { +type BoxFuture = Pin + Send + 'static>>; + +impl ConnectorBuilder for MqttConnectorBuilder { fn build<'a>( &'a self, db: &'a aimdb_core::builder::AimDb, - ) -> Pin< - Box< - dyn Future>> - + Send - + 'a, - >, - > { + ) -> Pin>> + Send + 'a>> { Box::pin(async move { // Collect inbound routes from database let inbound_routes = db.collect_inbound_routes("mqtt"); @@ -116,9 +112,9 @@ impl ConnectorBuilder for MqttConnectorBu #[cfg(feature = "tracing")] tracing::info!("MQTT router has {} topics", router.resource_ids().len()); - // Build the actual connector + // Build the client + event-loop future let runtime_ctx = db.runtime_any(); - let connector = MqttConnectorImpl::build_internal( + let (client, event_loop_future) = MqttConnectorImpl::build_internal( &self.broker_url, self.client_id.clone(), router, @@ -138,7 +134,6 @@ impl ConnectorBuilder for MqttConnectorBu } })?; - // NEW: Collect and spawn outbound publishers let outbound_routes = db.collect_outbound_routes("mqtt"); #[cfg(feature = "tracing")] @@ -147,9 +142,16 @@ impl ConnectorBuilder for MqttConnectorBu outbound_routes.len() ); - connector.spawn_outbound_publishers(db, outbound_routes)?; + let runtime_ctx: Arc = db.runtime_any(); + let mut futures: Vec = Vec::with_capacity(1 + outbound_routes.len()); + futures.push(event_loop_future); + futures.extend(MqttConnectorImpl::collect_outbound_futures( + client, + runtime_ctx, + outbound_routes, + )); - Ok(Arc::new(connector) as Arc) + Ok(futures) }) } @@ -182,7 +184,7 @@ impl MqttConnectorImpl { client_id: Option, router: Router, runtime_ctx: Option>, - ) -> Result { + ) -> Result<(Arc, BoxFuture), String> { // Parse the broker URL - we accept it with or without a topic let mut url = broker_url.to_string(); @@ -249,14 +251,12 @@ impl MqttConnectorImpl { let (client, event_loop) = AsyncClient::new(mqtt_opts, channel_capacity); let client_arc = Arc::new(client); - // CRITICAL: Spawn event loop BEFORE subscribing to topics - spawn_event_loop(event_loop, broker_key, router_arc.clone(), runtime_ctx); - - // Yield to ensure the event loop task is scheduled before we start subscribing - tokio::task::yield_now().await; + // Build the event-loop future (returned to the caller for the runner to + // drive). Per design 028 Β§"Connector futures", the event loop runs + // concurrently with outbound publishers under one `FuturesUnordered`. + let event_loop_future = + build_event_loop_future(event_loop, broker_key, router_arc.clone(), runtime_ctx); - // Subscribe to topics from the router - // Now safe for any number of topics since event loop is draining the channel let topics = router_arc.resource_ids(); #[cfg(feature = "tracing")] @@ -275,10 +275,7 @@ impl MqttConnectorImpl { #[cfg(feature = "tracing")] tracing::info!("MQTT subscriptions complete"); - Ok(Self { - client: client_arc, - router: router_arc, - }) + Ok((client_arc, event_loop_future)) } /// Get list of all MQTT topics this connector is subscribed to @@ -297,30 +294,21 @@ impl MqttConnectorImpl { self.router.route_count() } - /// Spawns outbound publisher tasks for all configured routes (internal) - /// - /// Called automatically during build() to start publishing data from AimDB to MQTT. - /// Each route spawns an independent task that subscribes to the record - /// and publishes to the MQTT broker. - /// - /// This method uses the ConsumerTrait + factory pattern to subscribe to - /// typed records without knowing the concrete type T at compile time. + /// Collects outbound publisher futures for all configured routes (internal). /// - /// If a topic provider is configured, it will be called for each value to - /// dynamically determine the MQTT topic. Otherwise, the static default topic is used. - fn spawn_outbound_publishers( - &self, - db: &aimdb_core::builder::AimDb, + /// Called automatically during `build()` to construct the per-route + /// publisher futures. Each subscribes to its record (type-erased), serializes + /// values, and publishes them to the MQTT broker. Returned futures are + /// appended to the `AimDbRunner` accumulator. + fn collect_outbound_futures( + client: Arc, + runtime_ctx: Arc, routes: Vec, - ) -> aimdb_core::DbResult<()> - where - R: aimdb_executor::Spawn + 'static, - { - let runtime = db.runtime(); - let runtime_ctx: Arc = db.runtime_any(); + ) -> Vec { + let mut futures: Vec = Vec::with_capacity(routes.len()); for (default_topic, consumer, serializer, config, topic_provider) in routes { - let client = self.client.clone(); + let client = client.clone(); let default_topic_clone = default_topic.clone(); let runtime_ctx = runtime_ctx.clone(); @@ -349,7 +337,7 @@ impl MqttConnectorImpl { } } - runtime.spawn(async move { + futures.push(Box::pin(async move { // Subscribe to typed values (type-erased) let mut reader = match consumer.subscribe_any().await { Ok(r) => r, @@ -422,10 +410,10 @@ impl MqttConnectorImpl { "MQTT outbound publisher stopped for topic: {}", default_topic_clone ); - })?; + })); } - Ok(()) + futures } } @@ -485,7 +473,7 @@ impl aimdb_core::transport::Connector for MqttConnectorImpl { // Inbound routing now uses the MqttRouter passed to new() } -/// Spawn the MQTT event loop in a background task with router-based dispatch +/// Builds the MQTT event-loop future with router-based dispatch. /// /// The event loop is required by rumqttc to handle: /// - Network I/O (reading/writing packets) @@ -493,18 +481,20 @@ impl aimdb_core::transport::Connector for MqttConnectorImpl { /// - QoS handshakes /// - Routing incoming publishes to AimDB producers /// +/// Returns a `BoxFuture` that is appended to the `AimDbRunner` accumulator. +/// /// # Arguments /// * `event_loop` - The rumqttc EventLoop to run /// * `_broker_key` - Broker identifier for logging (unused in release builds) /// * `router` - Router for dispatching messages to producers /// * `runtime_ctx` - Optional type-erased runtime for context-aware deserializers -fn spawn_event_loop( +fn build_event_loop_future( mut event_loop: EventLoop, _broker_key: String, router: Arc, runtime_ctx: Option>, -) { - tokio::spawn(async move { +) -> BoxFuture { + Box::pin(async move { #[cfg(feature = "tracing")] tracing::debug!("MQTT event loop started for {}", _broker_key); @@ -540,7 +530,7 @@ fn spawn_event_loop( } } } - }); + }) } #[cfg(test)] diff --git a/aimdb-persistence/src/builder_ext.rs b/aimdb-persistence/src/builder_ext.rs index ab9b126..6657304 100644 --- a/aimdb-persistence/src/builder_ext.rs +++ b/aimdb-persistence/src/builder_ext.rs @@ -4,7 +4,7 @@ use std::sync::Arc; use aimdb_core::builder::AimDbBuilder; use aimdb_core::remote::{QueryHandlerFn, QueryHandlerParams}; -use aimdb_executor::{Spawn, TimeOps}; +use aimdb_executor::{RuntimeAdapter, TimeOps}; use crate::backend::{PersistenceBackend, QueryParams}; @@ -20,7 +20,7 @@ pub struct PersistenceState { } /// Extension trait that adds `.with_persistence()` to [`AimDbBuilder`]. -pub trait AimDbBuilderPersistExt { +pub trait AimDbBuilderPersistExt { /// Configures a persistence backend with a retention window. /// /// Stores the backend in the builder's `Extensions` TypeMap (accessible to @@ -43,7 +43,7 @@ pub trait AimDbBuilderPersistExt { impl AimDbBuilderPersistExt for AimDbBuilder where - R: Spawn + TimeOps + 'static, + R: RuntimeAdapter + TimeOps + 'static, { fn with_persistence( mut self, diff --git a/aimdb-persistence/src/ext.rs b/aimdb-persistence/src/ext.rs index b8dfe38..1b44c45 100644 --- a/aimdb-persistence/src/ext.rs +++ b/aimdb-persistence/src/ext.rs @@ -3,7 +3,7 @@ use std::sync::Arc; use aimdb_core::typed_api::RecordRegistrar; -use aimdb_executor::Spawn; +use aimdb_executor::RuntimeAdapter; use crate::backend::PersistenceBackend; use crate::builder_ext::PersistenceState; @@ -16,7 +16,7 @@ use crate::builder_ext::PersistenceState; pub trait RecordRegistrarPersistExt<'a, T, R> where T: serde::Serialize + Send + Sync + Clone + core::fmt::Debug + 'static, - R: Spawn + 'static, + R: RuntimeAdapter + 'static, { /// Opt this record into persistence. /// @@ -29,7 +29,7 @@ where impl<'a, T, R> RecordRegistrarPersistExt<'a, T, R> for RecordRegistrar<'a, T, R> where T: serde::Serialize + Send + Sync + Clone + core::fmt::Debug + 'static, - R: Spawn + 'static, + R: RuntimeAdapter + 'static, { fn persist(&'a mut self, record_name: impl Into) -> &'a mut RecordRegistrar<'a, T, R> { let record_name: String = record_name.into(); diff --git a/aimdb-persistence/src/query_ext.rs b/aimdb-persistence/src/query_ext.rs index 070db62..416d37b 100644 --- a/aimdb-persistence/src/query_ext.rs +++ b/aimdb-persistence/src/query_ext.rs @@ -3,7 +3,7 @@ use std::sync::Arc; use aimdb_core::builder::AimDb; -use aimdb_executor::Spawn; +use aimdb_executor::RuntimeAdapter; use serde::de::DeserializeOwned; use crate::backend::{BoxFuture, PersistenceBackend, QueryParams}; @@ -56,7 +56,7 @@ pub trait AimDbQueryExt { } /// Helper: extract the backend from the `AimDb` extensions. -fn get_backend( +fn get_backend( db: &AimDb, ) -> Result, PersistenceError> { db.extensions() @@ -65,7 +65,7 @@ fn get_backend( .ok_or(PersistenceError::NotConfigured) } -impl AimDbQueryExt for AimDb { +impl AimDbQueryExt for AimDb { fn query_latest( &self, record_pattern: &str, diff --git a/aimdb-persistence/tests/query_skip_invalid.rs b/aimdb-persistence/tests/query_skip_invalid.rs index 4d6a708..f0af4c8 100644 --- a/aimdb-persistence/tests/query_skip_invalid.rs +++ b/aimdb-persistence/tests/query_skip_invalid.rs @@ -71,7 +71,7 @@ impl PersistenceBackend for MockBackend { /// configured β€” we only need the Extensions TypeMap and the query path). async fn build_db(mock: Arc) -> aimdb_core::AimDb { let adapter = Arc::new(TokioAdapter); - AimDbBuilder::new() + let (db, runner) = AimDbBuilder::new() .runtime(adapter) .with_persistence( mock as Arc, @@ -80,7 +80,9 @@ async fn build_db(mock: Arc) -> aimdb_core::AimDb { ) .build() .await - .expect("AimDb build failed") + .expect("AimDb build failed"); + tokio::spawn(runner.run()); + db } /// Shared fixture: four rows where rows at indices 1 and 3 cannot be diff --git a/aimdb-sync/src/handle.rs b/aimdb-sync/src/handle.rs index e9591f3..e39523f 100644 --- a/aimdb-sync/src/handle.rs +++ b/aimdb-sync/src/handle.rs @@ -176,9 +176,8 @@ impl AimDbHandle { // Build the database inside the async context runtime.block_on(async move { - // Build the database (now we're in Tokio context where it can spawn tasks!) - let db = match builder.build().await { - Ok(d) => Arc::new(d), + let (db, runner) = match builder.build().await { + Ok(d) => (Arc::new(d.0), d.1), Err(e) => { eprintln!("Failed to build database: {}", e); return; @@ -191,8 +190,14 @@ impl AimDbHandle { return; } - // Wait for shutdown signal, keeping database alive - let _ = shutdown_rx.recv().await; + // Drive the runner until shutdown. + // If runner.run() completes early (e.g. all tap futures finish), + // we must NOT drop the runtime β€” tasks spawned via runtime_handle + // would be aborted. Keep waiting for the explicit shutdown signal. + tokio::select! { + _ = runner.run() => { let _ = shutdown_rx.recv().await; } + _ = shutdown_rx.recv() => {} + } }); }) .map_err(|e| DbError::AttachFailed { diff --git a/aimdb-tokio-adapter/src/connector.rs b/aimdb-tokio-adapter/src/connector.rs deleted file mode 100644 index fc5ecbe..0000000 --- a/aimdb-tokio-adapter/src/connector.rs +++ /dev/null @@ -1,221 +0,0 @@ -//! Connector task spawning for Tokio runtime -//! -//! This module provides the infrastructure for spawning connector tasks -//! that bridge AimDB records to external systems (MQTT, Kafka, HTTP, etc.). - -#[cfg(feature = "tracing")] -use aimdb_core::RecordKey; -use aimdb_core::{AimDb, DbResult}; - -use crate::TokioAdapter; - -impl TokioAdapter { - /// Spawns connector tasks for all registered connectors - /// - /// This method iterates through all records in the database and spawns - /// a background task for each connector that was registered via `.link()`. - /// - /// Each connector task will: - /// 1. Subscribe to the record's buffer - /// 2. Create a protocol client (MQTT, Kafka, HTTP, etc.) - /// 3. Bridge values from the buffer to the external system - /// - /// # Arguments - /// * `db` - The database instance with registered records and connectors - /// - /// # Returns - /// `Ok(())` if all connectors were spawned successfully - /// - /// # Example - /// - /// ```rust,ignore - /// use aimdb_core::AimDb; - /// use aimdb_tokio_adapter::TokioAdapter; - /// use std::sync::Arc; - /// - /// #[tokio::main] - /// async fn main() -> DbResult<()> { - /// let runtime = Arc::new(TokioAdapter::new()?); - /// - /// // Build database with connector registrations - /// let db = AimDb::build_with(runtime.clone(), |builder| { - /// builder.configure::(|reg| { - /// reg.source(|_em, alert| async { /* ... */ }) - /// .link_to("mqtt://broker:1883").finish(); - /// }); - /// })?; - /// - /// // Spawn connector tasks - /// runtime.spawn_connectors(&db)?; - /// - /// Ok(()) - /// } - /// ``` - pub fn spawn_connectors(&self, db: &AimDb) -> DbResult<()> { - #[cfg(feature = "tracing")] - tracing::debug!("Spawning connector tasks for database"); - - let inner = db.inner(); - #[cfg(feature = "tracing")] - let mut total_connectors = 0; - - // Iterate through all registered records - for i in 0..inner.record_count() { - let record_id = aimdb_core::record_id::RecordId::new(i as u32); - let Some(record) = inner.storage(record_id) else { - continue; - }; - let connector_count = record.outbound_connector_count(); - - if connector_count > 0 { - #[cfg(feature = "tracing")] - { - let key = inner - .key_for(record_id) - .map(|k| k.as_str()) - .unwrap_or("unknown"); - tracing::info!("Record '{}' has {} connector(s)", key, connector_count); - - let urls = record.outbound_connector_urls(); - for url in urls { - tracing::debug!(" β†’ Connector URL: {}", url); - total_connectors += 1; - } - } - - #[cfg(not(feature = "tracing"))] - { - // Consume the connector_count to avoid unused warning - let _ = connector_count; - } - } - } - - #[cfg(feature = "tracing")] - if total_connectors > 0 { - tracing::info!("Total connectors discovered: {}", total_connectors); - } else { - tracing::debug!("No connectors registered in this database"); - } - - Ok(()) - } -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::TokioRecordRegistrarExt; - use aimdb_core::AimDbBuilder; - use std::sync::Arc; - - #[derive(Clone, Debug)] - struct TestMessage { - _content: String, - } - - #[tokio::test] - async fn test_spawn_connectors_no_connectors() -> DbResult<()> { - let adapter = TokioAdapter::new()?; - let mut builder = AimDbBuilder::new().runtime(Arc::new(adapter)); - - // Configure a test record type but don't add any connectors - builder.configure::("test.message", |reg| { - reg.source(|_ctx, _msg| async {}) - .tap(|_ctx, _consumer| async {}); - }); - - let db = builder.build().await?; - - // Should succeed even with no connectors - let result = adapter.spawn_connectors(&db); - assert!(result.is_ok()); - Ok(()) - } - - #[tokio::test] - async fn test_spawn_connectors_with_links() -> DbResult<()> { - use core::future::Future; - use core::pin::Pin; - - let adapter = TokioAdapter::new()?; - let mut builder = AimDbBuilder::new().runtime(Arc::new(adapter)); - - // Register a mock connector builder for testing - struct MockConnector; - - impl aimdb_core::transport::Connector for MockConnector { - fn publish( - &self, - _destination: &str, - _config: &aimdb_core::transport::ConnectorConfig, - _payload: &[u8], - ) -> Pin< - Box< - dyn Future> - + Send - + '_, - >, - > { - Box::pin(async move { Ok(()) }) - } - } - - struct MockBuilder; - - impl aimdb_core::connector::ConnectorBuilder for MockBuilder { - fn scheme(&self) -> &str { - "mqtt" - } - - fn build<'a>( - &'a self, - _db: &'a aimdb_core::builder::AimDb, - ) -> Pin< - Box< - dyn Future>> - + Send - + 'a, - >, - > { - Box::pin(async { - Ok(Arc::new(MockConnector) as Arc) - }) - } - } - - // Register the mock connector - builder = builder.with_connector(MockBuilder); - - // Register a record with connector links - builder.configure::("test.message", |reg| { - reg.source(|_ctx, _msg| async {}) - .tap(|_ctx, _consumer| async {}) - .link_to("mqtt://broker.example.com:1883") - .with_serializer_raw(|_msg: &TestMessage| { - // Dummy serializer for testing - Ok(vec![1, 2, 3]) - }) - .finish(); - }); - - let db = builder.build().await?; - - // Verify record was registered with the connector link - let inner = db.inner(); - let record_id = inner - .resolve_str("test.message") - .expect("Record should exist"); - let record = inner.storage(record_id).expect("Should have the record"); - - assert_eq!(record.outbound_connector_count(), 1); - - #[cfg(feature = "std")] - { - let urls = record.outbound_connector_urls(); - assert!(urls[0].contains("mqtt://")); - } - - Ok(()) - } -} diff --git a/aimdb-tokio-adapter/src/lib.rs b/aimdb-tokio-adapter/src/lib.rs index bcdb651..9259852 100644 --- a/aimdb-tokio-adapter/src/lib.rs +++ b/aimdb-tokio-adapter/src/lib.rs @@ -31,7 +31,6 @@ compile_error!("tokio-adapter requires the std feature"); pub mod buffer; -pub mod connector; pub mod error; #[cfg(feature = "tokio-runtime")] pub mod join_queue; diff --git a/aimdb-tokio-adapter/src/runtime.rs b/aimdb-tokio-adapter/src/runtime.rs index f386186..95e23b5 100644 --- a/aimdb-tokio-adapter/src/runtime.rs +++ b/aimdb-tokio-adapter/src/runtime.rs @@ -4,7 +4,7 @@ //! enabling async task spawning and execution in std environments using Tokio. use aimdb_core::{DbError, DbResult}; -use aimdb_executor::{ExecutorResult, Logger, RuntimeAdapter, Spawn, TimeOps}; +use aimdb_executor::{Logger, RuntimeAdapter, TimeOps}; use core::future::Future; use std::time::{Duration, Instant}; @@ -141,22 +141,6 @@ impl RuntimeAdapter for TokioAdapter { } } -// Implement Spawn trait for dynamic task spawning -#[cfg(feature = "tokio-runtime")] -impl Spawn for TokioAdapter { - type SpawnToken = tokio::task::JoinHandle<()>; - - fn spawn(&self, future: F) -> ExecutorResult - where - F: Future + Send + 'static, - { - #[cfg(feature = "tracing")] - tracing::debug!("Spawning future on Tokio runtime"); - - Ok(tokio::spawn(future)) - } -} - // New unified Runtime trait implementations #[cfg(feature = "tokio-runtime")] impl TimeOps for TokioAdapter { @@ -218,4 +202,4 @@ impl Logger for TokioAdapter { } } -// Runtime trait is auto-implemented when RuntimeAdapter + TimeOps + Logger + Spawn are implemented +// Runtime trait is auto-implemented when RuntimeAdapter + TimeOps + Logger are implemented diff --git a/aimdb-tokio-adapter/tests/drain_integration_tests.rs b/aimdb-tokio-adapter/tests/drain_integration_tests.rs index 1a45e4c..d4983e8 100644 --- a/aimdb-tokio-adapter/tests/drain_integration_tests.rs +++ b/aimdb-tokio-adapter/tests/drain_integration_tests.rs @@ -64,7 +64,9 @@ async fn setup_test_server(socket_path: &str) -> aimdb_core::AimDb reg.buffer(BufferCfg::SingleLatest).with_remote_access(); }); - builder.build().await.unwrap() + let (db, runner) = builder.build().await.unwrap(); + tokio::spawn(runner.run()); + db } /// Helper: set up server with a small ring to test overflow @@ -87,7 +89,9 @@ async fn setup_small_ring_server(socket_path: &str) -> aimdb_core::AimDb aimdb_core::AimDb("sensors.indoor", Temperature { celsius: 22.0 }) @@ -76,7 +77,8 @@ async fn test_producer() { reg.buffer(BufferCfg::SingleLatest); }); - let db = builder.build().await.unwrap(); + let (db, runner) = builder.build().await.unwrap(); + tokio::spawn(runner.run()); // Get key-bound producers let producer_a = db.producer::("sensor.a"); @@ -112,7 +114,8 @@ async fn test_consumer() { reg.buffer(BufferCfg::SingleLatest); }); - let db = builder.build().await.unwrap(); + let (db, runner) = builder.build().await.unwrap(); + tokio::spawn(runner.run()); // Get key-bound consumers let consumer_north = db.consumer::("zone.north"); @@ -139,7 +142,8 @@ async fn test_single_instance_key_lookup() { reg.buffer(BufferCfg::SingleLatest); }); - let db = builder.build().await.unwrap(); + let (db, runner) = builder.build().await.unwrap(); + tokio::spawn(runner.run()); // Key-based produce should work db.produce::("single.temp", Temperature { celsius: 30.0 }) @@ -161,7 +165,8 @@ async fn test_key_not_found_error() { reg.buffer(BufferCfg::SingleLatest); }); - let db = builder.build().await.unwrap(); + let (db, runner) = builder.build().await.unwrap(); + tokio::spawn(runner.run()); // Try to produce to non-existent key let result = db @@ -188,7 +193,8 @@ async fn test_type_mismatch_error() { reg.buffer(BufferCfg::SingleLatest); }); - let db = builder.build().await.unwrap(); + let (db, runner) = builder.build().await.unwrap(); + tokio::spawn(runner.run()); // Try to produce AppConfig to a Temperature record let result = db @@ -251,7 +257,8 @@ async fn test_record_id_stability() { reg.buffer(BufferCfg::SingleLatest); }); - let db = builder.build().await.unwrap(); + let (db, runner) = builder.build().await.unwrap(); + tokio::spawn(runner.run()); // RecordIds should be assigned in registration order let id_first = db.resolve_key("first").unwrap(); @@ -287,7 +294,8 @@ async fn test_records_of_type_introspection() { reg.buffer(BufferCfg::SingleLatest); }); - let db = builder.build().await.unwrap(); + let (db, runner) = builder.build().await.unwrap(); + tokio::spawn(runner.run()); // Should find 3 Temperature records let temp_ids = db.records_of_type::(); diff --git a/aimdb-tokio-adapter/tests/stage_profiling.rs b/aimdb-tokio-adapter/tests/stage_profiling.rs index e371781..8628c59 100644 --- a/aimdb-tokio-adapter/tests/stage_profiling.rs +++ b/aimdb-tokio-adapter/tests/stage_profiling.rs @@ -49,7 +49,8 @@ async fn source_and_tap_stages_are_timed_and_named() { .with_name("data_processor"); }); - let db = builder.build().await.expect("build"); + let (db, runner) = builder.build().await.expect("build"); + tokio::spawn(runner.run()); // Let the pipeline run for a bunch of iterations. tokio::time::sleep(Duration::from_millis(250)).await; @@ -108,7 +109,8 @@ async fn with_name_is_a_no_op_friendly_builder() { }) .with_name("idle_tap"); }); - let db = builder.build().await.expect("build"); + let (db, runner) = builder.build().await.expect("build"); + tokio::spawn(runner.run()); let rec = db .inner() .get_typed_record_by_key::("profiling::Unused") diff --git a/aimdb-tokio-adapter/tests/transform_join_integration_tests.rs b/aimdb-tokio-adapter/tests/transform_join_integration_tests.rs index d19336c..6fb6baf 100644 --- a/aimdb-tokio-adapter/tests/transform_join_integration_tests.rs +++ b/aimdb-tokio-adapter/tests/transform_join_integration_tests.rs @@ -64,7 +64,8 @@ async fn transform_join_produces_sum_on_both_inputs() { }); }); - let db = builder.build().await.unwrap(); + let (db, runner) = builder.build().await.unwrap(); + tokio::spawn(runner.run()); let mut sum_rx = db.subscribe::("test::Sum").unwrap(); // Yield to let the join transform task spawn its input forwarders and subscribe. @@ -131,7 +132,8 @@ async fn transform_join_bounded_fanin_backpressure_no_deadlock() { }); }); - let db = builder.build().await.unwrap(); + let (db, runner) = builder.build().await.unwrap(); + tokio::spawn(runner.run()); let mut echo_rx = db.subscribe::("stress::Echo").unwrap(); // Warm-up: keep producing a sentinel until its echo lands. SpmcRing buffers diff --git a/aimdb-wasm-adapter/src/bindings.rs b/aimdb-wasm-adapter/src/bindings.rs index 1e574ac..768629e 100644 --- a/aimdb-wasm-adapter/src/bindings.rs +++ b/aimdb-wasm-adapter/src/bindings.rs @@ -194,11 +194,18 @@ impl WasmDb { apply_record_config(&self.registry, &mut builder, config)?; } - let db = builder + let (db, runner) = builder .build() .await .map_err(|e| JsError::new(&format!("Build failed: {e:?}")))?; + // Drive the runner on the JS microtask queue. The runner future is + // !Send (it owns wasm-only state), but `wasm_bindgen_futures::spawn_local` + // doesn't require Send β€” WASM is single-threaded. + wasm_bindgen_futures::spawn_local(async move { + runner.run().await; + }); + self.db = Some(db); Ok(()) } diff --git a/aimdb-wasm-adapter/src/lib.rs b/aimdb-wasm-adapter/src/lib.rs index 870dc7c..7a7a632 100644 --- a/aimdb-wasm-adapter/src/lib.rs +++ b/aimdb-wasm-adapter/src/lib.rs @@ -54,7 +54,7 @@ pub use runtime::WasmAdapter; // Re-export executor traits for convenience pub use aimdb_executor::{ - ExecutorError, ExecutorResult, Logger as LoggerTrait, Runtime, RuntimeAdapter, Spawn, TimeOps, + ExecutorError, ExecutorResult, Logger as LoggerTrait, Runtime, RuntimeAdapter, TimeOps, }; // Re-export buffer types diff --git a/aimdb-wasm-adapter/src/runtime.rs b/aimdb-wasm-adapter/src/runtime.rs index df1d2cc..60cf555 100644 --- a/aimdb-wasm-adapter/src/runtime.rs +++ b/aimdb-wasm-adapter/src/runtime.rs @@ -1,62 +1,21 @@ -//! WasmAdapter struct and RuntimeAdapter + Spawn implementations +//! WasmAdapter struct and RuntimeAdapter implementation. //! -//! Single-threaded WASM runtime β€” tasks are spawned onto the browser's -//! microtask queue via `wasm_bindgen_futures::spawn_local`. +//! Single-threaded WASM runtime. After issue #88 there is no `Spawn` impl β€” +//! all database futures are driven by `AimDbRunner::run()`, which is awaited +//! at the WASM entry point. -use aimdb_executor::{ExecutorResult, RuntimeAdapter, Spawn}; -use core::future::Future; +use aimdb_executor::RuntimeAdapter; /// WASM runtime adapter for AimDB. /// -/// Implements the four executor traits required by `aimdb-core`: -/// [`RuntimeAdapter`], [`Spawn`], [`TimeOps`](crate::time), and -/// [`Logger`](crate::logger). -/// -/// # Safety -/// -/// `WasmAdapter` implements `Send + Sync` via `unsafe impl` because -/// `wasm32-unknown-unknown` is single-threaded β€” no concurrent access -/// is possible. This is the identical pattern used by `EmbassyAdapter`. +/// Implements the three executor traits required by `aimdb-core`: +/// [`RuntimeAdapter`], [`TimeOps`](crate::time), and [`Logger`](crate::logger). +/// As a zero-sized type with no fields, it auto-derives `Send + Sync`. #[derive(Clone, Copy, Debug)] pub struct WasmAdapter; -// SAFETY: wasm32-unknown-unknown is single-threaded. -// No concurrent access is possible β€” Send + Sync are trivially satisfied. -// This is the same pattern as aimdb-embassy-adapter/src/runtime.rs. -unsafe impl Send for WasmAdapter {} -unsafe impl Sync for WasmAdapter {} - impl RuntimeAdapter for WasmAdapter { fn runtime_name() -> &'static str { "wasm" } } - -impl Spawn for WasmAdapter { - type SpawnToken = (); // Same as Embassy β€” no join handle - - fn spawn(&self, future: F) -> ExecutorResult - where - F: Future + Send + 'static, - { - // spawn_local requires F: 'static but not F: Send. - // The Send bound on the trait is satisfied vacuously β€” - // all types are effectively Send in a single-threaded context. - #[cfg(feature = "wasm-runtime")] - { - wasm_bindgen_futures::spawn_local(future); - } - - #[cfg(not(feature = "wasm-runtime"))] - { - let _ = future; - // Without wasm-runtime, we can't spawn β€” this path is only - // hit during native-target unit tests. - return Err(aimdb_executor::ExecutorError::RuntimeUnavailable { - message: "wasm-runtime feature not enabled", - }); - } - - Ok(()) - } -} diff --git a/aimdb-websocket-connector/src/builder.rs b/aimdb-websocket-connector/src/builder.rs index 426615b..faab78d 100644 --- a/aimdb-websocket-connector/src/builder.rs +++ b/aimdb-websocket-connector/src/builder.rs @@ -31,7 +31,7 @@ use crate::{ client_manager::ClientManager, connector::WebSocketConnectorImpl, registry::StreamableRegistry, - server::start_server, + server::build_server_future, session::{NoQuery, NoSnapshot, QueryHandler, SessionContext, SnapshotProvider}, }; use aimdb_ws_protocol::TopicInfo; @@ -271,9 +271,11 @@ impl WebSocketConnectorBuilder { // ConnectorBuilder impl // ════════════════════════════════════════════════════════════════════ +type BoxFuture = Pin + Send + 'static>>; + impl ConnectorBuilder for WebSocketConnectorBuilder where - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { fn scheme(&self) -> &str { "ws" @@ -282,14 +284,8 @@ where fn build<'a>( &'a self, db: &'a aimdb_core::builder::AimDb, - ) -> Pin< - Box< - dyn core::future::Future< - Output = aimdb_core::DbResult>, - > + Send - + 'a, - >, - > { + ) -> Pin>> + Send + 'a>> + { Box::pin(async move { // ── Inbound routes ────────────────────────────────────── let inbound_routes = db.collect_inbound_routes("ws"); @@ -361,20 +357,24 @@ where runtime_ctx: Some(db.runtime_any()), }; - // ── Build connector & spawn outbound publishers ─────────────── + // ── Build connector & collect outbound publishers ─────────────── let connector = WebSocketConnectorImpl::new(client_mgr, self.raw_payload); - connector.spawn_outbound_publishers(db, outbound_routes, snapshot_map)?; + let outbound_futures = + connector.collect_outbound_futures(db, outbound_routes, snapshot_map); - // ── Start Axum server ───────────────────────────────── + // ── Build Axum server future ────────────────────────── let additional = self.additional_routes.clone(); - start_server( + let server_future = build_server_future( self.bind_addr, self.ws_path.clone(), session_ctx, additional, ); - Ok(Arc::new(connector) as Arc) + let mut futures: Vec = Vec::with_capacity(1 + outbound_futures.len()); + futures.push(server_future); + futures.extend(outbound_futures); + Ok(futures) }) } } diff --git a/aimdb-websocket-connector/src/client/builder.rs b/aimdb-websocket-connector/src/client/builder.rs index c89804c..be5c20c 100644 --- a/aimdb-websocket-connector/src/client/builder.rs +++ b/aimdb-websocket-connector/src/client/builder.rs @@ -130,9 +130,11 @@ impl WsClientConnectorBuilder { // ConnectorBuilder impl // ════════════════════════════════════════════════════════════════════ +type BoxFuture = Pin + Send + 'static>>; + impl ConnectorBuilder for WsClientConnectorBuilder where - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { fn scheme(&self) -> &str { "ws-client" @@ -141,14 +143,8 @@ where fn build<'a>( &'a self, db: &'a aimdb_core::builder::AimDb, - ) -> Pin< - Box< - dyn core::future::Future< - Output = aimdb_core::DbResult>, - > + Send - + 'a, - >, - > { + ) -> Pin>> + Send + 'a>> + { Box::pin(async move { // ── Inbound routes ────────────────────────────────────── let inbound_routes = db.collect_inbound_routes("ws-client"); @@ -194,21 +190,22 @@ where subscribe_topics: topics, }; - // ── Build the connector ───────────────────────────────── + // ── Build the connector (internal tasks are spawned via tokio::spawn β€” Group 4) ── let connector = WsClientConnectorImpl::connect(config, router, db) .await .map_err(|e| aimdb_core::DbError::RuntimeError { message: format!("WS client connect failed: {}", e), })?; - // ── Spawn outbound publishers ──────────────────────────── - connector - .spawn_outbound_publishers(db, outbound_routes) - .map_err(|e| aimdb_core::DbError::RuntimeError { - message: format!("WS client outbound setup failed: {}", e), - })?; + // ── Collect outbound publisher futures ─────────────────── + let futures = connector.collect_outbound_futures(db, outbound_routes); - Ok(Arc::new(connector) as Arc) + // The `WsClientConnectorImpl` itself is no longer returned (the + // `Arc` was discarded by `AimDbBuilder` anyway). + // Its background tasks live on inside the tokio spawns started in + // `connect()`; the only futures the runner drives from this connector + // are the per-route publishers. + Ok(futures) }) } } diff --git a/aimdb-websocket-connector/src/client/connector.rs b/aimdb-websocket-connector/src/client/connector.rs index a6b90cc..b8446c9 100644 --- a/aimdb-websocket-connector/src/client/connector.rs +++ b/aimdb-websocket-connector/src/client/connector.rs @@ -91,7 +91,7 @@ impl WsClientConnectorImpl { db: &aimdb_core::builder::AimDb, ) -> Result where - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { // Connect to the remote server let (ws_stream, _response) = tokio_tungstenite::connect_async(&config.url) @@ -123,7 +123,6 @@ impl WsClientConnectorImpl { } } - // ── Spawn write loop ──────────────────────────────────────── let reconnect_url = config.url.clone(); let reconnect_topics = config.subscribe_topics.clone(); let auto_reconnect = config.auto_reconnect; @@ -131,186 +130,174 @@ impl WsClientConnectorImpl { let router_for_reconnect = router.clone(); let runtime_ctx: Arc = db.runtime_any(); - db.runtime() - .spawn({ - let state = state.clone(); - async move { - Self::run_write_loop(ws_write, write_rx).await; + // Write loop. Bridge state: tokio::spawn directly (see method doc). + tokio::spawn({ + let state = state.clone(); + async move { + Self::run_write_loop(ws_write, write_rx).await; - // Write loop ended β€” connection closed - #[cfg(feature = "tracing")] - tracing::warn!("WS client: write loop ended"); + #[cfg(feature = "tracing")] + tracing::warn!("WS client: write loop ended"); - state.lock().await.status = ConnectionStatus::Disconnected; - } - }) - .map_err(|e| format!("Failed to spawn write loop: {e:?}"))?; + state.lock().await.status = ConnectionStatus::Disconnected; + } + }); - // ── Spawn read loop ───────────────────────────────────────── - db.runtime() - .spawn({ - let router = router.clone(); - let runtime_ctx = runtime_ctx.clone(); - async move { - Self::run_read_loop(ws_read, &router, Some(&runtime_ctx)).await; + // Read loop. + tokio::spawn({ + let router = router.clone(); + let runtime_ctx = runtime_ctx.clone(); + async move { + Self::run_read_loop(ws_read, &router, Some(&runtime_ctx)).await; - #[cfg(feature = "tracing")] - tracing::warn!("WS client: read loop ended"); - } - }) - .map_err(|e| format!("Failed to spawn read loop: {e:?}"))?; + #[cfg(feature = "tracing")] + tracing::warn!("WS client: read loop ended"); + } + }); - // ── Spawn keepalive ───────────────────────────────────────── + // Keepalive. if let Some(interval) = config.keepalive_interval { let ka_state = state.clone(); - db.runtime() - .spawn(async move { - Self::run_keepalive(ka_state, interval).await; - }) - .map_err(|e| format!("Failed to spawn keepalive: {e:?}"))?; + tokio::spawn(async move { + Self::run_keepalive(ka_state, interval).await; + }); } - // ── Spawn reconnect watcher ───────────────────────────────── + // Reconnect watcher (deferred to AimX follow-up β€” Group 4). if auto_reconnect { - db.runtime() - .spawn({ - let state = state.clone(); - let runtime_ctx = runtime_ctx.clone(); - async move { - Self::run_reconnect_watcher( - state, - reconnect_url, - reconnect_topics, - router_for_reconnect, - max_reconnect_attempts, - Some(runtime_ctx), - ) - .await; - } - }) - .map_err(|e| format!("Failed to spawn reconnect watcher: {e:?}"))?; + tokio::spawn({ + let state = state.clone(); + let runtime_ctx = runtime_ctx.clone(); + async move { + Self::run_reconnect_watcher( + state, + reconnect_url, + reconnect_topics, + router_for_reconnect, + max_reconnect_attempts, + Some(runtime_ctx), + ) + .await; + } + }); } Ok(Self { state, router }) } - /// Spawn one Tokio task per outbound route. + /// Collect one outbound publisher future per route. /// - /// Each task subscribes to a local record, serializes values, and sends - /// `ClientMessage::Write` to the remote server. - pub(crate) fn spawn_outbound_publishers( + /// Each future subscribes to a local record, serializes values, and sends + /// `ClientMessage::Write` to the remote server. Returned futures are appended + /// to the `AimDbRunner` accumulator. + pub(crate) fn collect_outbound_futures( &self, db: &aimdb_core::builder::AimDb, outbound_routes: Vec, - ) -> Result<(), String> + ) -> Vec + Send + 'static>>> where - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { - let runtime = db.runtime(); let runtime_ctx: Arc = db.runtime_any(); + let mut futures: Vec + Send + 'static>>> = + Vec::with_capacity(outbound_routes.len()); for (default_topic, consumer, serializer, _config, topic_provider) in outbound_routes { let state = self.state.clone(); let default_topic_clone = default_topic.clone(); let runtime_ctx = runtime_ctx.clone(); - runtime - .spawn(async move { - let mut reader = match consumer.subscribe_any().await { - Ok(r) => r, - Err(_e) => { - #[cfg(feature = "tracing")] - tracing::error!( - "WS client outbound: subscribe failed for '{}': {:?}", - default_topic_clone, - _e - ); - return; - } - }; + futures.push(Box::pin(async move { + let mut reader = match consumer.subscribe_any().await { + Ok(r) => r, + Err(_e) => { + #[cfg(feature = "tracing")] + tracing::error!( + "WS client outbound: subscribe failed for '{}': {:?}", + default_topic_clone, + _e + ); + return; + } + }; - #[cfg(feature = "tracing")] - tracing::info!( - "WS client outbound publisher started for topic: {}", - default_topic_clone - ); + #[cfg(feature = "tracing")] + tracing::info!( + "WS client outbound publisher started for topic: {}", + default_topic_clone + ); - while let Ok(value_any) = reader.recv_any().await { - // Resolve topic (dynamic or static) - let topic = topic_provider - .as_ref() - .and_then(|p| p.topic_any(&*value_any)) - .unwrap_or_else(|| default_topic_clone.clone()); - - // Serialize - let bytes = match &serializer { - aimdb_core::connector::SerializerKind::Raw(ser) => { - match ser(&*value_any) { - Ok(b) => b, - Err(_e) => { - #[cfg(feature = "tracing")] - tracing::error!( - "WS client outbound: serialize error for '{}': {:?}", - topic, - _e - ); - continue; - } - } + while let Ok(value_any) = reader.recv_any().await { + // Resolve topic (dynamic or static) + let topic = topic_provider + .as_ref() + .and_then(|p| p.topic_any(&*value_any)) + .unwrap_or_else(|| default_topic_clone.clone()); + + // Serialize + let bytes = match &serializer { + aimdb_core::connector::SerializerKind::Raw(ser) => match ser(&*value_any) { + Ok(b) => b, + Err(_e) => { + #[cfg(feature = "tracing")] + tracing::error!( + "WS client outbound: serialize error for '{}': {:?}", + topic, + _e + ); + continue; } - aimdb_core::connector::SerializerKind::Context(ser) => { - match ser(runtime_ctx.clone(), &*value_any) { - Ok(b) => b, - Err(_e) => { - #[cfg(feature = "tracing")] - tracing::error!( - "WS client outbound: serialize error for '{}': {:?}", - topic, - _e - ); - continue; - } + }, + aimdb_core::connector::SerializerKind::Context(ser) => { + match ser(runtime_ctx.clone(), &*value_any) { + Ok(b) => b, + Err(_e) => { + #[cfg(feature = "tracing")] + tracing::error!( + "WS client outbound: serialize error for '{}': {:?}", + topic, + _e + ); + continue; } } - }; + } + }; - // Build Write message - let payload: serde_json::Value = match serde_json::from_slice(&bytes) { - Ok(v) => v, - Err(_e) => { - // Fallback: wrap raw bytes as a JSON string - serde_json::Value::String( - String::from_utf8_lossy(&bytes).into_owned(), - ) - } - }; + // Build Write message + let payload: serde_json::Value = match serde_json::from_slice(&bytes) { + Ok(v) => v, + Err(_e) => { + // Fallback: wrap raw bytes as a JSON string + serde_json::Value::String(String::from_utf8_lossy(&bytes).into_owned()) + } + }; - let msg = ClientMessage::Write { - topic: topic.clone(), - payload, - }; + let msg = ClientMessage::Write { + topic: topic.clone(), + payload, + }; - if let Ok(json) = serde_json::to_string(&msg) { - let mut s = state.lock().await; - if s.status == ConnectionStatus::Connected { - let _ = s.write_tx.send(json); - } else if s.pending_writes.len() < s.max_offline_queue { - s.pending_writes.push_back(json); - } - // else: drop (overflow policy) + if let Ok(json) = serde_json::to_string(&msg) { + let mut s = state.lock().await; + if s.status == ConnectionStatus::Connected { + let _ = s.write_tx.send(json); + } else if s.pending_writes.len() < s.max_offline_queue { + s.pending_writes.push_back(json); } + // else: drop (overflow policy) } + } - #[cfg(feature = "tracing")] - tracing::info!( - "WS client outbound publisher stopped for topic: {}", - default_topic_clone - ); - }) - .map_err(|e| format!("Failed to spawn outbound publisher: {e:?}"))?; + #[cfg(feature = "tracing")] + tracing::info!( + "WS client outbound publisher stopped for topic: {}", + default_topic_clone + ); + })); } - Ok(()) + futures } // ════════════════════════════════════════════════════════════════ @@ -416,6 +403,9 @@ impl WsClientConnectorImpl { // Query results are handled by the WASM bridge; the native // client connector does not issue queries (yet). } + ServerMessage::TopicList { .. } => { + // Topic list responses are not used by the native client connector. + } } } } diff --git a/aimdb-websocket-connector/src/connector.rs b/aimdb-websocket-connector/src/connector.rs index 1a797be..9189095 100644 --- a/aimdb-websocket-connector/src/connector.rs +++ b/aimdb-websocket-connector/src/connector.rs @@ -20,13 +20,12 @@ use std::{collections::HashMap, pin::Pin, sync::Arc}; -use aimdb_core::{ - transport::{ConnectorConfig, PublishError}, - OutboundRoute, -}; +use aimdb_core::OutboundRoute; use crate::client_manager::ClientManager; +type BoxFuture = Pin + Send + 'static>>; + /// Live WebSocket connector returned by `build()`. pub struct WebSocketConnectorImpl { pub(crate) client_mgr: ClientManager, @@ -43,25 +42,25 @@ impl WebSocketConnectorImpl { } } - /// Spawn one Tokio task per outbound route. + /// Collects one outbound publisher future per route. /// - /// Each task: + /// Each future: /// 1. Calls `consumer.subscribe_any()` to get a type-erased reader. /// 2. Loops calling `reader.recv_any()`. /// 3. Runs the serializer. /// 4. Broadcasts the bytes via `ClientManager::broadcast()`. - pub(crate) fn spawn_outbound_publishers( + pub(crate) fn collect_outbound_futures( &self, db: &aimdb_core::builder::AimDb, outbound_routes: Vec, snapshot_map: Arc>>>, - ) -> aimdb_core::DbResult<()> + ) -> Vec where - R: aimdb_executor::Spawn + 'static, + R: aimdb_executor::RuntimeAdapter + 'static, { - let runtime = db.runtime(); let raw_payload = self.raw_payload; let runtime_ctx: Arc = db.runtime_any(); + let mut futures: Vec = Vec::with_capacity(outbound_routes.len()); for (default_topic, consumer, serializer, _config, topic_provider) in outbound_routes { let client_mgr = self.client_mgr.clone(); @@ -69,7 +68,7 @@ impl WebSocketConnectorImpl { let default_topic_clone = default_topic.clone(); let runtime_ctx = runtime_ctx.clone(); - runtime.spawn(async move { + futures.push(Box::pin(async move { let mut reader = match consumer.subscribe_any().await { Ok(r) => r, Err(_e) => { @@ -145,27 +144,9 @@ impl WebSocketConnectorImpl { "WS outbound publisher stopped for topic: {}", default_topic_clone ); - })?; + })); } - Ok(()) - } -} - -// ════════════════════════════════════════════════════════════════════ -// Connector trait -// ════════════════════════════════════════════════════════════════════ - -impl aimdb_core::transport::Connector for WebSocketConnectorImpl { - /// WebSocket inbound is driven by the session receive loop, not by - /// `publish()`. This implementation exists only to satisfy the trait and - /// will never be called in normal operation. - fn publish( - &self, - _destination: &str, - _config: &ConnectorConfig, - _payload: &[u8], - ) -> Pin> + Send + '_>> { - Box::pin(async move { Ok(()) }) + futures } } diff --git a/aimdb-websocket-connector/src/server.rs b/aimdb-websocket-connector/src/server.rs index 390238d..fa3062e 100644 --- a/aimdb-websocket-connector/src/server.rs +++ b/aimdb-websocket-connector/src/server.rs @@ -44,8 +44,14 @@ pub(crate) struct ServerState { // Server start // ════════════════════════════════════════════════════════════════════ -/// Start the WebSocket Axum server and return immediately (the server runs in -/// a background Tokio task). +type BoxFuture = std::pin::Pin + Send + 'static>>; + +/// Builds the WebSocket Axum server future. +/// +/// Returns a `BoxFuture` containing the `axum::serve()` accept loop. The +/// future is appended to the `AimDbRunner` accumulator (design 028 Β§"Connector +/// futures"). Per-connection handlers spawned by Axum internally continue to +/// use `tokio::spawn` β€” outside the scope of issue #88. /// /// # Arguments /// @@ -54,12 +60,12 @@ pub(crate) struct ServerState { /// * `session_ctx` β€” Shared session context (auth, router, client manager, …). /// * `additional_routes` β€” Optional user-supplied Axum `Router` that is merged /// into the server (useful for REST + WebSocket on the same port). -pub(crate) fn start_server( +pub(crate) fn build_server_future( bind_addr: SocketAddr, ws_path: String, session_ctx: SessionContext, additional_routes: Option, -) { +) -> BoxFuture { let state = ServerState { session_ctx, started_at: Instant::now(), @@ -80,7 +86,7 @@ pub(crate) fn start_server( ws_app }; - tokio::spawn(async move { + Box::pin(async move { let listener = match tokio::net::TcpListener::bind(bind_addr).await { Ok(l) => l, Err(_e) => { @@ -102,7 +108,7 @@ pub(crate) fn start_server( #[cfg(feature = "tracing")] tracing::error!("WebSocket server error: {}", _e); } - }); + }) } // ════════════════════════════════════════════════════════════════════ diff --git a/docs/design/028-M13-remove-spawn-trait.md b/docs/design/028-M13-remove-spawn-trait.md index 7f53254..7666a84 100644 --- a/docs/design/028-M13-remove-spawn-trait.md +++ b/docs/design/028-M13-remove-spawn-trait.md @@ -1,8 +1,9 @@ # Remove `Spawn` Trait β€” `build()` Collects, `run()` Drives -**Version:** 0.1 (draft) +**Version:** 0.2 (draft β€” revised after code audit) **Status:** πŸ“ Design **Issue:** [#88](https://github.com/aimdb-dev/aimdb/issues/88) +**Follow-up issue:** [AimX remote-access portability](../issues/aimx-remote-spawn-free.md) β€” TBD **Last Updated:** May 23, 2026 **Milestone:** M13 β€” Architectural clean-up @@ -16,12 +17,13 @@ - [Where `Spawn` propagates today](#where-spawn-propagates-today) - [Embassy workaround](#embassy-workaround) - [WASM workaround](#wasm-workaround) - - [The key audit finding](#the-key-audit-finding) + - [Runtime spawn sites β€” full inventory](#runtime-spawn-sites--full-inventory) - [Proposed Design](#proposed-design) - [New public API](#new-public-api) - [Internal mechanics](#internal-mechanics) - [Connector futures](#connector-futures) - [Remote supervisor](#remote-supervisor) + - [`AimDb::spawn_task` deletion](#aimdbspawn_task-deletion) - [Type-System Changes](#type-system-changes) - [Bound removals β€” summary table](#bound-removals--summary-table) - [unsafe impl Send/Sync analysis](#unsafe-impl-sendsync-analysis) @@ -32,10 +34,12 @@ - [WASM (wasm32-unknown-unknown)](#wasm-wasm32-unknown-unknown) - [Shutdown / Cancellation](#shutdown--cancellation) - [Connector API impact](#connector-api-impact) + - [Other affected crates](#other-affected-crates) - [Breaking Changes](#breaking-changes) - [Implementation Plan](#implementation-plan) - [Alternatives Considered](#alternatives-considered) -- [Open Questions](#open-questions) +- [Decisions](#decisions) +- [Out of Scope](#out-of-scope) --- @@ -47,9 +51,24 @@ inside `build()` are instead **collected** into a `Vec`. A new `AimDb::run()` method drives them via `FuturesUnordered`, blocking until shutdown. -The core invariant that makes this safe: **all `spawn()` calls in AimDB happen -inside `build()`.** No task is ever spawned after initialisation completes -(with one narrow exception β€” the remote-access supervisor, discussed below). +The intended invariant: **all `spawn()` calls reachable from `aimdb-core` happen +inside `build()` after this refactor.** A code audit (v0.2 of this doc) +identified five categories of runtime-spawn sites; the table in +[Runtime spawn sites β€” full inventory](#runtime-spawn-sites--full-inventory) +catalogues each and assigns it to one of: *converted to build-time +collection*, *deleted as part of this PR*, or *deferred to a follow-up +issue*. + +> **Scope note (v0.2):** The v0.1 framing of "one narrow exception" was an +> oversimplification. The AimX remote-access spawn calls (per-connection +> handler + per-subscription stream) are non-trivial to remove because they +> are inherently dynamic-fan-out. Rather than couple their refactor to the +> `Spawn`-trait removal, this design **defers** them to a separate issue +> ([AimX remote-access portability](../issues/aimx-remote-spawn-free.md)) +> so M13 can land as a focused trait-removal change. Within this PR, the AimX +> path keeps using bare `tokio::spawn` internally β€” which is fine because +> AimX is already `#[cfg(feature = "std")]`-gated. The follow-up issue both +> removes those spawn calls and prepares AimX for eventual un-gating. --- @@ -119,6 +138,12 @@ because they hold `Arc>` which transitively requires `R: Send + Sync`. | `aimdb-core/src/database.rs` | `Database` | high-level wrapper | | `aimdb-core/src/remote/supervisor.rs` | `R: Spawn` | supervisor spawning | | `aimdb-core/src/remote/handler.rs` | `R: Spawn` (Γ—14 bounds) | handler dispatch | +| `aimdb-codegen/src/rust.rs` | emits `` into generated `configure_schema` | code generation | +| `aimdb-persistence/src/builder_ext.rs` | `R: Spawn + TimeOps` | persistence builder extension trait | +| `aimdb-persistence/src/ext.rs` | `R: Spawn + 'static` (Γ—2) | persistence trait bounds | +| `aimdb-persistence/src/query_ext.rs` | `R: Spawn + 'static` (Γ—2) | query trait + backend helper | +| `aimdb-sync/src/handle.rs` | `R: Spawn` | sync handle bounds | +| `aimdb-tokio-adapter/src/connector.rs` | `TokioAdapter::spawn_connectors` (test-only helper) | unused helper, deletable | ### Embassy workaround @@ -155,42 +180,64 @@ unsafe impl Sync for WasmAdapter {} The `unsafe` exists solely to satisfy the `Spawn` trait's `F: Send` bound, which is vacuously satisfied on single-threaded WASM. -### The key audit finding +### Runtime spawn sites β€” full inventory -All `spawn()` call sites in AimDB are inside `build()`: +The v0.1 audit listed six build-time spawn sites and claimed they were the +total. A line-by-line code search produced the following complete picture. +Sites are grouped by disposition (what this PR does with them). + +**Group 1 β€” Build-time spawns converted to returned futures (in scope).** | Call site | File | One future per… | |---|---|---| -| `spawn_producer_service` | `typed_record.rs` | `.source()` | -| `spawn_consumer_tasks` | `typed_record.rs` | `.tap()` | -| `spawn_transform_task` | `typed_record.rs` | `.transform()` / `.transform_join()` | -| on_start tasks | `builder.rs` | `.on_start()` | -| Connector tasks | `builder.rs` (via `ConnectorBuilder::build`) | per-connector | -| Remote supervisor | `builder.rs` β†’ `supervisor.rs` | one per `with_remote_access()` | - -The total future count is **fully determined** by the database configuration at -the point `build()` is called. This is exactly the use case `FuturesUnordered` -is designed for. - -> **Exception β€” per-connection handlers:** The remote-access supervisor spawns -> **per-connection** handlers at runtime using bare `tokio::spawn`. These are -> already outside the `Spawn` trait abstraction and are unaffected by this -> change. See [Remote supervisor](#remote-supervisor) below. -> -> **Exception β€” connector infrastructure tasks:** Each Tokio connector also -> spawns one permanent infrastructure task via bare `tokio::spawn` β€” entirely -> separate from the outbound-publisher tasks that go through `runtime.spawn()`: -> -> | Connector | Infrastructure task | Current call | -> |---|---|---| -> | MQTT (Tokio) | `rumqttc` EventLoop poll loop | `spawn_event_loop()` β†’ `tokio::spawn` | -> | KNX (Tokio) | UDP connection + reconnect loop | `spawn_connection_task()` β†’ `tokio::spawn` | -> | WebSocket | Axum server (`axum::serve`) | `start_server()` β†’ `tokio::spawn` | -> -> Unlike the per-connection handlers, these infrastructure tasks **must be -> converted** to returned futures as part of Step 6. If left unchanged, -> connectors continue to call `tokio::spawn` directly and their infrastructure -> futures are invisible to `AimDbRunner`. +| `spawn_producer_service` | `aimdb-core/src/typed_record.rs:1228` | `.source()` | +| `spawn_consumer_tasks` | `aimdb-core/src/typed_record.rs:1155` | `.tap()` | +| `spawn_transform_task` | `aimdb-core/src/typed_record.rs:881` | `.transform()` / `.transform_join()` | +| on_start tasks | `aimdb-core/src/builder.rs:977,988` | `.on_start()` | +| Connector outbound publishers | per-connector `spawn_outbound_publishers()` | `.link_to()` route | +| Connector infrastructure | MQTT `spawn_event_loop()`, KNX `spawn_connection_task()`, WS `start_server()` | one per connector | +| Remote supervisor entry point | `aimdb-core/src/builder.rs` β†’ `supervisor.rs:108` | one per `with_remote_access()` | + +**Group 2 β€” Runtime spawn that hoists to build time (in scope, new in v0.2).** + +| Call site | File | Why it's actually a build-time fan-out | +|---|---|---| +| Join transform forwarders | `aimdb-core/src/transform/join.rs:329` | `inputs.len()` is fixed at `transform_join()` registration; lazy spawn is incidental | + +The forwarder count is statically known when the `JoinPipeline` is built. +Step 3a converts the lazy spawn into build-time collection alongside the +join transform future itself. + +**Group 3 β€” Runtime spawn deleted as part of this PR (in scope).** + +| Item | File | Disposition | +|---|---|---| +| `AimDb::spawn_task` public method | `aimdb-core/src/builder.rs:1096` | Delete. With `Spawn` gone there is no portable backing primitive; the method has no internal callers. | +| `TokioAdapter::spawn_connectors` | `aimdb-tokio-adapter/src/connector.rs:54` | Delete. Test-only helper, no production callers. | +| `BufferOps::spawn_dispatcher` | `aimdb-tokio-adapter/src/buffer.rs:205` | Keep (test-only utility), but mark for removal in a follow-up tidy if no external user adopts it. | + +**Group 4 β€” Runtime spawn deferred to follow-up issue (out of scope).** + +| Call site | File | Reason for deferral | +|---|---|---| +| AimX per-connection handler | `aimdb-core/src/remote/supervisor.rs:122` (bare `tokio::spawn`) | Dynamic fan-out by client count; needs nested-`FuturesUnordered` refactor β€” see [Out of Scope](#out-of-scope) | +| AimX per-subscription stream | `aimdb-core/src/remote/handler.rs:1042` (bare `tokio::spawn`) + `builder.rs:1409` (`runtime.spawn` inside `subscribe_record_updates`) | Same as above; coupled to handler refactor | +| WebSocket **client** reconnect | `aimdb-websocket-connector/src/client/connector.rs:505,510` (bare `tokio::spawn`) | Dynamic fan-out per reconnect; same nested-`FuturesUnordered` pattern as AimX. Tracked alongside follow-up. | + +**Group 5 β€” External / out-of-codebase (informational).** + +`wasm_bindgen_futures::spawn_local` calls in `aimdb-wasm-adapter/src/{ws_bridge.rs,bindings.rs}` are WASM-runtime glue invoked by JS callbacks; they are outside the `Spawn` trait surface and unaffected. Example binaries and tests that call `tokio::spawn` directly are user code, not core. + +#### Why these groupings make the refactor safe + +Groups 1, 2, and 3 cover **every** `runtime.spawn(...)` call within +`aimdb-core` and every connector β€” once they land, the `Spawn` trait has no +internal callers. Group 4 retains bare `tokio::spawn` calls inside +`aimdb-core/src/remote/`, but those calls do **not** depend on the trait +β€” they call Tokio directly through `#[cfg(feature = "std")]`. The trait +can therefore be deleted in this PR without breaking the remote-access +path; the follow-up issue removes the remaining `tokio::spawn` calls in a +separate, focused change. --- @@ -365,12 +412,41 @@ pub fn build_supervisor( ) -> DbResult> ``` -The supervisor loop internally calls `tokio::spawn` for per-connection handlers -β€” this is a Tokio-specific behaviour that already bypasses the `Spawn` trait -and is **unchanged** by this refactor. Per-connection spawning happens inside a -future that is driven by `FuturesUnordered`, which is fine because -`FuturesUnordered` is poll-based: the supervisor future yields control between -accepted connections. +**Bridge state inside this PR.** The supervisor body keeps two existing +`tokio::spawn` call sites untouched: the per-connection handler at +`supervisor.rs:122` and the per-subscription event streamer at +`handler.rs:1042`. The internal `AimDb::subscribe_record_updates` +([builder.rs:1409](../../aimdb-core/src/builder.rs#L1409)) β€” which today +uses `runtime.spawn(...)` β€” is rewritten in this PR to call `tokio::spawn` +directly, matching the rest of the AimX path. None of these depend on the +`Spawn` trait after the rewrite, so the trait can be deleted cleanly. + +This is a deliberate bridge: AimX is already `#[cfg(feature = "std")]`-gated, +so the temporary use of bare `tokio::spawn` does not regress portability. +The handler dispatch and supervisor still drop their `R: Spawn` bound (it +becomes `R: RuntimeAdapter`), which is the change that lets the rest of +the type system loosen. + +**Target state (follow-up issue).** The bare `tokio::spawn` calls are +replaced with nested `FuturesUnordered` driven by `select_biased!`, making +the AimX path runtime-agnostic. This is the prerequisite for ever lifting +the `std` gate from AimX. See +[AimX remote-access portability](../issues/aimx-remote-spawn-free.md) for +the full plan; not part of this PR. + +### `AimDb::spawn_task` deletion + +[`AimDb::spawn_task`](../../aimdb-core/src/builder.rs#L1096) is a public +convenience that forwards to `R::spawn`. With the trait removed there is +no portable backing primitive, and the method has no internal callers. +**Deleted in this PR** (Step 4). Downstream code that wants post-build task +creation must either: + +1. Register the future via `on_start()` (preferred β€” collected by `build()`). +2. Place a `FuturesUnordered` inside its own future and push children there. + +There is no third option. This is intentional: the surface area we are +removing is exactly the surface area that made the trait viral. --- @@ -392,6 +468,10 @@ accepted connections. | `Database` | `A: Spawn + 'static` | `A: RuntimeAdapter + 'static` | | `remote/handler.rs` (Γ—14) | `R: Spawn + 'static` | `R: RuntimeAdapter + 'static` | | `remote/supervisor.rs` | `R: Spawn + 'static` | `R: RuntimeAdapter + 'static` | +| `impl Clone for AimDb` | `R: Spawn + 'static` | `R: RuntimeAdapter + 'static` | +| `aimdb-codegen/src/rust.rs` emitted code | `` | `` (golden tests update) | +| `aimdb-persistence` (3 files) | `R: Spawn (+ TimeOps)` | `R: RuntimeAdapter (+ TimeOps)` | +| `aimdb-sync/src/handle.rs` | `R: Spawn` | `R: RuntimeAdapter` | ### `RecordSpawner` β†’ `RecordFutureCollector` @@ -443,6 +523,18 @@ only an `Option<&'static Stack<'static>>` (the network stack, gated by The `new_with_spawner()` constructor is removed (breaking change β€” see [Breaking Changes](#breaking-changes)). +`new_with_network(spawner, network)` ([runtime.rs:162](../../aimdb-embassy-adapter/src/runtime.rs#L162)) +currently also takes a `Spawner`. With `Spawn` removed the parameter is dead +weight. The signature changes to `new_with_network(network)`. This breaks +three example binaries and any aimdb-pro snippets that pass `spawner`: + +| Affected file | Current call | +|---|---| +| `examples/embassy-knx-connector-demo/src/main.rs:253` | `EmbassyAdapter::new_with_network(spawner, stack)` | +| `examples/embassy-mqtt-connector-demo/src/main.rs:317` | same | +| `examples/weather-mesh-demo/weather-station-gamma/src/main.rs:259` | same | +| aimdb-pro UI docs (llms-full.txt, 04-deployment.md) | same | + **`WasmAdapter`:** `WasmAdapter` is already a ZST (`#[derive(Clone, Copy, Debug)]` with no @@ -521,6 +613,18 @@ Heap usage is therefore unchanged. **Pool size feature flags** (`embassy-task-pool-8/16/32`) are **deleted**. +**Cooperative-scheduling implication.** Today each AimDB future runs in its +own Embassy task with its own stack: a producer that blocks between awaits +only starves itself. After this change, **all** collected futures share a +single Embassy task's stack and yield budget. In practice this is +benign β€” AimDB futures are async I/O loops that yield frequently β€” but it +is a real semantic shift. If an application registers a future that does +heavy synchronous work between awaits, that work now blocks every other +AimDB future, not just itself. Document this in the user-facing migration +note. (The eventual lift of `std` gating on AimX, when it lands via the +follow-up issue, will further amplify this β€” at that point the AimX +supervisor lives in the same shared stack.) + ### WASM (wasm32-unknown-unknown) WASM's single-threaded runtime presents no new concerns because @@ -578,6 +682,18 @@ async fn build( Affected connectors: `aimdb-mqtt-connector` (Tokio and Embassy), `aimdb-knx-connector`, `aimdb-websocket-connector`, and aimdb-pro call sites. +### Other affected crates + +Beyond the connector trait, three additional crates carry `R: Spawn` bounds +that must be loosened to `R: RuntimeAdapter` (mechanical, no behaviour change): + +- **`aimdb-codegen`** ([rust.rs:560,734,1253](../../aimdb-codegen/src/rust.rs#L560)) β€” the schema-codegen emits `use aimdb_executor::Spawn;` and `` into generated `configure_schema` functions. Update the emitter and the two golden tests at [rust.rs:1836,1941](../../aimdb-codegen/src/rust.rs#L1836). +- **`aimdb-persistence`** ([builder_ext.rs:23](../../aimdb-persistence/src/builder_ext.rs#L23), [ext.rs:19,32](../../aimdb-persistence/src/ext.rs#L19), [query_ext.rs:59,68](../../aimdb-persistence/src/query_ext.rs#L59)) β€” five `R: Spawn` bounds across three files. +- **`aimdb-sync`** ([handle.rs](../../aimdb-sync/src/handle.rs)) β€” sync-API handle types carry `R: Spawn`. + +None of these crates *call* `runtime.spawn`; they only propagate the bound. +Removing the bound is a find-replace. + --- ## Breaking Changes @@ -586,9 +702,16 @@ Affected connectors: `aimdb-mqtt-connector` (Tokio and Embassy), `aimdb-knx-conn |---|---| | Public API | `db.run().await` must be called after `build()` β€” tasks do not start until `run()` | | `Runtime` supertrait | `Spawn` removed β€” custom adapters no longer need to implement it | -| `EmbassyAdapter` | `new_with_spawner(spawner)` constructor removed | +| `Spawn` trait | Deleted entirely from `aimdb-executor` (`SpawnToken` associated type goes with it) | +| `ExecutorError::SpawnFailed` | Variant removed | +| `AimDb::spawn_task` | Public method **removed** β€” use `on_start()` or nested `FuturesUnordered` | +| `EmbassyAdapter::new_with_spawner` | Constructor removed | +| `EmbassyAdapter::new_with_network(spawner, network)` | Signature changes to `(network)` β€” `spawner` arg removed | | `EmbassyAdapter` feature flags | `embassy-task-pool-8/16/32` removed | | `ConnectorBuilder` trait | `build()` now returns `Vec` instead of `Box` | +| `TokioAdapter::spawn_connectors` | Removed (test-only helper, unused) | +| Generated code (codegen) | `configure_schema` β†’ `` β€” regenerate downstream | +| `aimdb-persistence`, `aimdb-sync` trait bounds | `R: Spawn` β†’ `R: RuntimeAdapter` on public traits | | `spawn_fns` in `AimDbBuilder` | internal β€” no public API change, but internal structure changes | --- @@ -640,6 +763,26 @@ Each method constructs the future and returns it rather than calling --- +### Step 3a β€” Core: hoist join-transform forwarder spawning + +**File:** `aimdb-core/src/transform/join.rs` + +`run_join_transform` ([join.rs:329](../../aimdb-core/src/transform/join.rs#L329)) +currently calls `runtime.spawn(forwarder_future)` lazily inside the +already-running transform task. The forwarder count equals `inputs.len()`, +which is known when the `JoinPipeline` is registered. + +- Change `JoinPipeline::into_descriptor()` to return both the transform + future *and* the forwarder futures: e.g. + `TransformDescriptor { task_future, fanin_futures: Vec> }`. +- Construct the `JoinTrigger` queue and forwarder futures at descriptor + construction time (build phase), not inside `run_join_transform`. +- `collect_transform_future` (Step 3) appends both `task_future` and every + `fanin_future` to the accumulator vec. +- Delete the `runtime.spawn(...)` call inside `run_join_transform`. + +--- + ### Step 4 β€” Core: `build()` accumulates, `run()` drives **File:** `aimdb-core/src/builder.rs` @@ -653,20 +796,24 @@ Each method constructs the future and returns it rather than calling - Implement `AimDbRunner::run(self)` using `FuturesUnordered`. - `AimDb` itself gains no new fields β€” it remains a plain clone-able handle. - Remove `R: Spawn` bound from `AimDb`, `AimDbBuilder`, - `AimDbInner::get_typed_record_by_key/id`. -- Collapse the `std`/`no_std` `on_start` bifurcation: unify - `StartFnType` and `NoStdStartFnType` into a single alias - `type StartFnType = Box) -> BoxFuture<'static, ()>>;` - (drop `+ Send` from the `FnOnce` β€” the closure is called once during - `build()`, not moved across threads; `Send` is already required by the - `BoxFuture` output type). Remove the `#[cfg(feature = "std")]` split. + `AimDbInner::get_typed_record_by_key/id`, **and the manual + `impl Clone for AimDb` at [builder.rs:1037](../../aimdb-core/src/builder.rs#L1037)**. +- **Delete `AimDb::spawn_task`** ([builder.rs:1096](../../aimdb-core/src/builder.rs#L1096)). + No internal callers; downstream callers migrate to `on_start()` or nested + `FuturesUnordered`. +- Collapse the `std`/`no_std` `on_start` bifurcation: the two type aliases + at [builder.rs:32-47](../../aimdb-core/src/builder.rs#L32-L47) are **already + identical** β€” the bifurcation is vestigial. Unify into a single + `type StartFnType = Box) -> BoxFuture<'static, ()> + Send>;` + and remove the `#[cfg(feature = "std")]` split. No closure-bound change + needed. --- ### Step 5 β€” Core: remote access bounds **Files:** `aimdb-core/src/remote/supervisor.rs`, -`aimdb-core/src/remote/handler.rs` +`aimdb-core/src/remote/handler.rs`, `aimdb-core/src/builder.rs` - Rename `spawn_supervisor` β†’ `build_supervisor_future`; return `DbResult>` instead of spawning. @@ -674,6 +821,12 @@ Each method constructs the future and returns it rather than calling with `R: RuntimeAdapter`. - `build()` in `builder.rs` calls `build_supervisor_future()` and pushes the returned future to the accumulator. +- **`AimDb::subscribe_record_updates`** ([builder.rs:1409](../../aimdb-core/src/builder.rs#L1409)) + currently calls `runtime.spawn(...)`. Rewrite to call `tokio::spawn` + directly under `#[cfg(feature = "std")]`. This is the bridge state β€” the + full nested-`FuturesUnordered` rewrite is deferred to the follow-up issue. +- Bare `tokio::spawn` calls at `supervisor.rs:122` and `handler.rs:1042` + remain **unchanged** in this PR; they are addressed in the follow-up. --- @@ -753,10 +906,41 @@ collection point. --- +### Step 9a β€” Codegen + +**File:** `aimdb-codegen/src/rust.rs` + +- Replace emitted `use aimdb_executor::Spawn;` with whatever the new bundle + exports (or drop it β€” `AimDbBuilder` no longer needs an explicit bound). +- Change emitted `` to `` in `configure_schema` signatures (lines 560, 734, 1253). +- Update the two golden-string tests at lines 1836 and 1941. + +### Step 9b β€” Persistence and sync + +- **`aimdb-persistence`** (`builder_ext.rs`, `ext.rs`, `query_ext.rs`): + replace every `R: Spawn` with `R: RuntimeAdapter`. No runtime calls + change. +- **`aimdb-sync`** (`handle.rs`): same find-replace. + +### Step 9c β€” Delete `TokioAdapter::spawn_connectors` + +`aimdb-tokio-adapter/src/connector.rs`: delete the helper and its tests. +Test-only, no external callers. + +--- + ### Step 10 β€” Examples and aimdb-pro call sites -Update all examples to add `db.run().await` after `build()`. Update aimdb-pro -demo binaries, connectors, and any internal tooling that calls `build()`. +Update all examples to add `runner.run().await` after `build()`. Update +aimdb-pro demo binaries, connectors, and any internal tooling that calls +`build()`. Three Embassy examples plus aimdb-pro docs require an +`EmbassyAdapter::new_with_network(spawner, stack)` β†’ `new_with_network(stack)` +edit: + +- `examples/embassy-knx-connector-demo/src/main.rs` +- `examples/embassy-mqtt-connector-demo/src/main.rs` +- `examples/weather-mesh-demo/weather-station-gamma/src/main.rs` +- aimdb-pro UI docs (`llms-full.txt`, `04-deployment.md`) --- @@ -795,6 +979,23 @@ spawning (e.g. a custom connector). **Deferred** β€” no current use case inside the AimDB codebase requires it. Can be re-introduced if a concrete need arises, backed by an explicit `Spawn` impl on the adapter. +### E β€” Channel-based `AimDbSpawner` handle + +A handle returned from `build()` that can submit new futures into the +running `FuturesUnordered` via an unbounded mpsc. **Rejected** β€” adds API +surface (`AimDbSpawner`, channel polling in `run()`) for a use case (AimX +per-connection / per-subscription fan-out) that has a strictly cleaner +local solution: nested `FuturesUnordered` inside the supervisor future +itself. See [Out of Scope](#out-of-scope). + +### F β€” Nested `FuturesUnordered` for AimX, inside this PR + +Combine the AimX portability refactor with the trait removal. **Deferred** β€” +two unrelated changes, doubles the review surface, and the AimX rewrite +touches subscription cancellation semantics (today: `oneshot::Sender` per +subscription; after: drop the future). Cleaner as a focused follow-up. +See [Out of Scope](#out-of-scope). + --- ## Decisions @@ -825,4 +1026,60 @@ All design questions raised during review have been resolved: reconsidered in a future milestone when a concrete use case exists. 6. **`on_start` std/no_std bifurcation** β†’ **Collapse.** - Unify into a single `StartFnType` alias (see Step 4 for details). + Unify into a single `StartFnType` alias (see Step 4 for details). The + two existing aliases are already byte-for-byte identical; this is a pure + simplification. + +7. **Join-transform forwarders** β†’ **Hoist to build time.** + `inputs.len()` is statically known when `transform_join()` registers the + pipeline. The lazy `runtime.spawn(forwarder)` inside `run_join_transform` + becomes build-time collection (Step 3a). This keeps the "no spawn + reachable from `aimdb-core` after `build()`" invariant clean β€” modulo + the AimX exception below. + +8. **AimX remote-access portability** β†’ **Defer to follow-up issue.** + The supervisor and handler currently call bare `tokio::spawn` for + per-connection and per-subscription tasks. These do **not** depend on + the `Spawn` trait, so they can stay as-is for this PR (AimX is + `std`-gated). A separate issue replaces them with nested + `FuturesUnordered` driven by `select_biased!`, which makes the AimX + path runtime-agnostic and is the prerequisite for eventually un-gating + AimX from `std`. See [Out of Scope](#out-of-scope). + +9. **`AimDb::spawn_task`** β†’ **Delete.** + Public convenience method with no internal callers. After the trait + removal there is no portable backing primitive. Downstream callers + migrate to `on_start()` (build-time collection) or to a private + `FuturesUnordered` inside their own future. No deprecation cycle; + already breaking release. + +--- + +## Out of Scope + +The following are explicitly **not** part of this PR / issue #88: + +### AimX remote-access spawn-free refactor + +**Tracked in:** [AimX remote-access portability](../issues/aimx-remote-spawn-free.md) (TBD on filing). + +The AimX path retains three bare `tokio::spawn` (or `runtime.spawn`-via-bridge) +calls that are not addressed here: + +- `aimdb-core/src/remote/supervisor.rs:122` β€” per-connection handler spawn +- `aimdb-core/src/remote/handler.rs:1042` β€” per-subscription event-stream spawn +- The temporary `tokio::spawn` inside the rewritten `AimDb::subscribe_record_updates` (this PR replaces `runtime.spawn` with `tokio::spawn`; the follow-up replaces the whole method with a `Stream`-returning helper) + +The follow-up issue replaces all three with nested `FuturesUnordered` driven +by `futures::select_biased!`, eliminates the per-subscription `oneshot` +cancel channel (cancellation = drop the future), and is a prerequisite for +ever lifting the `#[cfg(feature = "std")]` gate on AimX. + +### WebSocket client reconnect spawn + +The WS *client* connector's reconnect loop ([client/connector.rs:505,510](../../aimdb-websocket-connector/src/client/connector.rs#L505)) calls bare `tokio::spawn` on every reconnect. Same nested-`FuturesUnordered` pattern applies; bundled with the AimX follow-up. + +### Removing `R` from `Producer` and `Consumer` + +Already deferred (Alternative C). Touches every public API signature and is +a larger surface change with no functional benefit on top of this refactor. diff --git a/examples/embassy-knx-connector-demo/src/main.rs b/examples/embassy-knx-connector-demo/src/main.rs index 1bdf979..2494cbf 100644 --- a/examples/embassy-knx-connector-demo/src/main.rs +++ b/examples/embassy-knx-connector-demo/src/main.rs @@ -250,7 +250,7 @@ async fn main(spawner: Spawner) { info!("πŸ”Œ Initializing KNX client..."); - let runtime = alloc::sync::Arc::new(EmbassyAdapter::new_with_network(spawner, stack)); + let runtime = alloc::sync::Arc::new(EmbassyAdapter::new_with_network(stack)); use alloc::format; let gateway_url = format!("knx://{}:{}", KNX_GATEWAY_IP, KNX_GATEWAY_PORT); @@ -355,15 +355,19 @@ async fn main(spawner: Spawner) { info!("Press USER button to toggle light (1/0/6)"); static DB_CELL: StaticCell> = StaticCell::new(); - let _db = DB_CELL.init(builder.build().await.expect("Failed to build database")); + let (db, db_runner) = builder.build().await.expect("Failed to build database"); + let _db = DB_CELL.init(db); info!("βœ… Database running"); - // Main loop - blink LED to show system is alive - loop { - led.set_high(); - Timer::after(Duration::from_millis(100)).await; - led.set_low(); - Timer::after(Duration::from_millis(900)).await; - } + // Drive the AimDB runner and LED blink concurrently. + embassy_futures::join::join(db_runner.run(), async { + loop { + led.set_high(); + Timer::after(Duration::from_millis(100)).await; + led.set_low(); + Timer::after(Duration::from_millis(900)).await; + } + }) + .await; } diff --git a/examples/embassy-mqtt-connector-demo/Cargo.toml b/examples/embassy-mqtt-connector-demo/Cargo.toml index 9e46f7c..568e58e 100644 --- a/examples/embassy-mqtt-connector-demo/Cargo.toml +++ b/examples/embassy-mqtt-connector-demo/Cargo.toml @@ -18,7 +18,6 @@ std = [] aimdb-core = { path = "../../aimdb-core", default-features = false } aimdb-embassy-adapter = { path = "../../aimdb-embassy-adapter", default-features = false, features = [ "embassy-runtime", - "embassy-task-pool-16", # Need 13+ tasks: 8 user + 2 MQTT + 3 outbound publishers ] } aimdb-executor = { path = "../../aimdb-executor", default-features = false, features = [ "embassy-types", diff --git a/examples/embassy-mqtt-connector-demo/src/main.rs b/examples/embassy-mqtt-connector-demo/src/main.rs index 894f0ef..5d848b0 100644 --- a/examples/embassy-mqtt-connector-demo/src/main.rs +++ b/examples/embassy-mqtt-connector-demo/src/main.rs @@ -314,7 +314,7 @@ async fn main(spawner: Spawner) { info!("πŸ”Œ Initializing MQTT client..."); // Create AimDB database with Embassy adapter - let runtime = alloc::sync::Arc::new(EmbassyAdapter::new_with_network(spawner, stack)); + let runtime = alloc::sync::Arc::new(EmbassyAdapter::new_with_network(stack)); // Build MQTT broker URL use alloc::format; @@ -394,15 +394,19 @@ async fn main(spawner: Spawner) { info!(""); static DB_CELL: StaticCell> = StaticCell::new(); - let _db = DB_CELL.init(builder.build().await.expect("Failed to build database")); + let (db, db_runner) = builder.build().await.expect("Failed to build database"); + let _db = DB_CELL.init(db); info!("βœ… Database running with background services"); - // Main loop - blink LED to show system is alive - loop { - led.set_high(); - Timer::after(Duration::from_millis(100)).await; - led.set_low(); - Timer::after(Duration::from_millis(900)).await; - } + // Drive the AimDB runner (all connector/tap/source futures) and LED blink concurrently. + embassy_futures::join::join(db_runner.run(), async { + loop { + led.set_high(); + Timer::after(Duration::from_millis(100)).await; + led.set_low(); + Timer::after(Duration::from_millis(900)).await; + } + }) + .await; } diff --git a/examples/remote-access-demo/src/server.rs b/examples/remote-access-demo/src/server.rs index 1c7e130..e597e45 100644 --- a/examples/remote-access-demo/src/server.rs +++ b/examples/remote-access-demo/src/server.rs @@ -136,7 +136,8 @@ async fn main() -> Result<(), Box> { reg.buffer(BufferCfg::SingleLatest).with_remote_access(); }); - let db = builder.build().await?; + let (db, runner) = builder.build().await?; + tokio::spawn(runner.run()); info!("βœ… Database initialized with 5 record types"); info!(" - Temperature (SpmcRingΓ—100, has producer β€” drainable πŸ”„)"); diff --git a/examples/weather-mesh-demo/weather-station-alpha/src/main.rs b/examples/weather-mesh-demo/weather-station-alpha/src/main.rs index 133d66d..1580e76 100644 --- a/examples/weather-mesh-demo/weather-station-alpha/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-alpha/src/main.rs @@ -119,7 +119,8 @@ async fn main() -> Result<(), Box> { .finish(); }); - let db = builder.build().await?; + let (db, runner) = builder.build().await?; + tokio::spawn(runner.run()); info!("βœ… Database initialized with 3 record types"); info!(" - Temperature: {}", temp_topic); diff --git a/examples/weather-mesh-demo/weather-station-beta/src/main.rs b/examples/weather-mesh-demo/weather-station-beta/src/main.rs index 874b9f0..f762ac2 100644 --- a/examples/weather-mesh-demo/weather-station-beta/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-beta/src/main.rs @@ -119,7 +119,8 @@ async fn main() -> Result<(), Box> { .finish(); }); - let db = builder.build().await?; + let (db, runner) = builder.build().await?; + tokio::spawn(runner.run()); info!("βœ… Database initialized with 3 record types"); info!(" - Temperature: {} (synthetic)", temp_topic); diff --git a/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml b/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml index a99d5ef..6957312 100644 --- a/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml +++ b/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml @@ -24,7 +24,6 @@ aimdb-core = { path = "../../../aimdb-core", default-features = false, features aimdb-embassy-adapter = { path = "../../../aimdb-embassy-adapter", default-features = false, features = [ "alloc", "embassy-runtime", - "embassy-task-pool-16", # 2 producers + net + MQTT + join transform + 2 forwarders ] } aimdb-executor = { path = "../../../aimdb-executor", default-features = false } aimdb-data-contracts = { path = "../../../aimdb-data-contracts", default-features = false, features = [ diff --git a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs index 91c0e75..7ae2950 100644 --- a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs @@ -256,7 +256,7 @@ async fn main(spawner: Spawner) { info!("πŸ”Œ Initializing MQTT client..."); // Create AimDB database with Embassy adapter - let runtime = alloc::sync::Arc::new(EmbassyAdapter::new_with_network(spawner, stack)); + let runtime = alloc::sync::Arc::new(EmbassyAdapter::new_with_network(stack)); // Build MQTT broker URL use alloc::format; @@ -382,16 +382,20 @@ async fn main(spawner: Spawner) { info!(""); static DB_CELL: StaticCell> = StaticCell::new(); - let _db = DB_CELL.init(builder.build().await.expect("Failed to build database")); + let (db, db_runner) = builder.build().await.expect("Failed to build database"); + let _db = DB_CELL.init(db); info!("βœ… Database running"); info!("🎯 Weather Station Gamma ready!"); - // Main loop - blink LED to show system is alive - loop { - led.set_high(); - Timer::after(Duration::from_millis(100)).await; - led.set_low(); - Timer::after(Duration::from_millis(900)).await; - } + // Drive the AimDB runner and LED blink concurrently. + embassy_futures::join::join(db_runner.run(), async { + loop { + led.set_high(); + Timer::after(Duration::from_millis(100)).await; + led.set_low(); + Timer::after(Duration::from_millis(900)).await; + } + }) + .await; } From beeaf8fbc393a60322cb91c1c04136bc32bb03a6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Sat, 23 May 2026 20:56:07 +0000 Subject: [PATCH 03/14] feat: add configurable command channel capacity for KNX connector --- aimdb-knx-connector/src/embassy_client.rs | 39 ++++++++++++---- aimdb-knx-connector/src/tokio_client.rs | 55 +++++++++++++++-------- 2 files changed, 67 insertions(+), 27 deletions(-) diff --git a/aimdb-knx-connector/src/embassy_client.rs b/aimdb-knx-connector/src/embassy_client.rs index 49d3e85..fc4e0ab 100644 --- a/aimdb-knx-connector/src/embassy_client.rs +++ b/aimdb-knx-connector/src/embassy_client.rs @@ -70,12 +70,21 @@ use embassy_sync::blocking_mutex::raw::CriticalSectionRawMutex; use embassy_sync::channel::Channel; use static_cell::StaticCell; -/// Static channel for KNX commands (32 slots to match Tokio implementation) -static KNX_COMMAND_CHANNEL: StaticCell> = - StaticCell::new(); +/// Capacity of the static KNX command channel. +/// +/// Embassy requires a compile-time const generic β€” runtime configurability is +/// not possible with `StaticCell>`. Adjust this constant and +/// recompile if your installation needs a larger buffer. +const KNX_COMMAND_QUEUE_SIZE: usize = 32; + +/// Static channel for KNX commands (capacity: [`KNX_COMMAND_QUEUE_SIZE`]) +static KNX_COMMAND_CHANNEL: StaticCell< + Channel, +> = StaticCell::new(); /// Get or initialize the command channel -fn get_command_channel() -> &'static Channel { +fn get_command_channel( +) -> &'static Channel { KNX_COMMAND_CHANNEL.init(Channel::new()) } @@ -265,7 +274,7 @@ impl ChannelState { /// Internal KNX connector implementation pub struct KnxConnectorImpl { - command_channel: &'static Channel, + command_channel: &'static Channel, } impl KnxConnectorImpl { @@ -277,7 +286,7 @@ impl KnxConnectorImpl { runtime_ctx: Option>, ) -> Result< ( - &'static Channel, + &'static Channel, BoxFuture, ), &'static str, @@ -339,7 +348,11 @@ impl KnxConnectorImpl { gateway_addr: Ipv4Address, gateway_port: u16, router: Arc, - command_channel: &'static Channel, + command_channel: &'static Channel< + CriticalSectionRawMutex, + KnxCommand, + KNX_COMMAND_QUEUE_SIZE, + >, runtime_ctx: Option>, ) { loop { @@ -384,7 +397,11 @@ impl KnxConnectorImpl { gateway_addr: Ipv4Address, gateway_port: u16, router: &Router, - command_channel: &'static Channel, + command_channel: &'static Channel< + CriticalSectionRawMutex, + KnxCommand, + KNX_COMMAND_QUEUE_SIZE, + >, runtime_ctx: Option<&Arc>, ) -> Result<(), &'static str> { // Create UDP socket with static buffers @@ -1001,7 +1018,11 @@ impl KnxConnectorImpl { /// If a topic provider is configured, it will be called for each value to /// dynamically determine the KNX group address. Otherwise, the static default is used. fn collect_outbound_futures( - command_channel: &'static Channel, + command_channel: &'static Channel< + CriticalSectionRawMutex, + KnxCommand, + KNX_COMMAND_QUEUE_SIZE, + >, runtime_ctx: Arc, outbound_routes: Vec, ) -> Vec { diff --git a/aimdb-knx-connector/src/tokio_client.rs b/aimdb-knx-connector/src/tokio_client.rs index e51397f..dda000a 100644 --- a/aimdb-knx-connector/src/tokio_client.rs +++ b/aimdb-knx-connector/src/tokio_client.rs @@ -61,6 +61,9 @@ enum KnxCommand { /// automatically monitors all required KNX group addresses. pub struct KnxConnectorBuilder { gateway_url: String, + /// Capacity of the mpsc channel between outbound publishers and the + /// connection task. Defaults to 32. + command_queue_size: usize, } impl KnxConnectorBuilder { @@ -77,8 +80,19 @@ impl KnxConnectorBuilder { pub fn new(gateway_url: impl Into) -> Self { Self { gateway_url: gateway_url.into(), + command_queue_size: 32, } } + + /// Override the internal command channel capacity (default: 32). + /// + /// The channel sits between outbound publisher futures and the single + /// connection task that serializes UDP sends. Increase this for + /// installations with many outbound routes or bursty publish patterns. + pub fn with_command_queue_size(mut self, size: usize) -> Self { + self.command_queue_size = size; + self + } } type BoxFuture = Pin + Send + 'static>>; @@ -114,21 +128,25 @@ impl ConnectorBuilder for KnxCon // 2. Receiver captured by `connection_future`. // 3. Sender cloned into each outbound publisher future below. let runtime_ctx = db.runtime_any(); - let (command_tx, connection_future) = - KnxConnectorImpl::build_internal(&self.gateway_url, router, Some(runtime_ctx)) - .await - .map_err(|e| { - #[cfg(feature = "std")] - { - aimdb_core::DbError::RuntimeError { - message: format!("Failed to build KNX connector: {}", e), - } - } - #[cfg(not(feature = "std"))] - { - aimdb_core::DbError::RuntimeError { _message: () } - } - })?; + let (command_tx, connection_future) = KnxConnectorImpl::build_internal( + &self.gateway_url, + router, + Some(runtime_ctx), + self.command_queue_size, + ) + .await + .map_err(|e| { + #[cfg(feature = "std")] + { + aimdb_core::DbError::RuntimeError { + message: format!("Failed to build KNX connector: {}", e), + } + } + #[cfg(not(feature = "std"))] + { + aimdb_core::DbError::RuntimeError { _message: () } + } + })?; let outbound_routes = db.collect_outbound_routes("knx"); @@ -176,6 +194,7 @@ impl KnxConnectorImpl { gateway_url: &str, router: Router, runtime_ctx: Option>, + command_queue_size: usize, ) -> Result<(mpsc::Sender, BoxFuture), String> { // Parse the gateway URL let mut url = gateway_url.to_string(); @@ -202,7 +221,7 @@ impl KnxConnectorImpl { // 1. Create command channel; receiver goes to the connection future, // sender is returned for the publisher futures to clone. - let (command_tx, command_rx) = mpsc::channel::(32); + let (command_tx, command_rx) = mpsc::channel::(command_queue_size); // 2. Build the connection-task future (captures the receiver). let connection_future = build_connection_future( @@ -1065,7 +1084,7 @@ mod tests { async fn test_connector_creation_with_router() { let router = RouterBuilder::new().build(); let connector = - KnxConnectorImpl::build_internal("knx://192.168.1.19:3671", router, None).await; + KnxConnectorImpl::build_internal("knx://192.168.1.19:3671", router, None, 32).await; assert!(connector.is_ok()); } @@ -1073,7 +1092,7 @@ mod tests { async fn test_connector_with_port() { let router = RouterBuilder::new().build(); let connector = - KnxConnectorImpl::build_internal("knx://gateway.local:3672", router, None).await; + KnxConnectorImpl::build_internal("knx://gateway.local:3672", router, None, 32).await; assert!(connector.is_ok()); } From 8772bc168bfc81cecdfc4fe7bf9d8b69944a70e8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Sat, 23 May 2026 21:18:17 +0000 Subject: [PATCH 04/14] chore: update dependencies in Cargo.lock and embassy submodule --- Cargo.lock | 11 ++++++++--- _external/embassy | 2 +- 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index acd2c25..809a784 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1111,6 +1111,7 @@ dependencies = [ "futures-util", "heapless 0.9.1", "nb 1.1.0", + "once_cell", "proc-macro2", "quote", "rand_core 0.10.0", @@ -2246,9 +2247,13 @@ dependencies = [ [[package]] name = "once_cell" -version = "1.21.3" +version = "1.21.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" +dependencies = [ + "critical-section", + "portable-atomic", +] [[package]] name = "once_cell_polyfill" @@ -3027,7 +3032,7 @@ dependencies = [ [[package]] name = "stm32-metapac" version = "21.0.0" -source = "git+https://github.com/embassy-rs/stm32-data-generated?tag=stm32-data-fe91bec726df47c6467dafce3fcf64075f662a3c#1569e3dc33adb6924329da79ef7dc8ce64bc791d" +source = "git+https://github.com/embassy-rs/stm32-data-generated?tag=stm32-data-6d108b6d3695cafd30923b033bc82291da5859a7#afce0040f03f376ad34e62e78e201fbfe275a0a1" dependencies = [ "cortex-m", "cortex-m-rt", diff --git a/_external/embassy b/_external/embassy index 34d3684..823abdb 160000 --- a/_external/embassy +++ b/_external/embassy @@ -1 +1 @@ -Subproject commit 34d3684bd7c480a670af800bb65135367f8047bf +Subproject commit 823abdbe8512721b4797536b9d3151c94c4bbad0 From 1825ccc2873c1385adbb41e1df60c10636170ac5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Sat, 23 May 2026 22:01:21 +0000 Subject: [PATCH 05/14] feat: remove `Spawn` trait and update `AimDbBuilder::build()` to return `(AimDb, AimDbRunner)` across the workspace --- CHANGELOG.md | 4 ++++ README.md | 6 +++++- aimdb-codegen/CHANGELOG.md | 4 ++++ aimdb-core/CHANGELOG.md | 13 +++++++++++++ aimdb-core/src/builder.rs | 2 +- aimdb-embassy-adapter/CHANGELOG.md | 12 ++++++++++++ aimdb-executor/CHANGELOG.md | 9 +++++++++ aimdb-knx-connector/CHANGELOG.md | 7 +++++++ aimdb-mqtt-connector/CHANGELOG.md | 6 ++++++ aimdb-persistence/CHANGELOG.md | 4 ++++ aimdb-sync/CHANGELOG.md | 4 +++- aimdb-tokio-adapter/CHANGELOG.md | 9 +++++++++ aimdb-wasm-adapter/CHANGELOG.md | 5 +++++ aimdb-websocket-connector/CHANGELOG.md | 6 ++++++ examples/hello-single-latest-async/src/main.rs | 5 ++++- 15 files changed, 92 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4fe8946..4a18397 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,6 +27,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed (breaking) + +- **M13 β€” `Spawn` trait removed across the workspace; `AimDbBuilder::build()` now returns `(AimDb, AimDbRunner)` (Issue #88, [Design 028](docs/design/028-M13-remove-spawn-trait.md)).** Every future the database drives β€” producer services, taps, transforms, join forwarders, connector loops, the remote-access supervisor, `on_start` tasks β€” is collected at build time and driven by a single `FuturesUnordered` inside `runner.run().await`. Adapter implementations (`TokioAdapter`, `EmbassyAdapter`, `WasmAdapter`) drop their `impl Spawn`. The `embassy-task-pool-8/16/32` features are deleted and `EmbassyAdapter::new_with_network` no longer takes a `Spawner`. Connector authors must update `ConnectorBuilder::build()` to return `Vec` instead of `Arc`. See each crate's CHANGELOG for the per-crate impact. + ## [1.1.0] - 2026-05-22 ### Added diff --git a/README.md b/README.md index a569279..d7f791e 100644 --- a/README.md +++ b/README.md @@ -93,7 +93,11 @@ async fn main() -> Result<(), Box> { .finish(); }); - builder.build()?.run().await?; + // `.run()` builds the database, collects every producer/consumer/transform + // future, and drives them all on a single `FuturesUnordered`. It blocks + // until shutdown. For programmatic access to the `AimDb` handle, call + // `.build().await?` directly β€” it returns `(AimDb, AimDbRunner)`. + builder.run().await?; Ok(()) } ``` diff --git a/aimdb-codegen/CHANGELOG.md b/aimdb-codegen/CHANGELOG.md index 50bb368..12fa475 100644 --- a/aimdb-codegen/CHANGELOG.md +++ b/aimdb-codegen/CHANGELOG.md @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed (breaking) + +- Emitted `configure_schema` signature changed from `` to ``; emitted prelude now imports `aimdb_executor::RuntimeAdapter` instead of `Spawn` (Issue #88). Regenerate downstream schemas. + ## [0.2.0] - 2026-05-22 ### Changed diff --git a/aimdb-core/CHANGELOG.md b/aimdb-core/CHANGELOG.md index 1411147..dda8e36 100644 --- a/aimdb-core/CHANGELOG.md +++ b/aimdb-core/CHANGELOG.md @@ -7,6 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed (breaking) + +- **`Spawn` trait removed; `AimDbBuilder::build()` now returns `(AimDb, AimDbRunner)` (Issue #88, Design 028).** Every future the database needs β€” `.source()`/`.tap()`/`.transform()` tasks, on_start hooks, connector loops, the remote-access supervisor β€” is collected at build time into the new `AimDbRunner`, then driven by a single `FuturesUnordered` from `runner.run().await`. No background work runs until the runner is polled. + - `AimDb::spawn_task` is **deleted**. Migrate to `on_start()` (collected at build) or to a private `FuturesUnordered` inside your own future. + - The `Runtime` bundle no longer supertrait-requires `Spawn`. Custom adapters drop `impl Spawn`. + - `R: Spawn` bounds are gone everywhere in `aimdb-core` (`Producer`, `Consumer`, `TypedRecord`, `TransformDescriptor`, `RecordRegistrar`, `RecordT`, `AnyRecordExt::as_typed`, remote handler/supervisor, `Database`) β€” replaced by `R: RuntimeAdapter`. + - `RecordSpawner` renamed to `RecordFutureCollector`; its `spawn_all_tasks` β†’ `collect_all_futures`. Internal `spawn_consumer_tasks`/`spawn_producer_service`/`spawn_transform_task` on `TypedRecord` become `collect_consumer_futures`/`collect_producer_future`/`collect_transform_futures`. + - Join transforms now hoist their per-input forwarder construction to build time β€” `JoinPipeline::into_descriptor()` returns a `CollectedTransform { task_future, fanin_futures }` and the lazy `runtime.spawn(forwarder)` inside `run_join_transform` is gone. + - `ConnectorBuilder::build()` now returns `Vec>` instead of `Arc` (which `AimDbBuilder` already discarded). + - Unsafe `impl Send/Sync` blocks on `Producer` / `Consumer` deleted β€” they auto-derive now. + - On the AimX remote-access path, three `runtime.spawn(...)` call sites bridge to `tokio::spawn` directly under `#[cfg(feature = "std")]`. These (per-connection handler, per-subscription event stream, `subscribe_record_updates`) are addressed in the AimX portability follow-up. +- `on_start` no_std bifurcation collapsed: a single `StartFnType` alias replaces the byte-identical std/no_std pair. + ## [1.1.0] - 2026-05-22 ### Added diff --git a/aimdb-core/src/builder.rs b/aimdb-core/src/builder.rs index 8e48cac..bcce237 100644 --- a/aimdb-core/src/builder.rs +++ b/aimdb-core/src/builder.rs @@ -376,7 +376,7 @@ impl AimDbBuilder { /// /// These types are incompatible and cannot be transferred. However, this is not a bug /// because `.with_connector()` is only available AFTER calling `.runtime()` (it's defined - /// in the `impl where R: Spawn` block, not in `impl AimDbBuilder`). + /// in the `impl where R: RuntimeAdapter` block, not in `impl AimDbBuilder`). /// /// This means the type system **enforces** the correct call order: /// ```rust,ignore diff --git a/aimdb-embassy-adapter/CHANGELOG.md b/aimdb-embassy-adapter/CHANGELOG.md index cf4aa38..0c56eba 100644 --- a/aimdb-embassy-adapter/CHANGELOG.md +++ b/aimdb-embassy-adapter/CHANGELOG.md @@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Removed (breaking) + +- **`impl Spawn for EmbassyAdapter` deleted (Issue #88).** Static `generic_task_runner` task pool gone, along with `BoxedFuture` and the `unsafe Pin::new_unchecked` cast that fed the pool. Drive database futures by awaiting `AimDbRunner::run()` from inside the Embassy main task. +- **`embassy-task-pool-8` / `embassy-task-pool-16` / `embassy-task-pool-32` Cargo features deleted.** No pool β€” `FuturesUnordered` grows as needed within a single Embassy task's heap budget. +- **`EmbassyAdapter::new_with_spawner(spawner)` constructor deleted.** +- **`EmbassyAdapter::new_with_network(spawner, network)` signature changed** to `new_with_network(network)` β€” the `spawner` argument is gone. Update callers (three example binaries plus aimdb-pro docs). +- `spawner: Option` field and `spawner()` accessor deleted. + +### Notes + +- `unsafe impl Send/Sync for EmbassyAdapter` is **retained** when the `embassy-net-support` feature is enabled: `embassy_net::Stack` contains a `RefCell` and is `!Sync`. Embassy's single-threaded cooperative executor makes this sound, and the impl now has a smaller surface (no `Spawner` to justify it). + ## [0.6.0] - 2026-05-22 ### Added diff --git a/aimdb-executor/CHANGELOG.md b/aimdb-executor/CHANGELOG.md index 854cbac..1ba79d8 100644 --- a/aimdb-executor/CHANGELOG.md +++ b/aimdb-executor/CHANGELOG.md @@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Removed (breaking) + +- **`Spawn` trait deleted (Issue #88).** Including the `SpawnToken` associated type and the `ExecutorError::SpawnFailed` variant. The `Runtime` bundle trait is now `RuntimeAdapter + TimeOps + Logger` (no `Spawn` supertrait). Task execution moves to `AimDbRunner::run()` in `aimdb-core`, which drives every collected future via a single `FuturesUnordered`. Custom adapters must drop `impl Spawn`. +- `JoinFanInRuntime` supertrait relaxed from `Spawn` to `RuntimeAdapter`. + +### Added + +- `futures-util` (alloc-only) as a regular dependency β€” provides `FuturesUnordered` used by `aimdb-core`'s `AimDbRunner`. + ## [0.2.0] - 2026-05-22 ### Added diff --git a/aimdb-knx-connector/CHANGELOG.md b/aimdb-knx-connector/CHANGELOG.md index 2e1ea5c..65ece96 100644 --- a/aimdb-knx-connector/CHANGELOG.md +++ b/aimdb-knx-connector/CHANGELOG.md @@ -7,6 +7,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed (breaking) + +- **`ConnectorBuilder::build()` now returns `Vec>` instead of `Arc` (Issue #88).** Both Tokio and Embassy implementations updated. +- `spawn_connection_task()` β†’ `build_connection_future()`; the `mpsc::channel` for outbound commands is created up front, the receiver captured by the connection future, and the sender cloned into each outbound publisher future. `spawn_outbound_publishers()` β†’ `collect_outbound_futures()`. +- `R: Spawn` bounds dropped in favour of `R: RuntimeAdapter`. +- The `transport::Connector` impl on `KnxConnectorImpl` was removed alongside the discarded `Arc` return path. + ## [0.4.0] - 2026-05-22 ### Changed diff --git a/aimdb-mqtt-connector/CHANGELOG.md b/aimdb-mqtt-connector/CHANGELOG.md index 280ae92..3f4863c 100644 --- a/aimdb-mqtt-connector/CHANGELOG.md +++ b/aimdb-mqtt-connector/CHANGELOG.md @@ -7,6 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed (breaking) + +- **`ConnectorBuilder::build()` now returns `Vec>` instead of `Arc` (Issue #88).** Both Tokio and Embassy implementations updated. The MQTT event-loop, the Embassy event-router, and every outbound publisher are returned as futures that the `AimDbRunner` drives β€” no more `runtime.spawn` / `tokio::spawn` inside the connector. `R: Spawn` bounds dropped throughout in favour of `R: RuntimeAdapter`. +- `spawn_event_loop()` β†’ `build_event_loop_future()` (Tokio side). `spawn_outbound_publishers()` β†’ `collect_outbound_futures()` on both Tokio and Embassy. +- The `transport::Connector` impl on `MqttConnectorImpl` was removed alongside the discarded `Arc` return path; direct programmatic publish was already unreachable through the `AimDbBuilder` public API. + ## [0.6.0] - 2026-05-22 ### Changed diff --git a/aimdb-persistence/CHANGELOG.md b/aimdb-persistence/CHANGELOG.md index ec41e62..181aa5c 100644 --- a/aimdb-persistence/CHANGELOG.md +++ b/aimdb-persistence/CHANGELOG.md @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed (breaking) + +- All `R: Spawn` bounds replaced with `R: RuntimeAdapter` on public traits (`AimDbBuilderPersistExt`, `RecordRegistrarPersistExt`, `AimDbQueryExt`) and internal helpers (Issue #88). No behavioural change β€” `aimdb-persistence` never called `runtime.spawn` directly, the bound was just propagated. + ## [0.1.1] - 2026-05-22 ### Changed diff --git a/aimdb-sync/CHANGELOG.md b/aimdb-sync/CHANGELOG.md index 365da07..c5ace53 100644 --- a/aimdb-sync/CHANGELOG.md +++ b/aimdb-sync/CHANGELOG.md @@ -7,7 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -No changes yet. +### Changed + +- `attach()` updated to destructure the `(AimDb, AimDbRunner)` tuple returned by `AimDbBuilder::build()` after issue #88, and to drive the runner inside the runtime thread via `tokio::select!` against the shutdown signal. No public API change. ## [0.5.0] - 2026-02-21 diff --git a/aimdb-tokio-adapter/CHANGELOG.md b/aimdb-tokio-adapter/CHANGELOG.md index 4d2f709..efdcc3e 100644 --- a/aimdb-tokio-adapter/CHANGELOG.md +++ b/aimdb-tokio-adapter/CHANGELOG.md @@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Removed (breaking) + +- **`impl Spawn for TokioAdapter` deleted (Issue #88).** Adapter is now `RuntimeAdapter + TimeOps + Logger` only. Drive database futures by awaiting `AimDbRunner::run()` returned from `AimDbBuilder::build()`. +- **`TokioAdapter::spawn_connectors` test-only helper deleted** along with the `connector` module. It had no production callers; outbound connector futures are now collected by `ConnectorBuilder::build()` and driven by the runner. + +### Notes + +- `BufferOps::spawn_dispatcher` (a test-only utility) is unchanged β€” it calls `tokio::spawn` directly and does not depend on the deleted `Spawn` trait. + ## [0.6.0] - 2026-05-22 ### Added diff --git a/aimdb-wasm-adapter/CHANGELOG.md b/aimdb-wasm-adapter/CHANGELOG.md index 73a5ddd..233e4a0 100644 --- a/aimdb-wasm-adapter/CHANGELOG.md +++ b/aimdb-wasm-adapter/CHANGELOG.md @@ -7,6 +7,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Removed (breaking) + +- **`impl Spawn for WasmAdapter` deleted (Issue #88).** Also removed the `unsafe impl Send/Sync for WasmAdapter` β€” the adapter is a ZST and auto-derives both. +- `bindings.rs::Database::build()` now consumes the `AimDbRunner` returned from `AimDbBuilder::build()` and drives it via `wasm_bindgen_futures::spawn_local`, so existing JS callers keep working with no API change. + ## [0.2.0] - 2026-05-22 ### Added diff --git a/aimdb-websocket-connector/CHANGELOG.md b/aimdb-websocket-connector/CHANGELOG.md index 4fb38cf..f4b489f 100644 --- a/aimdb-websocket-connector/CHANGELOG.md +++ b/aimdb-websocket-connector/CHANGELOG.md @@ -7,6 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed (breaking) + +- **`ConnectorBuilder::build()` now returns `Vec>` instead of `Arc` (Issue #88).** Server-side: `start_server()` β†’ `build_server_future()` (the `axum::serve()` accept loop is collected, not spawned). Client-side: outbound publishers converted to `collect_outbound_futures()`. +- `R: Spawn` bounds dropped throughout in favour of `R: RuntimeAdapter`. The no-op `transport::Connector` impl on `WebSocketConnectorImpl` was removed. +- WS *client* internal background tasks (write loop, read loop, keepalive, reconnect watcher) are temporarily bridged to `tokio::spawn` directly (per design 028 Β§"Out of Scope" / Group 4). They will move to nested `FuturesUnordered` in the AimX portability follow-up. + ## [0.2.0] - 2026-05-22 ### Changed diff --git a/examples/hello-single-latest-async/src/main.rs b/examples/hello-single-latest-async/src/main.rs index 27e455f..de0a293 100644 --- a/examples/hello-single-latest-async/src/main.rs +++ b/examples/hello-single-latest-async/src/main.rs @@ -21,7 +21,10 @@ async fn main() -> Result<(), Box> { .tap(rollout_observer); }); - let _db = builder.build().await?; + let (_db, runner) = builder.build().await?; + // Drive the database futures concurrently with the demo's wait β€” without + // this the registered source/tap futures never run. + tokio::spawn(runner.run()); tokio::time::sleep(Duration::from_millis(700)).await; println!("Done. SingleLatest keeps only the current value for each subscriber."); From 63f69b1a70e164cfac82c9af19e2fff47387228a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Mon, 25 May 2026 07:01:26 +0000 Subject: [PATCH 06/14] feat: Remove runtime parameter `R` from `Producer` and `Consumer` - Updated `Producer` to `Producer` and `Consumer` to `Consumer`, simplifying user-facing signatures. - Removed the `.key()` method from `Producer`, as the record key is now captured at registration. - Adjusted tests and examples to reflect the new type signatures. - Updated documentation to describe the architectural changes and the motivation behind the removal of `R`. - Ensured backward compatibility for existing async function signatures while optimizing the hot path for production and consumption of records. --- aimdb-codegen/CHANGELOG.md | 1 + aimdb-codegen/src/rust.rs | 16 +- aimdb-core/CHANGELOG.md | 12 + aimdb-core/README.md | 4 +- aimdb-core/src/buffer/mod.rs | 5 + aimdb-core/src/buffer/traits.rs | 20 + aimdb-core/src/buffer/writer.rs | 77 +++ aimdb-core/src/builder.rs | 34 +- aimdb-core/src/ext_macros.rs | 16 +- aimdb-core/src/transform/join.rs | 48 +- aimdb-core/src/transform/mod.rs | 13 +- aimdb-core/src/transform/single.rs | 50 +- aimdb-core/src/typed_api.rs | 160 ++--- aimdb-core/src/typed_record.rs | 146 +++-- aimdb-data-contracts/CHANGELOG.md | 4 + aimdb-data-contracts/src/observable.rs | 2 +- aimdb-embassy-adapter/CHANGELOG.md | 4 + aimdb-embassy-adapter/src/lib.rs | 14 +- .../tests/topic_provider_tests.rs | 8 +- .../tests/topic_provider_tests.rs | 12 +- aimdb-tokio-adapter/CHANGELOG.md | 4 + .../tests/multi_instance_tests.rs | 22 +- docs/aimdb-usage-guide.md | 2 +- .../029-M14-remove-r-from-typed-handles.md | 554 ++++++++++++++++++ .../embassy-knx-connector-demo/src/main.rs | 2 +- .../embassy-mqtt-connector-demo/src/main.rs | 6 +- .../hello-single-latest-async/src/main.rs | 12 +- .../knx-connector-demo-common/src/monitors.rs | 8 +- .../src/monitors.rs | 4 +- examples/remote-access-demo/src/server.rs | 13 +- examples/tokio-knx-connector-demo/src/main.rs | 5 +- .../tokio-mqtt-connector-demo/src/main.rs | 6 +- .../weather-station-alpha/src/main.rs | 4 +- .../weather-station-beta/src/main.rs | 4 +- .../weather-station-gamma/src/main.rs | 7 +- 35 files changed, 1056 insertions(+), 243 deletions(-) create mode 100644 aimdb-core/src/buffer/writer.rs create mode 100644 docs/design/029-M14-remove-r-from-typed-handles.md diff --git a/aimdb-codegen/CHANGELOG.md b/aimdb-codegen/CHANGELOG.md index 12fa475..5e542ea 100644 --- a/aimdb-codegen/CHANGELOG.md +++ b/aimdb-codegen/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed (breaking) +- Emitted task scaffolds now use `Producer` / `Consumer` (no `, TokioAdapter` second parameter) and emitted doc tables show the same form, matching the M14 cleanup in `aimdb-core` (Design 029). Regenerate downstream scaffolds after upgrading. - Emitted `configure_schema` signature changed from `` to ``; emitted prelude now imports `aimdb_executor::RuntimeAdapter` instead of `Spawn` (Issue #88). Regenerate downstream schemas. ## [0.2.0] - 2026-05-22 diff --git a/aimdb-codegen/src/rust.rs b/aimdb-codegen/src/rust.rs index eaf4754..3993f15 100644 --- a/aimdb-codegen/src/rust.rs +++ b/aimdb-codegen/src/rust.rs @@ -387,12 +387,12 @@ pub fn generate_tasks_rs(state: &ArchitectureState, binary_name: &str) -> Option for input in &task.inputs { let arg_name = format_ident!("{}", to_snake_case(&input.record)); let value_type = format_ident!("{}Value", input.record); - params.push(quote! { #arg_name: Consumer<#value_type, TokioAdapter> }); + params.push(quote! { #arg_name: Consumer<#value_type> }); } for output in &task.outputs { let arg_name = format_ident!("{}", to_snake_case(&output.record)); let value_type = format_ident!("{}Value", output.record); - params.push(quote! { #arg_name: Producer<#value_type, TokioAdapter> }); + params.push(quote! { #arg_name: Producer<#value_type> }); } let todo_msg = match &task.task_type { @@ -1611,10 +1611,10 @@ fn build_transform_call(task: &TaskDef, variant_ident: &syn::Ident) -> TokenStre /// /// | Inputs | Outputs | API | Generated stub | /// |--------|---------|-----------------------|---------------------------| -/// | N > 1 | β‰₯ 1 | `.transform_join()` | `async fn task_handler(JoinEventRx, Producer)` | +/// | N > 1 | β‰₯ 1 | `.transform_join()` | `async fn task_handler(JoinEventRx, Producer)` | /// | 1 | β‰₯ 1 | `.transform().map()` | `fn task_transform(&Input) -> Option` | -/// | 0 | β‰₯ 1 | `.source()` | `async fn task(RuntimeContext, Producer)` | -/// | β‰₯ 1 | 0 | `.tap()` | `async fn task(RuntimeContext, Consumer)` | +/// | 0 | β‰₯ 1 | `.source()` | `async fn task(RuntimeContext, Producer)` | +/// | β‰₯ 1 | 0 | `.tap()` | `async fn task(RuntimeContext, Consumer)` | pub fn generate_hub_tasks_rs(state: &ArchitectureState) -> String { let project = state .project @@ -1660,7 +1660,7 @@ pub fn generate_hub_tasks_rs(state: &ArchitectureState) -> String { /// {inputs_doc}\n\ pub async fn {handler}(\n\ mut _rx: aimdb_core::transform::JoinEventRx,\n\ - _producer: aimdb_core::Producer<{out_t}, TokioAdapter>,\n\ + _producer: aimdb_core::Producer<{out_t}>,\n\ ) {{\n\ while let Ok(_trigger) = _rx.recv().await {{\n\ todo!(\"implement {handler}\")\n\ @@ -1689,7 +1689,7 @@ pub fn {handler}(input: &{in_t}) -> Option<{out_t}> {{\n\ fns.push_str(&format!( "pub async fn {}(\n\ _ctx: aimdb_core::RuntimeContext,\n\ - _producer: aimdb_core::Producer<{out_t}, TokioAdapter>,\n\ + _producer: aimdb_core::Producer<{out_t}>,\n\ ) {{\n\ todo!(\"implement {}\")\n\ }}\n\n", @@ -1700,7 +1700,7 @@ pub fn {handler}(input: &{in_t}) -> Option<{out_t}> {{\n\ fns.push_str(&format!( "pub async fn {}(\n\ _ctx: aimdb_core::RuntimeContext,\n\ - _consumer: aimdb_core::Consumer<{in_t}, TokioAdapter>,\n\ + _consumer: aimdb_core::Consumer<{in_t}>,\n\ ) {{\n\ todo!(\"implement {}\")\n\ }}\n\n", diff --git a/aimdb-core/CHANGELOG.md b/aimdb-core/CHANGELOG.md index dda8e36..aea7e95 100644 --- a/aimdb-core/CHANGELOG.md +++ b/aimdb-core/CHANGELOG.md @@ -9,6 +9,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed (breaking) +- **`Producer` / `Consumer` drop the runtime parameter `R` and pre-resolve the record at build time (Design 029, M14).** Producer/Consumer become handles to a buffer rather than tickets to look one up: `produce()` is one virtual call (no `HashMap` probe, no `TypeId` check, no downcast), and `subscribe()` collapses to `buffer.subscribe_boxed()`. The internal mechanic is a new crate-private `WriteHandle` trait backed by `RecordWriter` (in `aimdb-core/src/buffer/writer.rs`), pre-bound to the record's `Arc>` + snapshot mutex + metadata tracker. + - `Producer` β†’ `Producer`; `Consumer` β†’ `Consumer`. User code that names the two-parameter form must drop the trailing adapter arg. + - `Producer::key(&self) -> &str` is **removed**. Capture the record key at the registration site instead. + - `Producer::produce(value).await -> DbResult<()>` and `Consumer::subscribe() -> DbResult<…>` keep their signatures (bodies simplified, infallible internally β€” kept for source-level compat and backpressure headroom). + - `AimDb::producer(key)` / `AimDb::consumer(key)` now return `DbResult<…>` (was infallible). They resolve the typed record up front, so callers that previously assumed inference must add `?`. + - `Consumer` cannot exist without a buffer: `.tap()` on a record with no `.buffer(...)` now surfaces as `MissingConfiguration` at build time (was a deferred subscribe-time error). + - `TypedRecord::buffer` field is `Option>>` (was `Box`); `TypedRecord::set_buffer(Box<…>)` keeps its public signature and converts via `Arc::from(box_)` internally. + - `TypedRecord::create_producer_trait(&self)` no longer takes `db` / `record_key` β€” it uses the new `writer_handle()`. + - `ConnectorBuilder` cascade is zero-LOC: no connector struct carried `R` after M13. The outbound `consumer_factory` / inbound `producer_factory` callbacks now resolve the record once at link-startup time (via `db.inner().get_typed_record_by_key`) and construct the new handles. + - Codegen-emitted task scaffolds use `Producer` / `Consumer` (no `, TokioAdapter`). + - `data-contracts` `log_tap` parameter is `Consumer`. + - **`Spawn` trait removed; `AimDbBuilder::build()` now returns `(AimDb, AimDbRunner)` (Issue #88, Design 028).** Every future the database needs β€” `.source()`/`.tap()`/`.transform()` tasks, on_start hooks, connector loops, the remote-access supervisor β€” is collected at build time into the new `AimDbRunner`, then driven by a single `FuturesUnordered` from `runner.run().await`. No background work runs until the runner is polled. - `AimDb::spawn_task` is **deleted**. Migrate to `on_start()` (collected at build) or to a private `FuturesUnordered` inside your own future. - The `Runtime` bundle no longer supertrait-requires `Spawn`. Custom adapters drop `impl Spawn`. diff --git a/aimdb-core/README.md b/aimdb-core/README.md index 549cb6c..fa1a2fb 100644 --- a/aimdb-core/README.md +++ b/aimdb-core/README.md @@ -53,7 +53,7 @@ Portable code uses `R: Runtime` β€” no platform imports needed: ```rust /// Producer: reads a sensor and pushes typed values into AimDB. -async fn sensor_producer(ctx: RuntimeContext, producer: Producer) { +async fn sensor_producer(ctx: RuntimeContext, producer: Producer) { loop { let reading = read_sensor().await; producer.produce(Temperature { @@ -65,7 +65,7 @@ async fn sensor_producer(ctx: RuntimeContext, producer: Producer< } /// Consumer: subscribes to the buffer and reacts to every new value. -async fn temp_logger(ctx: RuntimeContext, consumer: Consumer) { +async fn temp_logger(ctx: RuntimeContext, consumer: Consumer) { let mut reader = consumer.subscribe().unwrap(); while let Ok(temp) = reader.recv().await { ctx.log().info(&format!("{}: {:.1}Β°C", temp.sensor_id, temp.celsius)); diff --git a/aimdb-core/src/buffer/mod.rs b/aimdb-core/src/buffer/mod.rs index 0c8d38c..1129a99 100644 --- a/aimdb-core/src/buffer/mod.rs +++ b/aimdb-core/src/buffer/mod.rs @@ -60,11 +60,16 @@ mod cfg; #[cfg(feature = "metrics")] mod counters; mod traits; +mod writer; // Public API exports pub use cfg::BufferCfg; pub use traits::{Buffer, BufferReader, DynBuffer}; +// Crate-private β€” used by Producer to push without per-call lookup +pub(crate) use traits::WriteHandle; +pub(crate) use writer::RecordWriter; + // JSON streaming support (std only) #[cfg(feature = "std")] pub use traits::JsonBufferReader; diff --git a/aimdb-core/src/buffer/traits.rs b/aimdb-core/src/buffer/traits.rs index 80cc90b..571ef48 100644 --- a/aimdb-core/src/buffer/traits.rs +++ b/aimdb-core/src/buffer/traits.rs @@ -93,6 +93,26 @@ pub trait DynBuffer: Send + Sync { fn reset_metrics(&self) {} } +/// Write-side handle for a single record (design 029, M14). +/// +/// `Producer` holds an `Arc>` so it can be parameterised +/// over `T` alone β€” no runtime adapter `R` and no per-call record-key string +/// lookup on the produce hot path. The implementor (`RecordWriter`) +/// pre-binds the underlying buffer, the latest-snapshot slot, and the metadata +/// tracker at build time. +/// +/// Crate-private on purpose. `Producer::new` is the only construction path; +/// external test code that needs a fake Producer should go through a future +/// `Producer::for_testing(...)` helper rather than implementing `WriteHandle` +/// directly. +pub(crate) trait WriteHandle: Send + Sync { + /// Push a value into the buffer, update the latest-snapshot cache, and + /// (when a buffer is present) mark the metadata `last_update` timestamp. + /// Infallible β€” all three operations are synchronous and lock-free or + /// spin-locked. + fn push(&self, value: T); +} + /// Reader trait for consuming values from a buffer /// /// All read operations are async. Each reader is independent with its own state. diff --git a/aimdb-core/src/buffer/writer.rs b/aimdb-core/src/buffer/writer.rs new file mode 100644 index 0000000..19e48c8 --- /dev/null +++ b/aimdb-core/src/buffer/writer.rs @@ -0,0 +1,77 @@ +//! `RecordWriter` β€” the sole implementor of `WriteHandle` (design 029). +//! +//! Pre-binds the three Arcs a `TypedRecord` already owns (buffer, +//! latest-snapshot, metadata tracker) so `Producer` can push values without +//! holding a `Arc>` or running a `HashMap` lookup per call. + +#[cfg(not(feature = "std"))] +extern crate alloc; + +#[cfg(not(feature = "std"))] +use alloc::sync::Arc; + +#[cfg(feature = "std")] +use std::sync::Arc; + +use super::traits::{DynBuffer, WriteHandle}; + +pub(crate) struct RecordWriter { + /// `None` for records that only support `latest()` (no buffer configured). + buffer: Option>>, + + /// Snapshot slot shared with `TypedRecord` and any `latest()` reader. + #[cfg(feature = "std")] + latest_snapshot: Arc>>, + #[cfg(not(feature = "std"))] + latest_snapshot: Arc>>, + + /// Metadata tracker (already `Clone` with shared inner `Arc` / + /// `Arc`). std-only. + #[cfg(feature = "std")] + metadata: crate::typed_record::RecordMetadataTracker, +} + +impl RecordWriter { + #[cfg(feature = "std")] + pub(crate) fn new( + buffer: Option>>, + latest_snapshot: Arc>>, + metadata: crate::typed_record::RecordMetadataTracker, + ) -> Self { + Self { + buffer, + latest_snapshot, + metadata, + } + } + + #[cfg(not(feature = "std"))] + pub(crate) fn new( + buffer: Option>>, + latest_snapshot: Arc>>, + ) -> Self { + Self { + buffer, + latest_snapshot, + } + } +} + +impl WriteHandle for RecordWriter { + fn push(&self, value: T) { + #[cfg(feature = "std")] + { + *self.latest_snapshot.lock().unwrap() = Some(value.clone()); + } + #[cfg(not(feature = "std"))] + { + *self.latest_snapshot.lock() = Some(value.clone()); + } + + if let Some(buf) = &self.buffer { + buf.push(value); + #[cfg(feature = "std")] + self.metadata.mark_updated(); + } + } +} diff --git a/aimdb-core/src/builder.rs b/aimdb-core/src/builder.rs index bcce237..279e221 100644 --- a/aimdb-core/src/builder.rs +++ b/aimdb-core/src/builder.rs @@ -1132,7 +1132,7 @@ impl AimDb { /// Creates a type-safe producer for a specific record by key /// - /// Returns a `Producer` bound to a specific record key. + /// Returns a `Producer` bound to a specific record key. /// /// # Arguments /// * `key` - The record key (e.g., "sensor.temperature") @@ -1150,16 +1150,20 @@ impl AimDb { pub fn producer( &self, key: impl Into, - ) -> crate::typed_api::Producer + ) -> DbResult> where T: Send + 'static + Debug + Clone, { - crate::typed_api::Producer::new(Arc::new(self.clone()), key.into()) + // Pre-resolve the typed record so the returned Producer holds a write + // handle to the record's buffer/snapshot/metadata directly + let key_str: alloc::string::String = key.into(); + let typed_rec = self.inner.get_typed_record_by_key::(&key_str)?; + Ok(crate::typed_api::Producer::new(typed_rec.writer_handle())) } /// Creates a type-safe consumer for a specific record by key /// - /// Returns a `Consumer` bound to a specific record key. + /// Returns a `Consumer` bound to a specific record key. /// /// # Arguments /// * `key` - The record key (e.g., "sensor.temperature") @@ -1176,11 +1180,29 @@ impl AimDb { pub fn consumer( &self, key: impl Into, - ) -> crate::typed_api::Consumer + ) -> DbResult> where T: Send + Sync + 'static + Debug + Clone, { - crate::typed_api::Consumer::new(Arc::new(self.clone()), key.into()) + // Pre-resolve the buffer Arc so the returned Consumer subscribes + // through a direct virtual call (design 029). A `.consumer()` without + // a configured buffer surfaces as `MissingConfiguration` here rather + // than panicking later inside `subscribe()`. + let key_str: alloc::string::String = key.into(); + let typed_rec = self.inner.get_typed_record_by_key::(&key_str)?; + let buffer = typed_rec.buffer_handle().ok_or({ + #[cfg(feature = "std")] + { + DbError::MissingConfiguration { + parameter: alloc::format!("buffer for record '{}'", key_str), + } + } + #[cfg(not(feature = "std"))] + { + DbError::MissingConfiguration { _parameter: () } + } + })?; + Ok(crate::typed_api::Consumer::new(buffer)) } /// Resolve a record key to its RecordId diff --git a/aimdb-core/src/ext_macros.rs b/aimdb-core/src/ext_macros.rs index 44c6d51..b746a2a 100644 --- a/aimdb-core/src/ext_macros.rs +++ b/aimdb-core/src/ext_macros.rs @@ -71,7 +71,7 @@ macro_rules! impl_record_registrar_ext { f: F, ) -> &'a mut $crate::RecordRegistrar<'a, T, $runtime> where - F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Producer) -> Fut + F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Producer) -> Fut + Send + Sync + 'static, @@ -83,7 +83,7 @@ macro_rules! impl_record_registrar_ext { f: F, ) -> &'a mut $crate::RecordRegistrar<'a, T, $runtime> where - F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Consumer) -> Fut + F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Consumer) -> Fut + Send + 'static, Fut: core::future::Future + Send + 'static; @@ -135,7 +135,7 @@ macro_rules! impl_record_registrar_ext { f: F, ) -> &'a mut $crate::RecordRegistrar<'a, T, $runtime> where - F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Producer) -> Fut + F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Producer) -> Fut + Send + Sync + 'static, @@ -152,7 +152,7 @@ macro_rules! impl_record_registrar_ext { f: F, ) -> &'a mut $crate::RecordRegistrar<'a, T, $runtime> where - F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Consumer) -> Fut + F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Consumer) -> Fut + Send + 'static, Fut: core::future::Future + Send + 'static, @@ -208,7 +208,7 @@ macro_rules! impl_record_registrar_ext { f: F, ) -> &'a mut $crate::RecordRegistrar<'a, T, $runtime> where - F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Producer) -> Fut + F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Producer) -> Fut + Send + Sync + 'static, @@ -220,7 +220,7 @@ macro_rules! impl_record_registrar_ext { f: F, ) -> &'a mut $crate::RecordRegistrar<'a, T, $runtime> where - F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Consumer) -> Fut + F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Consumer) -> Fut + Send + 'static, Fut: core::future::Future + Send + 'static; @@ -269,7 +269,7 @@ macro_rules! impl_record_registrar_ext { f: F, ) -> &'a mut $crate::RecordRegistrar<'a, T, $runtime> where - F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Producer) -> Fut + F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Producer) -> Fut + Send + Sync + 'static, @@ -286,7 +286,7 @@ macro_rules! impl_record_registrar_ext { f: F, ) -> &'a mut $crate::RecordRegistrar<'a, T, $runtime> where - F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Consumer) -> Fut + F: FnOnce($crate::RuntimeContext<$runtime>, $crate::Consumer) -> Fut + Send + 'static, Fut: core::future::Future + Send + 'static, diff --git a/aimdb-core/src/transform/join.rs b/aimdb-core/src/transform/join.rs index 12a3e39..9af9d49 100644 --- a/aimdb-core/src/transform/join.rs +++ b/aimdb-core/src/transform/join.rs @@ -166,8 +166,24 @@ where let factory: JoinInputFactory = Box::new( move |db: Arc>, index: usize, tx: Tx| { Box::pin(async move { - let consumer = - crate::typed_api::Consumer::::new(db, key_for_factory.clone()); + let consumer = match db.consumer::(&key_for_factory) { + Ok(c) => c, + Err(_e) => { + #[cfg(feature = "tracing")] + tracing::error!( + "πŸ”„ Join input '{}' (index {}) consumer resolution failed: {:?}", + key_for_factory, + index, + _e + ); + #[cfg(all(feature = "std", not(feature = "tracing")))] + eprintln!( + "AIMDB TRANSFORM ERROR: Join input '{}' (index {}) consumer resolution failed: {:?}", + key_for_factory, index, _e + ); + return; + } + }; let mut reader = match consumer.subscribe() { Ok(r) => r, Err(_e) => { @@ -206,7 +222,7 @@ where /// Complete the pipeline by providing an async task that owns the event loop and state. /// - /// The closure receives a [`JoinEventRx`] to read trigger events and a [`crate::Producer`] + /// The closure receives a [`JoinEventRx`] to read trigger events and a `Producer` /// to emit output values. Both are owned β€” moved into the `async move` block β€” so the /// closure can freely hold borrows across `.await` points and maintain any state it needs. /// @@ -230,7 +246,7 @@ where /// ``` pub fn on_triggers(self, handler: F) -> JoinPipeline where - F: FnOnce(JoinEventRx, crate::Producer) -> Fut + Send + 'static, + F: FnOnce(JoinEventRx, crate::Producer) -> Fut + Send + 'static, Fut: core::future::Future + Send + 'static, { let inputs = self.inputs; @@ -240,8 +256,8 @@ where JoinPipeline { spawn_factory: Box::new(move |_| TransformDescriptor { input_keys: input_keys_for_descriptor, - build_fn: Box::new(move |producer, db, runtime| { - build_join_collected(db, inputs, producer, handler, runtime) + build_fn: Box::new(move |producer, db, runtime, output_key| { + build_join_collected(db, inputs, producer, handler, runtime, output_key) }), }), } @@ -277,17 +293,23 @@ where fn build_join_collected( db: Arc>, inputs: Vec<(String, JoinInputFactory)>, - producer: crate::Producer, + producer: crate::Producer, handler: F, runtime: Arc, + output_key: &str, ) -> CollectedTransform where O: Send + Sync + Clone + Debug + 'static, R: JoinFanInRuntime + 'static, - F: FnOnce(JoinEventRx, crate::Producer) -> Fut + Send + 'static, + F: FnOnce(JoinEventRx, crate::Producer) -> Fut + Send + 'static, Fut: core::future::Future + Send + 'static, { - let output_key = producer.key().to_string(); + // Output key is threaded in from the descriptor so diagnostics stay + // unambiguous when multiple records share output type `O` (design 029). + #[cfg(feature = "tracing")] + let output_key_owned = output_key.to_string(); + #[cfg(not(feature = "tracing"))] + let _ = output_key; #[cfg(feature = "tracing")] { @@ -295,7 +317,7 @@ where tracing::info!( "πŸ”„ Join transform building: {:?} β†’ '{}'", input_keys, - output_key + output_key_owned ); } @@ -305,7 +327,7 @@ where #[cfg(feature = "tracing")] tracing::error!( "πŸ”„ Join transform '{}' FATAL: failed to create join queue", - output_key + output_key_owned ); // Empty collected transform β€” caller still receives a valid descriptor. return CollectedTransform { @@ -333,13 +355,13 @@ where #[cfg(feature = "tracing")] tracing::debug!( "βœ… Join transform '{}' handing receiver to user task", - output_key + output_key_owned ); handler(JoinEventRx::new(rx), producer).await; #[cfg(feature = "tracing")] - tracing::warn!("πŸ”„ Join transform '{}' user task exited", output_key); + tracing::warn!("πŸ”„ Join transform '{}' user task exited", output_key_owned); }); CollectedTransform { diff --git a/aimdb-core/src/transform/mod.rs b/aimdb-core/src/transform/mod.rs index b85115c..05aafe6 100644 --- a/aimdb-core/src/transform/mod.rs +++ b/aimdb-core/src/transform/mod.rs @@ -44,9 +44,20 @@ where { pub input_keys: Vec, + /// Build the transform's futures. + /// + /// Receives: + /// - `Producer` β€” the pre-resolved write handle for the output record. + /// - `Arc>` β€” used by input forwarders to resolve their input + /// consumers at startup time. + /// - `Arc` β€” the runtime adapter (e.g. for `create_join_queue`). + /// - `&str` β€” the output record's key, threaded through so the transform + /// task can include it in tracing messages even though `Producer` no + /// longer carries a `.key()` accessor (design 029, M14). Important when + /// multiple records share type `T` under different keys. #[allow(clippy::type_complexity)] pub build_fn: Box< - dyn FnOnce(crate::Producer, Arc>, Arc) -> CollectedTransform + dyn FnOnce(crate::Producer, Arc>, Arc, &str) -> CollectedTransform + Send, >, } diff --git a/aimdb-core/src/transform/single.rs b/aimdb-core/src/transform/single.rs index 9436d99..a9e85ee 100644 --- a/aimdb-core/src/transform/single.rs +++ b/aimdb-core/src/transform/single.rs @@ -132,16 +132,19 @@ where TransformDescriptor { input_keys, - build_fn: Box::new(move |producer, db, _runtime| CollectedTransform { - task_future: Box::pin(run_single_transform::( - db, - input_key_clone, - producer, - initial_state, - transform_fn, - )), - fanin_futures: alloc::vec::Vec::new(), - }), + build_fn: Box::new( + move |producer, db, _runtime, output_key| CollectedTransform { + task_future: Box::pin(run_single_transform::( + db, + input_key_clone, + output_key.to_string(), + producer, + initial_state, + transform_fn, + )), + fanin_futures: alloc::vec::Vec::new(), + }, + ), } } @@ -153,7 +156,8 @@ where pub(crate) async fn run_single_transform( db: Arc>, input_key: String, - producer: crate::Producer, + output_key: String, + producer: crate::Producer, mut state: S, transform_fn: impl Fn(&I, &mut S) -> Option + Send + Sync + 'static, ) where @@ -162,12 +166,30 @@ pub(crate) async fn run_single_transform( S: Send + 'static, R: aimdb_executor::RuntimeAdapter + 'static, { - let output_key = producer.key().to_string(); - + // `Producer` no longer exposes a `.key()` accessor (design 029) β€” the + // output key is threaded in by the transform descriptor so diagnostics + // remain unambiguous when multiple records share type `O`. #[cfg(feature = "tracing")] tracing::info!("πŸ”„ Transform started: '{}' β†’ '{}'", input_key, output_key); - let consumer = crate::typed_api::Consumer::::new(db, input_key.clone()); + let consumer = match db.consumer::(&input_key) { + Ok(c) => c, + Err(_e) => { + #[cfg(feature = "tracing")] + tracing::error!( + "πŸ”„ Transform '{}' β†’ '{}' FATAL: failed to resolve consumer: {:?}", + input_key, + output_key, + _e + ); + #[cfg(all(feature = "std", not(feature = "tracing")))] + eprintln!( + "AIMDB TRANSFORM ERROR: '{}' β†’ '{}' failed to resolve consumer: {:?}", + input_key, output_key, _e + ); + return; + } + }; let mut reader = match consumer.subscribe() { Ok(r) => r, Err(_e) => { diff --git a/aimdb-core/src/typed_api.rs b/aimdb-core/src/typed_api.rs index 3e70f05..4a6fce0 100644 --- a/aimdb-core/src/typed_api.rs +++ b/aimdb-core/src/typed_api.rs @@ -1,8 +1,8 @@ //! Type-safe Producer-Consumer API //! //! Provides the complete typed API for producer-consumer patterns including: -//! - `Producer` - Type-safe value production -//! - `Consumer` - Type-safe value consumption +//! - `Producer` - Type-safe value production +//! - `Consumer` - Type-safe value consumption //! - `RecordRegistrar` - Fluent record configuration API //! - `RecordT` trait - Self-registering records //! @@ -12,7 +12,7 @@ //! #[service] //! async fn temperature_producer( //! ctx: RuntimeContext, -//! producer: Producer, +//! producer: Producer, //! ) { //! loop { //! let temp = read_sensor().await; @@ -28,7 +28,7 @@ //! #[service] //! async fn temperature_monitor( //! ctx: RuntimeContext, -//! consumer: Consumer, +//! consumer: Consumer, //! ) { //! let mut rx = consumer.subscribe()?; //! while let Ok(temp) = rx.recv().await { @@ -64,6 +64,7 @@ use alloc::{ vec::Vec, }; +use crate::buffer::{DynBuffer, WriteHandle}; use crate::typed_record::TypedRecord; use crate::{AimDb, DbResult}; @@ -73,13 +74,17 @@ use crate::{AimDb, DbResult}; /// Type-safe producer for a specific record type /// -/// `Producer` provides scoped access to produce values of type `T` only. +/// `Producer` provides scoped access to produce values of type `T` only. /// This follows the principle of least privilege - services only get access /// to what they need, not the entire database. /// /// # Type Parameters /// * `T` - The record type this producer can emit -/// * `R` - The runtime adapter type (e.g., TokioAdapter, EmbassyAdapter) +/// +/// Pre-binds the record's buffer, latest-snapshot slot, and metadata tracker +/// (via an internal `Arc>`), so `produce()` is a single +/// virtual call rather than a `HashMap` lookup + downcast on each invocation +/// (design 029). /// /// # Benefits /// @@ -88,11 +93,9 @@ use crate::{AimDb, DbResult}; /// - **Clear Intent**: Function signature shows what it produces /// - **Decoupling**: No access to other record types /// - **Security**: Cannot misuse database for unintended operations -pub struct Producer { - /// Reference to the database - db: Arc>, - /// Record key for key-based routing (required - all records have keys) - record_key: String, +pub struct Producer { + /// Pre-resolved write handle to the record's buffer/snapshot/metadata. + write: Arc>, /// Stage profiling state (set by the spawn machinery for `.source()` stages). #[cfg(feature = "profiling")] profiling: Option>, @@ -101,16 +104,14 @@ pub struct Producer { _phantom: PhantomData T>, } -impl Producer +impl Producer where T: Send + 'static + Debug + Clone, - R: aimdb_executor::RuntimeAdapter + 'static, { - /// Create a new producer bound to a specific record key - pub(crate) fn new(db: Arc>, key: String) -> Self { + /// Create a new producer bound to a pre-resolved write handle. + pub(crate) fn new(write: Arc>) -> Self { Self { - db, - record_key: key, + write, #[cfg(feature = "profiling")] profiling: None, _phantom: PhantomData, @@ -135,28 +136,25 @@ where /// 1. All tap observers are notified /// 2. All link connectors are triggered /// 3. Buffers are updated (if configured) + /// + /// Signature is kept `async fn ... -> DbResult<()>` for source-level + /// compatibility with existing `producer.produce(x).await?` call sites and + /// to leave headroom for future backpressure-aware buffers β€” even though + /// the body is currently synchronous and infallible. pub async fn produce(&self, value: T) -> DbResult<()> { #[cfg(feature = "profiling")] if let Some(state) = &self.profiling { state.record_produce(); } - self.db.produce::(&self.record_key, value).await - } - - /// Returns the key this producer is bound to - pub fn key(&self) -> &str { - &self.record_key + self.write.push(value); + Ok(()) } } -impl Clone for Producer -where - R: aimdb_executor::RuntimeAdapter + 'static, -{ +impl Clone for Producer { fn clone(&self) -> Self { Self { - db: self.db.clone(), - record_key: self.record_key.clone(), + write: self.write.clone(), #[cfg(feature = "profiling")] profiling: self.profiling.clone(), _phantom: PhantomData, @@ -165,10 +163,9 @@ where } // Implement ProducerTrait for type-erased routing -impl crate::connector::ProducerTrait for Producer +impl crate::connector::ProducerTrait for Producer where T: Send + 'static + Debug + Clone, - R: aimdb_executor::RuntimeAdapter + 'static, { fn produce_any<'a>( &'a self, @@ -197,13 +194,12 @@ where /// Type-safe consumer for a specific record type /// -/// `Consumer` provides scoped access to subscribe to values of type `T` only. +/// `Consumer` provides scoped access to subscribe to values of type `T` only. /// This follows the principle of least privilege - services only get access /// to what they need, not the entire database. /// /// # Type Parameters /// * `T` - The record type this consumer can subscribe to -/// * `R` - The runtime adapter type (e.g., TokioAdapter, EmbassyAdapter) /// /// # Benefits /// @@ -212,29 +208,24 @@ where /// - **Clear Intent**: Function signature shows what it consumes /// - **Decoupling**: No access to other record types /// - **Security**: Cannot misuse database for unintended operations -#[derive(Clone)] -pub struct Consumer { - /// Reference to the database - db: Arc>, - /// Record key for key-based routing (required - all records have keys) - record_key: String, +pub struct Consumer { + /// Pre-resolved buffer handle to the record's buffer. + buffer: Arc>, /// Stage profiling state (set by the spawn machinery for `.tap()` / `.link()`). #[cfg(feature = "profiling")] profiling: Option<(Arc, crate::profiling::Clock)>, - // See Producer: `fn() -> T` keeps Send/Sync independent of T. + // See Producer: `fn() -> T` keeps Send/Sync independent of T. _phantom: PhantomData T>, } -impl Consumer +impl Consumer where T: Send + Sync + 'static + Debug + Clone, - R: aimdb_executor::RuntimeAdapter + 'static, { - /// Create a new consumer bound to a specific record key - pub(crate) fn new(db: Arc>, key: String) -> Self { + /// Create a new consumer bound to a pre-resolved buffer handle. + pub(crate) fn new(buffer: Arc>) -> Self { Self { - db, - record_key: key, + buffer, #[cfg(feature = "profiling")] profiling: None, _phantom: PhantomData, @@ -254,8 +245,11 @@ where /// Subscribe to updates for this record type /// /// Returns a reader that yields values when they are produced. + /// + /// Signature is kept as `DbResult<…>` for source-level compatibility; the + /// body is now infallible (the buffer is pre-resolved at construction). pub fn subscribe(&self) -> DbResult + Send>> { - let reader = self.db.subscribe::(&self.record_key)?; + let reader = self.buffer.subscribe_boxed(); #[cfg(feature = "profiling")] if let Some((metrics, clock)) = &self.profiling { return Ok(Box::new(crate::profiling::ProfilingBufferReader::new( @@ -266,10 +260,16 @@ where } Ok(reader) } +} - /// Returns the key this consumer is bound to - pub fn key(&self) -> &str { - &self.record_key +impl Clone for Consumer { + fn clone(&self) -> Self { + Self { + buffer: self.buffer.clone(), + #[cfg(feature = "profiling")] + profiling: self.profiling.clone(), + _phantom: PhantomData, + } } } @@ -301,10 +301,9 @@ impl crate::connector::AnyReader for TypedAnyReader crate::connector::ConsumerTrait for Consumer +impl crate::connector::ConsumerTrait for Consumer where T: Send + Sync + 'static + Debug + Clone, - R: aimdb_executor::RuntimeAdapter + 'static, { fn subscribe_any<'a>( &'a self, @@ -411,7 +410,7 @@ where /// - Advanced use cases requiring direct control pub fn source_raw(&'a mut self, f: F) -> &'a mut Self where - F: FnOnce(crate::Producer, Arc) -> Fut + F: FnOnce(crate::Producer, Arc) -> Fut + Send + Sync + 'static, @@ -443,7 +442,7 @@ where /// - Advanced use cases requiring direct control pub fn tap_raw(&'a mut self, f: F) -> &'a mut Self where - F: FnOnce(crate::Consumer, Arc) -> Fut + F: FnOnce(crate::Consumer, Arc) -> Fut + Send + 'static, Fut: Future + Send + 'static, @@ -858,22 +857,38 @@ where } // Store consumer factory that captures type T and record key - // This allows the connector to subscribe to values without knowing T at compile time + // This allows the connector to subscribe to values without knowing T at compile time. + // + // Resolves the record at link-startup time (not per-message) and constructs a + // `Consumer` bound to a pre-resolved buffer handle β€” same pattern as the + // build-time path in `TypedRecord::collect_consumer_futures` (design 029). { let record_key = self.registrar.record_key.clone(); link.consumer_factory = Some(Arc::new( move |db_any: Arc| { - // Downcast Arc to AimDb, then wrap in Arc let db_ref = db_any .downcast_ref::>() .expect("Invalid db type in consumer factory"); - let db = Arc::new(db_ref.clone()); + let typed_rec = db_ref + .inner() + .get_typed_record_by_key::(&record_key) + .unwrap_or_else(|e| { + panic!( + "outbound connector consumer factory: record '{}' lookup failed: {:?}", + record_key, e + ) + }); + let buffer = typed_rec.buffer_handle().unwrap_or_else(|| { + panic!( + "outbound connector for '{}' requires a buffer (call .buffer(...) before .link_to(...))", + record_key + ) + }); - // Create Consumer with captured type T and record key #[allow(unused_mut)] - let mut consumer = Consumer::::new(db.clone(), record_key.clone()); + let mut consumer = Consumer::::new(buffer); #[cfg(feature = "profiling")] - consumer.set_profiling(link_metrics.clone(), db.profiling_clock().clone()); + consumer.set_profiling(link_metrics.clone(), db_ref.profiling_clock().clone()); Box::new(consumer) as Box }, )); @@ -1096,16 +1111,23 @@ where // Wire through the topic resolver link.topic_resolver = self.topic_resolver; - // Add producer factory callback that captures type T and record key + // Add producer factory callback that captures type T and record key. { let record_key = self.registrar.record_key.clone(); link = link.with_producer_factory(move |db_any| { - // Downcast Arc to Arc> let db = db_any .downcast::>() .expect("Failed to downcast to AimDb"); - // Create Producer with captured type T and record key - Box::new(Producer::::new(db, record_key.clone())) + let typed_rec = db + .inner() + .get_typed_record_by_key::(&record_key) + .unwrap_or_else(|e| { + panic!( + "inbound connector producer factory: record '{}' lookup failed: {:?}", + record_key, e + ) + }); + Box::new(Producer::::new(typed_rec.writer_handle())) as Box }); } @@ -1593,10 +1615,12 @@ mod tests { { crate::transform::TransformDescriptor:: { input_keys: vec![], - build_fn: Box::new(|_p, _db, _ctx| crate::transform::CollectedTransform { - task_future: Box::pin(async {}), - fanin_futures: vec![], - }), + build_fn: Box::new( + |_p, _db, _ctx, _output_key| crate::transform::CollectedTransform { + task_future: Box::pin(async {}), + fanin_futures: vec![], + }, + ), } } diff --git a/aimdb-core/src/typed_record.rs b/aimdb-core/src/typed_record.rs index 7613c4e..db0c215 100644 --- a/aimdb-core/src/typed_record.rs +++ b/aimdb-core/src/typed_record.rs @@ -159,7 +159,7 @@ impl crate::buffer::JsonBufferReader for JsonReaderAd /// Metadata tracking for records (std only - used for remote access introspection) #[cfg(feature = "std")] #[derive(Debug, Clone)] -struct RecordMetadataTracker { +pub(crate) struct RecordMetadataTracker { /// Human-readable record name (type name) name: String, /// Creation timestamp (seconds, nanoseconds since UNIX_EPOCH) @@ -187,7 +187,7 @@ impl RecordMetadataTracker { } } - fn mark_updated(&self) { + pub(crate) fn mark_updated(&self) { use std::time::SystemTime; let duration = SystemTime::now() @@ -215,17 +215,17 @@ pub(crate) type BoxFuture<'a, T> = core::pin::Pin + Send + 'a>>; /// Type alias for consumer service closure stored in TypedRecord -/// Each consumer receives a Consumer handle for subscribing to the buffer +/// Each consumer receives a Consumer handle for subscribing to the buffer /// and a runtime context (Arc) for accessing runtime capabilities -type ConsumerServiceFn = Box< - dyn FnOnce(crate::Consumer, Arc) -> BoxFuture<'static, ()> + Send, +type ConsumerServiceFn = Box< + dyn FnOnce(crate::Consumer, Arc) -> BoxFuture<'static, ()> + Send, >; /// Type alias for producer service closure stored in TypedRecord -/// Takes (Producer, RuntimeContext) and returns a Future +/// Takes (Producer, RuntimeContext) and returns a Future /// This will be auto-spawned during build() -type ProducerServiceFn = Box< - dyn FnOnce(crate::Producer, Arc) -> BoxFuture<'static, ()> +type ProducerServiceFn = Box< + dyn FnOnce(crate::Producer, Arc) -> BoxFuture<'static, ()> + Send + Sync, >; @@ -532,22 +532,22 @@ pub struct TypedRecord< > { /// Optional producer service - a task that generates data /// This will be auto-spawned during build() if present - /// Stored as FnOnce that takes (Producer, RuntimeContext) and returns a Future + /// Stored as FnOnce that takes (Producer, RuntimeContext) and returns a Future /// Wrapped in Mutex for interior mutability (needed to take() during spawning) #[cfg(feature = "std")] - producer_service: std::sync::Mutex>>, + producer_service: std::sync::Mutex>>, #[cfg(not(feature = "std"))] - producer_service: spin::Mutex>>, + producer_service: spin::Mutex>>, /// List of consumer/tap tasks - wrapped in Mutex for Sync + taking out during spawn /// Each is spawned as an independent background task that subscribes to the buffer /// Using Mutex provides the Sync bound required by AnyRecord trait #[cfg(feature = "std")] - consumers: std::sync::Mutex>>, + consumers: std::sync::Mutex>>, #[cfg(not(feature = "std"))] - consumers: spin::Mutex>>, + consumers: spin::Mutex>>, /// Transform descriptor β€” mutually exclusive with producer_service. /// If set, this record is a reactive derivation from one or more input records. @@ -559,7 +559,10 @@ pub struct TypedRecord< /// Optional buffer for async dispatch /// When present, produce() enqueues to buffer instead of direct call - buffer: Option>>, + /// + /// Stored as `Arc` (not `Box`) so `Producer` and `Consumer` can hold + /// a pre-resolved handle to the same buffer β€” design 029 hot-path change. + buffer: Option>>, /// Buffer configuration (cached for metadata, std only) #[cfg(feature = "std")] @@ -669,7 +672,7 @@ impl(&mut self, f: F) where - F: FnOnce(crate::Producer, Arc) -> Fut + Send + Sync + 'static, + F: FnOnce(crate::Producer, Arc) -> Fut + Send + Sync + 'static, Fut: core::future::Future + Send + 'static, { // Check for existing transform (mutual exclusion) @@ -698,7 +701,7 @@ impl, + move |producer: crate::Producer, ctx: Arc| -> BoxFuture<'static, ()> { Box::pin(f(producer, ctx)) }, ); @@ -735,12 +738,12 @@ impl(&mut self, f: F) where - F: FnOnce(crate::Consumer, Arc) -> Fut + Send + 'static, + F: FnOnce(crate::Consumer, Arc) -> Fut + Send + 'static, Fut: core::future::Future + Send + 'static, { // Box the future to make it trait object compatible let boxed = Box::new( - move |consumer: crate::Consumer, + move |consumer: crate::Consumer, ctx_any: Arc| -> BoxFuture<'static, ()> { Box::pin(f(consumer, ctx_any)) }, ); @@ -918,10 +921,10 @@ impl bound to the specific record key - let producer = crate::typed_api::Producer::new(db.clone(), record_key.to_string()); + // Create Producer bound to a pre-resolved write handle for this record. + let producer = crate::typed_api::Producer::new(self.writer_handle()); - let collected = (desc.build_fn)(producer, db.clone(), runtime.clone()); + let collected = (desc.build_fn)(producer, db.clone(), runtime.clone(), record_key); let mut out = Vec::with_capacity(1 + collected.fanin_futures.len()); out.push(collected.task_future); @@ -943,7 +946,11 @@ impl)` reuses the existing heap allocation; the + // public API stays Box-flavoured to avoid churn at adapter / test call + // sites, while internally we share via Arc so producers/consumers can + // hold a pre-resolved handle. + self.buffer = Some(Arc::from(buffer)); } /// Sets the buffer configuration (for metadata tracking, std only) @@ -993,6 +1000,36 @@ impl>` bound to this record's buffer, + /// snapshot, and metadata. Used at build time by the spawn machinery to + /// pre-resolve `Producer` handles (design 029). + pub(crate) fn writer_handle(&self) -> Arc> + where + T: Send + Clone + 'static, + { + #[cfg(feature = "std")] + { + Arc::new(crate::buffer::RecordWriter::new( + self.buffer.clone(), + self.latest_snapshot.clone(), + self.metadata.clone(), + )) + } + #[cfg(not(feature = "std"))] + { + Arc::new(crate::buffer::RecordWriter::new( + self.buffer.clone(), + self.latest_snapshot.clone(), + )) + } + } + + /// Returns a clone of the buffer `Arc` (or `None` if no buffer is + /// configured). Used at build time to pre-resolve `Consumer` handles. + pub(crate) fn buffer_handle(&self) -> Option>> { + self.buffer.clone() + } + /// Subscribes to the buffer for this record type /// /// # Errors @@ -1167,6 +1204,14 @@ impl handle bound to the specific record key + // Create a Consumer bound to a pre-resolved buffer handle. #[allow(unused_mut)] - let mut consumer = crate::typed_api::Consumer::new(db.clone(), record_key.to_string()); + let mut consumer = crate::typed_api::Consumer::new(buffer_arc.clone()); #[cfg(feature = "profiling")] if let Some(entry) = self.profiling.tap(i) { @@ -1225,6 +1289,12 @@ impl() ); + #[cfg(not(feature = "tracing"))] + let _ = record_key; - // Create Producer bound to the specific record key + // Create Producer bound to a pre-resolved write handle for this record. #[allow(unused_mut)] - let mut producer = crate::typed_api::Producer::new(db.clone(), record_key.to_string()); + let mut producer = crate::typed_api::Producer::new(self.writer_handle()); #[cfg(feature = "profiling")] if let Some(entry) = self.profiling.source(0) { @@ -1344,19 +1417,14 @@ impl that can be used for routing + /// Backed by a pre-resolved `WriteHandle` (design 029) β€” no db / key lookup + /// is performed on each `produce_any` call. #[cfg(feature = "std")] - pub fn create_producer_trait( - &self, - db: Arc>, - record_key: String, - ) -> Box { - Box::new(crate::typed_api::Producer::::new(db, record_key)) + pub fn create_producer_trait(&self) -> Box + where + T: Send + 'static + Debug + Clone, + { + Box::new(crate::typed_api::Producer::::new(self.writer_handle())) } } diff --git a/aimdb-data-contracts/CHANGELOG.md b/aimdb-data-contracts/CHANGELOG.md index 74ecf79..da9e967 100644 --- a/aimdb-data-contracts/CHANGELOG.md +++ b/aimdb-data-contracts/CHANGELOG.md @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed (breaking) + +- `log_tap(ctx, consumer, node_id)` now takes `Consumer` instead of `Consumer` β€” `R` is still on `RuntimeContext` (Design 029, M14). User-visible signature shrink only; behaviour unchanged. + ## [0.1.1] - 2026-05-22 ### Changed diff --git a/aimdb-data-contracts/src/observable.rs b/aimdb-data-contracts/src/observable.rs index 501fc7b..53ac5c5 100644 --- a/aimdb-data-contracts/src/observable.rs +++ b/aimdb-data-contracts/src/observable.rs @@ -30,7 +30,7 @@ use crate::Observable; #[cfg(feature = "observable")] pub async fn log_tap( ctx: aimdb_core::RuntimeContext, - consumer: aimdb_core::typed_api::Consumer, + consumer: aimdb_core::typed_api::Consumer, node_id: &'static str, ) where T: Observable + Send + Sync + Clone + core::fmt::Debug + 'static, diff --git a/aimdb-embassy-adapter/CHANGELOG.md b/aimdb-embassy-adapter/CHANGELOG.md index 0c56eba..c107d2e 100644 --- a/aimdb-embassy-adapter/CHANGELOG.md +++ b/aimdb-embassy-adapter/CHANGELOG.md @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed (breaking) + +- **Generated extension trait emits `Producer` / `Consumer`** (no `, EmbassyAdapter`) via the updated `impl_record_registrar_ext!` macro from `aimdb-core` (Design 029, M14). Embassy demo signatures collapse from `Producer` to `Producer`. + ### Removed (breaking) - **`impl Spawn for EmbassyAdapter` deleted (Issue #88).** Static `generic_task_runner` task pool gone, along with `BoxedFuture` and the `unsafe Pin::new_unchecked` cast that fed the pool. Drive database futures by awaiting `AimDbRunner::run()` from inside the Embassy main task. diff --git a/aimdb-embassy-adapter/src/lib.rs b/aimdb-embassy-adapter/src/lib.rs index 5dbcbee..6aeae77 100644 --- a/aimdb-embassy-adapter/src/lib.rs +++ b/aimdb-embassy-adapter/src/lib.rs @@ -279,7 +279,7 @@ where /// /// async fn button_handler( /// ctx: RuntimeContext, - /// producer: Producer, + /// producer: Producer, /// button: ExtiInput<'static>, /// ) { /// // Use the button peripheral @@ -294,11 +294,7 @@ where ) -> &'a mut aimdb_core::RecordRegistrar<'a, T, EmbassyAdapter> where Ctx: Send + Sync + 'static, - F: FnOnce( - aimdb_core::RuntimeContext, - aimdb_core::Producer, - Ctx, - ) -> Fut + F: FnOnce(aimdb_core::RuntimeContext, aimdb_core::Producer, Ctx) -> Fut + Send + Sync + 'static, @@ -347,11 +343,7 @@ where ) -> &'a mut aimdb_core::RecordRegistrar<'a, T, EmbassyAdapter> where Ctx: Send + Sync + 'static, - F: FnOnce( - aimdb_core::RuntimeContext, - aimdb_core::Producer, - Ctx, - ) -> Fut + F: FnOnce(aimdb_core::RuntimeContext, aimdb_core::Producer, Ctx) -> Fut + Send + Sync + 'static, diff --git a/aimdb-knx-connector/tests/topic_provider_tests.rs b/aimdb-knx-connector/tests/topic_provider_tests.rs index 9f1d878..cc7792e 100644 --- a/aimdb-knx-connector/tests/topic_provider_tests.rs +++ b/aimdb-knx-connector/tests/topic_provider_tests.rs @@ -293,8 +293,7 @@ async fn test_knx_topic_provider_registration_api() { builder.configure::("knx.dimmer.living", |reg| { let counter = produced_count_clone.clone(); reg.buffer(BufferCfg::SingleLatest).source( - move |_ctx: RuntimeContext, - producer: Producer| { + move |_ctx: RuntimeContext, producer: Producer| { let counter = counter.clone(); async move { let dimmer = DimmerValue::new("living", 200); @@ -325,8 +324,7 @@ async fn test_knx_topic_provider_with_connector_registration() { builder.configure::("knx.dimmer.living", |reg| { reg.buffer(BufferCfg::SingleLatest) .source( - |_ctx: RuntimeContext, - producer: Producer| async move { + |_ctx: RuntimeContext, producer: Producer| async move { let dimmer = DimmerValue::new("living", 200); producer.produce(dimmer).await.ok(); }, @@ -383,7 +381,7 @@ async fn test_hvac_zone_routing() { reg.buffer(BufferCfg::SingleLatest) .source( |_ctx: RuntimeContext, - producer: Producer| async move { + producer: Producer| async move { // Different zones get routed to different group addresses for zone in 1..=4 { let setpoint = TemperatureSetpoint::new(zone, 21.0 + zone as f32 * 0.5); diff --git a/aimdb-mqtt-connector/tests/topic_provider_tests.rs b/aimdb-mqtt-connector/tests/topic_provider_tests.rs index 5377758..8148a4b 100644 --- a/aimdb-mqtt-connector/tests/topic_provider_tests.rs +++ b/aimdb-mqtt-connector/tests/topic_provider_tests.rs @@ -234,8 +234,7 @@ async fn test_topic_provider_registration_api() { builder.configure::("test.sensor.dynamic", |reg| { let counter = produced_count_clone.clone(); reg.buffer(BufferCfg::SingleLatest).source( - move |_ctx: RuntimeContext, - producer: Producer| { + move |_ctx: RuntimeContext, producer: Producer| { let counter = counter.clone(); async move { let temp = Temperature::new("test-001", 22.5); @@ -268,8 +267,7 @@ async fn test_topic_provider_with_connector_registration() { builder.configure::("test.sensor.dynamic", |reg| { reg.buffer(BufferCfg::SingleLatest) .source( - |_ctx: RuntimeContext, - producer: Producer| async move { + |_ctx: RuntimeContext, producer: Producer| async move { let temp = Temperature::new("kitchen", 22.5); producer.produce(temp).await.ok(); }, @@ -329,8 +327,7 @@ async fn test_mixed_static_and_dynamic_topics() { builder.configure::("test.sensor.static", |reg| { reg.buffer(BufferCfg::SingleLatest) .source( - |_ctx: RuntimeContext, - producer: Producer| async move { + |_ctx: RuntimeContext, producer: Producer| async move { producer .produce(Temperature::new("static", 20.0)) .await @@ -346,8 +343,7 @@ async fn test_mixed_static_and_dynamic_topics() { builder.configure::("test.sensor.dynamic", |reg| { reg.buffer(BufferCfg::SingleLatest) .source( - |_ctx: RuntimeContext, - producer: Producer| async move { + |_ctx: RuntimeContext, producer: Producer| async move { producer .produce(Temperature::new("dynamic", 25.0)) .await diff --git a/aimdb-tokio-adapter/CHANGELOG.md b/aimdb-tokio-adapter/CHANGELOG.md index efdcc3e..e7ead46 100644 --- a/aimdb-tokio-adapter/CHANGELOG.md +++ b/aimdb-tokio-adapter/CHANGELOG.md @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed (breaking) + +- **Generated extension trait emits `Producer` / `Consumer`** (no `, TokioAdapter`) via the updated `impl_record_registrar_ext!` macro from `aimdb-core` (Design 029, M14). User-side `.source(|ctx, producer| ...)` / `.tap(|ctx, consumer| ...)` callbacks now receive the simpler types. + ### Removed (breaking) - **`impl Spawn for TokioAdapter` deleted (Issue #88).** Adapter is now `RuntimeAdapter + TimeOps + Logger` only. Drive database futures by awaiting `AimDbRunner::run()` returned from `AimDbBuilder::build()`. diff --git a/aimdb-tokio-adapter/tests/multi_instance_tests.rs b/aimdb-tokio-adapter/tests/multi_instance_tests.rs index 97234b4..27f92b3 100644 --- a/aimdb-tokio-adapter/tests/multi_instance_tests.rs +++ b/aimdb-tokio-adapter/tests/multi_instance_tests.rs @@ -80,13 +80,10 @@ async fn test_producer() { let (db, runner) = builder.build().await.unwrap(); tokio::spawn(runner.run()); - // Get key-bound producers - let producer_a = db.producer::("sensor.a"); - let producer_b = db.producer::("sensor.b"); - - // Verify keys are bound correctly - assert_eq!(producer_a.key(), "sensor.a"); - assert_eq!(producer_b.key(), "sensor.b"); + // Get key-bound producers. Post-M14 these resolve the typed record up front + // (Producer no longer carries `R` and no longer exposes `.key()`). + let producer_a = db.producer::("sensor.a").unwrap(); + let producer_b = db.producer::("sensor.b").unwrap(); // Produce values producer_a @@ -117,13 +114,10 @@ async fn test_consumer() { let (db, runner) = builder.build().await.unwrap(); tokio::spawn(runner.run()); - // Get key-bound consumers - let consumer_north = db.consumer::("zone.north"); - let consumer_south = db.consumer::("zone.south"); - - // Verify keys are bound correctly - assert_eq!(consumer_north.key(), "zone.north"); - assert_eq!(consumer_south.key(), "zone.south"); + // Get key-bound consumers. Post-M14 these resolve the typed record up front + // (Consumer no longer carries `R` and no longer exposes `.key()`). + let consumer_north = db.consumer::("zone.north").unwrap(); + let consumer_south = db.consumer::("zone.south").unwrap(); // Subscribe should work let _reader_north = consumer_north.subscribe().unwrap(); diff --git a/docs/aimdb-usage-guide.md b/docs/aimdb-usage-guide.md index a2d43a5..6e2f569 100644 --- a/docs/aimdb-usage-guide.md +++ b/docs/aimdb-usage-guide.md @@ -200,7 +200,7 @@ struct ButtonPress { /// Button handler that produces events on button press async fn button_handler( ctx: RuntimeContext, - producer: Producer, + producer: Producer, mut button: Input<'static>, ) { let log = ctx.log(); diff --git a/docs/design/029-M14-remove-r-from-typed-handles.md b/docs/design/029-M14-remove-r-from-typed-handles.md new file mode 100644 index 0000000..cf43581 --- /dev/null +++ b/docs/design/029-M14-remove-r-from-typed-handles.md @@ -0,0 +1,554 @@ +# Remove `R` from `Producer` and `Consumer` + +**Version:** 0.3 (implemented) +**Status:** βœ… Implemented +**Issue:** TBD +**Depends on:** [M13 β€” Remove `Spawn` Trait](028-M13-remove-spawn-trait.md) (#88) +**Last Updated:** May 25, 2026 +**Milestone:** M14 β€” Architectural clean-up + +--- + +## Table of Contents + +- [Summary](#summary) +- [Motivation](#motivation) + - [The actual headline: hot-path lookup elimination](#the-actual-headline-hot-path-lookup-elimination) + - [Cosmetic win: simpler user-facing signatures](#cosmetic-win-simpler-user-facing-signatures) + - [What this is *not*](#what-this-is-not) +- [Current Architecture](#current-architecture) + - [Why `R` is present today](#why-r-is-present-today) + - [What does NOT need `R` at runtime](#what-does-not-need-r-at-runtime) + - [The full `R`-propagation surface](#the-full-r-propagation-surface) +- [Proposed Design](#proposed-design) + - [New `WriteHandle` indirection](#new-writehandlet-indirection) + - [`RecordWriter` construction (Option B)](#recordwritert-construction-option-b) + - [`Producer` β€” after](#producert--after) + - [`Consumer` β€” after](#consumert--after) + - [`ConnectorBuilder` β€” cascade (zero-LOC)](#connectorbuilder--cascade-zero-loc) +- [Type-System Changes](#type-system-changes) +- [Breaking Changes](#breaking-changes) +- [Implementation Plan](#implementation-plan) +- [Decisions](#decisions) +- [Out of Scope](#out-of-scope) + +--- + +## Summary + +Replace `Producer`'s `Arc>` field (and the string-keyed lookup +it forces on every `produce()`) with a pre-resolved `Arc>` +that points directly at the record's buffer + latest-snapshot + metadata. +Replace `Consumer`'s same `Arc>` with a pre-resolved +`Arc>`. The runtime parameter `R` disappears from both +public types. + +The substantive change is **on the hot path**: today every `producer.produce(value).await` +does a `HashMap` lookup, a bounds check, a `TypeId` +equality check, and a `dyn AnyRecord β†’ &TypedRecord` downcast before +the actual buffer push. After M14 it does one virtual call. The R-removal in +type signatures is the bonus, not the headline. + +`ConnectorBuilder` keeps its trait-level `R` parameter (Option A in +v0.1) β€” but the cascade is in practice zero-LOC because no connector struct +in the tree carries an `R` phantom today (M13 already cleaned them). + +--- + +## Motivation + +### The actual headline: hot-path lookup elimination + +Every `producer.produce(value).await` call today follows this path: + +```rust +// Producer::produce +self.db.produce::(&self.record_key, value).await + // β†’ AimDb::produce + β†’ self.inner.get_typed_record_by_key::(key)? + // HashMap lookup on key + // bounds check on id.index() < self.storages.len() + // TypeId::of::() == self.types[id.index()] equality check + // dyn AnyRecord β†’ &TypedRecord downcast + β†’ typed_rec.produce(value).await + // latest_snapshot.lock() = Some(value.clone()) + // buf.push(value) + // metadata.mark_updated() +``` + +That is one HashMap probe, two equality checks, and a downcast on **every +single produce** β€” regardless of where the call originates, because the +producer carries the record key as a `String` and has to resolve it each +time. For a sensor producing at 1 kHz, or a transform on a busy stream, +those operations dominate the actual buffer push. + +After M14 the producer holds a pre-resolved `Arc>` that +points straight at the buffer + snapshot + metadata for one specific record. +The same call becomes: + +```rust +// Producer::produce +self.write.push(value) // ← one virtual call, then the real work +``` + +Same story on the consume side: today every `consumer.subscribe()` does the +same key-lookup before calling `buffer.subscribe_boxed()`. After M14 the +buffer Arc is pre-resolved at build time β€” `subscribe()` collapses to one +virtual call. + +This is the substantive win and the reason to do M14. Pre-resolved at build, +not per-call: Producer/Consumer become "handles to a buffer", not "tickets to +look up a buffer". + +### Cosmetic win: simpler user-facing signatures + +After the hot-path change, `R` falls out of `Producer` / `Consumer` +naturally β€” the `Arc>` field is gone. That makes user-written +`.source()` / `.tap()` helper signatures shorter: + +```rust +// before +async fn produce_temperature( + ctx: RuntimeContext, + temperature: Producer, +) { ... } + +// after +async fn produce_temperature( + ctx: RuntimeContext, + temperature: Producer, +) { ... } +``` + +Codebase audit (M14-uncommitted): ~18 such occurrences across `examples/` +and `aimdb-pro/`. Shared helpers in `examples/{mqtt,knx}-connector-demo-common/` +become slightly less generic-soup too: + +```rust +// before +pub async fn light_monitor(ctx: RuntimeContext, consumer: Consumer) +// after +pub async fn light_monitor(ctx: RuntimeContext, consumer: Consumer) +``` + +`R` remains on `RuntimeContext` because it exposes `ctx.time()` / `ctx.log()` +which need runtime-specific types β€” see *Out of Scope*. + +### What this is *not* + +This refactor will not significantly reduce LOC. The trade is roughly +neutral: a new `WriteHandle` trait (~10 LOC) + a `RecordWriter` struct +(~30 LOC) replace ~30 LOC of `R` mentions across the codebase. If the goal +is line-count reduction, M14 will not deliver β€” measure success by *hot-path +cycles eliminated* and *type-signature simplification*, not by `git diff +--stat`. + +--- + +## Current Architecture + +### Why `R` is present today + +``` +Producer { + db: Arc>, // ← the only source of R at runtime + record_key: String, + profiling: Option>, // R-free +} + +produce(value: T): + db.produce::(&record_key, value) // key lookup β†’ TypedRecord + β†’ typed_record.produce(value) // push to buffer + snapshot +``` + +`AimDb::produce()` is the indirection: it resolves `record_key` to a +`TypedRecord` using `get_typed_record_by_key::()`. The `R` in the +return type is only needed for the downcast inside that lookup. + +`Consumer` follows the same pattern: `subscribe()` calls +`db.subscribe::(&record_key)` which returns `Box + Send>` +β€” already type-erased at the return boundary. + +### What does NOT need `R` at runtime + +| Component | R needed? | Why | +|---|---|---| +| `ProducerProfilingState` | βœ— | `Clock = Arc u64 + Send + Sync>`, erased at build time | +| Buffer write (`DynBuffer::push`) | βœ— | Already `dyn`, no R in trait | +| Buffer subscribe (`DynBuffer::subscribe_boxed`) | βœ— | Returns `Box>` | +| `latest_snapshot` update | βœ— | `Arc>>`, no R | +| Key β†’ buffer resolution | βœ“ | Currently lives inside `AimDb`, fixable by moving it to build time | + +### The full `R`-propagation surface + +After M13 (uncommitted), these public items still carry `R`: + +``` +Producer β€” user closures in .source(), .tap() +Consumer β€” user closures in .tap(), connector subscribe +RecordRegistrar β€” registration API (build time only) +OutboundConnectorBuilder +InboundConnectorBuilder +ConnectorBuilder β€” public connector-author trait + ↳ impl ConnectorBuilder for MqttConnector + ↳ impl ConnectorBuilder for KnxConnector + ↳ impl ConnectorBuilder for WsConnector +aimdb-codegen emitted types β€” on every generated registrar +``` + +--- + +## Proposed Design + +### New `WriteHandle` indirection + +Add a crate-private trait (alongside `DynBuffer`) in +`aimdb-core/src/buffer/traits.rs`: + +```rust +/// Write-side handle for a single record (issue #88 follow-up, design 029). +/// +/// `Producer` holds `Arc>` so it can be parameterised +/// over `T` only β€” no runtime adapter `R` and no record-key string lookup on +/// the produce hot path. The implementor (`RecordWriter`) pre-binds the +/// underlying buffer, the latest-snapshot slot, and the metadata tracker at +/// build time. +/// +/// Crate-private on purpose: `Producer::new` is the only construction +/// path. External callers that want to mock a Producer should go through +/// that constructor (or a future `Producer::for_testing(...)` helper) β€” +/// promoting the trait to `pub` is reserved for an explicit testing-DX +/// follow-up if/when there's demand. +pub(crate) trait WriteHandle: Send + Sync { + /// Push a value into the buffer, update the latest-snapshot cache, and + /// (when a buffer is present) mark the metadata `last_update` timestamp. + /// Infallible β€” all three operations are synchronous and lock-free or + /// spin-locked. + fn push(&self, value: T); +} +``` + +The visibility is deliberately `pub(crate)`. `Producer::new` is the only +construction path; external test code that needs a fake Producer goes through +a `for_testing(buffer: Arc>)` constructor we can add later +if demand materialises. + +### `RecordWriter` construction (Option B) + +The implementor of `WriteHandle` is `RecordWriter`, a plain struct +holding *clones of* the three shared Arcs the record already owns: + +```rust +// aimdb-core/src/buffer/writer.rs (new file) + +pub(crate) struct RecordWriter { + /// `None` for records that only support `latest()` (no buffer configured). + buffer: Option>>, + + /// Snapshot slot shared with `TypedRecord` and any `latest()` reader. + #[cfg(feature = "std")] + latest_snapshot: Arc>>, + #[cfg(not(feature = "std"))] + latest_snapshot: Arc>>, + + /// Metadata tracker (already `Clone` with shared inner `Arc` / + /// `Arc`). std-only. + #[cfg(feature = "std")] + metadata: RecordMetadataTracker, +} + +impl WriteHandle for RecordWriter { + fn push(&self, value: T) { + #[cfg(feature = "std")] + { *self.latest_snapshot.lock().unwrap() = Some(value.clone()); } + #[cfg(not(feature = "std"))] + { *self.latest_snapshot.lock() = Some(value.clone()); } + + if let Some(buf) = &self.buffer { + buf.push(value); + #[cfg(feature = "std")] + self.metadata.mark_updated(); + } + } +} +``` + +**Construction site:** `RecordFutureCollector::collect_all_futures` / +`TypedRecord::collect_producer_future`. The `RecordWriter` is built just +before the Producer is handed to the user closure β€” once per producer per +record. Producer clones share the same `Arc>`. + +**Ownership choice (vs. Option A).** Option A was "switch +`AimDbInner.storages: Vec>` to `Vec>` +and upcast `Arc>` to `Arc>`". Rejected: +that changes the ownership model of every record everywhere and touches +every consumer of `AimDbInner::storage() -> &dyn AnyRecord`. Option B +isolates the change to `TypedRecord`'s internals β€” from the outside, +`AimDbInner` still hands out `&dyn AnyRecord`. + +**Single structural change to `TypedRecord`.** Its +`buffer: Option>>` becomes +`Option>>`. Everything else (`latest_snapshot`, +`metadata`) is already shareable (`Arc>`, `Clone`). `TypedRecord` +gains two accessors used at build time only: + +```rust +fn writer_handle(&self) -> Arc>; // builds a fresh RecordWriter +fn buffer_handle(&self) -> Option>>; +``` + +`TypedRecord::produce()` (still callable via `AimDb::produce(key, value)`) +continues to work for the few legacy paths that have a key but not a +pre-built Producer β€” it internally constructs / reuses a writer. + +### `Producer` β€” after + +```rust +pub struct Producer { + write: Arc>, + #[cfg(feature = "profiling")] + profiling: Option>, + _phantom: PhantomData T>, // keeps Send/Sync independent of T +} +``` + +Gone: the `db: Arc>` field, the `record_key: String` field, the +`R` type parameter, the `Producer::key() -> &str` accessor. + +`produce()` becomes: + +```rust +pub async fn produce(&self, value: T) -> DbResult<()> { + #[cfg(feature = "profiling")] + if let Some(state) = &self.profiling { + state.record_produce(); + } + self.write.push(value); + Ok(()) +} +``` + +**Signature stability:** `async fn ... -> DbResult<()>` is *intentionally +kept* even though the body is synchronous and infallible. Reasons: + +1. Source-level compatibility β€” every existing `producer.produce(x).await?` + call site keeps working unchanged. +2. Forward compatibility β€” future backpressure-aware buffers (async + `push`) or shutdown-signalling semantics (`Err(ChannelClosed)`) can be + added without another breaking change. + +Cost is negligible (an `Ok` wrapper and a no-op await), both compile to +nothing. + +### `Consumer` β€” after + +```rust +pub struct Consumer { + buffer: Arc>, + #[cfg(feature = "profiling")] + profiling: Option<(Arc, Clock)>, + _phantom: PhantomData T>, +} +``` + +`subscribe()` collapses to: + +```rust +pub fn subscribe(&self) -> DbResult + Send>> { + let reader = self.buffer.subscribe_boxed(); + #[cfg(feature = "profiling")] + if let Some((metrics, clock)) = &self.profiling { + return Ok(Box::new(ProfilingBufferReader::new(reader, metrics.clone(), clock.clone()))); + } + Ok(reader) +} +``` + +`DynBuffer` already exists and is the natural read-side counterpart β€” no +new trait required. + +### `ConnectorBuilder` β€” cascade (zero-LOC) + +`ConnectorBuilder` keeps its trait-level `R` parameter (v0.1's Option A). +But the cascade in connector *struct* code is in practice **zero LOC**: a +grep of the workspace shows no connector struct carries an `R` phantom +today: + +```text +MqttConnectorBuilder, KnxConnectorBuilder, WebSocketConnectorBuilder, +MqttConnectorImpl, KnxConnectorImpl, WebSocketConnectorImpl +``` + +All six are non-generic. M13 already cleaned the field layer; M14 inherits +that. Connector authors do not see Producer/Consumer as typed values either +β€” routes are returned via `Box` / `Box` +(fully erased) from `db.collect_inbound_routes()` / +`db.collect_outbound_routes()`. So the only `R` mention left in connector +crates after M14 is on the `impl +ConnectorBuilder for X` line itself β€” unchanged. + +Full `ConnectorBuilder` non-generic-isation (v0.1's Option B) is still +deferred to a separate milestone (needs a `ConnectorContext` abstraction). + +--- + +## Type-System Changes + +| Before | After | +|---|---| +| `Producer` | `Producer` | +| `Consumer` | `Consumer` | +| `Producer::key(&self) -> &str` | **removed** | +| `Producer::produce(&self, T) -> impl Future>` | unchanged signature (body simplified) | +| `Consumer::subscribe(&self) -> DbResult<…>` | unchanged signature (body simplified) | +| `TypedRecord::buffer: Option>>` | `Option>>` | +| `RecordRegistrar` (build-time only) | unchanged β€” still needs `R` to resolve records | +| `ConnectorBuilder` | unchanged trait | +| `impl ConnectorBuilder for MqttConnectorBuilder` | unchanged (already R-free in the struct) | +| Codegen emitted `` on registrars | unchanged β€” registrars are build-time | +| `RuntimeContext` | unchanged β€” see *Out of Scope* | + +`RecordRegistrar` is intentionally kept generic: it is a build-time-only +object (not stored past `build()`) and needs `R` to call +`AimDb::register_record_typed()`. It does not appear in user code after +setup (inference handles `R` in `|reg| { ... }` closures). + +--- + +## Breaking Changes + +**Public API (semver minor bump):** + +- `Producer` β†’ `Producer` β€” any code that names the two-parameter form breaks. Code that uses `Producer` via type inference (the common case) compiles without changes. +- `Consumer` β†’ `Consumer` β€” same. +- `Producer::key()` is **removed**. Code that called `producer.key()` for diagnostic strings should capture the key at the registration site instead. Two internal callers in `aimdb-core` (`run_single_transform`, `run_join_transform`) are updated as part of step 7. + +**Codegen:** emitted code does not reference `Producer` / `Consumer` with type parameters directly. Golden-test strings update mechanically; no template changes. + +**Connector authors:** zero-LOC impact. Connector structs are already R-free (M13), and routes are already type-erased via `ConsumerTrait` / `ProducerTrait`. The `impl ConnectorBuilder for …` line stays as written. + +**`aimdb-pro` demo binaries:** ~6 weather-station files plus `weather-hub-streaming` have factored-out `async fn` producers/consumers that name `Producer` / `Consumer`. Drop the `, TokioAdapter` from each. No structural change. + +--- + +## Implementation Plan + +Listed in dependency order. Each step should pass `make check` before the +next begins (modulo the call-site updates in step 10, which are gated by +steps 1–9 landing first). + +### Step 1 β€” Add the write-side primitives (no external break) + +**Files:** `aimdb-core/src/buffer/traits.rs`, `aimdb-core/src/buffer/writer.rs` (new), `aimdb-core/src/buffer/mod.rs` + +- Add `pub(crate) trait WriteHandle` to `traits.rs`. +- Add `pub(crate) struct RecordWriter` + `impl WriteHandle for RecordWriter` to a new `writer.rs`. +- Re-export from `mod.rs` (crate-private). + +### Step 2 β€” `TypedRecord` Box β†’ Arc + accessors + +**File:** `aimdb-core/src/typed_record.rs` + +- Change `buffer: Option>>` β†’ `Option>>`. +- Add `pub(crate) fn writer_handle(&self) -> Arc>` β€” constructs a `RecordWriter` with cloned Arcs. +- Add `pub(crate) fn buffer_handle(&self) -> Option>>` β€” clones the buffer Arc. +- Leave `TypedRecord::produce()` callable for the legacy key-based path (`AimDb::produce(key, value)`). + +### Step 3 β€” `Producer` removes `R` + +**File:** `aimdb-core/src/typed_api.rs` + +- Drop type parameter `R` from `Producer` β†’ `Producer`. +- Replace `db: Arc>` with `write: Arc>`. +- Delete `record_key: String` field and the `pub fn key(&self) -> &str` accessor (two internal callers `run_single_transform` / `run_join_transform` are updated in step 7 to capture the key via closure instead). +- Keep `produce()` signature unchanged (`async fn produce(&self, value: T) -> DbResult<()>`). +- Update `impl ProducerTrait for Producer` likewise. + +### Step 4 β€” `Consumer` removes `R` + +**File:** `aimdb-core/src/typed_api.rs` + +- Drop type parameter `R` from `Consumer` β†’ `Consumer`. +- Replace `db: Arc>` field with `buffer: Arc>`. +- Update `subscribe()` to call `self.buffer.subscribe_boxed()` directly (no error path through `AimDb::subscribe`). +- Update `impl ConsumerTrait for Consumer` likewise. + +### Step 5 β€” `collect_*` construction sites + +**File:** `aimdb-core/src/typed_record.rs` + +- `collect_producer_future`: build `RecordWriter` from the typed record, construct `Producer::new(writer_handle)`. The producer no longer needs the `record_key` argument. +- `collect_consumer_futures`: clone `buffer_handle()`, construct `Consumer::new(buffer)`. +- `collect_transform_futures`: same as producer (transform produces into its own record). + +### Step 6 β€” `RecordRegistrar` internal Producer/Consumer construction + +**File:** `aimdb-core/src/typed_api.rs` + +- Update the (build-time-only) call sites inside `RecordRegistrar` that construct `Producer` / `Consumer` (e.g. for `.tap_raw()` factory closures, for the inbound-connector factory) to use the new constructors. +- The registrar keeps its `R` parameter β€” it still needs `R` to call `AimDb::register_record_typed()`. `R` does not leak past `build()`. + +### Step 7 β€” Transforms (`run_single_transform`, `run_join_transform`) + +**Files:** `aimdb-core/src/transform/single.rs`, `aimdb-core/src/transform/join.rs` + +- Update typed signatures: `Producer` β†’ `Producer`, `Consumer` β†’ `Consumer`. +- Where `producer.key()` was used for tracing strings, capture the key into the spawning closure instead and pass it as a parameter. + +### Step 8 β€” Connector cascade (zero structural changes) + +**Files:** `aimdb-mqtt-connector`, `aimdb-knx-connector`, `aimdb-websocket-connector` + +- No struct changes (verified: no connector struct carries `R`). +- `impl ConnectorBuilder for X` lines stay as-is. +- Routes are already type-erased via `ConsumerTrait` / `ProducerTrait`. + +### Step 9 β€” Codegen golden tests + +**File:** `aimdb-codegen/src/rust.rs` + +- Emitted `configure_schema(...)` is unchanged (registrar still has `R`). +- Producer/Consumer are not referenced in emitted code; no template changes. +- Golden-test strings: verify Producer/Consumer mentions (if any) are updated. + +### Step 10 β€” Examples + aimdb-pro + tests + +Drop `R` from Producer/Consumer in ~18 user-side signatures across: + +- `examples/{mqtt,knx,embassy-{mqtt,knx},tokio-{mqtt,knx},remote-access,hello-single-latest-async,weather-mesh-demo}-*` +- `aimdb-pro/demo/weather-station-*` +- Generic helpers in `examples/{mqtt,knx}-connector-demo-common/src/monitors.rs` (the `` parameter on the fn stays for `RuntimeContext`, the Producer/Consumer params drop their `R`). + +### Step 11 β€” CHANGELOG + README + +- Each affected crate's CHANGELOG gains an `[Unreleased]` entry. +- Lead with the **hot-path lookup elimination** narrative (per the Motivation section above), with R-removal as the secondary benefit. +- Workspace README quickstart already calls `builder.run().await?` (post-M13); add a `.source()` example to demonstrate `Producer` (no R). + +--- + +## Decisions + +All open questions raised in design review (May 24, 2026) are resolved: + +1. **`Producer::key()` accessor** β†’ **Delete**, along with the `record_key: String` field. Two internal callers (`run_single_transform`, `run_join_transform`) capture the key into the spawning closure instead. External users get the key from the registration site by construction, not from the Producer. + +2. **`Producer::produce()` return type** β†’ **Keep `async fn ... -> DbResult<()>` unchanged.** Even though the body is synchronous and infallible after M14, keeping the signature avoids breakage on every existing `.await?` call site and leaves room for future backpressure-aware buffers or shutdown-signalling semantics. Cost is two no-op compile-time abstractions. + +3. **`ConnectorBuilder` cascade** β†’ **v0.1 Option A**, but with the realisation that it is **zero LOC** for connector structs. Verified: no connector struct in `aimdb-mqtt-connector`, `aimdb-knx-connector`, or `aimdb-websocket-connector` carries an `R` phantom today (M13 already cleaned them). The only `R` left is on the `impl ConnectorBuilder for X` line and stays untouched. Full non-generic-isation is deferred to a separate milestone. + +4. **`RuntimeContext` removal** β†’ **Out of scope for M14.** `RuntimeContext` exposes `ctx.time()` / `ctx.log()` which depend on `R::TimeOps` / `R::Logger` associated types. Erasing those behind `dyn` is a larger change that warrants its own milestone. Users keep writing `ctx: RuntimeContext` next to `producer: Producer`. + +5. **`Arc>` construction** β†’ **Option B.** A `RecordWriter` struct lives in `aimdb-core/src/buffer/writer.rs`, holding cloned Arcs of the record's buffer, latest-snapshot, and metadata. Constructed at producer-creation time. `AimDbInner.storages: Vec>` is **unchanged** β€” only the Boxβ†’Arc on `TypedRecord::buffer` propagates. Rejected Option A (switching storages to `Arc`) was broader than necessary. + +6. **`WriteHandle` visibility** β†’ **`pub(crate)`.** External mocking goes through `Producer::new` (or a future `Producer::for_testing` helper) β€” no need to expose the trait. Promote to `pub` only if a concrete testing-DX request materialises. + +--- + +## Out of Scope + +- **`RuntimeContext` removal.** Same pattern as Producer/Consumer in spirit (R is the only generic), but `ctx.time()` / `ctx.log()` return runtime-specific associated types (`R::Instant`, `R::Duration`, etc.). Erasing those behind `dyn` is a structurally larger change that touches every user `ctx.time().sleep(d).await` call site. Worth its own milestone with a dedicated trait-object design (e.g. `DurationProvider`, `LoggerProvider`). Until then, users keep writing `ctx: RuntimeContext` next to `producer: Producer` β€” visually inconsistent but tolerable. +- **Full `ConnectorBuilder` non-generic-isation** (v0.1's Option B). Requires a `ConnectorContext` abstraction exposing type-erased record access (e.g. `fn producer(&self, key: &str) -> Producer`). Larger API design in its own right. The trait-level `` stays for M14. +- **`RecordRegistrar` removal.** Registrar is build-time only; its `R` bound causes no propagation into user runtime code (inference handles it in the `|reg| { ... }` closure form, which is the only form used in practice β€” verified across `examples/` and `aimdb-pro/`). Clean-up possible but low value. +- **AimX remote handler generics.** The remote handler still has `R` bounds from deferred bridge-state work (see M13 Β§"Out of Scope"). Orthogonal to this change; addressed by the AimX portability follow-up. +- **Promoting `WriteHandle` to `pub`.** Reserved for an explicit testing-DX follow-up if external users request mockable Producers. M14 keeps the trait `pub(crate)`. diff --git a/examples/embassy-knx-connector-demo/src/main.rs b/examples/embassy-knx-connector-demo/src/main.rs index 2494cbf..b145709 100644 --- a/examples/embassy-knx-connector-demo/src/main.rs +++ b/examples/embassy-knx-connector-demo/src/main.rs @@ -92,7 +92,7 @@ async fn net_task(mut runner: embassy_net::Runner<'static, Device>) -> ! { /// Button handler that toggles light on button press async fn button_handler( ctx: RuntimeContext, - producer: aimdb_core::Producer, + producer: aimdb_core::Producer, mut button: ExtiInput<'static, embassy_stm32::mode::Async>, ) { let log = ctx.log(); diff --git a/examples/embassy-mqtt-connector-demo/src/main.rs b/examples/embassy-mqtt-connector-demo/src/main.rs index 5d848b0..b00f0c5 100644 --- a/examples/embassy-mqtt-connector-demo/src/main.rs +++ b/examples/embassy-mqtt-connector-demo/src/main.rs @@ -101,7 +101,7 @@ async fn net_task(mut runner: embassy_net::Runner<'static, Device>) -> ! { /// Indoor temperature sensor producer async fn indoor_temp_producer( ctx: RuntimeContext, - temperature: Producer, + temperature: Producer, ) { let log = ctx.log(); log.info("🏠 Starting INDOOR temperature producer...\n"); @@ -127,7 +127,7 @@ async fn indoor_temp_producer( /// Outdoor temperature sensor producer async fn outdoor_temp_producer( ctx: RuntimeContext, - temperature: Producer, + temperature: Producer, ) { let log = ctx.log(); log.info("🌳 Starting OUTDOOR temperature producer...\n"); @@ -156,7 +156,7 @@ async fn outdoor_temp_producer( /// Server room temperature sensor producer async fn server_room_temp_producer( ctx: RuntimeContext, - temperature: Producer, + temperature: Producer, ) { let log = ctx.log(); log.info("πŸ–₯️ Starting SERVER ROOM temperature producer...\n"); diff --git a/examples/hello-single-latest-async/src/main.rs b/examples/hello-single-latest-async/src/main.rs index de0a293..5cdea5e 100644 --- a/examples/hello-single-latest-async/src/main.rs +++ b/examples/hello-single-latest-async/src/main.rs @@ -32,10 +32,7 @@ async fn main() -> Result<(), Box> { Ok(()) } -async fn rollout_source( - ctx: RuntimeContext, - producer: Producer, -) { +async fn rollout_source(ctx: RuntimeContext, producer: Producer) { let time = ctx.time(); time.sleep(time.millis(50)).await; @@ -50,7 +47,7 @@ async fn rollout_source( } } -async fn publish_rollout(producer: &Producer, rollout_percent: u8) { +async fn publish_rollout(producer: &Producer, rollout_percent: u8) { let gate = FeatureGate { rollout_percent }; match producer.produce(gate).await { Ok(()) => println!("source published rollout: {rollout_percent}%"), @@ -58,10 +55,7 @@ async fn publish_rollout(producer: &Producer, rollout } } -async fn rollout_observer( - ctx: RuntimeContext, - consumer: Consumer, -) { +async fn rollout_observer(ctx: RuntimeContext, consumer: Consumer) { let Ok(mut reader) = consumer.subscribe() else { eprintln!("failed to subscribe to config.checkout_rollout"); return; diff --git a/examples/knx-connector-demo-common/src/monitors.rs b/examples/knx-connector-demo-common/src/monitors.rs index a95d222..f26b626 100644 --- a/examples/knx-connector-demo-common/src/monitors.rs +++ b/examples/knx-connector-demo-common/src/monitors.rs @@ -26,10 +26,8 @@ use aimdb_core::{Consumer, Logger, Runtime, RuntimeContext}; /// .finish(); /// }); /// ``` -pub async fn temperature_monitor( - ctx: RuntimeContext, - consumer: Consumer, -) where +pub async fn temperature_monitor(ctx: RuntimeContext, consumer: Consumer) +where R: Runtime + Logger + Send + Sync + 'static, { let log = ctx.log(); @@ -58,7 +56,7 @@ pub async fn temperature_monitor( /// Light monitor that logs state changes from the buffer /// /// Uses the group address from the `LightState` data itself. -pub async fn light_monitor(ctx: RuntimeContext, consumer: Consumer) +pub async fn light_monitor(ctx: RuntimeContext, consumer: Consumer) where R: Runtime + Logger + Send + Sync + 'static, { diff --git a/examples/mqtt-connector-demo-common/src/monitors.rs b/examples/mqtt-connector-demo-common/src/monitors.rs index fd0bec3..bb7a9c7 100644 --- a/examples/mqtt-connector-demo-common/src/monitors.rs +++ b/examples/mqtt-connector-demo-common/src/monitors.rs @@ -25,7 +25,7 @@ use aimdb_core::{Consumer, Logger, Runtime, RuntimeContext}; /// .finish(); /// }); /// ``` -pub async fn temperature_logger(ctx: RuntimeContext, consumer: Consumer) +pub async fn temperature_logger(ctx: RuntimeContext, consumer: Consumer) where R: Runtime + Logger + Send + Sync + 'static, { @@ -53,7 +53,7 @@ where /// Command consumer that logs received commands /// /// Uses the action and sensor_id from the `TemperatureCommand` data. -pub async fn command_consumer(ctx: RuntimeContext, consumer: Consumer) +pub async fn command_consumer(ctx: RuntimeContext, consumer: Consumer) where R: Runtime + Logger + Send + Sync + 'static, { diff --git a/examples/remote-access-demo/src/server.rs b/examples/remote-access-demo/src/server.rs index e597e45..12b00ee 100644 --- a/examples/remote-access-demo/src/server.rs +++ b/examples/remote-access-demo/src/server.rs @@ -148,7 +148,7 @@ async fn main() -> Result<(), Box> { info!("πŸ“ Populating initial record data..."); - let config_producer = db.producer::("server::Config"); + let config_producer = db.producer::("server::Config")?; config_producer .produce(Config { app_name: "AimDB Demo".to_string(), @@ -214,7 +214,7 @@ async fn main() -> Result<(), Box> { /// timed by the `profiling` feature β€” see `get_stage_profiling`. async fn temperature_simulator( ctx: RuntimeContext, - temperature: Producer, + temperature: Producer, ) { let time = ctx.time(); let mut counter = 0u64; @@ -243,10 +243,7 @@ async fn temperature_simulator( /// Fast tap on Temperature β€” just logs. Stage profiling will show it as /// substantially faster than `slow_status_processor` on SystemStatus. -async fn temperature_logger( - _ctx: RuntimeContext, - consumer: Consumer, -) { +async fn temperature_logger(_ctx: RuntimeContext, consumer: Consumer) { let Ok(mut reader) = consumer.subscribe() else { tracing::error!("Failed to subscribe to Temperature"); return; @@ -263,7 +260,7 @@ async fn temperature_logger( /// Periodically produces SystemStatus readings. async fn system_status_simulator( ctx: RuntimeContext, - status: Producer, + status: Producer, ) { let time = ctx.time(); let mut uptime = 3600u64; @@ -288,7 +285,7 @@ async fn system_status_simulator( /// as the per-record bottleneck in `get_stage_profiling`. async fn slow_status_processor( ctx: RuntimeContext, - consumer: Consumer, + consumer: Consumer, ) { let time = ctx.time(); let Ok(mut reader) = consumer.subscribe() else { diff --git a/examples/tokio-knx-connector-demo/src/main.rs b/examples/tokio-knx-connector-demo/src/main.rs index a9c3b8a..9e53f29 100644 --- a/examples/tokio-knx-connector-demo/src/main.rs +++ b/examples/tokio-knx-connector-demo/src/main.rs @@ -39,10 +39,7 @@ use knx_connector_demo_common::{ // ============================================================================ /// Keyboard input handler - toggles light on ENTER -async fn input_handler( - ctx: RuntimeContext, - producer: Producer, -) { +async fn input_handler(ctx: RuntimeContext, producer: Producer) { let log = ctx.log(); log.info("\n⌨️ Input handler started. Press ENTER to toggle light on 1/0/6"); log.info(" (This sends GroupValueWrite to the KNX bus)\n"); diff --git a/examples/tokio-mqtt-connector-demo/src/main.rs b/examples/tokio-mqtt-connector-demo/src/main.rs index 12fbd18..46f3f95 100644 --- a/examples/tokio-mqtt-connector-demo/src/main.rs +++ b/examples/tokio-mqtt-connector-demo/src/main.rs @@ -51,7 +51,7 @@ use mqtt_connector_demo_common::{ /// Indoor temperature sensor producer async fn indoor_temp_producer( ctx: RuntimeContext, - temperature: Producer, + temperature: Producer, ) { let log = ctx.log(); let time = ctx.time(); @@ -79,7 +79,7 @@ async fn indoor_temp_producer( /// Outdoor temperature sensor producer async fn outdoor_temp_producer( ctx: RuntimeContext, - temperature: Producer, + temperature: Producer, ) { let log = ctx.log(); let time = ctx.time(); @@ -107,7 +107,7 @@ async fn outdoor_temp_producer( /// Server room temperature sensor producer async fn server_room_temp_producer( ctx: RuntimeContext, - temperature: Producer, + temperature: Producer, ) { let log = ctx.log(); let time = ctx.time(); diff --git a/examples/weather-mesh-demo/weather-station-alpha/src/main.rs b/examples/weather-mesh-demo/weather-station-alpha/src/main.rs index 1580e76..9dda4b0 100644 --- a/examples/weather-mesh-demo/weather-station-alpha/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-alpha/src/main.rs @@ -131,8 +131,8 @@ async fn main() -> Result<(), Box> { ); // Get producers - let temp_producer = db.producer::(TempKey::Alpha.as_str()); - let humidity_producer = db.producer::(HumidityKey::Alpha.as_str()); + let temp_producer = db.producer::(TempKey::Alpha.as_str())?; + let humidity_producer = db.producer::(HumidityKey::Alpha.as_str())?; // Spawn weather data producer info!("🌀️ Starting weather data producer..."); diff --git a/examples/weather-mesh-demo/weather-station-beta/src/main.rs b/examples/weather-mesh-demo/weather-station-beta/src/main.rs index f762ac2..0e38082 100644 --- a/examples/weather-mesh-demo/weather-station-beta/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-beta/src/main.rs @@ -131,8 +131,8 @@ async fn main() -> Result<(), Box> { ); // Get producers - let temp_producer = db.producer::(TempKey::Beta.as_str()); - let humidity_producer = db.producer::(HumidityKey::Beta.as_str()); + let temp_producer = db.producer::(TempKey::Beta.as_str())?; + let humidity_producer = db.producer::(HumidityKey::Beta.as_str())?; // Spawn synthetic data producer info!("🎲 Starting synthetic data producer..."); diff --git a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs index 7ae2950..925e4a2 100644 --- a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs @@ -69,7 +69,7 @@ async fn net_task(mut runner: embassy_net::Runner<'static, Device>) -> ! { /// Temperature producer - generates synthetic data async fn temperature_producer( ctx: RuntimeContext, - producer: Producer, + producer: Producer, ) { let log = ctx.log(); log.info("🌑️ Starting temperature producer..."); @@ -104,10 +104,7 @@ async fn temperature_producer( } /// Humidity producer - generates synthetic data -async fn humidity_producer( - ctx: RuntimeContext, - producer: Producer, -) { +async fn humidity_producer(ctx: RuntimeContext, producer: Producer) { let log = ctx.log(); log.info("πŸ’§ Starting humidity producer..."); From 6dce511419857ab70f1ee74186dcc2cb99f0c627 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Mon, 25 May 2026 08:00:35 +0000 Subject: [PATCH 07/14] Refactor produce method to be synchronous - Changed the `produce` method in various modules from async to synchronous, removing unnecessary await calls. - Updated related documentation and examples to reflect the new synchronous behavior. - Ensured that all instances of `producer.produce(value).await` were replaced with `producer.produce(value)`. - This change simplifies the API and improves performance by eliminating the overhead of async handling where it is not needed. --- aimdb-core/CHANGELOG.md | 7 +++ aimdb-core/src/builder.rs | 22 +++++---- aimdb-core/src/database.rs | 8 ++-- aimdb-core/src/transform/join.rs | 35 +++----------- aimdb-core/src/transform/single.rs | 26 +--------- aimdb-core/src/typed_api.rs | 48 ++++++++----------- aimdb-core/src/typed_record.rs | 4 +- aimdb-data-contracts/src/observable.rs | 5 +- .../tests/topic_provider_tests.rs | 6 +-- .../tests/topic_provider_tests.rs | 14 ++---- aimdb-persistence/src/ext.rs | 14 +----- aimdb-sync/src/handle.rs | 2 +- .../tests/drain_integration_tests.rs | 12 ----- .../tests/multi_instance_tests.rs | 37 +++++--------- aimdb-tokio-adapter/tests/stage_profiling.rs | 6 +-- .../tests/transform_join_integration_tests.rs | 14 +++--- aimdb-wasm-adapter/src/bindings.rs | 33 +------------ aimdb-wasm-adapter/src/ws_bridge.rs | 2 +- .../tests/transform_join_integration_tests.rs | 2 +- .../embassy-knx-connector-demo/src/main.rs | 12 +---- .../embassy-mqtt-connector-demo/src/main.rs | 18 ++----- .../hello-single-latest-async/src/main.rs | 10 +--- .../knx-connector-demo-common/src/monitors.rs | 10 +--- .../src/monitors.rs | 10 +--- examples/remote-access-demo/src/server.rs | 18 ++----- examples/sync-api-demo/src/main.rs | 5 +- examples/tokio-knx-connector-demo/src/main.rs | 12 +---- .../tokio-mqtt-connector-demo/src/main.rs | 12 ++--- .../weather-station-alpha/src/main.rs | 8 +--- .../weather-station-beta/src/main.rs | 8 +--- .../weather-station-gamma/src/main.rs | 8 +--- 31 files changed, 113 insertions(+), 315 deletions(-) diff --git a/aimdb-core/CHANGELOG.md b/aimdb-core/CHANGELOG.md index aea7e95..c697b87 100644 --- a/aimdb-core/CHANGELOG.md +++ b/aimdb-core/CHANGELOG.md @@ -9,6 +9,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed (breaking) +- **`Producer::produce` is now sync + infallible; `Consumer::subscribe` is now infallible (Design 029 follow-up, M14).** The pre-resolved `WriteHandle::push` cannot fail and the pre-resolved buffer Arc makes `subscribe()` infallible. Call sites collapse: `producer.produce(x).await?` β†’ `producer.produce(x);` and `let Ok(reader) = consumer.subscribe() else { ... }` β†’ `let reader = consumer.subscribe();`. The `ProducerTrait::produce_any` / `ConsumerTrait::subscribe_any` trait surfaces stay `Result`/`async` because the type-erasure downcast remains fallible. + - `AimDb::produce(key, value) -> DbResult<()>` is now sync; `.await` on the call site goes away. Only the key lookup can fail. + - `Database::produce` likewise sync. + - `TypedRecord::produce` is now `pub fn produce(&self, val: T)` (was `pub async fn produce`). + - `aimdb-wasm-adapter`: `bindings::poll_sync` helper deleted β€” no remaining callers now that `TypedRecord::produce` is sync. + - Dead `consumer.subscribe()` error arms in `transform/single.rs` and `transform/join.rs` removed (the `Err` branch was unreachable after M14). + - **`Producer` / `Consumer` drop the runtime parameter `R` and pre-resolve the record at build time (Design 029, M14).** Producer/Consumer become handles to a buffer rather than tickets to look one up: `produce()` is one virtual call (no `HashMap` probe, no `TypeId` check, no downcast), and `subscribe()` collapses to `buffer.subscribe_boxed()`. The internal mechanic is a new crate-private `WriteHandle` trait backed by `RecordWriter` (in `aimdb-core/src/buffer/writer.rs`), pre-bound to the record's `Arc>` + snapshot mutex + metadata tracker. - `Producer` β†’ `Producer`; `Consumer` β†’ `Consumer`. User code that names the two-parameter form must drop the trailing adapter arg. - `Producer::key(&self) -> &str` is **removed**. Capture the record key at the registration site instead. diff --git a/aimdb-core/src/builder.rs b/aimdb-core/src/builder.rs index 279e221..102fa5e 100644 --- a/aimdb-core/src/builder.rs +++ b/aimdb-core/src/builder.rs @@ -1081,9 +1081,13 @@ impl AimDb { b.run().await } - /// Produces a value to a specific record by key + /// Produces a value to a specific record by key. /// - /// Uses O(1) key-based lookup to find the correct record. + /// Uses O(1) key-based lookup to find the correct record. Sync + fallible β€” + /// only the lookup can fail; the buffer push itself is infallible. + /// + /// For hot paths, prefer `db.producer::(key)?` once and reuse the + /// returned handle. /// /// # Arguments /// * `key` - The record key (e.g., "sensor.temperature") @@ -1092,15 +1096,15 @@ impl AimDb { /// # Example /// /// ```rust,ignore - /// db.produce::("sensors.indoor", indoor_temp).await?; - /// db.produce::("sensors.outdoor", outdoor_temp).await?; + /// db.produce::("sensors.indoor", indoor_temp)?; + /// db.produce::("sensors.outdoor", outdoor_temp)?; /// ``` - pub async fn produce(&self, key: impl AsRef, value: T) -> DbResult<()> + pub fn produce(&self, key: impl AsRef, value: T) -> DbResult<()> where T: Send + 'static + Debug + Clone, { let typed_rec = self.inner.get_typed_record_by_key::(key)?; - typed_rec.produce(value).await; + typed_rec.produce(value); Ok(()) } @@ -1144,8 +1148,8 @@ impl AimDb { /// let outdoor_producer = db.producer::("sensors.outdoor"); /// /// // Each producer writes to its own record - /// indoor_producer.produce(indoor_temp).await?; - /// outdoor_producer.produce(outdoor_temp).await?; + /// indoor_producer.produce(indoor_temp); + /// outdoor_producer.produce(outdoor_temp); /// ``` pub fn producer( &self, @@ -1175,7 +1179,7 @@ impl AimDb { /// let outdoor_consumer = db.consumer::("sensors.outdoor"); /// /// // Each consumer reads from its own record - /// let mut rx = indoor_consumer.subscribe()?; + /// let mut rx = indoor_consumer.subscribe(); /// ``` pub fn consumer( &self, diff --git a/aimdb-core/src/database.rs b/aimdb-core/src/database.rs index ea54054..fec597c 100644 --- a/aimdb-core/src/database.rs +++ b/aimdb-core/src/database.rs @@ -72,16 +72,16 @@ impl Database { /// /// # Example /// ```rust,ignore - /// # async fn example(db: aimdb_core::Database) -> aimdb_core::DbResult<()> { - /// db.produce("sensor.temp", SensorData { temp: 23.5 }).await?; + /// # fn example(db: aimdb_core::Database) -> aimdb_core::DbResult<()> { + /// db.produce("sensor.temp", SensorData { temp: 23.5 })?; /// # Ok(()) /// # } /// ``` - pub async fn produce(&self, key: impl AsRef, data: T) -> DbResult<()> + pub fn produce(&self, key: impl AsRef, data: T) -> DbResult<()> where T: Send + 'static + Clone + core::fmt::Debug, { - self.aimdb.produce(key, data).await + self.aimdb.produce(key, data) } /// Subscribes to a record by key diff --git a/aimdb-core/src/transform/join.rs b/aimdb-core/src/transform/join.rs index 9af9d49..7a9b37d 100644 --- a/aimdb-core/src/transform/join.rs +++ b/aimdb-core/src/transform/join.rs @@ -65,7 +65,7 @@ impl JoinTrigger { /// _ => {} /// } /// if let (Some(a), Some(b)) = (last_a, last_b) { -/// producer.produce(compute(a, b)).await.ok(); +/// producer.produce(compute(a, b)); /// } /// } /// }) @@ -163,8 +163,8 @@ where type Tx = <::JoinQueue as JoinQueue>::Sender; - let factory: JoinInputFactory = Box::new( - move |db: Arc>, index: usize, tx: Tx| { + let factory: JoinInputFactory = + Box::new(move |db: Arc>, index: usize, tx: Tx| { Box::pin(async move { let consumer = match db.consumer::(&key_for_factory) { Ok(c) => c, @@ -176,32 +176,10 @@ where index, _e ); - #[cfg(all(feature = "std", not(feature = "tracing")))] - eprintln!( - "AIMDB TRANSFORM ERROR: Join input '{}' (index {}) consumer resolution failed: {:?}", - key_for_factory, index, _e - ); - return; - } - }; - let mut reader = match consumer.subscribe() { - Ok(r) => r, - Err(_e) => { - #[cfg(feature = "tracing")] - tracing::error!( - "πŸ”„ Join input '{}' (index {}) subscription failed: {:?}", - key_for_factory, - index, - _e - ); - #[cfg(all(feature = "std", not(feature = "tracing")))] - eprintln!( - "AIMDB TRANSFORM ERROR: Join input '{}' (index {}) subscription failed: {:?}", - key_for_factory, index, _e - ); return; } }; + let mut reader = consumer.subscribe(); while let Ok(value) = reader.recv().await { let trigger = JoinTrigger::Input { @@ -213,8 +191,7 @@ where } } }) as BoxFuture<'static, ()> - }, - ); + }); self.inputs.push((key_str, factory)); self @@ -239,7 +216,7 @@ where /// _ => {} /// } /// if let (Some(a), Some(b)) = (last_a, last_b) { - /// producer.produce(compute(a, b)).await.ok(); + /// producer.produce(compute(a, b)); /// } /// } /// }) diff --git a/aimdb-core/src/transform/single.rs b/aimdb-core/src/transform/single.rs index a9e85ee..8b74cbb 100644 --- a/aimdb-core/src/transform/single.rs +++ b/aimdb-core/src/transform/single.rs @@ -182,32 +182,10 @@ pub(crate) async fn run_single_transform( output_key, _e ); - #[cfg(all(feature = "std", not(feature = "tracing")))] - eprintln!( - "AIMDB TRANSFORM ERROR: '{}' β†’ '{}' failed to resolve consumer: {:?}", - input_key, output_key, _e - ); - return; - } - }; - let mut reader = match consumer.subscribe() { - Ok(r) => r, - Err(_e) => { - #[cfg(feature = "tracing")] - tracing::error!( - "πŸ”„ Transform '{}' β†’ '{}' FATAL: failed to subscribe to input: {:?}", - input_key, - output_key, - _e - ); - #[cfg(all(feature = "std", not(feature = "tracing")))] - eprintln!( - "AIMDB TRANSFORM ERROR: '{}' β†’ '{}' failed to subscribe to input: {:?}", - input_key, output_key, _e - ); return; } }; + let mut reader = consumer.subscribe(); #[cfg(feature = "tracing")] tracing::debug!( @@ -220,7 +198,7 @@ pub(crate) async fn run_single_transform( match reader.recv().await { Ok(input_value) => { if let Some(output_value) = transform_fn(&input_value, &mut state) { - let _ = producer.produce(output_value).await; + producer.produce(output_value); } } Err(crate::DbError::BufferLagged { .. }) => { diff --git a/aimdb-core/src/typed_api.rs b/aimdb-core/src/typed_api.rs index 4a6fce0..ec14c78 100644 --- a/aimdb-core/src/typed_api.rs +++ b/aimdb-core/src/typed_api.rs @@ -16,7 +16,7 @@ //! ) { //! loop { //! let temp = read_sensor().await; -//! producer.produce(temp).await?; +//! producer.produce(temp); //! ctx.time().sleep(ctx.time().secs(1)).await; //! } //! } @@ -30,7 +30,7 @@ //! ctx: RuntimeContext, //! consumer: Consumer, //! ) { -//! let mut rx = consumer.subscribe()?; +//! let mut rx = consumer.subscribe(); //! while let Ok(temp) = rx.recv().await { //! ctx.log().info(&format!("Temp: {:.1}Β°C", temp.celsius)); //! } @@ -132,22 +132,16 @@ where /// Produce a value of type T /// - /// This triggers the entire pipeline for this record type: - /// 1. All tap observers are notified - /// 2. All link connectors are triggered - /// 3. Buffers are updated (if configured) + /// Push to the record's buffer, update the latest-snapshot cache, and + /// notify tap observers + outbound link connectors. Synchronous and + /// infallible β€” the underlying `WriteHandle::push` cannot fail. /// - /// Signature is kept `async fn ... -> DbResult<()>` for source-level - /// compatibility with existing `producer.produce(x).await?` call sites and - /// to leave headroom for future backpressure-aware buffers β€” even though - /// the body is currently synchronous and infallible. - pub async fn produce(&self, value: T) -> DbResult<()> { + pub fn produce(&self, value: T) { #[cfg(feature = "profiling")] if let Some(state) = &self.profiling { state.record_produce(); } self.write.push(value); - Ok(()) } } @@ -172,18 +166,16 @@ where value: Box, ) -> Pin> + Send + 'a>> { Box::pin(async move { - // Downcast the Box to Box + // The only fallibility left is the type-erasure downcast; the + // produce itself is synchronous + infallible after M14. let value = value.downcast::().map_err(|_| { format!( "Failed to downcast value to type {}", core::any::type_name::() ) })?; - - // Produce the value - self.produce(*value) - .await - .map_err(|e| format!("Failed to produce value: {}", e)) + self.produce(*value); + Ok(()) }) } } @@ -242,23 +234,21 @@ where self.profiling = Some((metrics, clock)); } - /// Subscribe to updates for this record type - /// - /// Returns a reader that yields values when they are produced. + /// Subscribe to updates for this record type. /// - /// Signature is kept as `DbResult<…>` for source-level compatibility; the - /// body is now infallible (the buffer is pre-resolved at construction). - pub fn subscribe(&self) -> DbResult + Send>> { + /// Returns a reader that yields values as they are produced. + /// Infallible β€” the buffer is pre-resolved at `Consumer` construction. + pub fn subscribe(&self) -> Box + Send> { let reader = self.buffer.subscribe_boxed(); #[cfg(feature = "profiling")] if let Some((metrics, clock)) = &self.profiling { - return Ok(Box::new(crate::profiling::ProfilingBufferReader::new( + return Box::new(crate::profiling::ProfilingBufferReader::new( reader, metrics.clone(), clock.clone(), - ))); + )); } - Ok(reader) + reader } } @@ -310,7 +300,9 @@ where ) -> Pin>> + Send + 'a>> { Box::pin(async move { - let reader = self.subscribe()?; + // `subscribe()` is infallible after M14; keep the `DbResult` on + // the trait surface so connector code stays unchanged. + let reader = self.subscribe(); Ok(Box::new(TypedAnyReader:: { inner: reader }) as Box) }) diff --git a/aimdb-core/src/typed_record.rs b/aimdb-core/src/typed_record.rs index db0c215..2f3b16d 100644 --- a/aimdb-core/src/typed_record.rs +++ b/aimdb-core/src/typed_record.rs @@ -730,7 +730,7 @@ impl( { let log = ctx.log(); - let Ok(mut reader) = consumer.subscribe() else { - log.error(&alloc::format!("Failed to subscribe to {} buffer", T::NAME)); - return; - }; + let mut reader = consumer.subscribe(); while let Ok(value) = reader.recv().await { log.info(&value.format_log(node_id)); diff --git a/aimdb-knx-connector/tests/topic_provider_tests.rs b/aimdb-knx-connector/tests/topic_provider_tests.rs index cc7792e..bfaf44a 100644 --- a/aimdb-knx-connector/tests/topic_provider_tests.rs +++ b/aimdb-knx-connector/tests/topic_provider_tests.rs @@ -297,7 +297,7 @@ async fn test_knx_topic_provider_registration_api() { let counter = counter.clone(); async move { let dimmer = DimmerValue::new("living", 200); - producer.produce(dimmer).await.ok(); + producer.produce(dimmer); counter.fetch_add(1, Ordering::SeqCst); } }, @@ -326,7 +326,7 @@ async fn test_knx_topic_provider_with_connector_registration() { .source( |_ctx: RuntimeContext, producer: Producer| async move { let dimmer = DimmerValue::new("living", 200); - producer.produce(dimmer).await.ok(); + producer.produce(dimmer); }, ) .link_to("knx://1/0/0") // Fallback group address @@ -385,7 +385,7 @@ async fn test_hvac_zone_routing() { // Different zones get routed to different group addresses for zone in 1..=4 { let setpoint = TemperatureSetpoint::new(zone, 21.0 + zone as f32 * 0.5); - producer.produce(setpoint).await.ok(); + producer.produce(setpoint); } }, ) diff --git a/aimdb-mqtt-connector/tests/topic_provider_tests.rs b/aimdb-mqtt-connector/tests/topic_provider_tests.rs index 8148a4b..70ae7b4 100644 --- a/aimdb-mqtt-connector/tests/topic_provider_tests.rs +++ b/aimdb-mqtt-connector/tests/topic_provider_tests.rs @@ -238,7 +238,7 @@ async fn test_topic_provider_registration_api() { let counter = counter.clone(); async move { let temp = Temperature::new("test-001", 22.5); - producer.produce(temp).await.ok(); + producer.produce(temp); counter.fetch_add(1, Ordering::SeqCst); } }, @@ -269,7 +269,7 @@ async fn test_topic_provider_with_connector_registration() { .source( |_ctx: RuntimeContext, producer: Producer| async move { let temp = Temperature::new("kitchen", 22.5); - producer.produce(temp).await.ok(); + producer.produce(temp); }, ) .link_to("mqtt://sensors/temp/default") // Fallback topic @@ -328,10 +328,7 @@ async fn test_mixed_static_and_dynamic_topics() { reg.buffer(BufferCfg::SingleLatest) .source( |_ctx: RuntimeContext, producer: Producer| async move { - producer - .produce(Temperature::new("static", 20.0)) - .await - .ok(); + producer.produce(Temperature::new("static", 20.0)); }, ) .link_to("mqtt://sensors/temp/static-topic") @@ -344,10 +341,7 @@ async fn test_mixed_static_and_dynamic_topics() { reg.buffer(BufferCfg::SingleLatest) .source( |_ctx: RuntimeContext, producer: Producer| async move { - producer - .produce(Temperature::new("dynamic", 25.0)) - .await - .ok(); + producer.produce(Temperature::new("dynamic", 25.0)); }, ) .link_to("mqtt://sensors/temp/fallback") diff --git a/aimdb-persistence/src/ext.rs b/aimdb-persistence/src/ext.rs index 1b44c45..221f0a6 100644 --- a/aimdb-persistence/src/ext.rs +++ b/aimdb-persistence/src/ext.rs @@ -53,19 +53,7 @@ where // The second closure argument is the runtime context (Arc), // which we don't need β€” persistence is runtime-agnostic. self.tap_raw(move |consumer, _ctx| async move { - let mut reader = match consumer.subscribe() { - Ok(r) => r, - Err(e) => { - #[cfg(feature = "tracing")] - tracing::error!( - "Persistence subscriber for '{}' failed to subscribe: {:?}", - record_name, - e - ); - let _ = e; - return; - } - }; + let mut reader = consumer.subscribe(); loop { let value = match reader.recv().await { diff --git a/aimdb-sync/src/handle.rs b/aimdb-sync/src/handle.rs index e39523f..633b9cf 100644 --- a/aimdb-sync/src/handle.rs +++ b/aimdb-sync/src/handle.rs @@ -404,7 +404,7 @@ impl AimDbHandle { self.runtime_handle.spawn(async move { while let Some((value, result_tx)) = rx.recv().await { // Forward the value to the database's produce pipeline - let result = db.produce(&record_key, value).await; + let result = db.produce(&record_key, value); // Send the result back to the caller (may fail if caller dropped) let _ = result_tx.send(result); diff --git a/aimdb-tokio-adapter/tests/drain_integration_tests.rs b/aimdb-tokio-adapter/tests/drain_integration_tests.rs index d4983e8..347de22 100644 --- a/aimdb-tokio-adapter/tests/drain_integration_tests.rs +++ b/aimdb-tokio-adapter/tests/drain_integration_tests.rs @@ -136,7 +136,6 @@ async fn test_drain_cold_start_returns_empty() { sensor_id: "s1".to_string(), }, ) - .await .unwrap(); // Give the server time to start listening @@ -185,7 +184,6 @@ async fn test_drain_returns_accumulated_values() { sensor_id: format!("s{}", i), }, ) - .await .unwrap(); } @@ -239,7 +237,6 @@ async fn test_drain_sequential_only_new_values() { sensor_id: format!("batch1-{}", i), }, ) - .await .unwrap(); } tokio::time::sleep(std::time::Duration::from_millis(50)).await; @@ -261,7 +258,6 @@ async fn test_drain_sequential_only_new_values() { sensor_id: format!("batch2-{}", i), }, ) - .await .unwrap(); } tokio::time::sleep(std::time::Duration::from_millis(50)).await; @@ -311,7 +307,6 @@ async fn test_drain_with_limit() { sensor_id: format!("s{}", i), }, ) - .await .unwrap(); } tokio::time::sleep(std::time::Duration::from_millis(50)).await; @@ -371,7 +366,6 @@ async fn test_drain_single_latest_at_most_one() { // Write 5 values β€” SingleLatest overwrites, only last survives for i in 0..5 { db.produce::("test::Counter", Counter { value: i }) - .await .unwrap(); } tokio::time::sleep(std::time::Duration::from_millis(50)).await; @@ -460,7 +454,6 @@ async fn test_drain_with_ring_overflow() { // Write 20 values into capacity-4 ring β€” causes overflow for i in 0..20 { db.produce::("test::Counter", Counter { value: i }) - .await .unwrap(); } tokio::time::sleep(std::time::Duration::from_millis(50)).await; @@ -516,13 +509,11 @@ async fn test_drain_multiple_records_independent() { sensor_id: "temp".to_string(), }, ) - .await .unwrap(); } // Write to Counter only db.produce::("test::Counter", Counter { value: 42 }) - .await .unwrap(); tokio::time::sleep(std::time::Duration::from_millis(50)).await; @@ -567,7 +558,6 @@ async fn test_drain_response_structure() { sensor_id: "test-sensor".to_string(), }, ) - .await .unwrap(); tokio::time::sleep(std::time::Duration::from_millis(50)).await; @@ -614,7 +604,6 @@ async fn test_drain_with_zero_limit() { sensor_id: "s".to_string(), }, ) - .await .unwrap(); } tokio::time::sleep(std::time::Duration::from_millis(50)).await; @@ -664,7 +653,6 @@ async fn test_drain_independent_of_other_consumers() { sensor_id: "test".to_string(), }, ) - .await .unwrap(); } tokio::time::sleep(std::time::Duration::from_millis(100)).await; diff --git a/aimdb-tokio-adapter/tests/multi_instance_tests.rs b/aimdb-tokio-adapter/tests/multi_instance_tests.rs index 27f92b3..98dffbb 100644 --- a/aimdb-tokio-adapter/tests/multi_instance_tests.rs +++ b/aimdb-tokio-adapter/tests/multi_instance_tests.rs @@ -43,11 +43,9 @@ async fn test_multi_instance_same_type() { // Produce to each record separately db.produce::("sensors.indoor", Temperature { celsius: 22.0 }) - .await .unwrap(); db.produce::("sensors.outdoor", Temperature { celsius: -5.0 }) - .await .unwrap(); // Verify we can resolve both keys @@ -86,14 +84,8 @@ async fn test_producer() { let producer_b = db.producer::("sensor.b").unwrap(); // Produce values - producer_a - .produce(Temperature { celsius: 10.0 }) - .await - .unwrap(); - producer_b - .produce(Temperature { celsius: 20.0 }) - .await - .unwrap(); + producer_a.produce(Temperature { celsius: 10.0 }); + producer_b.produce(Temperature { celsius: 20.0 }); } /// Test: Key-based consumer works correctly @@ -120,8 +112,8 @@ async fn test_consumer() { let consumer_south = db.consumer::("zone.south").unwrap(); // Subscribe should work - let _reader_north = consumer_north.subscribe().unwrap(); - let _reader_south = consumer_south.subscribe().unwrap(); + let _reader_north = consumer_north.subscribe(); + let _reader_south = consumer_south.subscribe(); } /// Test: Single instance key-based lookup works @@ -141,7 +133,6 @@ async fn test_single_instance_key_lookup() { // Key-based produce should work db.produce::("single.temp", Temperature { celsius: 30.0 }) - .await .unwrap(); // Key-based subscribe should work @@ -163,9 +154,7 @@ async fn test_key_not_found_error() { tokio::spawn(runner.run()); // Try to produce to non-existent key - let result = db - .produce::("nonexistent.key", Temperature { celsius: 0.0 }) - .await; + let result = db.produce::("nonexistent.key", Temperature { celsius: 0.0 }); match result { Err(DbError::RecordKeyNotFound { key }) => { @@ -191,15 +180,13 @@ async fn test_type_mismatch_error() { tokio::spawn(runner.run()); // Try to produce AppConfig to a Temperature record - let result = db - .produce::( - "sensor.temp", - AppConfig { - debug: true, - name: "test".to_string(), - }, - ) - .await; + let result = db.produce::( + "sensor.temp", + AppConfig { + debug: true, + name: "test".to_string(), + }, + ); match result { Err(DbError::TypeMismatch { record_id, .. }) => { diff --git a/aimdb-tokio-adapter/tests/stage_profiling.rs b/aimdb-tokio-adapter/tests/stage_profiling.rs index 8628c59..123bdfb 100644 --- a/aimdb-tokio-adapter/tests/stage_profiling.rs +++ b/aimdb-tokio-adapter/tests/stage_profiling.rs @@ -32,14 +32,14 @@ async fn source_and_tap_stages_are_timed_and_named() { .source(|ctx, producer| async move { let mut n = 0u64; loop { - let _ = producer.produce(Reading { value: n }).await; + producer.produce(Reading { value: n }); n += 1; ctx.time().sleep(ctx.time().millis(10)).await; } }) .with_name("sensor_reader") .tap(|ctx, consumer| async move { - let mut reader = consumer.subscribe().expect("subscribe"); + let mut reader = consumer.subscribe(); while let Ok(reading) = reader.recv().await { // Simulate per-value processing work (wall-clock). let _ = reading.value; @@ -104,7 +104,7 @@ async fn with_name_is_a_no_op_friendly_builder() { builder.configure::("profiling::Unused", |reg| { reg.buffer(BufferCfg::SingleLatest) .tap(|_ctx, consumer| async move { - let mut reader = consumer.subscribe().expect("subscribe"); + let mut reader = consumer.subscribe(); let _ = reader.recv().await; }) .with_name("idle_tap"); diff --git a/aimdb-tokio-adapter/tests/transform_join_integration_tests.rs b/aimdb-tokio-adapter/tests/transform_join_integration_tests.rs index 6fb6baf..ba896b9 100644 --- a/aimdb-tokio-adapter/tests/transform_join_integration_tests.rs +++ b/aimdb-tokio-adapter/tests/transform_join_integration_tests.rs @@ -57,7 +57,7 @@ async fn transform_join_produces_sum_on_both_inputs() { _ => {} } if let (Some(a), Some(b)) = (last_a, last_b) { - let _ = producer.produce(Sum(a + b)).await; + producer.produce(Sum(a + b)); } } }) @@ -72,8 +72,8 @@ async fn transform_join_produces_sum_on_both_inputs() { tokio::task::yield_now().await; // A=1, B=10 β†’ Sum=11 (first time both are present) - db.produce::("test::A", ValueA(1)).await.unwrap(); - db.produce::("test::B", ValueB(10)).await.unwrap(); + db.produce::("test::A", ValueA(1)).unwrap(); + db.produce::("test::B", ValueB(10)).unwrap(); let s = tokio::time::timeout(Duration::from_secs(2), sum_rx.recv()) .await .unwrap() @@ -81,7 +81,7 @@ async fn transform_join_produces_sum_on_both_inputs() { assert_eq!(s.0, 11, "expected 1+10=11"); // A=2 β†’ Sum=12 (B stays 10) - db.produce::("test::A", ValueA(2)).await.unwrap(); + db.produce::("test::A", ValueA(2)).unwrap(); let s = tokio::time::timeout(Duration::from_secs(2), sum_rx.recv()) .await .unwrap() @@ -89,7 +89,7 @@ async fn transform_join_produces_sum_on_both_inputs() { assert_eq!(s.0, 12, "expected 2+10=12"); // B=20 β†’ Sum=22 (A stays 2) - db.produce::("test::B", ValueB(20)).await.unwrap(); + db.produce::("test::B", ValueB(20)).unwrap(); let s = tokio::time::timeout(Duration::from_secs(2), sum_rx.recv()) .await .unwrap() @@ -125,7 +125,7 @@ async fn transform_join_bounded_fanin_backpressure_no_deadlock() { // Yield between receives to keep the fan-in channel // pressured well above its 64-slot capacity. tokio::task::yield_now().await; - let _ = producer.produce(Sum(v.0)).await; + producer.produce(Sum(v.0)); } } }) @@ -146,7 +146,6 @@ async fn transform_join_bounded_fanin_backpressure_no_deadlock() { loop { warmup_db .produce::("stress::A", ValueA(SENTINEL)) - .await .unwrap(); tokio::time::sleep(Duration::from_millis(5)).await; } @@ -180,7 +179,6 @@ async fn transform_join_bounded_fanin_backpressure_no_deadlock() { for i in 0..N { producer_db .produce::("stress::A", ValueA(i)) - .await .unwrap(); } }); diff --git a/aimdb-wasm-adapter/src/bindings.rs b/aimdb-wasm-adapter/src/bindings.rs index 768629e..80cbbca 100644 --- a/aimdb-wasm-adapter/src/bindings.rs +++ b/aimdb-wasm-adapter/src/bindings.rs @@ -550,10 +550,7 @@ where .get_typed_record_by_key::(key) .map_err(|e| JsError::new(&format!("{e:?}")))?; - // TypedRecord::produce() is declared `async` but its body is synchronous: - // it updates `latest_snapshot` and calls `buf.push(val)` β€” both complete - // immediately on WasmBuffer. We poll the future exactly once. - poll_sync(typed.produce(val)); + typed.produce(val); Ok(()) } @@ -620,31 +617,3 @@ where } // ─── Sync future polling ────────────────────────────────────────────────── - -/// Poll a future that is known to resolve in a single poll (no real I/O). -/// -/// Used for `TypedRecord::produce()` whose body is synchronous despite being -/// declared `async fn` β€” it just updates a snapshot and pushes to a buffer. -/// -/// # Panics -/// -/// Panics if the future returns `Pending`. This should never happen for -/// operations on `WasmBuffer` (which are single-threaded, non-blocking). -pub(crate) fn poll_sync(f: F) -> F::Output { - use core::pin::Pin; - use core::task::{Context, Poll, Waker}; - - // SAFETY: the future is stack-local and will not be moved after pinning. - let mut f = f; - let f = unsafe { Pin::new_unchecked(&mut f) }; - - let waker = Waker::noop(); - let mut cx = Context::from_waker(waker); - - match f.poll(&mut cx) { - Poll::Ready(val) => val, - Poll::Pending => { - panic!("poll_sync: future returned Pending (expected synchronous completion)") - } - } -} diff --git a/aimdb-wasm-adapter/src/ws_bridge.rs b/aimdb-wasm-adapter/src/ws_bridge.rs index db86b21..17515bf 100644 --- a/aimdb-wasm-adapter/src/ws_bridge.rs +++ b/aimdb-wasm-adapter/src/ws_bridge.rs @@ -757,7 +757,7 @@ where let inner = db.inner(); match inner.get_typed_record_by_key::(key) { Ok(typed) => { - crate::bindings::poll_sync(typed.produce(val)); + typed.produce(val); } Err(e) => { web_sys::console::warn_1( diff --git a/aimdb-wasm-adapter/tests/transform_join_integration_tests.rs b/aimdb-wasm-adapter/tests/transform_join_integration_tests.rs index 9924222..a99199f 100644 --- a/aimdb-wasm-adapter/tests/transform_join_integration_tests.rs +++ b/aimdb-wasm-adapter/tests/transform_join_integration_tests.rs @@ -57,7 +57,7 @@ async fn transform_join_produces_sum_on_both_inputs() { _ => {} } if let (Some(a), Some(b)) = (last_a, last_b) { - let _ = producer.produce(Sum(a + b)).await; + producer.produce(Sum(a + b)); } } }) diff --git a/examples/embassy-knx-connector-demo/src/main.rs b/examples/embassy-knx-connector-demo/src/main.rs index b145709..7b8b980 100644 --- a/examples/embassy-knx-connector-demo/src/main.rs +++ b/examples/embassy-knx-connector-demo/src/main.rs @@ -115,17 +115,7 @@ async fn button_handler( let state = LightControl::new("1/0/6", light_on); - match producer.produce(state).await { - Ok(_) => { - log.info(&alloc::format!( - "βœ… Published to KNX: 1/0/6 = {}", - if light_on { "ON ✨" } else { "OFF" } - )); - } - Err(e) => { - log.error(&alloc::format!("❌ Failed to publish: {:?}", e)); - } - } + producer.produce(state); button.wait_for_rising_edge().await; time.sleep(time.millis(50)).await; diff --git a/examples/embassy-mqtt-connector-demo/src/main.rs b/examples/embassy-mqtt-connector-demo/src/main.rs index b00f0c5..789a6a5 100644 --- a/examples/embassy-mqtt-connector-demo/src/main.rs +++ b/examples/embassy-mqtt-connector-demo/src/main.rs @@ -114,9 +114,7 @@ async fn indoor_temp_producer( temp.celsius )); - if let Err(e) = temperature.produce(temp).await { - log.error(&alloc::format!("❌ Failed to produce indoor temp: {:?}", e)); - } + temperature.produce(temp); Timer::after(Duration::from_secs(2)).await; } @@ -140,12 +138,7 @@ async fn outdoor_temp_producer( temp.celsius )); - if let Err(e) = temperature.produce(temp).await { - log.error(&alloc::format!( - "❌ Failed to produce outdoor temp: {:?}", - e - )); - } + temperature.produce(temp); Timer::after(Duration::from_secs(2)).await; } @@ -169,12 +162,7 @@ async fn server_room_temp_producer( temp.celsius )); - if let Err(e) = temperature.produce(temp).await { - log.error(&alloc::format!( - "❌ Failed to produce server room temp: {:?}", - e - )); - } + temperature.produce(temp); Timer::after(Duration::from_secs(2)).await; } diff --git a/examples/hello-single-latest-async/src/main.rs b/examples/hello-single-latest-async/src/main.rs index 5cdea5e..2b3b124 100644 --- a/examples/hello-single-latest-async/src/main.rs +++ b/examples/hello-single-latest-async/src/main.rs @@ -49,17 +49,11 @@ async fn rollout_source(ctx: RuntimeContext, producer: Producer, rollout_percent: u8) { let gate = FeatureGate { rollout_percent }; - match producer.produce(gate).await { - Ok(()) => println!("source published rollout: {rollout_percent}%"), - Err(err) => eprintln!("failed to publish rollout {rollout_percent}%: {err}"), - } + producer.produce(gate); } async fn rollout_observer(ctx: RuntimeContext, consumer: Consumer) { - let Ok(mut reader) = consumer.subscribe() else { - eprintln!("failed to subscribe to config.checkout_rollout"); - return; - }; + let mut reader = consumer.subscribe(); let time = ctx.time(); let mut first = true; diff --git a/examples/knx-connector-demo-common/src/monitors.rs b/examples/knx-connector-demo-common/src/monitors.rs index f26b626..0336a97 100644 --- a/examples/knx-connector-demo-common/src/monitors.rs +++ b/examples/knx-connector-demo-common/src/monitors.rs @@ -33,10 +33,7 @@ where let log = ctx.log(); log.info("🌑️ Temperature monitor started\n"); - let Ok(mut reader) = consumer.subscribe() else { - log.error("Failed to subscribe to temperature buffer"); - return; - }; + let mut reader = consumer.subscribe(); while let Ok(temp) = reader.recv().await { let (whole, frac) = temp.display_parts(); @@ -63,10 +60,7 @@ where let log = ctx.log(); log.info("πŸ’‘ Light monitor started\n"); - let Ok(mut reader) = consumer.subscribe() else { - log.error("Failed to subscribe to light buffer"); - return; - }; + let mut reader = consumer.subscribe(); while let Ok(state) = reader.recv().await { log.info(&alloc::format!( diff --git a/examples/mqtt-connector-demo-common/src/monitors.rs b/examples/mqtt-connector-demo-common/src/monitors.rs index bb7a9c7..e2c8130 100644 --- a/examples/mqtt-connector-demo-common/src/monitors.rs +++ b/examples/mqtt-connector-demo-common/src/monitors.rs @@ -31,10 +31,7 @@ where { let log = ctx.log(); - let Ok(mut reader) = consumer.subscribe() else { - log.error("Failed to subscribe to temperature buffer"); - return; - }; + let mut reader = consumer.subscribe(); while let Ok(temp) = reader.recv().await { log.info(&alloc::format!( @@ -60,10 +57,7 @@ where let log = ctx.log(); log.info("πŸ“¨ Command consumer started\n"); - let Ok(mut reader) = consumer.subscribe() else { - log.error("Failed to subscribe to command buffer"); - return; - }; + let mut reader = consumer.subscribe(); while let Ok(cmd) = reader.recv().await { log.info(&alloc::format!( diff --git a/examples/remote-access-demo/src/server.rs b/examples/remote-access-demo/src/server.rs index 12b00ee..db09990 100644 --- a/examples/remote-access-demo/src/server.rs +++ b/examples/remote-access-demo/src/server.rs @@ -232,9 +232,7 @@ async fn temperature_simulator( timestamp, }; - if let Err(e) = temperature.produce(reading).await { - tracing::error!("Failed to produce temperature: {}", e); - } + temperature.produce(reading); counter += 1; time.sleep(time.secs(2)).await; @@ -244,10 +242,7 @@ async fn temperature_simulator( /// Fast tap on Temperature β€” just logs. Stage profiling will show it as /// substantially faster than `slow_status_processor` on SystemStatus. async fn temperature_logger(_ctx: RuntimeContext, consumer: Consumer) { - let Ok(mut reader) = consumer.subscribe() else { - tracing::error!("Failed to subscribe to Temperature"); - return; - }; + let mut reader = consumer.subscribe(); while let Ok(reading) = reader.recv().await { tracing::debug!( "🌑️ Logged temperature: {:.1} Β°C from {}", @@ -271,9 +266,7 @@ async fn system_status_simulator( memory_usage: 40.0 + ((uptime as f64 / 10.0) % 20.0), }; - if let Err(e) = status.produce(snapshot).await { - tracing::error!("Failed to produce system status: {}", e); - } + status.produce(snapshot); uptime += 5; time.sleep(time.secs(5)).await; @@ -288,10 +281,7 @@ async fn slow_status_processor( consumer: Consumer, ) { let time = ctx.time(); - let Ok(mut reader) = consumer.subscribe() else { - tracing::error!("Failed to subscribe to SystemStatus"); - return; - }; + let mut reader = consumer.subscribe(); while let Ok(snapshot) = reader.recv().await { time.sleep(time.millis(100)).await; tracing::debug!( diff --git a/examples/sync-api-demo/src/main.rs b/examples/sync-api-demo/src/main.rs index 92e2749..281d1ff 100644 --- a/examples/sync-api-demo/src/main.rs +++ b/examples/sync-api-demo/src/main.rs @@ -47,10 +47,7 @@ fn main() -> Result<(), Box> { reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) // Add a simple consumer that logs values .tap(|_ctx, consumer| async move { - let Ok(mut reader) = consumer.subscribe() else { - eprintln!("Failed to subscribe to temperature buffer"); - return; - }; + let mut reader = consumer.subscribe(); while let Ok(temp) = reader.recv().await { println!(" [Database] Received: {:.1}Β°C", temp.celsius); } diff --git a/examples/tokio-knx-connector-demo/src/main.rs b/examples/tokio-knx-connector-demo/src/main.rs index 9e53f29..8f53528 100644 --- a/examples/tokio-knx-connector-demo/src/main.rs +++ b/examples/tokio-knx-connector-demo/src/main.rs @@ -59,17 +59,7 @@ async fn input_handler(ctx: RuntimeContext, producer: Producer { - log.info(&format!( - "βœ… Published to KNX: 1/0/6 = {} (sent to bus)", - if light_on { "ON ✨" } else { "OFF" } - )); - } - Err(e) => { - log.error(&format!("❌ Failed to publish: {:?}", e)); - } - } + producer.produce(state); } Err(e) => { log.error(&format!("Error reading input: {}", e)); diff --git a/examples/tokio-mqtt-connector-demo/src/main.rs b/examples/tokio-mqtt-connector-demo/src/main.rs index 46f3f95..4c00cf3 100644 --- a/examples/tokio-mqtt-connector-demo/src/main.rs +++ b/examples/tokio-mqtt-connector-demo/src/main.rs @@ -66,9 +66,7 @@ async fn indoor_temp_producer( temp.celsius )); - if let Err(e) = temperature.produce(temp).await { - log.error(&format!("❌ Failed to produce indoor temp: {:?}", e)); - } + temperature.produce(temp); time.sleep(time.secs(2)).await; } @@ -94,9 +92,7 @@ async fn outdoor_temp_producer( temp.celsius )); - if let Err(e) = temperature.produce(temp).await { - log.error(&format!("❌ Failed to produce outdoor temp: {:?}", e)); - } + temperature.produce(temp); time.sleep(time.secs(2)).await; } @@ -122,9 +118,7 @@ async fn server_room_temp_producer( temp.celsius )); - if let Err(e) = temperature.produce(temp).await { - log.error(&format!("❌ Failed to produce server room temp: {:?}", e)); - } + temperature.produce(temp); time.sleep(time.secs(2)).await; } diff --git a/examples/weather-mesh-demo/weather-station-alpha/src/main.rs b/examples/weather-mesh-demo/weather-station-alpha/src/main.rs index 9dda4b0..a6ba886 100644 --- a/examples/weather-mesh-demo/weather-station-alpha/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-alpha/src/main.rs @@ -151,13 +151,9 @@ async fn main() -> Result<(), Box> { timestamp: now, }; - if let Err(e) = temp_producer.produce(temp).await { - tracing::warn!("Failed to produce temperature: {:?}", e); - } + temp_producer.produce(temp); - if let Err(e) = humidity_producer.produce(humidity).await { - tracing::warn!("Failed to produce humidity: {:?}", e); - } + humidity_producer.produce(humidity); tracing::info!( "πŸ“Š Published: {:.1}Β°C, {:.1}%", diff --git a/examples/weather-mesh-demo/weather-station-beta/src/main.rs b/examples/weather-mesh-demo/weather-station-beta/src/main.rs index 0e38082..20a3017 100644 --- a/examples/weather-mesh-demo/weather-station-beta/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-beta/src/main.rs @@ -174,13 +174,9 @@ async fn main() -> Result<(), Box> { Humidity::simulate(&humidity_config, prev_humidity.as_ref(), &mut rng, now); // Produce the readings - if let Err(e) = temp_producer.produce(temp.clone()).await { - tracing::warn!("Failed to produce temperature: {:?}", e); - } + temp_producer.produce(temp.clone()); - if let Err(e) = humidity_producer.produce(humidity.clone()).await { - tracing::warn!("Failed to produce humidity: {:?}", e); - } + humidity_producer.produce(humidity.clone()); tracing::info!( "πŸ“Š Published: {:.1}Β°C, {:.1}%", diff --git a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs index 925e4a2..a41986b 100644 --- a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs @@ -94,9 +94,7 @@ async fn temperature_producer( log.info(&alloc::format!("πŸ“Š Temp: {:.1}Β°C", temp.celsius)); - if let Err(e) = producer.produce(temp.clone()).await { - log.error(&alloc::format!("❌ Failed to produce: {:?}", e)); - } + producer.produce(temp.clone()); prev = Some(temp); Timer::after(Duration::from_secs(5)).await; @@ -128,9 +126,7 @@ async fn humidity_producer(ctx: RuntimeContext, producer: Produc log.info(&alloc::format!("πŸ“Š Humidity: {:.1}%", humidity.percent)); - if let Err(e) = producer.produce(humidity.clone()).await { - log.error(&alloc::format!("❌ Failed to produce: {:?}", e)); - } + producer.produce(humidity.clone()); prev = Some(humidity); Timer::after(Duration::from_secs(5)).await; From e7ed5da835fe42c2f96b6679b387a1397f2885de Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Mon, 25 May 2026 08:06:52 +0000 Subject: [PATCH 08/14] feat: simplify DewPoint production by removing await and error handling --- .../weather-station-alpha/src/main.rs | 13 ++++--------- .../weather-station-beta/src/main.rs | 13 ++++--------- .../weather-station-gamma/src/main.rs | 9 ++------- 3 files changed, 10 insertions(+), 25 deletions(-) diff --git a/examples/weather-mesh-demo/weather-station-alpha/src/main.rs b/examples/weather-mesh-demo/weather-station-alpha/src/main.rs index a6ba886..6817977 100644 --- a/examples/weather-mesh-demo/weather-station-alpha/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-alpha/src/main.rs @@ -98,15 +98,10 @@ async fn main() -> Result<(), Box> { t.celsius, h.percent ); - if let Err(e) = producer - .produce(DewPoint { - celsius: dew_point, - timestamp, - }) - .await - { - tracing::warn!("DewPoint produce failed: {:?}", e); - } + producer.produce(DewPoint { + celsius: dew_point, + timestamp, + }); } } }) diff --git a/examples/weather-mesh-demo/weather-station-beta/src/main.rs b/examples/weather-mesh-demo/weather-station-beta/src/main.rs index 20a3017..6c956dc 100644 --- a/examples/weather-mesh-demo/weather-station-beta/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-beta/src/main.rs @@ -98,15 +98,10 @@ async fn main() -> Result<(), Box> { t.celsius, h.percent ); - if let Err(e) = producer - .produce(DewPoint { - celsius: dew_point, - timestamp, - }) - .await - { - tracing::warn!("DewPoint produce failed: {:?}", e); - } + producer.produce(DewPoint { + celsius: dew_point, + timestamp, + }); } } }) diff --git a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs index a41986b..962c704 100644 --- a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs @@ -335,15 +335,10 @@ async fn main(spawner: Spawner) { let frac = ((mag - whole as f32) * 10.0 + 0.5) as i32 % 10; let sign = if neg { "-" } else { "" }; info!("πŸ“Š DewPoint: {}{}.{}Β°C", sign, whole, frac); - if let Err(_e) = producer - .produce(DewPoint { + producer.produce(DewPoint { celsius: dew_point, timestamp, - }) - .await - { - defmt::warn!("DewPoint produce failed"); - } + }); } } }) From be36514993e69a71217d417138d3a29a37823cb5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Mon, 25 May 2026 19:34:31 +0000 Subject: [PATCH 09/14] feat: add "macros" feature to Tokio dependency in Cargo.toml --- aimdb-sync/Cargo.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aimdb-sync/Cargo.toml b/aimdb-sync/Cargo.toml index 52fb92e..671c113 100644 --- a/aimdb-sync/Cargo.toml +++ b/aimdb-sync/Cargo.toml @@ -15,7 +15,7 @@ aimdb-core = { path = "../aimdb-core", version = "1.1.0" } aimdb-tokio-adapter = { path = "../aimdb-tokio-adapter", version = "0.6.0" } # Tokio for channels and runtime -tokio = { version = "1.40", features = ["sync", "rt", "time"] } +tokio = { version = "1.40", features = ["sync", "rt", "time", "macros"] } # Error handling thiserror = "1.0" From 19d44572a4bbfc41531e1b603183092442625fb8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Mon, 25 May 2026 20:00:10 +0000 Subject: [PATCH 10/14] feat: update documentation and design notes for removing `Spawn` trait and `R` parameter from `Producer` and `Consumer` --- aimdb-core/CHANGELOG.md | 2 +- aimdb-core/src/builder.rs | 9 +-- aimdb-core/src/typed_api.rs | 1 - aimdb-core/src/typed_record.rs | 46 ++----------- docs/design/028-M13-remove-spawn-trait.md | 28 ++++++-- .../029-M14-remove-r-from-typed-handles.md | 67 ++++++++++++------- 6 files changed, 80 insertions(+), 73 deletions(-) diff --git a/aimdb-core/CHANGELOG.md b/aimdb-core/CHANGELOG.md index c697b87..eccd491 100644 --- a/aimdb-core/CHANGELOG.md +++ b/aimdb-core/CHANGELOG.md @@ -19,7 +19,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **`Producer` / `Consumer` drop the runtime parameter `R` and pre-resolve the record at build time (Design 029, M14).** Producer/Consumer become handles to a buffer rather than tickets to look one up: `produce()` is one virtual call (no `HashMap` probe, no `TypeId` check, no downcast), and `subscribe()` collapses to `buffer.subscribe_boxed()`. The internal mechanic is a new crate-private `WriteHandle` trait backed by `RecordWriter` (in `aimdb-core/src/buffer/writer.rs`), pre-bound to the record's `Arc>` + snapshot mutex + metadata tracker. - `Producer` β†’ `Producer`; `Consumer` β†’ `Consumer`. User code that names the two-parameter form must drop the trailing adapter arg. - `Producer::key(&self) -> &str` is **removed**. Capture the record key at the registration site instead. - - `Producer::produce(value).await -> DbResult<()>` and `Consumer::subscribe() -> DbResult<…>` keep their signatures (bodies simplified, infallible internally β€” kept for source-level compat and backpressure headroom). + - `Producer::produce(value) -> ()` and `Consumer::subscribe() -> Box + Send>` (v0.4 revision β€” see the sync/infallible bullet above for the rationale and migration). The `ProducerTrait::produce_any` / `ConsumerTrait::subscribe_any` trait surfaces retain `async`/`Result` for the type-erased downcast that can still fail. - `AimDb::producer(key)` / `AimDb::consumer(key)` now return `DbResult<…>` (was infallible). They resolve the typed record up front, so callers that previously assumed inference must add `?`. - `Consumer` cannot exist without a buffer: `.tap()` on a record with no `.buffer(...)` now surfaces as `MissingConfiguration` at build time (was a deferred subscribe-time error). - `TypedRecord::buffer` field is `Option>>` (was `Box`); `TypedRecord::set_buffer(Box<…>)` keeps its public signature and converts via `Arc::from(box_)` internally. diff --git a/aimdb-core/src/builder.rs b/aimdb-core/src/builder.rs index 102fa5e..02d236b 100644 --- a/aimdb-core/src/builder.rs +++ b/aimdb-core/src/builder.rs @@ -938,8 +938,7 @@ where } } - // Unwrap the Arc to return the owned AimDb (safe: we just created it and hold the only ref). - let db_owned = Arc::try_unwrap(db).unwrap_or_else(|arc| (*arc).clone()); + let db_owned = (*db).clone(); Ok((db_owned, AimDbRunner::new(futures_acc))) } @@ -1028,8 +1027,10 @@ impl AimDbRunner { /// Drives every collected future to completion. /// /// Returns when all futures complete (normally: never, while producer and - /// consumer loops are alive). Drop the runner β€” or wrap this call in a - /// `select!` with a shutdown signal β€” to cancel. + /// consumer loops are alive). To cancel, wrap this call in `tokio::select!` + /// (or the analogous primitive on Embassy / WASM) with a shutdown signal β€” + /// when the `select!` arm fires, this future is dropped and every + /// `FuturesUnordered` entry is cancelled with it. pub async fn run(self) { use futures_util::stream::{FuturesUnordered, StreamExt}; diff --git a/aimdb-core/src/typed_api.rs b/aimdb-core/src/typed_api.rs index ec14c78..77f7d03 100644 --- a/aimdb-core/src/typed_api.rs +++ b/aimdb-core/src/typed_api.rs @@ -404,7 +404,6 @@ where where F: FnOnce(crate::Producer, Arc) -> Fut + Send - + Sync + 'static, Fut: Future + Send + 'static, { diff --git a/aimdb-core/src/typed_record.rs b/aimdb-core/src/typed_record.rs index 2f3b16d..fb3e5b4 100644 --- a/aimdb-core/src/typed_record.rs +++ b/aimdb-core/src/typed_record.rs @@ -223,11 +223,12 @@ type ConsumerServiceFn = Box< /// Type alias for producer service closure stored in TypedRecord /// Takes (Producer, RuntimeContext) and returns a Future -/// This will be auto-spawned during build() +/// This will be auto-spawned during build(). +/// `Send` only β€” `Sync` is not required because the closure is taken out of +/// `Mutex>` exactly once via `.take()`, and `Mutex: Sync` +/// already holds when `T: Send`. Matches the consumer-side `ConsumerServiceFn`. type ProducerServiceFn = Box< - dyn FnOnce(crate::Producer, Arc) -> BoxFuture<'static, ()> - + Send - + Sync, + dyn FnOnce(crate::Producer, Arc) -> BoxFuture<'static, ()> + Send, >; /// Type-erased trait for records @@ -672,7 +673,7 @@ impl(&mut self, f: F) where - F: FnOnce(crate::Producer, Arc) -> Fut + Send + Sync + 'static, + F: FnOnce(crate::Producer, Arc) -> Fut + Send + 'static, Fut: core::future::Future + Send + 'static, { // Check for existing transform (mutual exclusion) @@ -1713,40 +1714,7 @@ impl() ); - // Check if buffer exists before trying to produce - if self.buffer.is_none() { - return Err(DbError::RuntimeError { - message: format!( - "Record '{}' has no buffer configured. \ - Cannot produce value without buffer.", - core::any::type_name::() - ), - }); - } - - // Use the existing produce() method which handles: - // 1. Updating latest_snapshot - // 2. Pushing to buffer - // 3. Updating metadata timestamp (std only) - // This is a synchronous wrapper around the async produce() - { - #[cfg(feature = "std")] - { - *self.latest_snapshot.lock().unwrap() = Some(value.clone()); - } - - #[cfg(not(feature = "std"))] - { - *self.latest_snapshot.lock() = Some(value.clone()); - } - - if let Some(buf) = &self.buffer { - buf.push(value); - - #[cfg(feature = "std")] - self.metadata.mark_updated(); - } - } + self.produce(value); #[cfg(feature = "tracing")] tracing::info!( diff --git a/docs/design/028-M13-remove-spawn-trait.md b/docs/design/028-M13-remove-spawn-trait.md index 7666a84..5eea1cb 100644 --- a/docs/design/028-M13-remove-spawn-trait.md +++ b/docs/design/028-M13-remove-spawn-trait.md @@ -1,7 +1,7 @@ # Remove `Spawn` Trait β€” `build()` Collects, `run()` Drives -**Version:** 0.2 (draft β€” revised after code audit) -**Status:** πŸ“ Design +**Version:** 0.3 (implemented β€” `unsafe impl` outcome on Embassy corrected) +**Status:** βœ… Implemented **Issue:** [#88](https://github.com/aimdb-dev/aimdb/issues/88) **Follow-up issue:** [AimX remote-access portability](../issues/aimx-remote-spawn-free.md) β€” TBD **Last Updated:** May 23, 2026 @@ -517,8 +517,28 @@ used in `impl Spawn for EmbassyAdapter`). **Remove the `spawner` field.** After removing `spawner`, `EmbassyAdapter` becomes a simple struct holding only an `Option<&'static Stack<'static>>` (the network stack, gated by -`embassy-net-support`). `&'static Stack<'static>` is `Send + Sync`, so -`EmbassyAdapter` auto-derives `Send + Sync`. **Remove the `unsafe impl`.** +`embassy-net-support`). + +> **Implementation note (v0.3).** The earlier draft assumed +> `&'static Stack<'static>` would be `Send + Sync` and the `unsafe impl` +> blocks could be deleted outright. In practice `embassy_net::Stack` +> contains a `RefCell` and is `!Sync`, so the adapter still needs: +> +> ```rust +> // SAFETY: Embassy executors run cooperatively on a single core. The +> // `embassy_net::Stack` (when present) is `!Sync` only because of its +> // internal `RefCell`; in a single-threaded executor no concurrent access +> // is possible. +> unsafe impl Send for EmbassyAdapter {} +> unsafe impl Sync for EmbassyAdapter {} +> ``` +> +> The *justification* changes (single-threaded cooperative executor, no +> longer "satisfy the `Spawn` trait's `F: Send` bound") and the audit-trail +> comment is rewritten accordingly, but the `unsafe` blocks themselves +> stay. The net `unsafe`-block count on the adapter goes from +> [Send, Sync, Pin::new_unchecked in task pool] to just [Send, Sync] β€” +> the task-pool `Pin::new_unchecked` is the one that actually disappears. The `new_with_spawner()` constructor is removed (breaking change β€” see [Breaking Changes](#breaking-changes)). diff --git a/docs/design/029-M14-remove-r-from-typed-handles.md b/docs/design/029-M14-remove-r-from-typed-handles.md index cf43581..d63c222 100644 --- a/docs/design/029-M14-remove-r-from-typed-handles.md +++ b/docs/design/029-M14-remove-r-from-typed-handles.md @@ -1,6 +1,6 @@ # Remove `R` from `Producer` and `Consumer` -**Version:** 0.3 (implemented) +**Version:** 0.4 (implemented β€” produce/subscribe collapsed to sync) **Status:** βœ… Implemented **Issue:** TBD **Depends on:** [M13 β€” Remove `Spawn` Trait](028-M13-remove-spawn-trait.md) (#88) @@ -318,27 +318,35 @@ Gone: the `db: Arc>` field, the `record_key: String` field, the `produce()` becomes: ```rust -pub async fn produce(&self, value: T) -> DbResult<()> { +pub fn produce(&self, value: T) { #[cfg(feature = "profiling")] if let Some(state) = &self.profiling { state.record_produce(); } self.write.push(value); - Ok(()) } ``` -**Signature stability:** `async fn ... -> DbResult<()>` is *intentionally -kept* even though the body is synchronous and infallible. Reasons: - -1. Source-level compatibility β€” every existing `producer.produce(x).await?` - call site keeps working unchanged. -2. Forward compatibility β€” future backpressure-aware buffers (async - `push`) or shutdown-signalling semantics (`Err(ChannelClosed)`) can be - added without another breaking change. - -Cost is negligible (an `Ok` wrapper and a no-op await), both compile to -nothing. +**Signature change (v0.4 revision).** The earlier draft preserved +`async fn ... -> DbResult<()>` for source-level compatibility. After +implementation we reversed that decision: every reachable code path is +synchronous and infallible (one virtual `WriteHandle::push`), so the wrapper +was pure ceremony. Keeping `async`/`Result` would have meant `.await?` at +every call site forever β€” propagating fictional fallibility to please a +type signature. + +**Trade-off accepted.** `producer.produce(x).await?` no longer compiles β€” +call sites drop both the `.await` and the `?`. This is a one-time edit on +~50 examples / aimdb-pro sites and is called out as a breaking change in +the aimdb-core 1.1.0 CHANGELOG. If backpressure-aware buffers or +shutdown-signalling semantics show up in a future milestone, they can be +introduced as a new `produce_async(&self, value: T) -> impl Future>` method alongside the sync one β€” additive, not breaking. + +The type-erased trait surface keeps a `Future>`: +`ProducerTrait::produce_any` still returns a pinned future because its +fallibility is real (the `Any` downcast can fail when a route delivers the +wrong type). ### `Consumer` β€” after @@ -354,19 +362,28 @@ pub struct Consumer { `subscribe()` collapses to: ```rust -pub fn subscribe(&self) -> DbResult + Send>> { +pub fn subscribe(&self) -> Box + Send> { let reader = self.buffer.subscribe_boxed(); #[cfg(feature = "profiling")] if let Some((metrics, clock)) = &self.profiling { - return Ok(Box::new(ProfilingBufferReader::new(reader, metrics.clone(), clock.clone()))); + return Box::new(ProfilingBufferReader::new(reader, metrics.clone(), clock.clone())); } - Ok(reader) + reader } ``` `DynBuffer` already exists and is the natural read-side counterpart β€” no new trait required. +Same v0.4 revision as `produce()`: the buffer Arc is pre-resolved at +`Consumer::new`, so the lookup that previously produced `DbResult` is gone +and there is nothing left to fail. The `DbResult<>` wrapper was dropped to +avoid forcing a `.unwrap()` / `?` at every call site. The type-erased +`ConsumerTrait::subscribe_any` keeps its `DbResult` (factories still resolve +records through a downcast that can fail) β€” see `impl ConsumerTrait for +Consumer` in `typed_api.rs`, which wraps the infallible inner call in +`Ok(...)`. + ### `ConnectorBuilder` β€” cascade (zero-LOC) `ConnectorBuilder` keeps its trait-level `R` parameter (v0.1's Option A). @@ -399,8 +416,8 @@ deferred to a separate milestone (needs a `ConnectorContext` abstraction). | `Producer` | `Producer` | | `Consumer` | `Consumer` | | `Producer::key(&self) -> &str` | **removed** | -| `Producer::produce(&self, T) -> impl Future>` | unchanged signature (body simplified) | -| `Consumer::subscribe(&self) -> DbResult<…>` | unchanged signature (body simplified) | +| `Producer::produce(&self, T) -> impl Future>` | `pub fn produce(&self, T)` β€” synchronous, infallible (v0.4 revision) | +| `Consumer::subscribe(&self) -> DbResult<…>` | `pub fn subscribe(&self) -> Box + Send>` β€” infallible (v0.4 revision) | | `TypedRecord::buffer: Option>>` | `Option>>` | | `RecordRegistrar` (build-time only) | unchanged β€” still needs `R` to resolve records | | `ConnectorBuilder` | unchanged trait | @@ -422,6 +439,8 @@ setup (inference handles `R` in `|reg| { ... }` closures). - `Producer` β†’ `Producer` β€” any code that names the two-parameter form breaks. Code that uses `Producer` via type inference (the common case) compiles without changes. - `Consumer` β†’ `Consumer` β€” same. - `Producer::key()` is **removed**. Code that called `producer.key()` for diagnostic strings should capture the key at the registration site instead. Two internal callers in `aimdb-core` (`run_single_transform`, `run_join_transform`) are updated as part of step 7. +- **`Producer::produce()` is now `pub fn produce(&self, value: T)` β€” synchronous and infallible** (v0.4 revision). Every `producer.produce(x).await?` call site must drop both the `.await` and the `?`. ProducerTrait::produce_any keeps its async/Result surface for type-erased routing where the Any downcast can still fail. +- **`Consumer::subscribe()` is now `pub fn subscribe(&self) -> Box + Send>`** β€” the `DbResult<>` wrapper is gone because the buffer Arc is pre-resolved at `Consumer::new`. Call sites drop the `?`. ConsumerTrait::subscribe_any keeps its `DbResult` for factory-side fallibility. **Codegen:** emitted code does not reference `Producer` / `Consumer` with type parameters directly. Golden-test strings update mechanically; no template changes. @@ -461,8 +480,8 @@ steps 1–9 landing first). - Drop type parameter `R` from `Producer` β†’ `Producer`. - Replace `db: Arc>` with `write: Arc>`. - Delete `record_key: String` field and the `pub fn key(&self) -> &str` accessor (two internal callers `run_single_transform` / `run_join_transform` are updated in step 7 to capture the key via closure instead). -- Keep `produce()` signature unchanged (`async fn produce(&self, value: T) -> DbResult<()>`). -- Update `impl ProducerTrait for Producer` likewise. +- Change `produce()` signature to `pub fn produce(&self, value: T)` (v0.4 revision β€” synchronous and infallible; see Decision 2). +- Update `impl ProducerTrait for Producer` likewise β€” keeps async/Result for type-erased fallibility. ### Step 4 β€” `Consumer` removes `R` @@ -470,8 +489,8 @@ steps 1–9 landing first). - Drop type parameter `R` from `Consumer` β†’ `Consumer`. - Replace `db: Arc>` field with `buffer: Arc>`. -- Update `subscribe()` to call `self.buffer.subscribe_boxed()` directly (no error path through `AimDb::subscribe`). -- Update `impl ConsumerTrait for Consumer` likewise. +- Update `subscribe()` to call `self.buffer.subscribe_boxed()` directly (no error path through `AimDb::subscribe`). Return `Box + Send>` directly β€” drop the `DbResult<>` wrapper (v0.4 revision). +- Update `impl ConsumerTrait for Consumer` likewise β€” keeps `DbResult` on the trait surface for factory-side fallibility. ### Step 5 β€” `collect_*` construction sites @@ -533,7 +552,7 @@ All open questions raised in design review (May 24, 2026) are resolved: 1. **`Producer::key()` accessor** β†’ **Delete**, along with the `record_key: String` field. Two internal callers (`run_single_transform`, `run_join_transform`) capture the key into the spawning closure instead. External users get the key from the registration site by construction, not from the Producer. -2. **`Producer::produce()` return type** β†’ **Keep `async fn ... -> DbResult<()>` unchanged.** Even though the body is synchronous and infallible after M14, keeping the signature avoids breakage on every existing `.await?` call site and leaves room for future backpressure-aware buffers or shutdown-signalling semantics. Cost is two no-op compile-time abstractions. +2. **`Producer::produce()` / `Consumer::subscribe()` return types** β†’ **Collapse to sync, infallible** (v0.4 revision β€” reversing the v0.3 decision). The body has no async or fallible operations after M14: `produce` is one virtual `WriteHandle::push`, `subscribe` is one virtual `DynBuffer::subscribe_boxed`. Keeping `async fn ... -> DbResult<()>` would have forced `.await?` at every call site forever, propagating fictional fallibility purely to please a signature. The one-time `.await?` removal across ~50 examples / aimdb-pro sites is worth the cleaner steady state. Future backpressure-aware variants can be added as additional methods (`produce_async`, `try_produce`) β€” additive, not breaking. The type-erased trait surfaces (`ProducerTrait::produce_any`, `ConsumerTrait::subscribe_any`) keep async/Result for genuine fallibility (downcast, factory resolution). 3. **`ConnectorBuilder` cascade** β†’ **v0.1 Option A**, but with the realisation that it is **zero LOC** for connector structs. Verified: no connector struct in `aimdb-mqtt-connector`, `aimdb-knx-connector`, or `aimdb-websocket-connector` carries an `R` phantom today (M13 already cleaned them). The only `R` left is on the `impl ConnectorBuilder for X` line and stays untouched. Full non-generic-isation is deferred to a separate milestone. From 6ea9573529b2e17341abfd0458e58dc387e1a8c5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Mon, 25 May 2026 20:07:07 +0000 Subject: [PATCH 11/14] feat: add build command for remote-access-demo in Makefile and update config producer in server --- Makefile | 2 ++ examples/remote-access-demo/src/server.rs | 12 +++++------- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Makefile b/Makefile index 65811ba..9135561 100644 --- a/Makefile +++ b/Makefile @@ -295,6 +295,8 @@ examples: cargo build --package weather-station-beta @printf "$(YELLOW) β†’ Building weather-station-gamma (embedded, embassy runtime)$(NC)\n" cargo build --package weather-station-gamma --target thumbv7em-none-eabihf + @printf "$(YELLOW) β†’ Building remote-access-demo (AimX server + client)$(NC)\n" + cargo build --package remote-access-demo @printf "$(YELLOW) β†’ Building hello-mailbox (sync)$(NC)\n" cargo build --package hello-mailbox @printf "$(YELLOW) β†’ Building hello-single-latest-async$(NC)\n" diff --git a/examples/remote-access-demo/src/server.rs b/examples/remote-access-demo/src/server.rs index db09990..b57f271 100644 --- a/examples/remote-access-demo/src/server.rs +++ b/examples/remote-access-demo/src/server.rs @@ -149,13 +149,11 @@ async fn main() -> Result<(), Box> { info!("πŸ“ Populating initial record data..."); let config_producer = db.producer::("server::Config")?; - config_producer - .produce(Config { - app_name: "AimDB Demo".to_string(), - version: "0.1.0".to_string(), - debug_mode: true, - }) - .await?; + config_producer.produce(Config { + app_name: "AimDB Demo".to_string(), + version: "0.1.0".to_string(), + debug_mode: true, + }); // Initialize AppSettings WITHOUT creating a producer // This makes it writable via remote access (record.set) From 3a79c530391edc9a6213bee3c681d82363a9cfe6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Tue, 26 May 2026 17:19:03 +0000 Subject: [PATCH 12/14] feat: update dependencies and remove alloc feature checks across the codebase --- Cargo.lock | 3 ++- Makefile | 4 ++++ _external/embassy | 2 +- aimdb-core/src/builder.rs | 7 +------ aimdb-core/src/connector.rs | 17 ++--------------- aimdb-core/src/lib.rs | 3 --- aimdb-core/src/transform/join.rs | 6 ------ aimdb-core/src/transform/mod.rs | 1 - aimdb-core/src/typed_api.rs | 2 -- aimdb-core/src/typed_record.rs | 3 +-- aimdb-knx-connector/src/embassy_client.rs | 2 +- examples/embassy-knx-connector-demo/Cargo.toml | 1 + .../rust-toolchain.toml | 2 +- examples/embassy-mqtt-connector-demo/Cargo.toml | 1 + examples/knx-connector-demo-common/src/lib.rs | 1 - 15 files changed, 15 insertions(+), 40 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 809a784..f25860c 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1149,6 +1149,7 @@ dependencies = [ "defmt 1.0.1", "document-features", "embassy-time-driver", + "embassy-time-queue-utils", "embedded-hal 0.2.7", "embedded-hal 1.0.0", "embedded-hal-async", @@ -1180,7 +1181,7 @@ dependencies = [ [[package]] name = "embassy-usb-synopsys-otg" -version = "0.3.3" +version = "0.4.0" dependencies = [ "critical-section", "defmt 1.0.1", diff --git a/Makefile b/Makefile index 9135561..cdcec40 100644 --- a/Makefile +++ b/Makefile @@ -277,10 +277,14 @@ examples: @printf "$(GREEN)Building all example projects...$(NC)\n" @printf "$(YELLOW) β†’ Building sync-api-demo (synchronous API wrapper)$(NC)\n" cargo build --package sync-api-demo + @printf "$(YELLOW) β†’ Building mqtt-connector-demo-common (shared MQTT demo code, runtime-agnostic)$(NC)\n" + cargo build --package mqtt-connector-demo-common @printf "$(YELLOW) β†’ Building tokio-mqtt-connector-demo (native, tokio runtime)$(NC)\n" cargo build --package tokio-mqtt-connector-demo @printf "$(YELLOW) β†’ Building embassy-mqtt-connector-demo (embedded, embassy runtime)$(NC)\n" cargo build --package embassy-mqtt-connector-demo --target thumbv7em-none-eabihf + @printf "$(YELLOW) β†’ Building knx-connector-demo-common (shared KNX demo code, runtime-agnostic)$(NC)\n" + cargo build --package knx-connector-demo-common @printf "$(YELLOW) β†’ Building tokio-knx-connector-demo (native, tokio runtime)$(NC)\n" cargo build --package tokio-knx-connector-demo @printf "$(YELLOW) β†’ Building embassy-knx-connector-demo (embedded, embassy runtime)$(NC)\n" diff --git a/_external/embassy b/_external/embassy index 823abdb..4bf217e 160000 --- a/_external/embassy +++ b/_external/embassy @@ -1 +1 @@ -Subproject commit 823abdbe8512721b4797536b9d3151c94c4bbad0 +Subproject commit 4bf217e8b59df60f34820d62dc01094714c6dae2 diff --git a/aimdb-core/src/builder.rs b/aimdb-core/src/builder.rs index 02d236b..277a2ff 100644 --- a/aimdb-core/src/builder.rs +++ b/aimdb-core/src/builder.rs @@ -15,7 +15,7 @@ use hashbrown::HashMap; #[cfg(not(feature = "std"))] use alloc::{boxed::Box, sync::Arc}; -#[cfg(all(not(feature = "std"), feature = "alloc"))] +#[cfg(not(feature = "std"))] use alloc::string::{String, ToString}; #[cfg(feature = "std")] @@ -54,7 +54,6 @@ use crate::{DbError, DbResult}; /// - `SerializerKind` - User-provided serializer for the record type (raw or context-aware) /// - `Vec<(String, String)>` - Configuration options from the URL query /// - `Option` - Optional dynamic topic provider -#[cfg(feature = "alloc")] pub type OutboundRoute = ( String, Box, @@ -579,7 +578,6 @@ where let mut reg = RecordRegistrar { rec, connector_builders: &self.connector_builders, - #[cfg(feature = "alloc")] record_key: record_key.as_str().to_string(), extensions: &self.extensions, last_stage: None, @@ -1499,7 +1497,6 @@ impl AimDb { /// let router = RouterBuilder::from_routes(routes).build(); /// connector.set_router(router).await?; /// ``` - #[cfg(feature = "alloc")] pub fn collect_inbound_routes( &self, scheme: &str, @@ -1577,7 +1574,6 @@ impl AimDb { /// /// The returned TypeId is the `TypeId::of::()` for the record type `T` /// that was used in the corresponding `configure::()` call. - #[cfg(feature = "alloc")] pub fn collect_outbound_topic_type_ids(&self, scheme: &str) -> Vec<(String, TypeId)> { let mut result = Vec::new(); @@ -1595,7 +1591,6 @@ impl AimDb { result } - #[cfg(feature = "alloc")] pub fn collect_outbound_routes(&self, scheme: &str) -> Vec { let mut routes = Vec::new(); diff --git a/aimdb-core/src/connector.rs b/aimdb-core/src/connector.rs index b9e4287..68d7dec 100644 --- a/aimdb-core/src/connector.rs +++ b/aimdb-core/src/connector.rs @@ -480,7 +480,6 @@ pub struct ConnectorLink { /// Mirrors the producer_factory pattern used for inbound connectors. /// /// Available in both `std` and `no_std + alloc` environments. - #[cfg(feature = "alloc")] pub consumer_factory: Option, /// Optional dynamic topic provider @@ -507,10 +506,7 @@ impl Debug for ConnectorLink { ) .field( "consumer_factory", - #[cfg(feature = "alloc")] &self.consumer_factory.as_ref().map(|_| ""), - #[cfg(not(feature = "alloc"))] - &None::<()>, ) .field( "topic_provider", @@ -527,7 +523,6 @@ impl ConnectorLink { url, config: Vec::new(), serializer: None, - #[cfg(feature = "alloc")] consumer_factory: None, topic_provider: None, } @@ -547,7 +542,6 @@ impl ConnectorLink { /// Returns None if no factory is configured. /// /// Available in both `std` and `no_std + alloc` environments. - #[cfg(feature = "alloc")] pub fn create_consumer( &self, db_any: Arc, @@ -598,7 +592,6 @@ pub enum DeserializerKind { /// the factory in a type-erased InboundConnectorLink. /// /// Available in both `std` and `no_std + alloc` environments. -#[cfg(feature = "alloc")] pub type ProducerFactoryFn = Arc) -> Box + Send + Sync>; @@ -651,7 +644,6 @@ pub trait ProducerTrait: Send + Sync { /// Mirrors the ProducerFactoryFn pattern for symmetry between inbound and outbound. /// /// Available in both `std` and `no_std + alloc` environments. -#[cfg(feature = "alloc")] pub type ConsumerFactoryFn = Arc) -> Box + Send + Sync>; @@ -719,7 +711,6 @@ pub struct InboundConnectorLink { /// Captures the record type T at link_from() call time. /// /// Available in both `std` and `no_std + alloc` environments. - #[cfg(feature = "alloc")] pub producer_factory: Option, /// Optional dynamic topic resolver (late-binding) @@ -737,7 +728,6 @@ impl Clone for InboundConnectorLink { url: self.url.clone(), config: self.config.clone(), deserializer: self.deserializer.clone(), - #[cfg(feature = "alloc")] producer_factory: self.producer_factory.clone(), topic_resolver: self.topic_resolver.clone(), } @@ -765,16 +755,14 @@ impl InboundConnectorLink { url, config: Vec::new(), deserializer, - #[cfg(feature = "alloc")] producer_factory: None, topic_resolver: None, } } - /// Sets the producer factory callback (alloc feature) + /// Sets the producer factory callback. /// /// Available in both `std` and `no_std + alloc` environments. - #[cfg(feature = "alloc")] pub fn with_producer_factory(mut self, factory: F) -> Self where F: Fn(Arc) -> Box @@ -786,10 +774,9 @@ impl InboundConnectorLink { self } - /// Creates a producer using the stored factory (alloc feature) + /// Creates a producer using the stored factory. /// /// Available in both `std` and `no_std + alloc` environments. - #[cfg(feature = "alloc")] pub fn create_producer( &self, db_any: Arc, diff --git a/aimdb-core/src/lib.rs b/aimdb-core/src/lib.rs index b415fa1..c8ffbff 100644 --- a/aimdb-core/src/lib.rs +++ b/aimdb-core/src/lib.rs @@ -16,7 +16,6 @@ #![cfg_attr(not(feature = "std"), no_std)] -#[cfg(feature = "alloc")] extern crate alloc; pub mod buffer; @@ -74,7 +73,6 @@ pub use aimdb_executor::{ pub use database::Database; // Producer-Consumer Pattern exports -#[cfg(feature = "alloc")] pub use builder::OutboundRoute; pub use builder::{AimDb, AimDbBuilder}; pub use connector::ConnectorBuilder; @@ -101,6 +99,5 @@ pub use record_id::{RecordId, RecordKey, StringKey}; pub use graph::{DependencyGraph, EdgeType, GraphEdge, GraphNode, RecordGraphInfo, RecordOrigin}; // Transform API exports -#[cfg(feature = "alloc")] pub use transform::{JoinBuilder, JoinEventRx, JoinPipeline, JoinTrigger}; pub use transform::{TransformBuilder, TransformPipeline}; diff --git a/aimdb-core/src/transform/join.rs b/aimdb-core/src/transform/join.rs index 7a9b37d..8c16cdf 100644 --- a/aimdb-core/src/transform/join.rs +++ b/aimdb-core/src/transform/join.rs @@ -114,7 +114,6 @@ impl + Send> DynJoinRx for R { // ============================================================================ /// Type-erased factory for creating a forwarder task for one join input. -#[cfg(feature = "alloc")] type JoinInputFactory = Box< dyn FnOnce( Arc>, @@ -133,13 +132,11 @@ type JoinInputFactory = Box< /// internal constant chosen per adapter (Tokio: 64, Embassy: 8, WASM: 64). /// /// Obtain via [`RecordRegistrar::transform_join`]. -#[cfg(feature = "alloc")] pub struct JoinBuilder { inputs: Vec<(String, JoinInputFactory)>, _phantom: PhantomData<(O, R)>, } -#[cfg(feature = "alloc")] impl JoinBuilder where O: Send + Sync + Clone + Debug + 'static, @@ -245,12 +242,10 @@ where /// /// Produced by [`JoinBuilder::on_triggers`] and consumed by /// [`RecordRegistrar::transform_join`]. Not normally constructed directly. -#[cfg(feature = "alloc")] pub struct JoinPipeline { pub(crate) spawn_factory: Box TransformDescriptor + Send>, } -#[cfg(feature = "alloc")] impl JoinPipeline where O: Send + Sync + Clone + Debug + 'static, @@ -265,7 +260,6 @@ where // Join Transform Build (forwarders + handler future, both collected at build time) // ============================================================================ -#[cfg(feature = "alloc")] #[allow(unused_variables)] fn build_join_collected( db: Arc>, diff --git a/aimdb-core/src/transform/mod.rs b/aimdb-core/src/transform/mod.rs index 05aafe6..39c83b7 100644 --- a/aimdb-core/src/transform/mod.rs +++ b/aimdb-core/src/transform/mod.rs @@ -22,7 +22,6 @@ pub mod single; // Public re-exports pub use single::{StatefulTransformBuilder, TransformBuilder, TransformPipeline}; -#[cfg(feature = "alloc")] pub use join::{JoinBuilder, JoinEventRx, JoinPipeline, JoinTrigger}; // ============================================================================ diff --git a/aimdb-core/src/typed_api.rs b/aimdb-core/src/typed_api.rs index 77f7d03..0f34f89 100644 --- a/aimdb-core/src/typed_api.rs +++ b/aimdb-core/src/typed_api.rs @@ -548,7 +548,6 @@ where /// Register a multi-input join transform (low-level API). /// /// Panics if a `.source()` or another `.transform()` is already registered. - #[cfg(feature = "alloc")] pub fn transform_join_raw(&'a mut self, build_fn: F) -> &'a mut Self where R: aimdb_executor::JoinFanInRuntime, @@ -567,7 +566,6 @@ where /// Derives this record from multiple input records. Available on any runtime /// that implements `JoinFanInRuntime`. Panics if a `.source()` or another /// `.transform()` is already registered. - #[cfg(feature = "alloc")] pub fn transform_join(&'a mut self, build_fn: F) -> &'a mut Self where R: aimdb_executor::JoinFanInRuntime, diff --git a/aimdb-core/src/typed_record.rs b/aimdb-core/src/typed_record.rs index fb3e5b4..d94ac28 100644 --- a/aimdb-core/src/typed_record.rs +++ b/aimdb-core/src/typed_record.rs @@ -20,8 +20,7 @@ extern crate alloc; #[cfg(not(feature = "std"))] use alloc::{boxed::Box, sync::Arc, vec::Vec}; -// ToString is only needed when alloc is enabled (for record_key.to_string()) -#[cfg(all(not(feature = "std"), feature = "alloc"))] +#[cfg(not(feature = "std"))] use alloc::string::ToString; #[cfg(feature = "std")] diff --git a/aimdb-knx-connector/src/embassy_client.rs b/aimdb-knx-connector/src/embassy_client.rs index fc4e0ab..ddcb492 100644 --- a/aimdb-knx-connector/src/embassy_client.rs +++ b/aimdb-knx-connector/src/embassy_client.rs @@ -637,7 +637,7 @@ impl KnxConnectorImpl { let timed_out = state.check_ack_timeouts(); if !timed_out.is_empty() { #[cfg(feature = "defmt")] - defmt::warn!("⚠️ ACK timeouts for sequences: {:?}", timed_out); + defmt::warn!("⚠️ ACK timeouts for sequences: {:?}", timed_out.as_slice()); } } } diff --git a/examples/embassy-knx-connector-demo/Cargo.toml b/examples/embassy-knx-connector-demo/Cargo.toml index b155e6a..5ce8451 100644 --- a/examples/embassy-knx-connector-demo/Cargo.toml +++ b/examples/embassy-knx-connector-demo/Cargo.toml @@ -51,6 +51,7 @@ embassy-time = { workspace = true, features = [ "defmt", "defmt-timestamp-uptime", "tick-hz-32_768", + "generic-queue-16", ] } embassy-net = { workspace = true, features = [ "defmt", diff --git a/examples/embassy-knx-connector-demo/rust-toolchain.toml b/examples/embassy-knx-connector-demo/rust-toolchain.toml index b4f3adf..750ee6f 100644 --- a/examples/embassy-knx-connector-demo/rust-toolchain.toml +++ b/examples/embassy-knx-connector-demo/rust-toolchain.toml @@ -1,4 +1,4 @@ [toolchain] -channel = "1.90" +channel = "1.95" components = ["rust-src", "rustfmt", "llvm-tools"] targets = ["thumbv8m.main-none-eabihf"] diff --git a/examples/embassy-mqtt-connector-demo/Cargo.toml b/examples/embassy-mqtt-connector-demo/Cargo.toml index 568e58e..f4dc7bf 100644 --- a/examples/embassy-mqtt-connector-demo/Cargo.toml +++ b/examples/embassy-mqtt-connector-demo/Cargo.toml @@ -51,6 +51,7 @@ embassy-time = { workspace = true, features = [ "defmt", "defmt-timestamp-uptime", "tick-hz-32_768", + "generic-queue-16", ] } embassy-net = { workspace = true, features = [ "defmt", diff --git a/examples/knx-connector-demo-common/src/lib.rs b/examples/knx-connector-demo-common/src/lib.rs index c54e697..4888c7f 100644 --- a/examples/knx-connector-demo-common/src/lib.rs +++ b/examples/knx-connector-demo-common/src/lib.rs @@ -30,7 +30,6 @@ #![cfg_attr(not(feature = "std"), no_std)] -#[cfg(feature = "alloc")] extern crate alloc; pub mod monitors; From b9223e463553e3125e8e1cf5cb407b03bfbe7068 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Tue, 26 May 2026 17:21:52 +0000 Subject: [PATCH 13/14] feat: add "generic-queue-16" feature to embassy-time dependency in Cargo.toml --- examples/weather-mesh-demo/weather-station-gamma/Cargo.toml | 1 + 1 file changed, 1 insertion(+) diff --git a/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml b/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml index 6957312..2d1ddef 100644 --- a/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml +++ b/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml @@ -52,6 +52,7 @@ embassy-time = { workspace = true, features = [ "defmt", "defmt-timestamp-uptime", "tick-hz-32_768", + "generic-queue-16", ] } embassy-net = { workspace = true, features = [ "defmt", From d871fd5d0533f64db102b5dc44b6f73b7d8aa17b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Tue, 26 May 2026 17:24:04 +0000 Subject: [PATCH 14/14] feat: update embassy-time dependency to include generic-queue-16 feature for FuturesUnordered compatibility --- docs/aimdb-usage-guide.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/docs/aimdb-usage-guide.md b/docs/aimdb-usage-guide.md index 6e2f569..27a1fc2 100644 --- a/docs/aimdb-usage-guide.md +++ b/docs/aimdb-usage-guide.md @@ -159,7 +159,7 @@ aimdb-embassy-adapter = { version = "0.3", features = ["embassy-runtime", "embas # Embassy runtime (example for RP2040) embassy-executor = { version = "0.6", features = ["arch-cortex-m", "executor-thread"] } -embassy-time = { version = "0.3" } +embassy-time = { version = "0.3", features = ["generic-queue-16"] } # REQUIRED β€” see note below embassy-rp = { version = "0.2", features = ["time-driver"] } # CRITICAL: Patch dependencies for compatibility @@ -179,6 +179,18 @@ mountain-mqtt = { git = "https://github.com/aimdb-dev/mountain-mqtt.git", branch mountain-mqtt-embassy = { git = "https://github.com/aimdb-dev/mountain-mqtt.git", branch = "main" } ``` +> **Required: `embassy-time` generic timer queue.** AimDB drives every +> registered future from a single Embassy task via `FuturesUnordered`, so +> `embassy_time::Timer` and `Ticker` are polled with `FuturesUnordered`'s +> waker rather than an Embassy-executor waker. The default +> executor-integrated timer queue rejects that waker and panics with +> *"Found waker not created by the Embassy executor…"* at the first timer +> await. Enable one of the `generic-queue-N` features on `embassy-time` in +> your binary β€” `generic-queue-16` is a reasonable starting point; raise it +> if you have many concurrent timers in flight. This is a binary-level +> choice: AimDB's own crates deliberately do **not** enable a queue size, +> following [embassy-time's guidance for libraries](https://docs.embassy.dev/embassy-time/git/default/index.html#generic-queue). + ### Basic Example (main.rs) ```rust