Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 28 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

> **Note**: This is the global changelog for the AimDB project. For detailed changes to individual crates, see their respective CHANGELOG.md files:
>
> - [aimdb-core/CHANGELOG.md](aimdb-core/CHANGELOG.md)
> - [aimdb-codegen/CHANGELOG.md](aimdb-codegen/CHANGELOG.md)
> - [aimdb-data-contracts/CHANGELOG.md](aimdb-data-contracts/CHANGELOG.md)
Expand Down Expand Up @@ -105,6 +106,10 @@ multi-step migration round-trip suite and a trybuild compile-fail harness.

### Added

- **Zero-allocation generated Postcard encoding (Issue #177).** `Linkable` gains a source-compatible `encode_into(&mut [u8])` fast path and generated Postcard records implement it with `postcard::to_slice`. Core reuses onebounded 256-byte scratch allocation per generated Postcard outbound route (raw builders choose their capacity) and falls back to the existing owned serializer when a value does not fit. The codec seam is gated by a 10,000-iteration allocation-counting bench (0 allocations, 0 bytes) plus
host, `no_std` Cortex-M, and codegen-drift tests. The claim intentionally excludes connector-internal payload copies and the existing boxed connector reader future. ([aimdb-core](aimdb-core/CHANGELOG.md),
[aimdb-data-contracts](aimdb-data-contracts/CHANGELOG.md),
[aimdb-codegen](aimdb-codegen/CHANGELOG.md))
- **Embassy MQTT client TLS — `mqtts://` for embedded, WP7 ([design 044](docs/design/044-embassy-mqtt-tls.md)).** The Embassy MQTT connector can now dial a broker over TLS 1.3 (`embedded-tls`, pure-Rust `rustpki` certificate verification with `rsa`/`p384` so public CA chains verify out of the box), with SNI/hostname verification, DNS resolution, and a connector-internal SNTP time source that gates the first handshake so certificate validity is always checked against real time. Behind the new `embassy-tls` feature; `MqttConnectorBuilder::new` picks the transport from the URL scheme (`mqtt://` plain, `mqtts://` TLS) and gains `with_tls(TlsOptions)` (entropy, record buffers, SNTP server) and `with_credentials(username, password)` (MQTT CONNECT auth, both transports). The plain `mqtt://` path is untouched. The `embassy-mqtt-connector-demo` example gains a `tls` feature demonstrating the full flow against the repo's `dev/mosquitto` bench broker. ([aimdb-mqtt-connector](aimdb-mqtt-connector/CHANGELOG.md))
- **Design 034 Phase 3 — sans-io KNX/IP tunneling engine shared by both transports (Issue #135, [review doc §3.7](docs/design/034-technical-debt-review.md)).** The entire tunneling lifecycle — CONNECT_REQUEST/RESPONSE handshake, TUNNELING_REQUEST/ACK sequence + pending-ACK bookkeeping, keepalive (CONNECTIONSTATE_REQUEST) scheduling, ACK-timeout sweeps, and reconnect-with-backoff — now lives **once**, in the new runtime-neutral `aimdb_knx_connector::tunnel` module (`no_std + alloc`, no tokio/embassy imports), driven as a poll-based state machine (events in, `Action`s out, `next_deadline()` for timer arming). `tokio_client.rs` (988 → ~530 lines incl. a new fake-gateway integration test) and `embassy_client.rs` (1,055 → ~450 lines) are reduced to socket shims; the previously untestable handshake/ACK/keepalive/reconnect paths now have 15 host-run unit tests plus a scripted localhost-UDP roundtrip test. Behavioral unifications: Embassy gains the 5 s CONNECT_RESPONSE timeout (previously waited forever), both shims reconnect on fatal socket errors, and the dead tokio-only per-publish ACK oneshot is dropped (it was always `None`); the CONNECT_REQUEST HPAI stays per-transport (`LocalEndpoint`: tokio = real bound address, Embassy = NAT mode). ([aimdb-knx-connector](aimdb-knx-connector/CHANGELOG.md))

Expand Down Expand Up @@ -418,6 +423,7 @@ builder.configure::<Temperature>("sensor.temperature", |reg| {
```

**Key naming conventions:**

- Use dot-separated hierarchical names: `"sensors.indoor"`, `"config.app"`
- Keys must be unique across all records (duplicate keys panic at registration)
- For single-instance records, any descriptive string works (e.g., `"sensor.temperature"`, `"app.config"`)
Expand Down Expand Up @@ -513,6 +519,7 @@ This release introduces **bidirectional connector support**, enabling true two-w
### Modified Crates

See individual changelogs for detailed changes:

- **[aimdb-core](aimdb-core/CHANGELOG.md)**: Core connector architecture, router system, bidirectional APIs
- **[aimdb-tokio-adapter](aimdb-tokio-adapter/CHANGELOG.md)**: Connector builder integration
- **[aimdb-embassy-adapter](aimdb-embassy-adapter/CHANGELOG.md)**: Network stack access, connector support
Expand All @@ -522,6 +529,7 @@ See individual changelogs for detailed changes:
### Migration Guide

**1. Update connector registration:**

```rust
// Old (v0.1.0)
let mqtt = MqttConnector::new("mqtt://broker:1883").await?;
Expand All @@ -532,6 +540,7 @@ builder.with_connector(MqttConnectorBuilder::new("mqtt://broker:1883"))
```

**2. Make build() async:**

```rust
// Old (v0.1.0)
let db = builder.build()?;
Expand All @@ -541,6 +550,7 @@ let db = builder.build().await?;
```

**3. Use directional link methods:**

```rust
// Old (v0.1.0)
.link("mqtt://sensors/temp")
Expand All @@ -551,6 +561,7 @@ let db = builder.build().await?;
```

**4. Remove manual MQTT task spawning (Embassy):**

```rust
// Old (v0.1.0) - Manual task spawning required
let mqtt_result = MqttConnector::create(...).await?;
Expand Down Expand Up @@ -627,12 +638,14 @@ impl MyConnector {
```

**Why this change?** The new trait-based architecture provides:

- ✅ Symmetry with inbound routing (`ProducerTrait` ↔ `ConsumerTrait`)
- ✅ Testability (can mock `ConsumerTrait` without real records)
- ✅ Type safety via factory pattern (type capture at configuration time)
- ✅ Maintainability (connector logic stays in connector crate)

**Migration checklist for custom connectors:**

- [ ] Add `spawn_outbound_publishers()` method to connector implementation
- [ ] Call `db.collect_outbound_routes(protocol_name)` in `ConnectorBuilder::build()`
- [ ] Call `connector.spawn_outbound_publishers(db, routes)?` before returning
Expand All @@ -653,6 +666,7 @@ See [Connector Development Guide](docs/design/012-M5-connector-development-guide
### Added

#### Core Database (`aimdb-core`)

- Initial release of AimDB async in-memory database engine
- Type-safe record system using `TypeId`-based routing
- Three buffer types for different data flow patterns:
Expand All @@ -666,6 +680,7 @@ See [Connector Development Guide](docs/design/012-M5-connector-development-guide
- Remote access protocol (AimX v1) for cross-process introspection

#### Runtime Adapters

- **Tokio Adapter** (`aimdb-tokio-adapter`): Full-featured std runtime support
- Lock-free buffer implementations
- Configurable buffer capacities
Expand All @@ -676,6 +691,7 @@ See [Connector Development Guide](docs/design/012-M5-connector-development-guide
- Compatible with ARM Cortex-M targets

#### MQTT Connector (`aimdb-mqtt-connector`)

- Dual runtime support for both Tokio and Embassy
- Automatic consumer registration via builder pattern
- Topic mapping with QoS and retain configuration
Expand All @@ -685,6 +701,7 @@ See [Connector Development Guide](docs/design/012-M5-connector-development-guide
- Uses `mountain-mqtt` for embedded environments

#### Developer Tools

- **MCP Server** (`aimdb-mcp`): LLM-powered introspection and debugging
- Discover running AimDB instances
- Query record values and schemas
Expand All @@ -702,12 +719,14 @@ See [Connector Development Guide](docs/design/012-M5-connector-development-guide
- Clean error handling

#### Synchronous API (`aimdb-sync`)

- Blocking wrapper around async AimDB core
- Thread-safe synchronous record access
- Automatic Tokio runtime management
- Ideal for gradual migration from sync to async

#### Documentation & Examples

- Comprehensive README with architecture overview
- Individual crate documentation with examples
- 12 detailed design documents in `/docs/design`
Expand All @@ -718,6 +737,7 @@ See [Connector Development Guide](docs/design/012-M5-connector-development-guide
- `remote-access-demo`: Cross-process introspection server

#### Build & CI Infrastructure

- Comprehensive Makefile with color-coded output
- GitHub Actions workflows:
- Continuous integration (format, lint, test)
Expand All @@ -729,28 +749,33 @@ See [Connector Development Guide](docs/design/012-M5-connector-development-guide
- Dev container setup for consistent development environment

### Design Goals Achieved

- ✅ Sub-50ms latency for data synchronization
- ✅ Lock-free buffer operations
- ✅ Cross-platform support (MCU → edge → cloud)
- ✅ Type safety with zero-cost abstractions
- ✅ Protocol-agnostic connector architecture

### Known Limitations

- Kafka and DDS connectors planned for future releases
- CLI tool is currently skeleton implementation
- Performance benchmarks not yet included
- Limited to Unix domain sockets for remote access (no TCP yet)

### Dependencies

- Rust 1.75+ required
- Tokio 1.47+ for std environments
- Embassy 0.9+ for embedded environments
- See `deny.toml` for approved dependency licenses

### Breaking Changes

None (initial release)

### Migration Guide

Not applicable (initial release)

---
Expand All @@ -762,6 +787,7 @@ Not applicable (initial release)
AimDB v0.1.0 establishes the foundational architecture for async, in-memory data synchronization across MCU → edge → cloud environments. This release focuses on core functionality, dual runtime support, and developer tooling.

**Highlights:**

- 🚀 Dual runtime support: Works on both standard library (Tokio) and embedded (Embassy)
- 🔒 Type-safe record system eliminates runtime string lookups
- 📦 Three buffer types cover most data patterns
Expand All @@ -770,6 +796,7 @@ AimDB v0.1.0 establishes the foundational architecture for async, in-memory data
- ✅ 27+ core tests, comprehensive CI/CD, security auditing

**Get Started:**

```bash
cargo add aimdb-core aimdb-tokio-adapter
```
Expand All @@ -778,7 +805,7 @@ See [README.md](README.md) for quickstart guide and examples.

**Feedback Welcome:**
This is an early release. Please report issues, suggest features, or contribute at:
https://github.com/aimdb-dev/aimdb
<https://github.com/aimdb-dev/aimdb>

---

Expand Down
30 changes: 29 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions aimdb-bench/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ harness = false
name = "b0_alloc_migration"
harness = false

[[bench]]
name = "b0_alloc_linkable"
harness = false

[features]
default = ["std"]
# Gates `profiles`/`reports`/`harness` and their criterion/serde_json/
Expand Down Expand Up @@ -89,6 +93,7 @@ serde_json = { workspace = true, optional = true }
# criterion/serde_json above, since that's the only bench that needs it.
aimdb-data-contracts = { path = "../aimdb-data-contracts", default-features = false, features = [
"alloc",
"linkable",
"migratable",
], optional = true }

Expand Down Expand Up @@ -124,3 +129,4 @@ critical-section = { version = "1.1", features = ["std"] }
# defmt logger / panic-handler + embassy-time driver stubs for the host bench
defmt = { workspace = true }
embassy-time-driver = { path = "../_external/embassy/embassy-time-driver" }
postcard = { version = "1.0", default-features = false, features = ["alloc"] }
11 changes: 9 additions & 2 deletions aimdb-bench/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ Plus two informational benches that exercise the full runner-driven pipeline.
**Adapters covered:**

- **Tokio** — `b0_alloc_tokio`, `b1_b2_tokio` (host).
- **postcard `Linkable` codec** — `b0_alloc_linkable` (host). This isolates the
issue #177 `encode_into` seam; it is not a whole-connector allocation claim.
- **Embassy** — `b0_alloc_embassy`, `b1_b2_embassy`
(host). These drive the real [`EmbassyBuffer`] backend via
`futures::executor::block_on` over embassy-sync's poll methods — no
Expand Down Expand Up @@ -53,6 +55,9 @@ Always run from the workspace root (`/aimdb_ws/aimdb`).
# B0 — allocation gate (buffer layer)
cargo bench -p aimdb-bench --bench b0_alloc_tokio

# B0 — allocation gate (Postcard Linkable::encode_into codec seam)
cargo bench -p aimdb-bench --bench b0_alloc_linkable

# B1 + B2 — latency (time/iter) and throughput (msgs/sec), one Criterion suite
cargo bench -p aimdb-bench --bench b1_b2_tokio

Expand Down Expand Up @@ -91,6 +96,7 @@ Criterion writes HTML reports to `target/criterion/`.
`b0_alloc_tokio` does not use Criterion. It runs a fixed warmup + batch cycle and writes JSON results to `aimdb-bench/target/bench-results/b0_alloc_tokio.json` (the path is anchored to the crate dir, so it is the same regardless of the directory you run from).

**Measurement model:**

1. Create buffer + reader.
2. Warmup ≥ 200 push → recv cycles (excluded from counters).
3. Reset allocation counters.
Expand All @@ -103,6 +109,9 @@ The committed baseline lives in `data/baselines/b0_alloc_tokio.json`. When a cha

`b0_alloc_embassy` mirrors this against the Embassy buffer backend and writes `data/baselines/b0_alloc_embassy.json` — also **0 allocs/msg** across all three profiles, confirming the Embassy `poll_recv` path is allocation-free on the host. The on-target B3 bench (`examples/embassy-bench-stm32h5`) re-checks the same 0-alloc claim against the real embedded allocator.

`b0_alloc_linkable` warms up for 1,000 iterations, then measures 10,000 generated-shape postcard `Linkable::encode_into` calls into one stack buffer.
The required result is **0 allocation calls and 0 allocated bytes**. It isolates the codec seam: `SerializedReader` still returns a boxed future, dynamic topics may allocate and connector implementations may copy payload ownership after the core pump lends them the scratch slice.

> **Embassy eager registration (design 039 F8/F9).** An Embassy `SpmcRing` reader registers its embassy `Subscriber` eagerly, at `subscribe()` time — matching Tokio's `broadcast` — so no separate priming step is needed before the first `push`.

**Noise reduction:** a `new_current_thread()` Tokio executor is used so there are no work-stealing threads and Tokio's scheduler does not allocate per-poll in the hot path.
Expand All @@ -127,5 +136,3 @@ Use them as a comparison point, not a regression gate. If they regress, `b0_allo
- Criterion p99 can vary ±5–10% on noisy CI runners. Use p50 medians for trend comparisons.
- Always specify `--release` or debug build consistently when comparing runs; optimizations differ by 5–50×.
- `b_alloc_pipeline` uses a paced source: per-message pace tokens and notification channels. The coordination overhead is included in the measured window.


Loading