feat(eventing): add typed signal-to-event projection architecture

[robbiemu]

Summary

This PR introduces a host-neutral, typed signal-to-event projection architecture and uses it to connect three event-producing paths that previously had different shapes:

• immediate events derived from incoming telemetry, demonstrated with a GitLab issue-created OTLP log in Maple Local;
• verified provider events, demonstrated by moving PlanetScale webhooks through the same source/selector/projector contracts before durable queueing; and
• scheduled aggregate alerts, which retain their query-driven lifecycle but now also produce the common CloudEvents envelope.

The existing alert-core extraction remains part of the change, but it is now one producer within the larger event model rather than the whole design.

Related to #222

How the design evolved

The first version of this PR took the simplest route suggested by #222: extract Maple's hosted threshold and incident-lifecycle logic into a host-neutral package, then eventually supply Local versions of the scheduler, query adapter, persistence, and delivery adapters.

That remains the right model for aggregate alerts, such as “the error rate has remained above 5% for ten minutes.” It turned out not to be the right model for every behavior we were trying to enable.

The motivating small-deployment example is deliberately ordinary: a self-hosted GitLab instance emits an OTLP record when somebody creates an issue, Maple receives it, and a later downstream consumer might post a Matrix message that an agent can act on. In ELI5 terms, Maple already has a filing cabinet for telemetry. An alert periodically asks a question about a group of filed records and produces one answer. The GitLab case instead needs Maple to notice each new envelope as it arrives and turn that individual fact into an event.

Trying to implement the second behavior with the first mechanism would require polling chDB windows, inventing cursors over data that has no ingestion sequence, handling overlap and late arrivals, re-discovering and deduplicating rows, and recovering scalar types after arbitrary OTLP attributes have already been flattened to strings. It would also make every configured event rule compete with ingest and analytical queries.

Scoping that out exposed a broader reusable boundary:

1. source adapters normalize authenticated inputs into typed signals;
2. selectors decide which signals qualify;
3. pure projectors turn qualifying signals into factual typed events; and
4. consumers subscribe downstream of a durable outbox.

That model handles GitLab, PlanetScale, and future providers without making any of them the event abstraction. It also lets query-alert lifecycle transitions join the same event stream without pretending an aggregate conclusion is an incoming signal.

Architecture

There are intentionally two production paths, converging only after each has produced a factual CloudEvent.

Immediate per-occurrence path

authenticated OTLP or provider input → decode once → typed normalized signal → bounded selector evaluation → registered projector → durable outbox

The original telemetry continues separately through the existing warehouse encoder and into chDB or the hosted warehouse.

For Maple Local, a matched event is staged before the chDB write, marked ready only after the warehouse write succeeds, and the OTLP request is acknowledged only after readiness succeeds. A retry recomputes the same event ID when the source furnished stable occurrence identity, so outbox insertion is idempotent.

Scheduled aggregate path

warehouse query → AlertObservation → alert evaluation and incident lifecycle → alert lifecycle projector → durable delivery outbox

This path remains query-driven because rates, percentiles, absence, thresholds, and recovery are conclusions about a time window rather than individual incoming facts.

The distinction is semantic, not just an implementation optimization: an ingest-time selector can emit one event for every matching occurrence, while a scheduled alert evaluates a set and normally emits one lifecycle transition.

Core patterns

1. Typed source adapters

A source adapter owns the mapping from one authenticated wire format to NormalizedSignal records. It declares a field catalog, scalar types, permitted operators, sensitivity, and replay capability.

The eventing core does not know about GitLab, PlanetScale, OTLP transport, HTTP signatures, chDB, PostgreSQL, Cloudflare, or Matrix. Provider authentication remains outside the adapter boundary. “Plugin” here means a compile-time registered module behind this interface; this PR does not load arbitrary runtime code.

The first Local adapter handles OTLP logs. PlanetScale now registers a provider-specific webhook adapter using the same interface.

2. Small structured selector AST

Projection configuration stores a typed predicate tree rather than SQL or a new textual DSL.

Version 1 supports:

• all, any, and not composition;
• exists;
• eq and neq;
• gt, gte, lt, and lte;
• string contains; and
• typed membership with in.

String, boolean, exact signed int64, finite float64, RFC 3339 timestamp, and integer-nanosecond duration values retain distinct semantics. There is no implicit coercion: the string "12" does not satisfy an integer comparison, timestamps compare as instants, and integers above JavaScript's safe-number range remain exact decimal values.

The evaluator is bounded to depth 8, 64 predicate nodes, 100 in members, and 4 KiB string literals. Version 1 intentionally has no functions, regular expressions, arithmetic, joins, aggregation, user code, or raw SQL.

3. Immutable compiled registry snapshots

Each semantic projection edit creates an immutable revision. Activation validates the runtime schema, verifies every referenced source field/operator and projector registration, decodes projector configuration, compiles the complete candidate registry, commits the revision, and atomically swaps the runtime snapshot while Local ingest is quiesced.

An ingest request therefore observes exactly one registry version. Several projections can match the same signal, and all matches run. One failing projector is isolated and recorded without suppressing successful sibling projections.

4. Pure versioned projectors

A projector is deterministic code registered with:

• an ID and version;
• accepted source kinds;
• a declared output event type;
• a declared data schema;
• a configuration decoder; and
• a pure signal-to-event-data function.

Projectors do not create issues, call Matrix, send webhooks, mutate provider state, or perform I/O. Those are downstream consumer responsibilities.

The GitLab semantic projector currently produces dev.maple.gitlab.issue.created.v1. PlanetScale produces the provider-factual dev.maple.planetscale.webhook.received.v1 event and leaves issue/timeline policy downstream.

5. Canonical CloudEvents and deterministic identity

Projected output uses CloudEvents 1.0 structured representation with Maple extension attributes for tenant, projection revision, and projector version.

Event IDs are SHA-256 hashes over a length-delimited identity tuple:

• tenant;
• source kind;
• source;
• source occurrence ID;
• projection ID; and
• projection revision.

This makes retries effectively once at the logical event-creation boundary when source identity is stable. It does not claim exactly-once external side effects across arbitrary consumers.

The package includes generated JSON schemas and versioned conformance fixtures so a future Rust adapter can prove the same predicate and identity semantics rather than independently approximating the TypeScript implementation.

6. Durable Local control state and outbox

Maple Local stores projection revisions, active pointers, bounded projection failures, and staged/ready events in a private SQLite control database at control/eventing.sqlite.

The store uses WAL, synchronous=FULL, strict tables, unique event IDs, atomic revision writes, atomic readiness transitions, collision detection, size limits, symlink refusal, and bounded reads.

A staged event is not blindly promoted after a crash because the control database alone cannot prove whether the corresponding chDB write committed. Safe recovery is source re-delivery: the same ID deduplicates at staging and becomes ready only after a successful warehouse write. Authenticated health and outbox inspection expose staged records for diagnosis.

7. Checkpoint participation

The control database is now part of Maple Local checkpoint correctness instead of being an unprotected sidecar.

Checkpoint manifest version 2 binds the control snapshot's path, byte count, SHA-256 digest, schema version, and row counts alongside the chDB backup. Restore validates those bindings before use. Version 1 checkpoints remain readable and restore an empty eventing control store because they predate this state.

8. Bounded failure and data handling

The Local adapter bounds attribute counts, string sizes, nested depth and nodes, normalized source data, and canonical event size. Secret-like attribute names are excluded from selector and projection views.

Malformed configuration is rejected before activation. Missing selector fields and runtime type mismatches are total non-matches rather than exceptions. Projector failures are persisted idempotently by occurrence where possible and retained under a bounded per-tenant policy. Infrastructure failures that would lose required state remain retryable.

First vertical: GitLab issue-created

The first end-to-end fixture uses an OTLP LogRecord whose eventName is gitlab.issue.created.

The source contract prefers event.id for stable occurrence identity and accepts cloudevents.id or gitlab.event.id aliases. The semantic projector requires:

• gitlab.project.path; and
• integer gitlab.issue.iid.

It can also include project and issue IDs, title, URL, actor identity, service name, and an explicitly enabled body.

GitLab does not emit this exact contract merely because Maple is running. Instrumentation or an adapter must produce the OTLP LogRecord. The important architectural result is that the event name is ordinary projection configuration, not a hard-coded GitLab branch in the eventing core.

This PR intentionally stops that vertical at the durable outbox. Matrix transport, agent authorization, action policy, and external side effects are separate downstream concerns.

Existing producer convergence

PlanetScale

Every verified non-test PlanetScale webhook now runs through a registered source adapter, selector, and projector before the route acknowledges it. The resulting CloudEvent is carried durably by the existing dedicated Cloudflare Queue.

The queue job temporarily retains the provider payload so the current timeline and issue-hub consumers preserve their behavior during migration. Those behaviors classify the typed event downstream; they are no longer what defines ingestion.

Provider webhooks with a stable source timestamp derive stable event identity across delivery retries. When PlanetScale supplies no source timestamp, receipt time participates in the derived identity so two envelopes never share an ID while carrying different event times.

Query alerts

The host-neutral alert core still owns aggregate observation evaluation, trigger/resolve/renotify planning, flap suppression, no-data recovery safety, tenant-fair scheduling helpers, delivery idempotency, and bounded retry policy.

Hosted alert-delivery rows remain this producer's durable outbox. Their payload now includes an additive deterministic dev.maple.alert.lifecycle trigger, resolve, renotify, or test CloudEvent while retaining the existing top-level payload fields and delivery behavior.

This is a compatibility bridge: existing destinations keep working, while future event-aware consumers get the common factual envelope.

Headless Local operation

Projection management does not depend on the web UI or an open browser.

The Local server exposes maintenance-token-authenticated endpoints for:

• activating an immutable projection revision;
• listing active projections;
• inspecting eventing health; and
• reading bounded ready or staged outbox records.

With no active projection for a source kind, Local exits the event path before normalization and retains the existing warehouse behavior.

Deliberate boundaries

This PR does not:

• add NATS, JetStream, Kafka, or another required broker;
• implement Matrix delivery or agent actions;
• replace the Collector's filtering, queueing, or authentication;
• replace scheduled aggregate alerts with ingest selectors;
• expose arbitrary SQL or executable user configuration;
• add dynamic third-party module loading;
• claim exactly-once warehouse storage after ambiguous OTLP failures; or
• implement automatic historical replay.

Replay capability metadata is present, but replay execution is deliberately deferred. Current Local attribute maps have lost original scalar types and the warehouse rows do not have a native occurrence ID. A later bounded, operator-invoked replay adapter must require explicit acknowledgement for coerced fields and pass evaluator-versus-warehouse conformance tests. The live path never falls back to chDB polling.

The TypeScript Local log adapter is the reference first implementation. Hosted Rust ingest, additional OTLP signal kinds, public CRUD/UI, and downstream delivery are follow-up adapters against the shared contracts.

Compatibility and operational impact

• Existing hosted alert evaluation and destination delivery remain in place.
• Existing PlanetScale issue and timeline behavior remains in place behind the queued event.
• Original Local telemetry is still encoded and stored in chDB.
• No browser or new broker is required.
• Local checkpoint manifests advance to version 2 while retaining version 1 restore support.
• New Local eventing endpoints require the existing maintenance credential.
• The work does not deploy or activate a projection automatically; operators must install an explicit revision.

Review guide

The main architectural surfaces are:

• packages/eventing-core — schemas, typed selector evaluator, source/projector registries, canonical event identity, and conformance fixtures;
• apps/cli/src/server/eventing — Local OTLP normalization, compiled runtime, SQLite control state, and outbox;
• apps/cli/src/server/serve.ts — decode-once pre-chDB integration and authenticated maintenance endpoints;
• apps/cli/src/server/checkpoints.ts — checkpoint-v2 control-state binding;
• packages/alerting-core and AlertsService — query-alert lifecycle convergence; and
• the PlanetScale webhook route, queue, and runtime — provider-event convergence.

The detailed acceptance contract and design rationale live in docs/signal-to-event-projection.md.

Validation

• Whole-repository production typecheck: 38/38 tasks passed, including cargo check for the Rust ingest crate.
• Full API suite: 1,575 passed, 126 skipped.
• Eventing core: 23 tests passed, including generated-schema drift checks and predicate/identity fixtures.
• Alerting core: 10 tests passed.
• Local eventing and checkpoint focus: 49 tests passed.
• PlanetScale final focus: 35 tests passed.
• Alert service and delivery focus: 83 tests passed.
• Targeted oxlint and git diff --check passed.
• Exhaustive CLI run: 437 tests passed. Three listener tests could not bind Bun's ephemeral port 0 in the execution sandbox; the one real CORS expectation exposed by that run was updated, and its four-test browser-origin subset passes.