IDC: Asynchronous reducer.

[Shubham8287]

Async Reducer Execution (Outbox/Inbox)

Adds a message-driven execution model for reducers with idempotency and transactional guarantees.

Maybe start from looking at smoketest

Outbox Macro

outbox macros mark tables that emit reducer calls.

• Limited implementation for testing purpose.
• Single on_result callback per table (not per row, this difffers from proposal) - discussion
• It should merge behing unstable feature flag.

System Tables

ST_OUTBOUND_MSG

• Maintains an auto-incrementing msg_id per sender.
• Stores in-flight outbound reducer calls (outbox pattern).
• Used by IdcActor to issue remote calls with strictly increasing msg_id.

ST_INBOUND_MSG

• Tracks latest processed msg_id per sender along with reducer result
• For duplicate requests where the msg_id is already present, the result is returned without executing the reducer.
• Requests with a msg_id older than the latest seen for a sender are dropped

Actor

IDCActor handles execution.

• Drives reducer execution from ST_OUTBOUND_MSG
• Calls on_result reducer and delete entry from ST_OUTBOUND_MSG in same transaction.
• If reducer fails, delete entry in next transaction, somewhat similar to SchedulerActror
• Reducer failure are only considered if reducer actually ran and returned error, all other cases of error will result in retrying.

API

call_from_database(database_identity, msg_id, params)

• Caller should ensure msg_id never decrease for same sender database_identity else API returns InvalidRequest.
• If reducer ran and failed it returns StatusCode::UNPROCESSABLE_ENTITY, which caller shouln't retry.
• All other errors, including network errors are retried by IdcActor.

API and ABI breaking changes

• New APIs only, but we may not want to make macro implementation public, so it has to be behind unstable feature flag.

Expected complexity level and risk

4/5, Exactly once semantic require closer look and can be tricky.

Testing

New smoketests.