Goal
When a user inference request lands on a node, route it across the peers that host the model's modules, orchestrate the forward pass, and return the result.
Scope
- Define the request envelope (input data hash, requested model hash/manifest, return-to address).
- Look up which peers host which modules (via P2P capability gossip — already exists in
p2p.py).
- Plan a forward-pass execution path across peers (linear pipeline first; tensor parallelism later).
- Drive the tensor-transport protocol step by step.
- Handle peer dropout gracefully (pick alternate host or fail with a clear error).
- Emit the events that
RPB.recordInference() listens for (provider, requester, units, normalized cost).
Depends on
- Tensor transport over libp2p (separate issue)
- A loadable trained-model manifest (separate issue, currently a single-CID stub on
RPB.setMatureModel)
Acceptance
- One end-to-end test: user submits an inference request → router orchestrates → result returns →
InferenceServed event fires → recordInference accounting matches expectation.
Goal
When a user inference request lands on a node, route it across the peers that host the model's modules, orchestrate the forward pass, and return the result.
Scope
p2p.py).RPB.recordInference()listens for (provider, requester, units, normalized cost).Depends on
RPB.setMatureModel)Acceptance
InferenceServedevent fires →recordInferenceaccounting matches expectation.