Skip to content

Build inference request router for distributed forward pass #8

@EightRice

Description

@EightRice

Goal

When a user inference request lands on a node, route it across the peers that host the model's modules, orchestrate the forward pass, and return the result.

Scope

  • Define the request envelope (input data hash, requested model hash/manifest, return-to address).
  • Look up which peers host which modules (via P2P capability gossip — already exists in p2p.py).
  • Plan a forward-pass execution path across peers (linear pipeline first; tensor parallelism later).
  • Drive the tensor-transport protocol step by step.
  • Handle peer dropout gracefully (pick alternate host or fail with a clear error).
  • Emit the events that RPB.recordInference() listens for (provider, requester, units, normalized cost).

Depends on

  • Tensor transport over libp2p (separate issue)
  • A loadable trained-model manifest (separate issue, currently a single-CID stub on RPB.setMatureModel)

Acceptance

  • One end-to-end test: user submits an inference request → router orchestrates → result returns → InferenceServed event fires → recordInference accounting matches expectation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    track:infraP2P, libp2p, blob store, contracts deploymenttype:featureNew capability

    Type

    No type

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions