Skip to content

channel-integration-plan.md

Latest commit

 

History

History
255 lines (197 loc) · 7.68 KB

channel-integration-plan.md

File metadata and controls

255 lines (197 loc) · 7.68 KB
Raw

Channel Integration Plan - Reliable-Ordered Messages and Jitter Estimation

Status

Accepted - 2026-06-09

1. Gap Analysis

After porting netcode, reliable, and serialize, three features remain unimplemented:

Feature Gap Source reference
Reliable-ordered messages + aggressive resend No message channel layer above ReliableEndpoint yojimbo_reliable_ordered_channel.h/.cpp
Large data blocks on reliable messages No block fragmentation at message layer yojimbo_reliable_ordered_channel.h SendBlockData / ReceiveBlockData
Jitter estimation per-connection ReliableEndpoint tracks RTT but not jitter yojimbo_network_info.h NetworkInfo.RTT

2. Architecture

flowchart TD
    App[Application] -->|sendMessage / sendBlock| ROC[ReliableOrderedChannel]
    ROC -->|generatePacketData - WriteStream| Conn[Connection]
    Conn -->|sendPacket| RE[ReliableEndpoint]
    RE -->|transmitHandler| Transport[UdpTransport]

    Transport -->|recv| RE2[ReliableEndpoint]
    RE2 -->|processHandler - receivePacket| Conn2[Connection]
    Conn2 -->|processPacketData - ReadStream| ROC2[ReliableOrderedChannel]
    ROC2 -->|receiveMessage| App2[Application]

    RE2 -->|getAcks| Conn3[Connection]
    Conn3 -->|processAck| ROC3[ReliableOrderedChannel]
Loading

Layer contract

Application
   |  sendMessage(msgType, data, off, len)
   |  sendBlock(msgType, blockData, off, len)
   |  receiveMessage(...) -> msgType or -1
   v
ReliableOrderedChannel  <- this plan
   |  generatePacketData(WriteStream, packetSeq, availBits) -> bitsWritten
   |  processPacketData(ReadStream, packetSeq)
   |  processAck(packetSeq)
   |  advanceTime(time)
   v
Connection              <- this plan (thin glue)
   |  sendPendingData() -> calls endpoint.sendPacket()
   |  onPacketReceived(data, off, len, packetSeq)
   |  onAcksReceived(acks, numAcks)
   v
ReliableEndpoint        <- already exists; add jitter

3. Key Differences from Yojimbo

Yojimbo uses reference-counted Message* objects allocated per send. This is hostile to the allocation-free constraint. The Java port uses:

Yojimbo Java port
Message* ref-counted objects Raw byte[] per slot (pre-allocated at construction)
MessageFactory creating typed objects msgType int tag; caller serializes before enqueue
BlockMessage with attached heap block Dedicated pre-allocated byte[] sendBlockData (single in-flight block)
BitArray class for fragment ack tracking long[] bit arrays (256 fragments max = 4 longs)
Dynamic sentPacket->messageIds alloc Flat pre-allocated short[] sentMsgIds[sentCap * maxPerPacket]

4. Wire Format

Channel data is written into the CONNECTION_PAYLOAD plaintext using WriteStream / ReadStream. It sits between the reliable header (added by ReliableEndpoint) and the UDP packet boundary.

Message-mode packet (no block)

numMessages : 8 bits                        (0..maxMessagesPerPacket)
for each message i:
  messageId  : 16 bits                      (uint16, wrapping)
  messageType: 16 bits
  messageLen : 16 bits                      (byte count, 0..maxMessageSize)
  align to byte boundary
  messageData: messageLen bytes

Block-mode packet (one fragment per packet)

messageId     : 16 bits                     (message id the block is attached to)
fragmentId    : 16 bits                     (0..numFragments-1)
numFragments  : 16 bits
messageType   : 16 bits
fragmentSize  : 16 bits                     (bytes in this fragment)
hasMsgData    : 1 bit                       (1 only when fragmentId == 0)
if hasMsgData:
  msgDataLen  : 16 bits                     (header bytes, 0..maxMessageSize)
  align to byte boundary
  msgData     : msgDataLen bytes
align to byte boundary
fragmentData  : fragmentSize bytes

Multi-channel envelope (for Connection with multiple channels)

for each channel c in [0, numChannels):
  hasData  : 1 bit
  if hasData:
    isBlock: 1 bit
    ... message-mode or block-mode data above ...

5. Data Structures

Send queue (per-slot primitives, indexed by msgId & mask)

int[]     sendSeq;         // 0xFFFF_FFFF = empty; else uint32 of msgId
double[]  sendTimeLastSent;
boolean[] sendIsBlock;
int[]     sendMsgType;
int[]     sendMsgLen;
byte[][]  sendData;        // [sendCap][maxMessageSize]  - non-block header bytes or full message

Receive queue (per-slot primitives)

int[]     recvSeq;
int[]     recvMsgType;
int[]     recvMsgLen;
boolean[] recvIsBlock;
byte[][]  recvData;        // [recvCap][maxMessageSize]  - non-block messages

Sent packet tracking (indexed by packetSeq & mask)

int[]    sentPktSeq;            // 0xFFFF_FFFF = empty
boolean[] sentPktIsBlock;
int[]    sentPktNumMsgIds;
short[]  sentPktMsgIds;         // [sentCap * maxMessagesPerPacket]
short[]  sentPktBlockMsgId;
short[]  sentPktBlockFragId;

Block send state (one in-flight block at a time)

boolean  sendBlockActive;
int      sendBlockMsgId;        // uint16
int      sendBlockSize;
int      sendBlockNumFrags;
int      sendBlockNumAckedFrags;
long[]   sendBlockAcked;        // bit array, 256 max fragments = 4 longs
double[] sendBlockFragTime;     // last send time per fragment
byte[]   sendBlockData;         // pre-alloc maxBlockSize bytes
int      sendBlockMsgType;

Block receive state

boolean  recvBlockActive;
int      recvBlockMsgId;
int      recvBlockNumFrags;
int      recvBlockNumRecvFrags;
int      recvBlockSize;
int      recvBlockMsgType;
long[]   recvBlockReceived;     // bit array
byte[]   recvBlockData;         // pre-alloc maxBlockSize bytes

6. Jitter Estimation

Add jitter (float, ms) to ReliableEndpoint. Computed in processAckBits() using mean absolute deviation from RTT:

rttDeviation = |rttSample - smoothedRtt|
jitter += (rttDeviation - jitter) * rttSmoothingFactor

API addition: ReliableEndpoint.jitter() returns float in ms.

7. Constraints and Invariants

  • All buffer sizes MUST be power-of-two (enforced by constructor assertion)
  • Only ONE block message can be in-flight at a time (send side); returns error otherwise
  • Only ONE block can be in reassembly at a time (recv side); mismatched messageId -> desync
  • maxBlockFragments = maxBlockSize / blockFragmentSize - MUST be <= 256
  • sentPktMsgIds flat array: index = (packetSeq & sentMask) * maxMessagesPerPacket + i
  • Message ID arithmetic uses wrapping uint16: (id & 0xFFFF)
  • Sequence comparisons use sequenceGreaterThan (half-space rule, same as reliable)

8. Package Layout

New package: net.ztrust.channel

net.ztrust.channel/
  ChannelConfig.java               - configuration (all public fields, defaults from yojimbo)
  ChannelError.java                - error level constants
  ReliableOrderedChannel.java      - main class: send queue, recv queue, block send/recv
  Connection.java                  - glues ReliableOrderedChannel[] to ReliableEndpoint

New tests: src/test/java/.../channel/

  ReliableOrderedChannelTest.java  - send/recv, resend, ack, block send/recv
  ConnectionTest.java              - end-to-end round trip

9. Performance Budget

Operation Target
sendMessage (enqueue) < 50 ns
generatePacketData (64 messages, no block) < 2 us
processPacketData (64 messages) < 2 us
processAck < 200 ns
sendBlock fragment selection < 300 ns
Allocation on any hot-path call 0 bytes

10. File Creation Sequence

  1. ChannelConfig.java
  2. ChannelError.java
  3. ReliableOrderedChannel.java
  4. Connection.java
  5. Patch ReliableEndpoint.java - add jitter
  6. ReliableOrderedChannelTest.java
  7. ConnectionTest.java