Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
363 changes: 172 additions & 191 deletions .beads/issues.jsonl

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions planning/collab/crypto-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,9 +109,9 @@ On the wire, the envelope stores `nonce`, `ciphertext`, and the AAD fields in cl

- XChaCha20's 24-byte nonce space (192 bits) is large enough that random nonces are safe without coordination. Birthday-collision risk is negligible up to ~2^48 envelopes per key.
- **Never** reuse a nonce with the same key.
- For `snapshot_blob` with R2 spillover, the AEAD encrypts the *blob bytes themselves* (not the BlobRef). The envelope in the DO carries only the `BlobRef` (also encrypted, separately, with `eventKey` since it's metadata-shaped) — pin the convention:
- R2 object body = `nonce || ciphertext || tag` of the snapshot bytes under `snapshotKey`.
- DO envelope ciphertext = AEAD-encrypt of the canonical-JSON BlobRef under `eventKey`.
- For `snapshot_blob` with R2 spillover, the AEAD encrypts the *blob bytes themselves* (not the BlobRef). The envelope in the DO carries only the `BlobRef` (also encrypted, separately) — pin the convention:
- R2 object body = `nonce || ciphertext || tag` of the snapshot bytes under `snapshotKey`, with the AAD bound to the wrapper envelope's cleartext header (same `EnvelopeAad` shape as the envelope itself) so a blob body cannot be swapped between envelopes.
- DO envelope ciphertext = AEAD-encrypt of the canonical-JSON BlobRef under `snapshotKey`. (Amended 2026-06-10 from the original `eventKey` choice: the inbound pipeline keys strictly by `kind`, and a receiver cannot know whether a `snapshot_blob` plaintext is bytes or a BlobRef until after decrypting — a per-shape key split would force trial decryption. One key per kind keeps the dispatch table total.)
- The relay sees neither.

## Hashcash Proof-of-Work
Expand Down
18 changes: 14 additions & 4 deletions relay/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -330,7 +330,16 @@ export default {
return handleBlobPut(request, env, url, roomId, envelopeId);
}
if (request.method === "GET") {
return handleBlobGet(request, env, url, roomId, envelopeId);
// Cap-bearing GET → serve bytes straight from R2. Cap-less GET →
// the DO's download-presign endpoint (admission-auth'd), which
// mints the cap this branch later consumes.
if (url.searchParams.has("cap")) {
return handleBlobGet(request, env, url, roomId, envelopeId);
}
const id = env.RELAY_ROOMS.idFromName(roomId);
const stub = env.RELAY_ROOMS.get(id);
const response = await stub.fetch(request);
return corsMiddleware(request, env, response);
}
return Response.json(
{ error: { code: "ATTN_METHOD_NOT_ALLOWED", message: `${request.method} not allowed on /blobs/:envelopeId` } },
Expand Down Expand Up @@ -474,9 +483,10 @@ async function handleBlobPut(
/**
* `GET /v2/rooms/:roomId/blobs/:envelopeId?cap=...`
*
* Download a previously-uploaded blob. The cap was minted by an explicit
* (forthcoming) `GET /v2/rooms/:roomId/blobs/:envelopeId` DO endpoint — for
* now tests mint the cap directly via the r2.ts helper.
* Download a previously-uploaded blob. The cap is minted by the cap-less
* form of the same route, which the Worker forwards to the DO's
* admission-auth'd download-presign endpoint (room-do.ts
* `handleBlobDownloadPresign`).
*/
async function handleBlobGet(
request: Request,
Expand Down
69 changes: 68 additions & 1 deletion relay/src/room-do.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ import { canonicalize, type CanonicalValue } from "./canonical";
import type { Env } from "./env";
import { OwnerSigError, verifyOwnerSignature } from "./owner-sig";
import { parsePow, POW_MAX_LIFETIME_MS, PowError, verifyPow } from "./pow";
import { presignBlobUpload, type PresignedUploadResult } from "./r2";
import { blobObjectKey, presignBlobDownload, presignBlobUpload, type PresignedUploadResult } from "./r2";
import { DurableObjectRateLimit, type RateLimitResult } from "./rate-limit";
import {
acksRequestSchema,
Expand All @@ -49,6 +49,7 @@ const ROOM_ENVELOPES_PATH_RE = /^\/v2\/rooms\/([^/]+)\/envelopes\/?$/;
const ROOM_ACKS_PATH_RE = /^\/v2\/rooms\/([^/]+)\/acks\/?$/;
const ROOM_SOCKET_PATH_RE = /^\/v2\/rooms\/([^/]+)\/socket\/?$/;
const ROOM_BLOBS_PATH_RE = /^\/v2\/rooms\/([^/]+)\/blobs\/?$/;
const ROOM_BLOB_OBJECT_PATH_RE = /^\/v2\/rooms\/([^/]+)\/blobs\/([^/]+)\/?$/;
/** Any path starting with `/v2/rooms/:roomId` (optionally followed by a subroute). */
const ROOM_PATH_LOOSE_RE = /^\/v2\/rooms\/([^/]+)(?:\/.*)?$/;

Expand Down Expand Up @@ -343,6 +344,22 @@ export class RoomDO extends DurableObject<Env> {
return errorResponse(405, "ATTN_METHOD_NOT_ALLOWED", `${request.method} not allowed on /acks`);
}

// Must precede ROOM_BLOBS_PATH_RE — `/blobs/:envelopeId` would otherwise
// never match (`/blobs` is a prefix of it only with the optional trailing
// slash, but ordering keeps the intent explicit).
const blobObjectMatch = url.pathname.match(ROOM_BLOB_OBJECT_PATH_RE);
if (blobObjectMatch) {
const roomId = blobObjectMatch[1];
const envelopeId = blobObjectMatch[2];
if (roomId === undefined || roomId === "" || envelopeId === undefined || envelopeId === "") {
return errorResponse(400, "ATTN_ROOM_ID_INVALID", "roomId and envelopeId required");
}
if (request.method === "GET") {
return this.handleBlobDownloadPresign(request, roomId, decodeURIComponent(envelopeId), url.pathname);
}
return errorResponse(405, "ATTN_METHOD_NOT_ALLOWED", `${request.method} not allowed on /blobs/:envelopeId`);
}

const blobsMatch = url.pathname.match(ROOM_BLOBS_PATH_RE);
if (blobsMatch) {
const roomId = blobsMatch[1];
Expand Down Expand Up @@ -1752,6 +1769,56 @@ export class RoomDO extends DurableObject<Env> {
return Response.json(presigned, { status: 200 });
}

// -- GET /v2/rooms/:roomId/blobs/:envelopeId (download presign) ----------

/**
* Mint a download capability for a previously-uploaded R2 spillover blob,
* per relay-spec.md §R2 spillover: "Reads use a presigned GET URL fetched
* via GET /v2/rooms/:roomId/blobs/:envelopeId".
*
* The Worker routes a cap-less GET on this path here (a cap-bearing GET is
* served bytes directly from R2 — see index.ts `handleBlobGet`). Auth is
* admission-HMAC only: per crypto-spec.md §Hashcash Proof-of-Work, PoW
* does not apply to GETs (rate limits handle those).
*
* Existence is checked against R2 so clients get a 404 instead of a cap
* that will 404 at fetch time — distinguishing "blob not uploaded yet"
* (retry later) from "cap expired" (re-presign).
*/
private async handleBlobDownloadPresign(
request: Request,
roomId: string,
envelopeId: string,
urlPath: string,
): Promise<Response> {
const storedAdmissionKey = await this.ctx.storage.get<Uint8Array>(META.admissionKey);
if (storedAdmissionKey === undefined) {
return errorResponse(404, "ATTN_ROOM_NOT_FOUND", `room ${roomId} does not exist`);
}
try {
// GET bodies are always empty; hand the original request straight to
// verifyAdmission (no double-read concern).
await verifyAdmission(request, urlPath, { roomId, admissionKey: storedAdmissionKey });
} catch (err) {
if (err instanceof AdmissionError) {
return errorResponse(401, err.code, err.message);
}
throw err;
}

const head = await this.env.RELAY_BLOBS.head(blobObjectKey(roomId, envelopeId));
if (head === null) {
return errorResponse(404, "ATTN_BLOB_NOT_FOUND", `blob for envelope ${envelopeId} not found`);
}

try {
const presigned = await presignBlobDownload(this.env, roomId, envelopeId);
return Response.json(presigned, { status: 200 });
} catch (err) {
return errorResponse(500, "ATTN_BLOB_PRESIGN_FAILED", `presign failed: ${(err as Error).message}`);
}
}

// -- DELETE /v2/rooms/:roomId -------------------------------------------

/**
Expand Down
80 changes: 77 additions & 3 deletions relay/test/integration/blobs.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,31 @@ interface PresignedUploadResponse {
blobKey: string;
}

/** Cap-less GET on the blob object path → DO download-presign endpoint. */
async function getBlobDownloadPresign(opts: {
roomId: string;
admissionKey: Uint8Array;
envelopeId: string;
omitAdmission?: boolean;
}): Promise<Response> {
const url = `${URL_BASE}/v2/rooms/${opts.roomId}/blobs/${encodeURIComponent(opts.envelopeId)}`;
const headers: Record<string, string> = {};
if (!opts.omitAdmission) {
headers["Attn-Admission"] = await admissionHeaderFor({
method: "GET",
url,
admissionKey: opts.admissionKey,
});
}
return SELF.fetch(url, { method: "GET", headers });
}

interface PresignedDownloadResponse {
downloadUrl: string;
method: "GET";
expiresAt: number;
}

interface ErrorResponse {
error: { code: string; message: string };
}
Expand Down Expand Up @@ -396,9 +421,18 @@ describe("POST /v2/rooms/:roomId/blobs — happy path", () => {
});
expect(putRes.status).toBe(204);

// Mint a download cap (in v2 the GET endpoint owning this lives in 5.10/5.x;
// for now tests mint directly via the r2.ts helper).
const download = await presignBlobDownload(env, roomId, "blob-rt-1");
// Mint a download cap via the real endpoint: cap-less GET on the blob
// path routes to the DO's admission-auth'd download-presign handler.
const presignDlRes = await getBlobDownloadPresign({
roomId,
admissionKey,
envelopeId: "blob-rt-1",
});
expect(presignDlRes.status).toBe(200);
const download = (await presignDlRes.json()) as PresignedDownloadResponse;
expect(download.method).toBe("GET");
expect(download.downloadUrl).toContain("cap=");
expect(download.expiresAt).toBeGreaterThan(Date.now());
const getRes = await SELF.fetch(`${URL_BASE}${download.downloadUrl}`, { method: "GET" });
expect(getRes.status).toBe(200);
const fetched = new Uint8Array(await getRes.arrayBuffer());
Expand Down Expand Up @@ -557,6 +591,46 @@ describe("GET /v2/rooms/:roomId/blobs/:envelopeId — missing object", () => {
});
});

describe("GET /v2/rooms/:roomId/blobs/:envelopeId — download presign endpoint", () => {
it("rejects a cap-less GET without admission with 401", async () => {
const roomId = uniqueRoomId("blob-dl-noadm");
const owner = await generateEd25519Keypair();
const admissionKey = await createRoom({ roomId, ownerKp: owner });
const res = await getBlobDownloadPresign({
roomId,
admissionKey,
envelopeId: "whatever",
omitAdmission: true,
});
expect(res.status).toBe(401);
});

it("returns 404 ATTN_ROOM_NOT_FOUND for a room that does not exist", async () => {
const res = await getBlobDownloadPresign({
roomId: uniqueRoomId("blob-dl-noroom"),
admissionKey: makeAdmissionKey(0x99),
envelopeId: "whatever",
});
expect(res.status).toBe(404);
const err = (await res.json()) as ErrorResponse;
expect(err.error.code).toBe("ATTN_ROOM_NOT_FOUND");
});

it("returns 404 ATTN_BLOB_NOT_FOUND when the blob was never uploaded", async () => {
const roomId = uniqueRoomId("blob-dl-noblob");
const owner = await generateEd25519Keypair();
const admissionKey = await createRoom({ roomId, ownerKp: owner });
const res = await getBlobDownloadPresign({
roomId,
admissionKey,
envelopeId: "never-uploaded",
});
expect(res.status).toBe(404);
const err = (await res.json()) as ErrorResponse;
expect(err.error.code).toBe("ATTN_BLOB_NOT_FOUND");
});
});

describe("Room delete sweeps blobs (cross-check with 5.10)", () => {
it("removes a blob uploaded under the room from R2 after DELETE /v2/rooms/:roomId", async () => {
const roomId = uniqueRoomId("blob-sweep");
Expand Down
136 changes: 136 additions & 0 deletions scripts/test-snapshot-blob-e2e.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
#!/usr/bin/env bash
# Snapshot blob-lane E2E — guards the large-file-share regression (2026-06-10):
# a ~100 KB markdown file produced a ~1 MB SnapshotCreated event envelope,
# which the relay rejected with 413 (HARD_MAX_EVENT_BYTES = 256 KiB). The
# poison envelope blocked the outbox — including WebRTC signaling — forever,
# so sharing silently never worked.
#
# The fix routes snapshot bytes through the `kind=snapshot_blob` lane
# (5 MiB cap): inline through the mailbox at ≤ 1 MiB sealed, R2 presign+PUT
# above. This script proves both lanes end-to-end against a real local relay:
#
# #1 owner shares a folder with a mid-size doc (inline blob lane) and a
# large doc (R2 spillover lane) — both publish without a 413
# #2 the owner log shows one storage=Mailbox and one storage=R2 publish
# #3 the reviewer (windowed daemon join) receives BOTH snapshots with
# full markdown — blob → relay → reviewer → rehydration → frontend
# #4 no "413"/"Payload Too Large" anywhere in either daemon log
#
# Every wait is a polled condition (per CLAUDE.md: never sleep on a condition).
# ATTN_SKIP_SNAPSHOT_BLOB_E2E=1 → clean skip (no relay/daemon infra).

set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
PROJECT_DIR="$(dirname "$SCRIPT_DIR")"
cd "$PROJECT_DIR"

if [ "${ATTN_SKIP_SNAPSHOT_BLOB_E2E:-0}" = "1" ]; then
echo "test-snapshot-blob-e2e: ATTN_SKIP_SNAPSHOT_BLOB_E2E=1 — skipping (clean exit)"; exit 0
fi

: "${RELAY_PORT:=8801}"
: "${ATTN_BIN:=$PROJECT_DIR/target/debug/attn}"
RELAY_URL="${ATTN_EXTERNAL_RELAY:-http://localhost:${RELAY_PORT}}"
OWNER_HOME="/tmp/attn-blob-owner"; REV_HOME="/tmp/attn-blob-reviewer"
WORK="/tmp/attn-blob-work"; DOCS="$WORK/owner-docs"; REVLOCAL="$WORK/reviewer-local"
RELAY_LOG="$WORK/relay.log"; OWNER_LOG="$WORK/owner.log"; REV_LOG="$WORK/reviewer.log"
RELAY_PID=""; OWNER_PID=""; REV_PID=""; PASS=0; FAIL=0

log() { printf '\n==> %s\n' "$*"; }
ok() { PASS=$((PASS+1)); printf ' \033[32mPASS\033[0m %s\n' "$*"; }
bad() { FAIL=$((FAIL+1)); printf ' \033[31mFAIL\033[0m %s\n' "$*"; }
info(){ printf ' %s\n' "$*"; }
attn_owner() { ATTN_HOME="$OWNER_HOME" ATTN_RELAY_URL="$RELAY_URL" "$ATTN_BIN" "$@"; }
attn_rev() { ATTN_HOME="$REV_HOME" ATTN_RELAY_URL="$RELAY_URL" "$ATTN_BIN" "$@"; }
ev_rev() { attn_rev --eval "$1" 2>/dev/null | sed 's/^"//; s/"$//'; }
poll() { local t="$1"; shift; local d=$(( $(date +%s)*1000 + t )); while [ "$(($(date +%s)*1000))" -lt "$d" ]; do if "$@" >/dev/null 2>&1; then return 0; fi; sleep 0.25; done; return 1; }
kill_pid() { local p="$1"; [ -z "$p" ] && return 0; kill "$p" 2>/dev/null || true; wait "$p" 2>/dev/null || true; }
cleanup() { log "Cleaning up"; kill_pid "$OWNER_PID"; kill_pid "$REV_PID"; if [ -n "$RELAY_PID" ]; then pkill -P "$RELAY_PID" 2>/dev/null || true; kill_pid "$RELAY_PID"; fi; pkill -f "wrangler dev --local --port $RELAY_PORT" 2>/dev/null || true; }
trap cleanup EXIT INT TERM

[ -x "$ATTN_BIN" ] || { log "Building attn"; cargo build || exit 1; }

json_decode() { node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{try{process.stdout.write(String(JSON.parse(s)))}catch{process.stdout.write("")}})'; }

rm -rf "$OWNER_HOME" "$REV_HOME" "$WORK"
mkdir -p "$OWNER_HOME" "$REV_HOME" "$DOCS" "$REVLOCAL"

# Fixture shape mirrors the original failing file (GPU-CLI-OUTREACH-100.md):
# many short heading/paragraph blocks. The anchor index balloons the sealed
# snapshot ~20x for this block-dense shape, so: mid.md (~28 KB markdown →
# ~0.6 MiB sealed) rides the inline mailbox lane; big.md (~150 KB markdown →
# ~3 MiB sealed) crosses the 1 MiB threshold → R2 spillover lane.
gen_doc() { # gen_doc <path> <title> <blocks>
local p="$1" t="$2" n="$3" i=1
{ printf '# %s\n\n' "$t"
while [ "$i" -le "$n" ]; do
printf '## Target %04d\n\n- handle: @dev_%04d\n- focus: GPU kernels, CLI tooling, outreach batch %d\n\nShort pitch paragraph for target %04d with enough words to look like the real outreach notes doc.\n\n' "$i" "$i" $((i % 7)) "$i"
i=$((i+1))
done
} > "$p"
}
gen_doc "$DOCS/mid.md" "Mid outreach doc (inline blob lane)" 150
gen_doc "$DOCS/big.md" "Big outreach doc (R2 spillover lane)" 800
printf '# Reviewer Local Scratch\n' > "$REVLOCAL/mine.md"
info "mid.md: $(wc -c < "$DOCS/mid.md") bytes, big.md: $(wc -c < "$DOCS/big.md") bytes"

if [ -n "${ATTN_EXTERNAL_RELAY:-}" ]; then
curl -fsS "$RELAY_URL/health" >/dev/null 2>&1 || { log "external relay unreachable"; exit 1; }
else
[ -d "$PROJECT_DIR/relay/node_modules" ] || (cd relay && npm ci >/dev/null)
log "Starting relay on :$RELAY_PORT"
( cd "$PROJECT_DIR/relay" && exec npx wrangler dev --local --port "$RELAY_PORT" ) >"$RELAY_LOG" 2>&1 & RELAY_PID=$!
deadline=$(( $(date +%s) + 60 )); while [ "$(date +%s)" -lt "$deadline" ]; do curl -fsS "$RELAY_URL/health" >/dev/null 2>&1 && break; kill -0 "$RELAY_PID" 2>/dev/null || { log "relay died"; tail -20 "$RELAY_LOG"; exit 1; }; sleep 0.3; done
fi
log "Relay healthy"

log "Owner daemon on owner-docs/mid.md; reviewer daemon on reviewer-local/mine.md"
ATTN_HOME="$OWNER_HOME" ATTN_RELAY_URL="$RELAY_URL" "$ATTN_BIN" --no-fork "$DOCS/mid.md" >"$OWNER_LOG" 2>&1 & OWNER_PID=$!
ATTN_HOME="$REV_HOME" ATTN_RELAY_URL="$RELAY_URL" "$ATTN_BIN" --no-fork "$REVLOCAL/mine.md" >"$REV_LOG" 2>&1 & REV_PID=$!
poll 30000 attn_owner --wait-for 'h1' --timeout 1000 || { bad "owner never rendered"; tail -20 "$OWNER_LOG"; exit 1; }
poll 30000 attn_rev --wait-for 'h1' --timeout 1000 || { bad "reviewer never rendered"; tail -20 "$REV_LOG"; exit 1; }

log "Owner: attn review share owner-docs/ (mid + big)"
attn_owner review share "$DOCS" 2>&1 | sed 's/^/ /'

# #1/#2 — both lanes published, visible in the owner daemon log
# (--no-fork logs to stdout, i.e. $OWNER_LOG, not $ATTN_HOME/attn.log).
mailbox_published() { grep -q "published snapshot.*storage=Mailbox" "$OWNER_LOG"; }
r2_published() { grep -q "published snapshot.*storage=R2" "$OWNER_LOG"; }
if poll 60000 mailbox_published; then ok "#2 inline mailbox-lane snapshot published (mid.md)"; else bad "#2 no storage=Mailbox publish in owner log"; fi
if poll 60000 r2_published; then ok "#2 R2 spillover-lane snapshot published (big.md)"; else bad "#2 no storage=R2 publish in owner log"; fi

INVITE="$(attn_owner --eval 'window.__attn_review_store__?.currentShare?.inviteUrl || ""' 2>/dev/null | json_decode)"
[ -n "$INVITE" ] && ok "#1 owner minted an invite" || { bad "#1 no invite url on owner"; log "Result: $PASS passed, $FAIL failed"; exit 1; }

log "Reviewer: attn review join <invite> (windowed daemon)"
attn_rev review join "$INVITE" 2>&1 | sed 's/^/ /'

# #3 — the reviewer receives BOTH snapshots with full markdown bodies. The
# wire form never inlines plaintext, so non-empty markdown here proves the
# whole chain: snapshot_blob envelope (and the R2 fetch for big.md) →
# reviewer blob store → rehydration at the IPC boundary → frontend store.
rev_has_both() {
local n
n="$(ev_rev 'String((window.__attn_review_store__?.snapshots||[]).filter(s=>(s.markdown||"").length>10000).length)')"
[ "$n" -ge 2 ] 2>/dev/null
}
if poll 60000 rev_has_both; then
ok "#3 reviewer holds both snapshots with full markdown"
info "sizes: $(ev_rev '(window.__attn_review_store__?.snapshots||[]).map(s=>(s.markdown||"").length).join(",")')"
else
bad "#3 reviewer missing snapshot markdown"
info "snapshots: $(ev_rev 'JSON.stringify((window.__attn_review_store__?.snapshots||[]).map(s=>({f:s.fileId,len:(s.markdown||"").length})))')"
tail -20 "$REV_LOG" 2>/dev/null | sed 's/^/ rev: /'
fi

# #4 — the original failure signature must be gone from both daemons.
if grep -q -e "413" -e "Payload Too Large" "$OWNER_LOG" "$REV_LOG" 2>/dev/null; then
bad "#4 a relay 413 appeared in a daemon log"
grep -h -e "413" -e "Payload Too Large" "$OWNER_LOG" "$REV_LOG" | head -3 | sed 's/^/ /'
else
ok "#4 no 413 / Payload Too Large in daemon logs"
fi

echo ""; log "Result: $PASS passed, $FAIL failed"
[ "$FAIL" -eq 0 ]
Loading
Loading