Original Spec

[rawkode]

proto — A Git-Native, Single-Binary Forge

Context

proto is a self-hostable git forge in Rust whose defining principle is
operational minimalism: one binary, no central database, and everything
configured through git. It must be trivial to run, back up (it's just git plus a
few SQLite files), and migrate.

The forge core is deliberately small — git hosting + access control + a read-only
web UI. Everything else (Pull Requests, Issues, Epics, …) ships as WebAssembly
Component extensions loaded at startup and configured per-repo. Each extension
gets its own isolated libSQL/Turso embedded SQLite database; the host exposes a
single Apollo-Federation-v2 /graphql that composes extension subgraphs.

This document is the umbrella architecture spec for the whole system plus a
detailed plan for sub-project 1 (forge core + config reconciliation). Remaining
sub-projects get their own spec → plan → build cycles.

Decisions captured during design

• Primary driver: self-host simplicity (single binary, zero external services).
• MVP core: git hosting + access, read-only web browse. PRs/Issues/Epics are WASM extensions.
• Mutable extension state: per-extension libSQL (Turso) embedded SQLite (NOT one central DB).
• Config: admin repo (global identities/keys/teams/repo-catalog/extension-registry) + per-repo .forge/config.toml.
• Extension surface: HTTP routes, git hooks, UI slots/widgets, scheduled jobs, federated GraphQL subgraph.
• WASM: Wasmtime + Component Model (WIT world).
• Federation: Apollo Federation v2, with real composition + query planning (pure Rust crates).
• Extension distribution: OCI artifacts (ref + digest).
• Git transport: both SSH and smart-HTTPS — pure Rust, no git binary, no shelling out.
• Storage: local filesystem only — object storage (R2) is explicitly out of scope / future.

---

Part A — Architecture Spec (whole system)

A1. North star & non-goals

North star: a single static-ish Rust binary you can drop on a host, point at an
admin git repo, and have a working forge — backups are git + a state/ dir.

Non-goals (v1): object storage backends (R2/S3), HA/clustering, in-browser git
editing, CI/CD runners. (A real federation query planner IS in scope — supplied by
the Hive Router crate, not hand-rolled.)

A2. On-disk layout (local filesystem only)

<data-root>/
  admin/                     # clone of the admin repo (source of truth for config)
  repos/<name>.git/          # bare git repos (one per managed repo)
  extensions/
    blobs/sha256/<digest>    # content-addressed, verified .wasm bytes (OCI cache)
    manifests/<reg>/<repo>/<digest>.json
  state/
    <ext>/<repo>.db          # per-extension × per-repo libSQL files
    host_ed25519             # SSH host key (forge-local secret)
    session.key              # web session signing key (forge-local secret)
  config-snapshot.json       # last reconciled DesiredState (debug/audit)

A3. Component map

Layer	Responsibility	Key crates	Sub-project
Core + reconcile	binary, CLI, admin-repo clone/fetch, config model, desired-vs-actual reconcile, permission resolution	tokio, clap, gix (network-client), toml+serde, arc-swap	1
Git hosting	SSH (russh) + smart-HTTPS transport, auth, permission enforcement, in-process hook dispatch	russh, axum, gix plumbing (gix-pack, gix-protocol, gix-ref, gix-negotiate) — pure Rust, no git binary	2
Web UI (browse)	repo/file/commit/branch/tag browse, diff, blame	axum, gix, maud/askama	3
WASM host	Wasmtime engine, WIT world, host capabilities (db/git/http/registration), OCI pull+cache, lifecycle	wasmtime, wit-bindgen, oci-client+oci-wasm, libsql	4
Extension dispatch	wire routes/hooks/jobs/UI-slots to loaded extensions	(host)	5
GraphQL gateway	/graphql, runtime Fed-v2 composition + query planning, plan execution into WASM	graphql-composition, hive-router query-planner, custom WASM executor; async-graphql (subgraph side)	6
First-party extensions	Issues → PRs → Epics, as WASM components	guest-side, target the WIT world	7+

A4. Configuration model (git as source of truth)

Admin repo (<data-root>/admin/, TOML): identities (display name, SSH public
keys, OIDC subject/email mapping, argon2id-hashed PATs), teams, global settings,
repo catalog, per-repo permission grants, and the extension registry (OCI ref +
digest + which repos enable it). The forge hosts its own admin repo so pushes
trigger reconciliation.

Per-repo (.forge/config.toml at the default branch tip): collaborators,
enabled extensions, and extension settings — read during reconciliation.

Reconciliation: clone-or-fetch admin repo (pure-gix client) → parse to a typed
DesiredState → validate the whole state (reject the entire push if invalid) →
diff against actual on-disk → apply idempotently (create bare repos, ensure
extension artifacts pulled, create/migrate per-extension DBs) → swap the
in-memory snapshot via ArcSwap so live requests always see a consistent view.
Never auto-delete repos — require an explicit archived/deleted marker
(tombstone dir). Optionally require signed commits on the admin repo so
reconfiguration is authenticated.

A5. Auth & secrets model (keeps "config in git" honest)

• SSH public keys → in admin repo (git). Public keys aren't secrets; matched in
  russh auth_publickey, mapped key→identity→permissions. Primary git auth.
• HTTP git PATs → argon2id hash stored in git. Forge mints the token (shown
  once), commits only the hash to the admin repo; constant-time compare on request.
• Web-UI login → OIDC to an external IdP (openidconnect + tower-sessions).
  Only non-secret OIDC client config in git; client secret + session key are
  forge-local (state/).
• Forge-local secrets (never in git): SSH host key, OIDC client secret, session
  signing key, registry credentials, the libSQL files.

A6. WASM extension contract (WIT world — sketch)

Pattern: guest export init() calls host import register-*() once to declare its
routes/hooks/jobs/UI-slots/subgraph; the host records handler-ids and later
dispatches handle-route / handle-hook / run-job / render-slot /
graphql-resolve. Guests stay synchronous; host functions are async (so
db.query can drive libSQL). DB connections and read-only git handles are WIT
resources — never pass file paths/fds across the boundary.

package forge:ext@0.1.0;

interface db {                       // host import; per-extension isolated libSQL
  resource connection {
    query: func(sql: string, params: list<value>) -> result<rows, db-error>;
    execute: func(sql: string, params: list<value>) -> result<u64, db-error>;
    transaction: func() -> result<txn, db-error>;
  }
  resource txn  { execute: func(sql: string, params: list<value>) -> result<u64, db-error>;
                  commit: func() -> result<_, db-error>; rollback: func(); }
  resource rows { next: func() -> option<list<value>>; columns: func() -> list<string>; }
  variant value { null, integer(s64), real(f64), text(string), blob(list<u8>) }
}

interface git-read {                 // host import; READ-ONLY
  resource repo {
    resolve-ref: func(name: string) -> option<oid>;
    read-object: func(id: oid) -> result<git-object, git-error>;
    list-tree:   func(tree: oid) -> result<list<tree-entry>, git-error>;
    read-blob:   func(id: oid) -> result<blob, git-error>;
  }
  record oid { hex: string }
}

interface http-types {
  record request  { method: string, path: string, query: string,
                     headers: list<tuple<string,string>>, body: list<u8> }
  record response { status: u16, headers: list<tuple<string,string>>, body: list<u8> }
}

interface registration {             // host import; called from init()
  register-route:   func(method: string, path-pattern: string, handler-id: string);
  register-hook:    func(event: hook-event, handler-id: string);
  register-job:     func(name: string, schedule: string, handler-id: string);
  register-ui-slot: func(slot: string, widget-id: string);
  register-subgraph: func(sdl: string);   // federation SDL emitted here
}

world extension {
  import db; import git-read; import http-types; import registration;
  export init: func();
  export handle-route:    func(handler-id: string, req: http-types.request) -> http-types.response;
  export handle-hook:     func(handler-id: string, ev: hook-payload) -> hook-result;
  export run-job:         func(handler-id: string) -> result<_, string>;
  export render-slot:     func(widget-id: string, ctx: string) -> string;
  export graphql-resolve: func(req: gql-request) -> gql-response;
}

A7. GraphQL federation strategy (real composition + planning, all pure Rust)

The "hard pillar" is mostly off-the-shelf via maintained Rust crates — no
hand-rolled stitching, no JS, no external router process. Four stages, all
in-process in the single binary:

1. Subgraph SDL: each extension emits Fed-v2 subgraph SDL via async-graphql
   (enable_federation, #[graphql(entity)] → _service/_entities).
2. Composition (runtime): host composes the supergraph with graphql-composition
   (Grafbase, Apache-2.0): Subgraphs::ingest per extension → compose →
   render_federated_sdl. This fills the gap that Hive Router itself requires a
   pre-composed supergraph.
3. Query planning: host plans operations with Hive Router's query-planner
   crate (MIT, usable standalone) against the composed supergraph — real
   @key/@requires/@provides/entity planning.
4. Execution (custom WASM executor): host walks the query plan and, for each
   subgraph fetch node, dispatches the sub-operation into the owning WASM
   component's graphql-resolve export (instead of Hive's default HTTP transport),
   resolving _entities representations per the plan and stitching results.

• Recompose + replan-cache-invalidate when extensions load/unload (swap under lock).
• Build-time spikes (SP6): (a) confirm hive-router query-planner is consumable
  as a crate (crates.io vs git dep / vendored); (b) confirm its plan IR drives our
  custom executor (we own execution, avoiding coupling to Hive's HTTP executor);
  (c) version-compatibility between graphql-composition's join-spec output and the
  planner's expected supergraph dialect. Do NOT embed apollo-federation
  (unstable/internal) or harmonizer (bundles JS) at runtime.

A8. WASM runtime policy

• Per-request / per-event instantiation via InstancePre + pooling allocator
  (µs-scale instantiation) for clean isolation; long-lived instances only if
  measured necessary.
• CPU: epoch interruption (background increment_epoch timer). Memory:
  ResourceLimiter caps + OS/cgroup backstop. Host async via func_wrap_async;
  keep Store data Send.
• Supported guest tiers: Rust (gold); JS (jco/componentize-js), Python
  (componentize-py), TinyGo — best-effort. Keep the world on the sync-guest path.

A9. Git serving policy — pure Rust, no git binary, no shelling out

gix is client-only for the network protocol, but it re-exports the plumbing, so
we implement the server responders ourselves:

• Reads (browse/diff/log/refs/tags): gix high-level API. Blame: gix is
  correct but slow — cache aggressively (no git fallback, by design).
• upload-pack (fetch/clone) responder: advertise refs via gix::refs, parse
  client wants/haves over pkt-line via gix::protocol + gix-negotiate, then
  generate the packfile with gix-pack's generate feature
  (data::output count→entry→write).
• receive-pack (push) responder: read commands + incoming pack over pkt-line,
  ingest/resolve the pack with gix-pack's streaming-input (Bundle::write_to
  / index creation), then apply ref updates via gix::refs transactions.
• Transports: the same in-process responders are driven by both the SSH
  channel (russh exec of git-upload-pack/git-receive-pack) and the smart-HTTPS
  axum handlers (info/refs, git-upload-pack, git-receive-pack), which only add
  pkt-line framing + auth gating around the shared responder core.
• Hooks are in-process: the receive-pack responder calls policy/permission
  checks and (later) WASM extension pre-receive/post-receive hooks directly —
  no native hook scripts installed, nothing to repair on disk.
• Version pins: gix 0.73 → gix-pack 0.60 / gix-ref 0.53 family, used via the gix
  facade re-exports (gix::odb::pack, gix::protocol, gix::refs, gix::negotiate)
  to avoid version-matched direct sub-crate deps. Admin-repo clone/fetch uses gix's
  client network feature (blocking-network-client or async variant) — still
  pure Rust, no subprocess.

A10. Decomposition & build order

1. Forge core + config reconciliation ← detailed below.
2. Git hosting (SSH + smart-HTTPS + in-process hooks).
3. Web UI (browse).
4. WASM host runtime (engine, WIT world, capabilities, OCI pull, libSQL).
5. Extension dispatch (routes/hooks/jobs/UI-slots).
6. GraphQL federation gateway.
7. First-party extensions: Issues → PRs → Epics.

---

Part B — Sub-Project 1: Forge Core + Config Reconciliation

Goal: a binary that, given an admin repo, models identities/repos/permissions,
reconciles desired→actual on-disk state idempotently, and exposes a permission
-resolution API the later transport/UI/WASM layers consume. No transport, web UI,
or WASM yet — but clean seams (traits) for them.

B1. Crates

tokio, clap (derive), gix (with a network-client feature —
blocking-network-client or async variant — for pure-Rust admin-repo clone/fetch;
no git subprocess), serde + toml, arc-swap, thiserror/anyhow,
tracing + tracing-subscriber, argon2 (PAT hashing), tempfile (tests).
Pin gix 0.73 (→ gix-pack 0.60 / gix-ref 0.53 family) via facade re-exports.
Workspace laid out so later sub-projects add crates without churn.

B2. Module layout (new files under repo root)

• src/main.rs — CLI entry (clap), subcommands, tracing init.
• src/cli.rs — forge serve | reconcile | hook <type> | admin init.
• src/config/mod.rs — typed DesiredState (identities, teams, repos, grants,
  global settings, extension registry) + RepoConfig (.forge/config.toml).
• src/config/parse.rs — TOML→types via serde; schema validation.
• src/config/validate.rs — whole-state validation (dup identities, malformed
  keys, dangling team/extension refs, bad OCI digests). Returns aggregated errors.
• src/reconcile/mod.rs — orchestrator: acquire admin repo → parse → validate →
  diff → apply → ArcSwap swap.
• src/reconcile/admin_repo.rs — clone-or-init + fetch/fast-forward via gix client
  (pure Rust, no subprocess); pin to a resolved commit; optional signed-commit verification.
• src/reconcile/diff.rs — desired-vs-actual: repos to create, tombstones to honor
  (never destructive delete). No hook-script bookkeeping — hooks are in-process (SP2).
• src/reconcile/apply.rs — create bare repos via gix (gix::init_bare),
  idempotent re-runs. (Hook dispatch lives in the SP2 receive-pack responder, not on disk.)
• src/state.rs — ForgeState snapshot behind ArcSwap; live read handle.
• src/permissions.rs — resolve(identity, repo, op) -> Allow/Deny; consumed later
  by transport. Pure, table-driven, unit-testable.
• src/layout.rs — <data-root> path helpers (A2).
• src/identity.rs — identity model, SSH-key fingerprinting, PAT hash/verify (argon2id).

B3. Reconciliation flow (idempotent)

1. Acquire: clone admin repo on first run (or forge admin init scaffolds a
   minimal one); else fetch + fast-forward. Resolve to a concrete commit.
2. Parse admin TOML + each managed repo's .forge/config.toml (tip of default
   branch) into DesiredState.
3. Validate the entire state; on failure, abort without mutating anything (and,
   when invoked on an admin-repo push, reject the push).
4. Diff vs actual on-disk (existing bare repos, extension cache entries).
5. Apply under a process lock: create missing bare repos (gix init_bare),
   honor tombstones (move to tombstone dir, never rm -rf), persist
   config-snapshot.json. (No hook scripts — hooks fire in-process in SP2.)
6. Swap the ArcSwap<ForgeState> so handlers see the new snapshot atomically.

Triggers: forge serve startup; admin-repo push (detected in-process by the
SP2 receive-pack responder, which calls reconcile); a periodic fetch backstop for
out-of-band updates. (In SP1, forge reconcile is the manual entry point.)

B4. Testing (TDD — write tests first)

• Unit: config parse (valid/invalid TOML), validation error aggregation,
  permission resolution truth table, PAT hash/verify round-trip, diff logic
  (create/tombstone classification).
• Integration: spin up a temp admin repo (real git via tempfile + gix), run a
  full reconcile, assert bare repos created; mutate config, re-reconcile, assert
  idempotency and correct diffs; assert a tombstoned repo is moved not deleted;
  assert an invalid config is rejected with no partial mutation.
• CLI smoke: forge admin init then forge reconcile on a temp data-root.

B5. Seams left for later sub-projects (no premature abstraction)

• permissions::resolve is the single entry transport (SP2) will call.
• A reconcile() entry point SP2's in-process receive-pack hook can invoke on
  admin-repo push (no on-disk hook scripts).
• ForgeState carries the extension registry so SP4 can pull/cache without reshaping.
• layout.rs already reserves extensions/, state/ paths.

---

Verification (end-to-end for sub-project 1)

1. cargo test — all unit + integration tests green (the TDD suite above).
2. cargo run -- admin init --data-root /tmp/forge scaffolds a minimal admin repo.
3. Add an identity (SSH pubkey) + a repo to the admin repo, commit, then
   cargo run -- reconcile --data-root /tmp/forge:
   • assert /tmp/forge/repos/<name>.git exists and opens as a bare repo via
     gix (gix::open → is_bare() true) — no git binary used;
   • assert config-snapshot.json reflects desired state.
4. Re-run reconcile — assert no changes (idempotent), exit 0.
5. Mark a repo archived, reconcile — assert it moved to tombstone dir, not deleted.
6. Provide an invalid admin config — assert reconcile aborts with a clear aggregated
   error and the previous snapshot is untouched.
7. cargo run -- --help shows the documented subcommands.

Open decisions

1. Binary/product name — keep proto or rename?
2. Signed commits on the admin repo: require (recommended) or optional in v1?
3. Repo deletion: tombstone dir (recommended) vs archive flag only.
4. Web template engine (SP3): maud vs askama — defer to SP3.
5. Federation stack (SP6): graphql-composition (runtime compose) +
   hive-router query-planner (plan) + a custom WASM executor (execute plan into
   graphql-resolve). Build-time spike to confirm the planner is consumable as a
   crate and its plan IR drives our executor.

Validated technical findings (research-grounded)

• WASM host (wasmtime + Component Model): production-ready; async host fns via
  func_wrap_async, per-request instantiation + pooling allocator, epoch
  interruption, ResourceLimiter, WIT resource handles. Keep guests sync.
• Git serving: pure-Rust on gix plumbing — gix-pack generate (serve packs)
  and streaming-input (ingest packs), gix::protocol/gix-negotiate (pkt-line +
  negotiation), gix::refs (advertise + transactional updates). No git binary.
• Federation: graphql-composition (Grafbase, runtime compose) + hive-router
  query-planner (Rust, standalone) + custom WASM executor. Avoid apollo-federation
  (unstable) and harmonizer (JS).
• SSH: russh (auth_publickey → admin-repo keys; exec allowlist for
  git-upload-pack/git-receive-pack; channel↔responder bridge).
• Smart-HTTP: axum streaming around the shared responders; basic/bearer auth;
  handle gzip request bodies; correct pkt-line # service= framing on info/refs.
• libSQL: libsql 0.9.x core feature, WAL, user_version migrations,
  connection LRU for many small DBs.
• OCI: oci-client (+ oci-wasm), verify digest, content-addressed cache.
• Secrets: SSH pubkeys + argon2id PAT hashes in git; OIDC + real secrets forge-local.

[rawkode]

The SPEC.md you're asking about is a new, uncommitted rewrite

git diff shows your working-tree SPEC.md replaces the committed "Comtrya v3 Specification" with a new vision titled "proto — A
Git-Native, Single-Binary Forge." The existing codebase was built against the old v3 spec (GOAL.md says that cutover is
"complete"). So this isn't "how far has the build progressed" — it's "how far does today's Comtrya already satisfy a
freshly-pivoted target." That distinction explains why some pillars are absent by design rather than unfinished.

Scorecard against the new "proto" spec

┌──────────────────────────────────┬───────────────┬──────────────────────────────────────────────────────────────────────────┐
│ Pillar (proto spec) │ Status │ Reality │
├──────────────────────────────────┼───────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Single binary, local-FS only, no │ ✅ Aligned │ Rust workspace, filesystem layout, no external services │
│ central DB │ │ │
├──────────────────────────────────┼───────────────┼──────────────────────────────────────────────────────────────────────────┤
│ WASM host: Wasmtime + Component │ ✅ Aligned │ Wasmtime 43, per-request instantiation, fuel limits, 64 MiB memory cap │
│ Model + WIT │ │ │
├──────────────────────────────────┼───────────────┼──────────────────────────────────────────────────────────────────────────┤
│ OCI artifact distribution + │ ✅ Aligned │ crates/extension-oci does content-addressed fetch+verify │
│ digest cache │ │ │
├──────────────────────────────────┼───────────────┼──────────────────────────────────────────────────────────────────────────┤
│ First-party extensions │ ✅ Aligned │ All built to dist/*.wasm: issues (824 LOC), PRs (623), epics (468), │
│ (Issues→PRs→Epics) │ (+more) │ checks (265); docs/workspace_home are stubs │
├──────────────────────────────────┼───────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Config from git ("admin repo" → │ 🟡 Partial / │ config_sync.rs + reconcile.rs do clone→eval→desired-vs-actual→live-apply │
│ reconcile) │ different │ — but via CUE (comtrya.cue/schema/), not TOML .forge/config.toml, no │
│ │ shape │ ArcSwap, no tombstones (it hard-deletes) │
├──────────────────────────────────┼───────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Read-only web browse │ 🟡 Partial │ Vue shell browses tree/commits/diffs; blame, branch/tag listing, file │
│ (files/commits/diff) │ │ history all missing │
├──────────────────────────────────┼───────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Smart-HTTPS git transport, pure │ 🟡 Partial │ Pure-Rust upload-pack + gix-pack packfile gen exists, but has an opt-in │
│ Rust │ │ git-binary fallback path, and push (receive-pack) is intentionally 403'd │
├──────────────────────────────────┼───────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Per-extension isolated │ │ Storage is JSONL files scoped by extension ID (one shared file, │
│ libSQL/Turso DB │ ❌ Divergent │ in-memory filtering); kernel uses rusqlite. No libSQL, no per-ext×repo │
│ │ │ DB │
├──────────────────────────────────┼───────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Apollo Federation v2 GraphQL │ ❌ Divergent │ The whole federation subsystem is absent. Extensions dispatch via │
│ (subgraphs, hive-router, │ by design │ /api/ops WIT routes; /graphql is kernel-only hardcoded resolvers. The │
│ graphql-composition) │ │ old spec explicitly chose this. │
├──────────────────────────────────┼───────────────┼──────────────────────────────────────────────────────────────────────────┤
│ SSH transport (russh) │ ❌ Missing │ No russh anywhere; HTTPS only │
├──────────────────────────────────┼───────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Auth: OIDC web login │ ✅ Aligned │ openidconnect + SQLite sessions, JIT provisioning │
├──────────────────────────────────┼───────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Auth: SSH-key + argon2id PAT in │ ❌ Missing │ No argon2, no PAT issuance, no SSH-key matching │
│ git │ │ │
└──────────────────────────────────┴───────────────┴──────────────────────────────────────────────────────────────────────────┘

Bottom line

Roughly half-aligned, and the gaps cluster on the pivot's headline claims. What Comtrya already nails is the
operational-minimalism substrate: single binary, Wasmtime Component-Model extension host, OCI distribution, git-as-config
reconciliation, and a working Issues/PRs/Epics extension suite — those are real and substantial (~32k LOC across crates).

The four things that make the new spec the new spec are the ones not there:

1. Federated GraphQL (Apollo Fed v2 + hive-router + graphql-composition) — currently /api/ops, deliberately the opposite choice.
2. Per-extension libSQL/Turso — currently JSONL shim.
3. SSH transport + write/push — HTTPS-read-only today.
4. PAT/SSH-key auth — OIDC-only today.