diff --git a/.github/ISSUE_TEMPLATE/bug.yml b/.github/ISSUE_TEMPLATE/bug.yml new file mode 100644 index 000000000..733cd0a80 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug.yml @@ -0,0 +1,65 @@ +name: Bug report +description: Something in the Cortex control plane is wrong +labels: ["bug"] +body: + - type: markdown + attributes: + value: | + Use this for defects in **this** repo (gateway, validator, challenges, deploy). + Security issues: do **not** file here — see [SECURITY.md](../SECURITY.md). + Miner-facing public repos are separate (design-challenge / prism). + - type: textarea + id: summary + attributes: + label: Summary + description: What broke, in one or two sentences. + validations: + required: true + - type: dropdown + id: component + attributes: + label: Component + options: + - gateway + - validator + - design-challenge + - prism-challenge + - deploy / compose + - docs + - other + validations: + required: true + - type: input + id: version + attributes: + label: Version + description: Commit SHA on main, or annotated tag (v*.*.*). + validations: + required: true + - type: textarea + id: repro + attributes: + label: Reproduction + description: Commands you ran (cargo test, local-e2e, curl). No secrets, no exploit payloads. + render: bash + validations: + required: true + - type: textarea + id: expected + attributes: + label: Expected + validations: + required: true + - type: textarea + id: actual + attributes: + label: Actual + validations: + required: true + - type: checkboxes + id: naming + attributes: + label: Naming + options: + - label: I am not asking to rename BASE_* env vars, /opt/base paths, or base-*-v1 domain tags (see docs/NAMING.md). + required: true diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 000000000..1e813dbe9 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,8 @@ +blank_issues_enabled: true +contact_links: + - name: Miner submit docs + url: https://github.com/CortexLM/cortex/blob/main/docs/external-miner/README.md + about: HTTP submit guides live in docs/external-miner (and the public design-challenge / prism repos). + - name: Security report + url: https://github.com/CortexLM/cortex/security/advisories/new + about: Private vulnerability reporting — do not file a public issue. diff --git a/.github/ISSUE_TEMPLATE/feature.yml b/.github/ISSUE_TEMPLATE/feature.yml new file mode 100644 index 000000000..93bd56756 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature.yml @@ -0,0 +1,46 @@ +name: Feature request +description: Propose a change to the Cortex control plane +labels: ["enhancement"] +body: + - type: markdown + attributes: + value: | + Frozen specs (`docs/BUNDLE_SPEC.md`, `docs/DESIGN_CHALLENGE.md`) pin + incentive / scoring / consensus semantics. A feature that changes those + needs an explicit spec + xtask pin update, not a drive-by PR. + - type: textarea + id: problem + attributes: + label: Problem + description: What is hard or missing today? + validations: + required: true + - type: textarea + id: proposal + attributes: + label: Proposal + description: What should Cortex do instead? Name crates/binaries if you know them. + validations: + required: true + - type: dropdown + id: area + attributes: + label: Area + options: + - challenge (design / prism) + - gateway / weights + - validator / chain + - deploy / ops + - docs / branding + - other + validations: + required: true + - type: checkboxes + id: constraints + attributes: + label: Constraints + options: + - label: I have read AGENTS.md and docs/NAMING.md (BASE_* / domain tags stay). + required: true + - label: This is not a request to fork cryptographic domain tags or on-chain names. + required: true diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 000000000..474defb47 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,20 @@ +## Summary + + + +## Test plan + +- [ ] `cargo test --workspace` (or note the subset and why) +- [ ] `cargo fmt --all -- --check` +- [ ] Clippy / deny / xtask gates if this PR touches crates they cover + +## Risk + + + +## Naming + +I did **not** rename `BASE_*` environment variables, deployed host paths +(`/opt/base`, `/run/base`, …), GHCR `baseintelligence/base` package names, or +`base-*-v1` cryptographic domain tags, unless this PR’s purpose is a coordinated +cutover documented in `docs/NAMING.md`. diff --git a/AGENTS.md b/AGENTS.md index 680288f1d..92e4c49b2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,7 +1,9 @@ -# AGENTS.md — BASE control plane +# AGENTS.md — Cortex control plane Short contract for agents and operators. Prefer linking over restating runbooks. +**Product:** Cortex ([`CortexLM/cortex`](https://github.com/CortexLM/cortex)) — Bittensor subnet control plane for decentralized collaborative AI research via multiple challenges. Naming split (Cortex vs leftover `base` / `BASE_*`): [`docs/NAMING.md`](docs/NAMING.md). + ## Monorepo map | Path | Role | @@ -22,6 +24,9 @@ Working branch: **`main`**. Prod ships from annotated tags `v*.*.*` cut on `main - **Gateway runs on master only** (`--profile master` / `role-master.yml`). Validators point at the master gateway over VPC. - **`evil-gateway` is test-only** — never enable on prod hosts; assert with `deploy/scripts/assert-evil-gateway-not-default.sh`. - Platform is **DigitalOcean Droplets + Docker Compose**, not App Platform / DOKS. +- **Do not rename `BASE_*` env vars, deployed paths, or crypto domain tags.** They are measured into miner CVM `app-compose.json` and live on droplets / RTMR3 pin continuity. `CORTEX_*` is an accepted alias in `crates/config` only. See [`docs/NAMING.md`](docs/NAMING.md). +- Frozen specs (`docs/BUNDLE_SPEC.md`, `docs/DESIGN_CHALLENGE.md`) are pinned by xtask. Do not weaken gates or rewrite incentive / scoring / consensus semantics. +- `unsafe_code = forbid`. No `unwrap` / `expect` in non-test code. ## Wallet / key roles (do not conflate) @@ -44,7 +49,7 @@ Each live challenge has a **separate public GitHub repo** for miners. Those repo | Design | [`BaseIntelligence/design-challenge`](https://github.com/BaseIntelligence/design-challenge) | Miner docs + baseline harness | | Prism | [`BaseIntelligence/prism`](https://github.com/BaseIntelligence/prism) | Miner docs + recipe examples (publish / keep in sync; no control-plane code) | -Monorepo mirror for CI and operators: [`docs/external-miner/`](docs/external-miner/). Frozen contracts stay in this repo (`docs/DESIGN_CHALLENGE.md`, `docs/PRISM.md`, …). +Those public URLs are historical org names; this control-plane repo is `CortexLM/cortex`. Monorepo mirror for CI and operators: [`docs/external-miner/`](docs/external-miner/). Frozen contracts stay in this repo (`docs/DESIGN_CHALLENGE.md`, `docs/PRISM.md`, …). **When a challenge product or public API changes**, agents **must** update: @@ -68,6 +73,22 @@ When verifying a challenge (local-e2e, staging, or focused tests), **simulate a Local smoke automates the weights seal step via `weights-smoke` inside `./deploy/scripts/local-e2e.sh --smoke` (see [`deploy/AGENTS.md`](deploy/AGENTS.md) and [`docs/runbooks/local-testnet-e2e.md`](docs/runbooks/local-testnet-e2e.md)). +## Commands (local) + +```bash +cargo fmt --all -- --check +cargo clippy --workspace --all-targets -- -D warnings +cargo test --workspace +cargo deny check +cargo run -p xtask -- loc-cap +cargo run -p xtask -- consensus-lint +cargo run -p xtask -- spec-check +cargo run -p xtask -- design-check +cargo run -p xtask -- external-docs-check +``` + +Commit subjects: `type(scope): summary` (lowercase, ≤72 chars). Hooks: `./scripts/install-githooks.sh`. + ## Required gates (before merge) Match CI (`.github/workflows/ci.yml`): @@ -86,6 +107,7 @@ Match CI (`.github/workflows/ci.yml`): | Need | Start here | |------|------------| | System map / process topology | [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | +| Cortex vs leftover `base` names | [`docs/NAMING.md`](docs/NAMING.md) | | Deploy / Compose / DO topology | [`deploy/README.md`](deploy/README.md) + [`deploy/AGENTS.md`](deploy/AGENTS.md) | | **Local full-subnet test** (master+gateway+validator on testnet 541 + tunnel) | [`docs/runbooks/local-testnet-e2e.md`](docs/runbooks/local-testnet-e2e.md) · [`deploy/AGENTS.md`](deploy/AGENTS.md) § Local testnet E2E · `./deploy/scripts/local-e2e.sh --help` | | Doc authority vs evidence | [`docs/AGENTS.md`](docs/AGENTS.md) | diff --git a/CODEOWNERS b/CODEOWNERS index a3e9cb6b8..9884f070f 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -1,2 +1,2 @@ -# Default owners for BaseIntelligence/base +# Default owners for CortexLM/cortex * @echobt diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 000000000..532a65e2c --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,131 @@ +# Contributor Covenant Code of Conduct + +## Our Pledge + +We as members, contributors, and leaders pledge to make participation in our +community a harassment-free experience for everyone, regardless of age, body +size, visible or invisible disability, ethnicity, sex characteristics, gender +identity and expression, level of experience, education, socio-economic status, +nationality, personal appearance, race, caste, color, religion, or sexual +identity and orientation. + +We pledge to act and interact in ways that contribute to an open, welcoming, +diverse, inclusive, and healthy community. + +## Our Standards + +Examples of behavior that contributes to a positive environment for our +community include: + +* Demonstrating empathy and kindness toward other people +* Being respectful of differing opinions, viewpoints, and experiences +* Giving and gracefully accepting constructive feedback +* Accepting responsibility and apologizing to those affected by our mistakes, + and learning from the experience +* Focusing on what is best not just for us as individuals, but for the overall + community + +Examples of unacceptable behavior include: + +* The use of sexualized language or imagery, and sexual attention or advances of + any kind +* Trolling, insulting or derogatory comments, and personal or political attacks +* Public or private harassment +* Publishing others' private information, such as a physical or email address, + without their explicit permission +* Other conduct which could reasonably be considered inappropriate in a + professional setting + +## Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our standards of +acceptable behavior and will take appropriate and fair corrective action in +response to any behavior that they deem inappropriate, threatening, offensive, +or harmful. + +Community leaders have the right and responsibility to remove, edit, or reject +comments, commits, code, wiki edits, issues, and other contributions that are +not aligned to this Code of Conduct, and will communicate reasons for moderation +decisions when appropriate. + +## Scope + +This Code of Conduct applies within all community spaces, and also applies when +an individual is officially representing the community in public spaces. +Examples of representing our community include using an official e-mail address, +posting via an official social media account, or acting as an appointed +representative at an online or offline event. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be +reported by opening a GitHub issue on +[`CortexLM/cortex`](https://github.com/CortexLM/cortex) or by contacting the +maintainers listed in [CODEOWNERS](CODEOWNERS) via GitHub (mention or +maintainer tools). GitHub Discussions are not enabled on this repository. +There is no public conduct inbox. + +All complaints will be reviewed and investigated promptly and fairly. + +All community leaders are obligated to respect the privacy and security of the +reporter of any incident. + +## Enforcement Guidelines + +Community leaders will follow these Community Impact Guidelines in determining +the consequences for any action they deem in violation of this Code of Conduct: + +### 1. Correction + +**Community Impact**: Use of inappropriate language or other behavior deemed +unprofessional or unwelcome in the community. + +**Consequence**: A private, written warning from community leaders, providing +clarity around the nature of the violation and an explanation of why the +behavior was inappropriate. A public apology may be requested. + +### 2. Warning + +**Community Impact**: A violation through a single incident or series of +actions. + +**Consequence**: A warning with consequences for continued behavior. No +interaction with the people involved, including unsolicited interaction with +those enforcing the Code of Conduct, for a specified period of time. This +includes avoiding interactions in community spaces as well as external channels +like social media. Violating these terms may lead to a temporary or permanent +ban. + +### 3. Temporary Ban + +**Community Impact**: A serious violation of community standards, including +sustained inappropriate behavior. + +**Consequence**: A temporary ban from any sort of interaction or public +communication with the community for a specified period of time. No public or +private interaction with the people involved, including unsolicited interaction +with those enforcing the Code of Conduct, is allowed during this period. +Violating these terms may lead to a permanent ban. + +### 4. Permanent Ban + +**Community Impact**: Demonstrating a pattern of violation of community +standards, including sustained inappropriate behavior, harassment of an +individual, or aggression toward or disparagement of classes of individuals. + +**Consequence**: A permanent ban from any sort of public interaction within the +community. + +## Attribution + +This Code of Conduct is adapted from the +[Contributor Covenant](https://www.contributor-covenant.org), version 2.1, +available at +. + +Community Impact Guidelines were inspired by +[Mozilla's code of conduct enforcement ladder](https://github.com/mozilla/inclusion). + +For answers to common questions about this code of conduct, see the FAQ at +. Translations are available at +. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 000000000..4dced9b00 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,83 @@ +# Contributing to Cortex + +This is the Rust control-plane monorepo for the Cortex Bittensor subnet +([`CortexLM/cortex`](https://github.com/CortexLM/cortex)). + +## Before you start + +1. Read [AGENTS.md](AGENTS.md) (repo map, gates, what must not break). +2. Read [docs/NAMING.md](docs/NAMING.md) before renaming anything that looks + like `base` / `BASE_*`. +3. Frozen specs (`docs/BUNDLE_SPEC.md`, `docs/DESIGN_CHALLENGE.md`) are + pinned by xtask. Do not change incentive, scoring, or consensus semantics + unless that is the explicit task. + +## Development setup + +- Rust **1.96.0** via `rust-toolchain.toml` (`rustfmt`, `clippy`). +- Optional: `./scripts/install-githooks.sh` so `commit-msg` / `pre-commit` + match CI. + +```bash +cargo test --workspace +``` + +That is the core gate. Before opening a PR, also run what CI runs: + +```bash +cargo fmt --all -- --check +cargo clippy --workspace --all-targets -- -D warnings +cargo deny check +cargo run -p xtask -- loc-cap +cargo run -p xtask -- consensus-lint +cargo run -p xtask -- spec-check +cargo run -p xtask -- design-check +cargo run -p xtask -- external-docs-check +``` + +Local subnet stack (Docker Compose, secrets via age): see +[deploy/README.md](deploy/README.md) and +[docs/runbooks/local-testnet-e2e.md](docs/runbooks/local-testnet-e2e.md). + +```bash +./deploy/scripts/materialize-env.sh +./deploy/scripts/local-e2e.sh --smoke +``` + +## Pull requests + +- Target **`main`**. +- Use a [pull request template](.github/PULL_REQUEST_TEMPLATE.md). +- Keep diffs scoped. Branding and docs PRs must not rewrite protocol bytes. +- `unsafe_code` is forbidden. No `unwrap` / `expect` outside tests. + +## Commit messages + +A `commit-msg` hook enforces Conventional Commits: + +```text +type(scope): summary +``` + +- `type` is one of: `feat`, `fix`, `refactor`, `perf`, `test`, `docs`, + `chore`, `build`, `ci`, `style`, `revert`. +- Subject starts with a **lowercase** letter. +- Entire subject ≤ **72** characters. +- `Merge` and `Revert` subjects are allowed as-is. + +Examples: `docs(readme): describe cortex miner http path`, +`feat(config): accept CORTEX_* env aliases`. + +## Issues + +Use the GitHub issue templates (bug / feature). GitHub Discussions are +**not** enabled on this repository. + +## Security + +Do not file public issues for vulnerabilities. See [SECURITY.md](SECURITY.md). + +## Code of conduct + +[CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md). Contact owners listed in +[CODEOWNERS](CODEOWNERS) via GitHub. diff --git a/Cargo.lock b/Cargo.lock index f2447c32d..a4a034199 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1594,16 +1594,6 @@ dependencies = [ "thiserror 2.0.19", ] -[[package]] -name = "design-rating" -version = "0.1.0" -dependencies = [ - "design-challenge-task", - "hex", - "sha2 0.10.9", - "thiserror 2.0.19", -] - [[package]] name = "design-sandbox" version = "0.1.0" diff --git a/Cargo.toml b/Cargo.toml index 0698c47ee..d8646506a 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -6,7 +6,7 @@ members = ["crates/*", "bins/*", "xtask"] version = "0.1.0" edition = "2021" license = "Apache-2.0" -repository = "https://github.com/BaseIntelligence/base" +repository = "https://github.com/CortexLM/cortex" rust-version = "1.96" [workspace.lints.rust] diff --git a/LICENSE b/LICENSE index 877a3c4fb..3496f6eaa 100644 --- a/LICENSE +++ b/LICENSE @@ -175,7 +175,7 @@ END OF TERMS AND CONDITIONS - Copyright 2024 BASE Intelligence + Copyright 2024-2026 CortexLM Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. diff --git a/README.md b/README.md index 9331b81a0..91e3961dd 100644 --- a/README.md +++ b/README.md @@ -1,51 +1,73 @@
-# BASE +# Cortex -**Multi-challenge Bittensor subnet control plane (Rust).** +**Bittensor subnet control plane for decentralized collaborative AI research (Rust).** -[![CI](https://github.com/BaseIntelligence/base/actions/workflows/ci.yml/badge.svg)](https://github.com/BaseIntelligence/base/actions/workflows/ci.yml) -[![License](https://img.shields.io/github/license/BaseIntelligence/base)](https://github.com/BaseIntelligence/base/blob/main/LICENSE) +[![CI](https://github.com/CortexLM/cortex/actions/workflows/ci.yml/badge.svg)](https://github.com/CortexLM/cortex/actions/workflows/ci.yml) +[![License](https://img.shields.io/github/license/CortexLM/cortex)](https://github.com/CortexLM/cortex/blob/main/LICENSE) [![Bittensor](https://img.shields.io/badge/Bittensor-subnet-black.svg)](https://bittensor.com/) -![BASE Banner](assets/banner.jpg) +![Cortex Banner](assets/banner.jpg)
## What it is -BASE is the Bittensor subnet control plane for BaseIntelligence. This branch (`main`) -is the **Rust** workspace: gateway, validator, agent-challenge, miner -CVM templates, and shared crates. It coordinates agent challenges (native pack -executor — no Harbor product CLI), seals a final weight vector, and serves it to -validators. The gateway is the sole TLS / public edge process. +Cortex ([`CortexLM/cortex`](https://github.com/CortexLM/cortex)) is the Rust +control plane for a multi-challenge Bittensor subnet. Challenge services on +the **master** host accept miner work over HTTP, sign score leaves, and the +**gateway** (master-only) seals an epoch weight bundle. Validators **fetch** +`GET /v1/weights/latest` and submit on-chain weights. They do not execute +challenges. + +Live challenges today: + +| Challenge | How miners submit | Spec | +|-----------|-------------------|------| +| **Design** | ZIP harness (`agent.py` + `pyproject.toml`) → sandboxed pages + admin winners | [`docs/DESIGN_CHALLENGE.md`](docs/DESIGN_CHALLENGE.md) | +| **Prism** | AutoModel pin + patch → operator-owned GPU recipe eval | [`docs/PRISM.md`](docs/PRISM.md) | + +There is **no miner Phala/CVM path** on this branch (agent-v1 / Harbor pack +executors were removed). Operator-facing map: [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md). + +Some env vars, host paths, GHCR package names, and crypto domain tags still +spell `BASE_*` / `base`. That is intentional — see [`docs/NAMING.md`](docs/NAMING.md). + +## Architecture (short) + +```text +Miners --HTTP--> gateway (TLS) --proxy--> design-challenge / prism-challenge + | signed leaves + v + gateway seals EpochBundleV1 + | +Validators <--- GET /v1/weights/latest ---+ + | + +--> on-chain set_weights / CRV4 timelock +``` -- **Agent packs**: Harbor-format task workspaces from the pinned catalog - ([BaseIntelligence/deepagent](https://huggingface.co/datasets/BaseIntelligence/deepagent) / - git pin), graded by the in-tree native executor + Docker socket-proxy. -- **Miner CVM**: Phala / dstack measured compose — `socket-proxy` + `agent` + - `attest-helper` (digest-pinned images on GHCR). -- **Emission**: validators call on-chain `set_weights` from sealed - `GET /v1/weights/latest` — never the gateway. +- Gateway is the sole public edge and **only** runs on the subnet-owner host + (`docker compose --profile master`). +- Trust roots (`config/challenges.toml`, `config/measurements.toml`) are + owner-signed **local files**, never fetched from the gateway. +- Unsealed / decode-error latest weights are a **burn vector** (uid 0 = 100%, + `sealed: false`), not a 404. -## Branch +## Miners -| Branch | Role | -|--------|------| -| **`main`** | Active Rust control plane (this tree). PRs target `main`. | -| `dev` | Kept alive for in-flight work; merges back into `main`. | -| `legacy/main-pre-rust` (tag) | Legacy / prior stack — do not mix secret material across histories. | +HTTP submit only. Start at [docs/external-miner/](docs/external-miner/). -## Miners +```text +https:///challenge/design/... +https:///challenge/prism/... +``` -Day-1: [docs/external-miner/](docs/external-miner/) +Public miner docs (examples only — no control-plane code): +[design-challenge](https://github.com/BaseIntelligence/design-challenge), +[prism](https://github.com/BaseIntelligence/prism). -1. Deploy a measured CVM (`miner deploy`) with digest-pinned `base-agent` + - `base-attest-helper` + socket-proxy. -2. Fund your own Phala account; hotkey + launch token + receipt sk are **files** - under `/run/base/` (never env secret values). -3. Certify each epoch (`miner certify`) via loopback attest-helper - `GET /v1/quote` → validator attest API. +Never put mnemonics or challenge signing keys in miner clients. ## Validators @@ -55,7 +77,7 @@ Weight-only path after seal: curl -fsS "$GATEWAY/v1/weights/latest" ``` -Then `set_weights` with your wallet. Operator compose: +Then `set_weights` / CRV4 with the validator wallet. Operator compose: ```bash ./deploy/scripts/materialize-env.sh @@ -63,39 +85,59 @@ docker compose up -d # postgres, validator, updater, socket-pro docker compose --profile master up -d # + gateway (subnet owner host only) ``` -Details: [deploy/README.md](deploy/README.md), [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md). +Local full-stack smoke (testnet 541 + optional tunnel): + +```bash +./deploy/scripts/local-e2e.sh --help +./deploy/scripts/local-e2e.sh --smoke +``` + +Details: [deploy/README.md](deploy/README.md), [docs/runbooks/local-testnet-e2e.md](docs/runbooks/local-testnet-e2e.md). ## Images (GHCR) -CI workflow [`.github/workflows/images.yml`](.github/workflows/images.yml) builds -and pushes digest-pinned service images to `ghcr.io/baseintelligence/base/*`. -Never `:latest` in measured compose. +CI [`.github/workflows/images.yml`](.github/workflows/images.yml) builds +digest-pinned images. The registry path is still +`ghcr.io/baseintelligence/base/` (historical package name; see +[docs/NAMING.md](docs/NAMING.md)). Never `:latest` in measured compose. | Target | Image suffix | |--------|----------------| | validator | `validator` | | gateway | `gateway` | | updater | `updater` | -| agent-challenge | `agent-challenge` | -| base-agent | `base-agent` (miner runner) | -| base-attest-helper | `base-attest-helper` (quote helper) | +| prism-challenge | `prism-challenge` | +| design-challenge | `design-challenge` | +| design-egress-proxy | `design-egress-proxy` | -## Toolchain +## Toolchain and gates - Rust **1.96.0** (`rust-toolchain.toml`) - Workspace: `crates/*`, `bins/*`, `xtask` -- Gates: `fmt`, `clippy -D warnings`, `test`, `cargo deny`, `xtask loc-cap`, - `xtask consensus-lint`, `xtask spec-check`, `xtask agent-challenge-check` +- Core gate: `cargo test --workspace` +- CI also runs `fmt`, `clippy -D warnings`, `cargo deny`, and + +```bash +cargo run -p xtask -- loc-cap +cargo run -p xtask -- consensus-lint +cargo run -p xtask -- spec-check +cargo run -p xtask -- design-check +cargo run -p xtask -- external-docs-check +``` ## Docs | Doc | Content | |-----|---------| +| [AGENTS.md](AGENTS.md) | Agent / operator contract | +| [docs/NAMING.md](docs/NAMING.md) | Cortex vs leftover `base` identifiers | +| [CONTRIBUTING.md](CONTRIBUTING.md) | How to change this repo | | [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | System map | -| [docs/AGENT_CHALLENGE.md](docs/AGENT_CHALLENGE.md) | Pack grade + miner CVM contract | -| [docs/BUNDLE_SPEC.md](docs/BUNDLE_SPEC.md) | Sealed weight bundle | +| [docs/BUNDLE_SPEC.md](docs/BUNDLE_SPEC.md) | Sealed weight bundle (frozen) | +| [docs/DESIGN_CHALLENGE.md](docs/DESIGN_CHALLENGE.md) | Design challenge (frozen) | +| [docs/PRISM.md](docs/PRISM.md) | Prism challenge | | [docs/THREAT_MODEL.md](docs/THREAT_MODEL.md) | Security claims | -| [docs/runbooks/](docs/runbooks/) | Ops cutovers (incl. measurement re-pin) | +| [docs/runbooks/](docs/runbooks/) | Ops procedures | ## License diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 000000000..cd7928905 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,41 @@ +# Security policy + +## Reporting a vulnerability + +Use **GitHub private vulnerability reporting** on this repository: + + + +Do **not** open a public issue for a vulnerability. + +There is no public security email. If private reporting is unavailable, +contact the maintainers listed in [CODEOWNERS](CODEOWNERS) through GitHub +maintainer tools (do not paste secrets into a public mention). + +Please include: + +- Affected component (`gateway`, `validator`, a challenge binary, a crate) +- Cortex commit SHA or release tag (`v*.*.*`) +- Impact and a minimal reproduction **without** exploit payload dumps +- Whether the issue is already public + +## Scope + +In scope: this control-plane repo — gateway, validator, challenge services, +deploy compose/scripts as documented, and the sealed-weight path. + +Out of scope for this document: miner-submitted harnesses (untrusted by +design), third-party GPU clouds, and public miner-doc repos that contain +examples only. + +## What we will not change for a “security rename” + +Do not treat leftover `BASE_*` environment names, `/opt/base` host paths, or +`base-*-v1` domain tags as vulnerabilities. Those strings are measured into +live miner CVMs and on-chain/signature domains. See +[docs/NAMING.md](docs/NAMING.md) and [docs/THREAT_MODEL.md](docs/THREAT_MODEL.md). + +## Supported versions + +Fixes land on `main` and ship in annotated tags `v*.*.*`. Staging tracks +`main`; production is digest-pinned from those tags. diff --git a/SUPPORT.md b/SUPPORT.md new file mode 100644 index 000000000..f5b8b2844 --- /dev/null +++ b/SUPPORT.md @@ -0,0 +1,27 @@ +# Support + +## How to get help + +| Kind | Where | +|------|--------| +| Bug in this control plane | [GitHub issue — bug](https://github.com/CortexLM/cortex/issues/new?template=bug.yml) | +| Feature or docs change | [GitHub issue — feature](https://github.com/CortexLM/cortex/issues/new?template=feature.yml) | +| Miner submit / scoring questions | [docs/external-miner/](docs/external-miner/) and the public miner repos linked there | +| Operator deploy | [deploy/README.md](deploy/README.md), [docs/runbooks/](docs/runbooks/) | +| Security | [SECURITY.md](SECURITY.md) — private vulnerability reporting only | + +GitHub **Discussions are disabled** on this repository. Use issues. + +There is no public support email and no Slack/Discord operated from this +repo. Mentions of maintainers go through GitHub and +[CODEOWNERS](CODEOWNERS). + +## What this repo is (and is not) + +- **Is:** Cortex Rust workspace — gateway, validator, design/prism + challenges, deploy matrix. +- **Is not:** a miner SDK rewrite, a protocol fork, or the public + design-challenge / prism documentation repos. + +If you are about to rename `BASE_*` or a `base-*-v1` domain tag, stop and +read [docs/NAMING.md](docs/NAMING.md). diff --git a/crates/config/Cargo.toml b/crates/config/Cargo.toml index d7ea632d8..205c1e495 100644 --- a/crates/config/Cargo.toml +++ b/crates/config/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "config" -description = "Typed layered configuration: defaults → base.toml → BASE_*" +description = "Typed layered configuration: defaults → base.toml → BASE_* (CORTEX_* aliases)" version.workspace = true edition.workspace = true license.workspace = true diff --git a/crates/config/src/lib.rs b/crates/config/src/lib.rs index 824a7e639..5894e2682 100644 --- a/crates/config/src/lib.rs +++ b/crates/config/src/lib.rs @@ -1,10 +1,11 @@ -//! Typed layered configuration for base binaries. +//! Typed layered configuration for Cortex binaries. //! //! # Layers (later wins) //! //! 1. **Defaults** — chain endpoint, epoch length, D6/D21/D26 knobs. -//! 2. **`base.toml`** (or path from `BASE_CONFIG`) — optional file. -//! 3. **`BASE_*` environment variables** — highest precedence. +//! 2. **`base.toml`** (or path from `BASE_CONFIG` / `CORTEX_CONFIG`) — optional file. +//! 3. **`BASE_*` environment variables** — highest precedence. Matching +//! `CORTEX_*` names are aliases; `BASE_*` wins when both are set. //! //! # Validation //! @@ -20,8 +21,8 @@ mod role; pub use error::{Error, Issue, ValidationReport}; pub use load::{ - base_env_from_process, database_url_from, keys, load, load_from, load_from_toml_str, - resolve_toml_path, DEFAULT_CHAIN_ENDPOINT, DEFAULT_EPOCH_LENGTH, + base_env_from_process, cortex_alias_key, database_url_from, keys, load, load_from, + load_from_toml_str, resolve_toml_path, DEFAULT_CHAIN_ENDPOINT, DEFAULT_EPOCH_LENGTH, DEFAULT_MAX_COLLATERAL_AGE_SECS, DEFAULT_MIN_PEER_SAMPLE, DEFAULT_MIN_SHARE_MASS_BPS, DEFAULT_ROTATION_EPOCHS, MAX_SHARE_MASS_BPS, }; @@ -382,4 +383,53 @@ domain = "ignored-for-updater.example" None ); } + + #[test] + fn cortex_alias_loads_when_base_unset() { + let env = BTreeMap::from([ + ("CORTEX_ROLE".to_owned(), "validator".to_owned()), + ("CORTEX_NETUID".to_owned(), "7".to_owned()), + ( + "CORTEX_DATABASE_URL".to_owned(), + "postgres://cortex/db".to_owned(), + ), + ( + "CORTEX_CHAIN_ENDPOINT".to_owned(), + "wss://alias.example/ws".to_owned(), + ), + ]); + let cfg = load_from_toml_str(None, &env).expect("cortex alias"); + assert_eq!(cfg.role, Role::Validator); + assert_eq!(cfg.netuid.get(), 7); + assert_eq!(cfg.database_url.as_deref(), Some("postgres://cortex/db")); + assert_eq!(cfg.chain_endpoint, "wss://alias.example/ws"); + } + + #[test] + fn base_env_wins_over_cortex_alias() { + let env = BTreeMap::from([ + (keys::ROLE.to_owned(), "miner".to_owned()), + ("CORTEX_ROLE".to_owned(), "validator".to_owned()), + (keys::NETUID.to_owned(), "3".to_owned()), + ("CORTEX_NETUID".to_owned(), "99".to_owned()), + ]); + let cfg = load_from_toml_str(None, &env).expect("base wins"); + assert_eq!(cfg.role, Role::Miner); + assert_eq!(cfg.netuid.get(), 3); + } + + #[test] + fn cortex_config_path_alias_resolves() { + let env = BTreeMap::from([("CORTEX_CONFIG".to_owned(), "/tmp/cortex.toml".to_owned())]); + assert_eq!( + resolve_toml_path(&env), + Some(std::path::PathBuf::from("/tmp/cortex.toml")) + ); + } + + #[test] + fn cortex_alias_key_maps_base_prefix() { + assert_eq!(cortex_alias_key(keys::ROLE).as_deref(), Some("CORTEX_ROLE")); + assert_eq!(cortex_alias_key("OTHER"), None); + } } diff --git a/crates/config/src/load.rs b/crates/config/src/load.rs index 036e13689..357dfaf7d 100644 --- a/crates/config/src/load.rs +++ b/crates/config/src/load.rs @@ -1,4 +1,5 @@ -//! Layered configuration loading: defaults → TOML → `BASE_*` env. +//! Layered configuration loading: defaults → TOML → `BASE_*` env +//! (`CORTEX_*` accepted as aliases; `BASE_*` wins when both are set). use std::collections::BTreeMap; use std::fs; @@ -34,6 +35,11 @@ pub const DEFAULT_MAX_COLLATERAL_AGE_SECS: u64 = 7 * 24 * 60 * 60; pub const MAX_SHARE_MASS_BPS: u16 = 10_000; /// Env / file key names (stable contract for operators). +/// +/// Canonical wire spelling is `BASE_*`. Matching `CORTEX_*` names are +/// accepted as aliases (`CORTEX_ROLE` ≡ `BASE_ROLE`, …). When both are set, +/// `BASE_*` wins. Do not rename `BASE_*` — those names are measured into +/// miner CVM `app-compose.json` and live droplet env. pub mod keys { /// Process role. pub const ROLE: &str = "BASE_ROLE"; @@ -66,6 +72,22 @@ pub mod keys { pub const CONFIG_PATH: &str = "BASE_CONFIG"; } +/// `CORTEX_*` alias for a canonical `BASE_*` key, if the name is in that family. +#[must_use] +pub fn cortex_alias_key(base_key: &str) -> Option { + base_key + .strip_prefix("BASE_") + .map(|rest| format!("CORTEX_{rest}")) +} + +/// Read `base_key`, then its `CORTEX_*` alias. `BASE_*` wins when both exist. +fn env_lookup<'a>(env: &'a BTreeMap, base_key: &str) -> Option<&'a String> { + if let Some(v) = env.get(base_key) { + return Some(v); + } + cortex_alias_key(base_key).and_then(|alias| env.get(&alias)) +} + /// Optional overrides from a single layer (TOML file or env). #[derive(Debug, Clone, Default, Deserialize, PartialEq, Eq)] #[serde(deny_unknown_fields, default)] @@ -183,33 +205,27 @@ impl Builder { fn apply_env_map(&mut self, env: &BTreeMap) { let layer = Layer { - role: env.get(keys::ROLE).cloned(), - netuid: env - .get(keys::NETUID) + role: env_lookup(env, keys::ROLE).cloned(), + netuid: env_lookup(env, keys::NETUID) .and_then(|s| parse_u64_field("netuid", s, &mut self.early_issues)), - chain_endpoint: env - .get(keys::CHAIN_ENDPOINTS) - .or_else(|| env.get(keys::CHAIN_ENDPOINT)) + chain_endpoint: env_lookup(env, keys::CHAIN_ENDPOINTS) + .or_else(|| env_lookup(env, keys::CHAIN_ENDPOINT)) .cloned(), - gateway_endpoint: env.get(keys::GATEWAY_ENDPOINT).cloned(), - database_url: env.get(keys::DATABASE_URL).cloned(), - database_url_file: env.get(keys::DATABASE_URL_FILE).cloned(), - epoch_length: env - .get(keys::EPOCH_LENGTH) + gateway_endpoint: env_lookup(env, keys::GATEWAY_ENDPOINT).cloned(), + database_url: env_lookup(env, keys::DATABASE_URL).cloned(), + database_url_file: env_lookup(env, keys::DATABASE_URL_FILE).cloned(), + epoch_length: env_lookup(env, keys::EPOCH_LENGTH) .and_then(|s| parse_u64_field("epoch_length", s, &mut self.early_issues)), - min_share_mass_bps: env - .get(keys::MIN_SHARE_MASS_BPS) + min_share_mass_bps: env_lookup(env, keys::MIN_SHARE_MASS_BPS) .and_then(|s| parse_u64_field("min_share_mass_bps", s, &mut self.early_issues)), - rotation_epochs: env - .get(keys::ROTATION_EPOCHS) + rotation_epochs: env_lookup(env, keys::ROTATION_EPOCHS) .and_then(|s| parse_u64_field("rotation_epochs", s, &mut self.early_issues)), - min_peer_sample: env - .get(keys::MIN_PEER_SAMPLE) + min_peer_sample: env_lookup(env, keys::MIN_PEER_SAMPLE) .and_then(|s| parse_u64_field("min_peer_sample", s, &mut self.early_issues)), - max_collateral_age_secs: env.get(keys::MAX_COLLATERAL_AGE_SECS).and_then(|s| { + max_collateral_age_secs: env_lookup(env, keys::MAX_COLLATERAL_AGE_SECS).and_then(|s| { parse_u64_field("max_collateral_age_secs", s, &mut self.early_issues) }), - domain: env.get(keys::DOMAIN).cloned(), + domain: env_lookup(env, keys::DOMAIN).cloned(), }; self.apply_layer(layer); } @@ -468,17 +484,29 @@ pub fn load_from_toml_str( } /// Collect `BASE_*` variables from the process environment. +/// +/// `CORTEX_*` names are folded into the corresponding `BASE_*` keys when the +/// canonical name is unset, so callers that only read `BASE_*` keep working. #[must_use] pub fn base_env_from_process() -> BTreeMap { - std::env::vars() - .filter(|(k, _)| k.starts_with("BASE_")) - .collect() + let mut out = BTreeMap::new(); + for (k, v) in std::env::vars() { + if let Some(rest) = k.strip_prefix("CORTEX_") { + out.entry(format!("BASE_{rest}")).or_insert(v); + } + } + for (k, v) in std::env::vars() { + if k.starts_with("BASE_") { + out.insert(k, v); + } + } + out } -/// Resolve the TOML path: `BASE_CONFIG` env, else `./base.toml` if it exists. +/// Resolve the TOML path: `BASE_CONFIG` / `CORTEX_CONFIG`, else `./base.toml`. #[must_use] pub fn resolve_toml_path(env: &BTreeMap) -> Option { - if let Some(p) = env.get(keys::CONFIG_PATH) { + if let Some(p) = env_lookup(env, keys::CONFIG_PATH) { return Some(PathBuf::from(p)); } let default = PathBuf::from("base.toml"); diff --git a/crates/db/Cargo.toml b/crates/db/Cargo.toml index a1dd1abc6..579bbae4b 100644 --- a/crates/db/Cargo.toml +++ b/crates/db/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "db" -description = "Postgres schema, sqlx migrations, and pool helpers for base" +description = "Postgres schema, sqlx migrations, and pool helpers for Cortex" version.workspace = true edition.workspace = true license.workspace = true diff --git a/crates/design-rating/Cargo.toml b/crates/design-rating/Cargo.toml deleted file mode 100644 index b71325e40..000000000 --- a/crates/design-rating/Cargo.toml +++ /dev/null @@ -1,18 +0,0 @@ -[package] -name = "design-rating" -description = "Integer Elo, pairing, bottom-20% elimination for design challenge" -version.workspace = true -edition.workspace = true -license.workspace = true -repository.workspace = true -rust-version.workspace = true -publish = false - -[dependencies] -design-challenge-task = { path = "../design-challenge-task" } -hex = "0.4" -sha2 = "0.10" -thiserror = "2" - -[lints] -workspace = true diff --git a/crates/design-rating/src/lib.rs b/crates/design-rating/src/lib.rs deleted file mode 100644 index a38523fa8..000000000 --- a/crates/design-rating/src/lib.rs +++ /dev/null @@ -1,361 +0,0 @@ -//! Integer Elo, deterministic pairing, elimination. - -#![forbid(unsafe_code)] -#![allow(clippy::missing_errors_doc)] -#![allow(clippy::doc_markdown)] -#![allow(clippy::unreadable_literal)] -#![allow(clippy::cast_possible_truncation)] - -use std::collections::{BTreeMap, BTreeSet}; - -use design_challenge_task::{ - ELIMINATION_BOTTOM_BPS, ELIMINATION_COOLDOWN_ROUNDS, PAIR_ID_DOMAIN, SCORE_MAX, -}; -use sha2::{Digest, Sha256}; -use thiserror::Error; - -/// Starting Elo. -pub const ELO_INITIAL: i32 = 1000; -/// Fixed K-factor (integer). -pub const ELO_K: i32 = 32; - -/// One pairwise outcome (winner beat loser). -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct AnnotationOutcome { - /// Winner run id. - pub winner_run_id: String, - /// Loser run id. - pub loser_run_id: String, - /// Created ordering key. - pub created_at_ms: u64, - /// Stable id for tie-break. - pub id: String, -} - -/// Per-miner rating aggregate. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct MinerRating { - /// Hotkey. - pub miner_hotkey: String, - /// Elo. - pub rating: i32, - /// Wins. - pub wins: u32, - /// Losses. - pub losses: u32, -} - -/// Pairing candidate run. -#[derive(Debug, Clone)] -pub struct PairCandidate { - /// Run id. - pub run_id: String, - /// Miner hotkey. - pub miner_hotkey: String, - /// Prompt id. - pub prompt_id: String, -} - -/// Generated pair. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct PairSpec { - /// Pair id. - pub id: String, - /// Prompt. - pub prompt_id: String, - /// Run A. - pub run_a_id: String, - /// Run B. - pub run_b_id: String, -} - -/// Rating errors. -#[derive(Debug, Error, PartialEq, Eq)] -pub enum RatingError { - /// Bad input. - #[error("rating: {0}")] - Invalid(String), -} - -/// Integer expected score × 1000 for Elo (logistic approximation table-free). -/// -/// `expected_a = 1000 / (1 + 10^((rb-ra)/400))` via integer pow approx. -#[must_use] -pub fn expected_score_millis(ra: i32, rb: i32) -> i32 { - // Use: 1000 / (1 + 10^((rb-ra)/400)) ≈ via scaled integer. - let diff = rb.saturating_sub(ra); - // Approximate 10^(d/400) with 2^(d*10/400*log2(10)) ≈ 2^(d/120) roughly (log2(10)≈3.32 → 400/3.32≈120). - // Clamp for stability. - let clamped = diff.clamp(-1200, 1200); - let exp_approx = if clamped >= 0 { - // 10^(x/400) ≈ (1000 + 4*x)/1000 for modest x; better: use millidiv table. - millipow10(clamped) - } else { - let inv = millipow10(-clamped); - // 1 / inv_pow → millipow returns millis of 10^; so 10^(-x) = 1000^2 / millipow(x) / 1000 - if inv == 0 { - 1000 - } else { - 1_000_000 / inv - } - }; - // expected = 1000 / (1 + 10^diff) = 1000 * 1000 / (1000 + exp_approx) - let denom = 1000i64.saturating_add(i64::from(exp_approx)); - if denom <= 0 { - return 500; - } - i32::try_from(1_000_000i64 / denom).unwrap_or(500) -} - -/// `round(1000 * 10^(x/400))` for x in 0..=1200. -fn millipow10(x: i32) -> i32 { - // Piecewise linear on decades of 100 Elo points. - // 10^(0)=1, 10^(0.25)≈1.778, 10^(0.5)≈3.162, 10^(0.75)≈5.623, 10^1=10, ... - const TABLE: &[(i32, i32)] = &[ - (0, 1000), - (100, 1778), - (200, 3162), - (300, 5623), - (400, 10000), - (500, 17783), - (600, 31623), - (700, 56234), - (800, 100000), - (900, 177828), - (1000, 316228), - (1100, 562341), - (1200, 1000000), - ]; - let x = x.clamp(0, 1200); - for w in TABLE.windows(2) { - let (x0, y0) = w[0]; - let (x1, y1) = w[1]; - if x <= x1 { - let t = i64::from(x - x0); - let span = i64::from(x1 - x0).max(1); - let y = i64::from(y0) + (i64::from(y1 - y0) * t) / span; - return i32::try_from(y).unwrap_or(y0); - } - } - 1_000_000 -} - -/// Apply one game: winner gains, loser loses (integer Elo). -#[must_use] -pub fn apply_game(winner: i32, loser: i32) -> (i32, i32) { - let exp_w = expected_score_millis(winner, loser); - // score_w = 1000, delta = K * (score - expected) / 1000 - let delta_w = (ELO_K * (1000 - exp_w)) / 1000; - let delta_l = (ELO_K * (0 - (1000 - exp_w))) / 1000; - ( - winner.saturating_add(delta_w), - loser.saturating_add(delta_l), - ) -} - -/// Aggregate Elo over a deterministic annotation sequence. -/// -/// `run_to_miner` maps run_id → miner_hotkey. Annotations must be pre-sorted -/// by `(created_at_ms, id)`. -pub fn aggregate_elo( - run_to_miner: &BTreeMap, - outcomes: &[AnnotationOutcome], -) -> Result, RatingError> { - let mut ratings: BTreeMap = BTreeMap::new(); - let ensure = |map: &mut BTreeMap, hk: &str| { - map.entry(hk.to_owned()).or_insert_with(|| MinerRating { - miner_hotkey: hk.to_owned(), - rating: ELO_INITIAL, - wins: 0, - losses: 0, - }); - }; - for o in outcomes { - let w_hk = run_to_miner - .get(&o.winner_run_id) - .ok_or_else(|| RatingError::Invalid(format!("unknown winner {}", o.winner_run_id)))?; - let l_hk = run_to_miner - .get(&o.loser_run_id) - .ok_or_else(|| RatingError::Invalid(format!("unknown loser {}", o.loser_run_id)))?; - if w_hk == l_hk { - continue; - } - ensure(&mut ratings, w_hk); - ensure(&mut ratings, l_hk); - let rw = ratings[w_hk].rating; - let rl = ratings[l_hk].rating; - let (nw, nl) = apply_game(rw, rl); - if let Some(r) = ratings.get_mut(w_hk) { - r.rating = nw; - r.wins = r.wins.saturating_add(1); - } - if let Some(r) = ratings.get_mut(l_hk) { - r.rating = nl; - r.losses = r.losses.saturating_add(1); - } - } - Ok(ratings.into_values().collect()) -} - -/// Normalize ratings onto `[0, SCORE_MAX]` lattice (integer). -#[must_use] -pub fn ratings_to_scores(ratings: &[MinerRating]) -> BTreeMap { - if ratings.is_empty() { - return BTreeMap::new(); - } - let min_r = ratings.iter().map(|r| r.rating).min().unwrap_or(0); - let max_r = ratings.iter().map(|r| r.rating).max().unwrap_or(0); - let span = (max_r - min_r).max(1); - ratings - .iter() - .map(|r| { - let num = i64::from(r.rating - min_r) * i64::try_from(SCORE_MAX).unwrap_or(1); - let v = u64::try_from(num / i64::from(span)).unwrap_or(0); - (r.miner_hotkey.clone(), v.min(SCORE_MAX)) - }) - .collect() -} - -/// Build round-robin pairs per prompt (order by run_id for determinism). -#[must_use] -pub fn build_pairs(round_id: u64, candidates: &[PairCandidate]) -> Vec { - let mut by_prompt: BTreeMap> = BTreeMap::new(); - for c in candidates { - by_prompt.entry(c.prompt_id.clone()).or_default().push(c); - } - let mut out = Vec::new(); - for (prompt_id, mut group) in by_prompt { - group.sort_by_key(|c| &c.run_id); - for i in 0..group.len() { - for j in (i + 1)..group.len() { - let a = group[i]; - let b = group[j]; - if a.miner_hotkey == b.miner_hotkey { - continue; - } - let id = pair_id(round_id, &prompt_id, &a.run_id, &b.run_id); - out.push(PairSpec { - id, - prompt_id: prompt_id.clone(), - run_a_id: a.run_id.clone(), - run_b_id: b.run_id.clone(), - }); - } - } - } - out.sort_by(|x, y| x.id.cmp(&y.id)); - out -} - -/// Pair id digest. -#[must_use] -pub fn pair_id(round_id: u64, prompt_id: &str, run_a: &str, run_b: &str) -> String { - let (a, b) = if run_a <= run_b { - (run_a, run_b) - } else { - (run_b, run_a) - }; - let mut h = Sha256::new(); - h.update(PAIR_ID_DOMAIN); - h.update(round_id.to_be_bytes()); - h.update(prompt_id.as_bytes()); - h.update(a.as_bytes()); - h.update(b.as_bytes()); - hex::encode(h.finalize()) -} - -/// Bottom 20% (at least 1) miners to eliminate; returns hotkeys + until_round. -#[must_use] -pub fn elimination_set(ratings: &[MinerRating], current_round: u64) -> BTreeSet<(String, u64)> { - if ratings.is_empty() { - return BTreeSet::new(); - } - let mut sorted = ratings.to_vec(); - sorted.sort_by(|a, b| { - a.rating - .cmp(&b.rating) - .then(a.miner_hotkey.cmp(&b.miner_hotkey)) - }); - let n = sorted.len(); - let mut k = (n as u64 * u64::from(ELIMINATION_BOTTOM_BPS) / 10_000) as usize; - k = k.max(1).min(n); - let until = current_round.saturating_add(ELIMINATION_COOLDOWN_ROUNDS); - sorted - .into_iter() - .take(k) - .map(|r| (r.miner_hotkey, until)) - .collect() -} - -#[cfg(test)] -mod tests { - #![allow(clippy::unwrap_used)] - use super::*; - - #[test] - fn elo_deterministic() { - let mut map = BTreeMap::new(); - map.insert("r1".into(), "m1".into()); - map.insert("r2".into(), "m2".into()); - let outcomes = vec![ - AnnotationOutcome { - winner_run_id: "r1".into(), - loser_run_id: "r2".into(), - created_at_ms: 1, - id: "a".into(), - }, - AnnotationOutcome { - winner_run_id: "r1".into(), - loser_run_id: "r2".into(), - created_at_ms: 2, - id: "b".into(), - }, - ]; - let a = aggregate_elo(&map, &outcomes).unwrap(); - let b = aggregate_elo(&map, &outcomes).unwrap(); - assert_eq!(a, b); - let m1 = a.iter().find(|r| r.miner_hotkey == "m1").unwrap(); - let m2 = a.iter().find(|r| r.miner_hotkey == "m2").unwrap(); - assert!(m1.rating > m2.rating); - } - - #[test] - fn elimination_bottom_20() { - let ratings: Vec<_> = (0..10) - .map(|i| MinerRating { - miner_hotkey: format!("m{i:02}"), - rating: 1000 + i * 10, - wins: 0, - losses: 0, - }) - .collect(); - let elim = elimination_set(&ratings, 5); - assert_eq!(elim.len(), 2); // 20% of 10 - let until = 5 + ELIMINATION_COOLDOWN_ROUNDS; - assert!(elim.contains(&("m00".into(), until))); - assert!(elim.contains(&("m01".into(), until))); - } - - #[test] - fn pairing_skips_same_miner() { - let c = vec![ - PairCandidate { - run_id: "a".into(), - miner_hotkey: "m1".into(), - prompt_id: "p".into(), - }, - PairCandidate { - run_id: "b".into(), - miner_hotkey: "m1".into(), - prompt_id: "p".into(), - }, - PairCandidate { - run_id: "c".into(), - miner_hotkey: "m2".into(), - prompt_id: "p".into(), - }, - ]; - let pairs = build_pairs(1, &c); - assert_eq!(pairs.len(), 2); // a-c, b-c - } -} diff --git a/crates/prism-registry/src/hf.rs b/crates/prism-registry/src/hf.rs index 8494b6946..e1c10e24f 100644 --- a/crates/prism-registry/src/hf.rs +++ b/crates/prism-registry/src/hf.rs @@ -30,8 +30,8 @@ const DEFAULT_REPO: &str = "BaseIntelligence/top-prism-architecture"; const DEFAULT_REVISION: &str = "main"; /// Hub regular (non-LFS) file ceiling used by the ndjson commit API. const REGULAR_FILE_MAX: usize = 5 * 1024 * 1024; -/// Org banner mirrored from the Base monorepo (model card + org profile). -const BANNER_URL: &str = "https://github.com/BaseIntelligence/base/raw/main/assets/banner.jpg"; +/// Org banner mirrored from the Cortex monorepo (model card + org profile). +const BANNER_URL: &str = "https://github.com/CortexLM/cortex/raw/main/assets/banner.jpg"; /// GPT-2 Large (774M) Prism-protocol public-pack reference (eval-only). const GPT2_LARGE_LABEL: &str = "GPT-2 Large (774M)"; diff --git a/crates/site-types/src/frames.rs b/crates/site-types/src/frames.rs index bd98efbc9..c1da6f3f6 100644 --- a/crates/site-types/src/frames.rs +++ b/crates/site-types/src/frames.rs @@ -24,7 +24,7 @@ pub fn coding_arena() -> Arena { weight: 0.0, rewards_per_day: 0.0, references: Vec::new(), - source_url: "https://github.com/BaseIntelligence/base".into(), + source_url: "https://github.com/CortexLM/cortex".into(), plate: "/plates/coding.svg".into(), round_id: None, round_ends_at: None, @@ -55,10 +55,10 @@ pub fn design_frame() -> Arena { rewards_per_day: 0.0, references: vec![ProjectReference { name: "Design challenge".into(), - repo: "BaseIntelligence/base".into(), - repo_url: "https://github.com/BaseIntelligence/base".into(), + repo: "CortexLM/cortex".into(), + repo_url: "https://github.com/CortexLM/cortex".into(), }], - source_url: "https://github.com/BaseIntelligence/base".into(), + source_url: "https://github.com/CortexLM/cortex".into(), plate: "/plates/design.svg".into(), round_id: None, round_ends_at: None, @@ -89,10 +89,10 @@ pub fn prism_frame() -> Arena { rewards_per_day: 0.0, references: vec![ProjectReference { name: "PRISM recipe".into(), - repo: "BaseIntelligence/base".into(), - repo_url: "https://github.com/BaseIntelligence/base".into(), + repo: "CortexLM/cortex".into(), + repo_url: "https://github.com/CortexLM/cortex".into(), }], - source_url: "https://github.com/BaseIntelligence/base".into(), + source_url: "https://github.com/CortexLM/cortex".into(), plate: "/plates/prism.svg".into(), round_id: None, round_ends_at: None, diff --git a/deploy/README.md b/deploy/README.md index 207dca8ab..0ac2e4247 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -1,4 +1,4 @@ -# base deploy (compose) +# Cortex deploy (compose) ## Services diff --git a/docs/AGENTS.md b/docs/AGENTS.md index 1d3aa8afe..0a0424a5e 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -6,7 +6,7 @@ How to treat documentation in this repo. | Kind | Paths | Treat as | |------|-------|----------| -| **Normative** | `ARCHITECTURE.md`, frozen specs (`BUNDLE_SPEC.md`, `DESIGN_CHALLENGE.md`, `PRISM.md`, …), `THREAT_MODEL.md`, `OPERATOR_SECURITY.md`, `COMPLETENESS.md`, `runbooks/`, `external-miner/` | Source of truth for contracts, ops, and status | +| **Normative** | `ARCHITECTURE.md`, `NAMING.md`, frozen specs (`BUNDLE_SPEC.md`, `DESIGN_CHALLENGE.md`, `PRISM.md`, …), `THREAT_MODEL.md`, `OPERATOR_SECURITY.md`, `COMPLETENESS.md`, `runbooks/`, `external-miner/` | Source of truth for contracts, ops, naming, and status | | **Non-normative** | `evidence/`, `spikes/` | Historical ops notes / experiments. **Do not** implement against them as spec; **do not** delete in cleanup passes without an explicit ops decision | When a spike or evidence report conflicts with a frozen spec or runbook, the normative doc wins. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index f500bb751..7b5181ef3 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -1,4 +1,4 @@ -# base architecture +# Cortex architecture Operator-facing map of the control plane. Normative byte contracts live in the frozen specs: @@ -19,7 +19,7 @@ Miner-facing docs (version-pinned): [`external-miner/`](./external-miner/). ## 1. Goals -- Lighter Rust control plane than the Python BASE stack. +- Lighter Rust control plane than the prior Python stack. - Gateway runs **only** as subnet owner (master). Startup asserts hotkey == on-chain `SubnetOwnerHotkey` or exits `2` before bind. - Validators **recompute** the weight vector from a signed, merkle-rooted epoch bundle. Challenge keys and measurements come from **owner-signed local files**, never from gateway HTTP. - CRV4 timelock commit-reveal on Bittensor testnet/mainnet as configured. Reveal is automatic on-chain. diff --git a/docs/BUNDLE_SPEC.md b/docs/BUNDLE_SPEC.md index 55df604b9..17a2ec33c 100644 --- a/docs/BUNDLE_SPEC.md +++ b/docs/BUNDLE_SPEC.md @@ -1,4 +1,4 @@ -# base Epoch Bundle Specification +# Cortex Epoch Bundle Specification **Status:** FROZEN (task 8 wave gate) **Normative for:** `bundle`, `aggregate`, gateway seal, validator verify/recompute diff --git a/docs/COMPLETENESS.md b/docs/COMPLETENESS.md index 6e667cee4..e7aedddf8 100644 --- a/docs/COMPLETENESS.md +++ b/docs/COMPLETENESS.md @@ -1,4 +1,4 @@ -# Base completeness matrix +# Cortex completeness matrix Honest per-component status as of `main` HEAD. Updated as phases land. @@ -53,7 +53,7 @@ Removed (replaced by design + prism HTTP paths; no Phala/CVM miner). | Component | Status | Notes | |-----------|--------|-------| -| Crates (`crates/design-*`) | done | task, harness, prompts, sandbox, sanitize, rating, store, egress-proxy, challenge. | +| Crates (`crates/design-*`) | done | task, harness, prompts, sandbox, sanitize, store, egress-proxy, challenge. Elo lives in `design_rating` Postgres via `design-db` / `design-store-pg` — not a standalone crate. | | Binary (`bins/design-challenge`) | done | HTTP API on `:8093`. | | Binary (`bins/design-egress-proxy`) | done | Open egress proxy (internal blocklist) + budgeted LLM path. | | Spec + checklist | done | [`DESIGN_CHALLENGE.md`](DESIGN_CHALLENGE.md) + checklist; `xtask design-check`. | diff --git a/docs/DESIGN_CHALLENGE.md b/docs/DESIGN_CHALLENGE.md index 6a2da998f..b945b37c1 100644 --- a/docs/DESIGN_CHALLENGE.md +++ b/docs/DESIGN_CHALLENGE.md @@ -612,8 +612,7 @@ returns 403). Operators hit `design-challenge:8093` on the master host | `design-prompts` | Pinned prompt bank + deterministic weighted selection | | `design-sandbox` | Two-phase Docker + `SimSandbox` | | `design-sanitize` | HTML/CSS sanitize + viewer headers | -| `design-rating` | Legacy Elo helpers (not on leaf path) | -| `design-store` | `DesignStore` trait, memory + DB adapter | +| `design-store` | `DesignStore` trait, memory + DB adapter (`design_rating` table) | | `design-egress-proxy` | Open egress proxy (internal blocklist) + budgeted LLM path | | `design-http` | Miner/viewer/admin winners/stats HTTP API | | `design-challenge` | Orchestrator, agentic, scoring, leaf emit | diff --git a/docs/NAMING.md b/docs/NAMING.md new file mode 100644 index 000000000..a7c680eb0 --- /dev/null +++ b/docs/NAMING.md @@ -0,0 +1,110 @@ +# Naming: Cortex vs leftover `base` identifiers + +**Product name:** Cortex +**Org / repo:** [`CortexLM/cortex`](https://github.com/CortexLM/cortex) + +This repo used to ship as “BASE” / `BaseIntelligence/base`. Human-facing +docs and GitHub metadata now say Cortex. A large set of **wire, deploy, and +crypto identifiers still spell `base` / `BASE_*`**. Changing those forks the +subnet or breaks measured miner CVMs and live droplets. Do not “fix” them +in a drive-by rename. + +## What to write in new code and docs + +| Use | Spelling | +|-----|----------| +| Product, README, PR titles, comments that mean the project | **Cortex** | +| This GitHub repository | `CortexLM/cortex` | +| Typed config crate keys (canonical) | `BASE_*` | +| Typed config crate aliases (optional) | `CORTEX_*` (same suffix) | +| Cryptographic domain tags, signing context | `base-*-v1` (frozen) | +| Compose project, host paths, GHCR package path | `base` / `/opt/base` / `ghcr.io/baseintelligence/base/*` | + +## Environment variables + +`crates/config` reads **`BASE_*` as the canonical names**. Matching +`CORTEX_*` names are accepted as aliases (`CORTEX_ROLE` ≡ `BASE_ROLE`, +`CORTEX_CONFIG` ≡ `BASE_CONFIG`, …). + +- If both are set, **`BASE_*` wins**. +- Deploy compose, droplet env files, and miner CVM `app-compose.json` must + keep emitting `BASE_*`. Those strings are measured (RTMR3 / pin continuity). +- Other crates and scripts may still read `BASE_*` directly. Do not rename + those readers unless you also keep the `BASE_*` spelling. + +Canonical keys today: `BASE_ROLE`, `BASE_NETUID`, `BASE_CHAIN_ENDPOINT`, +`BASE_CHAIN_ENDPOINTS`, `BASE_GATEWAY_ENDPOINT`, `BASE_DATABASE_URL`, +`BASE_DATABASE_URL_FILE`, `BASE_EPOCH_LENGTH`, `BASE_MIN_SHARE_MASS_BPS`, +`BASE_ROTATION_EPOCHS`, `BASE_MIN_PEER_SAMPLE`, `BASE_MAX_COLLATERAL_AGE_SECS`, +`BASE_DOMAIN`, `BASE_CONFIG`. Many other `BASE_*` knobs exist outside +`crates/config` (gateway owner flag, challenge secrets, deploy paths). Those +are also frozen spellings. + +Optional local config file remains `base.toml` (path from `BASE_CONFIG` / +`CORTEX_CONFIG`). + +## Cryptographic domain tags + +`crates/crypto` binds signatures to static tags. Examples (not exhaustive): + +- signing context: `base-sr25519-v1` +- `base-bundle-v1`, `base-rawweight-v1`, `base-dissent-v1`, `base-root-v1` +- `base-attest-v1`, `base-trustroot-v1` +- design: `base-design-round-id-v1`, `base-design-submission-v1`, `base-design-pair-id-v1` + +Renaming a tag makes old signatures unverifiable. Leave them. + +## Deployed filesystem and image paths + +Leave these as `base` even in new docs that mention them: + +| Kind | Example | +|------|---------| +| Droplet checkout | `/opt/base` | +| Miner CVM secret dir | `/run/base/` | +| Validator LKG default | `/var/lib/base/last-sealed.bundle` | +| Age identity | `/etc/base/age-identity.txt` | +| Compose project | `COMPOSE_PROJECT_NAME=base` | +| Sandbox name prefix | `base-design-` (pinned in `DESIGN_CHALLENGE.md`) | +| GHCR packages | `ghcr.io/baseintelligence/base/` | +| systemd units | `base-real-seal.timer`, `base-burn-seal.timer` | +| Host nicknames | `base-staging`, `base-prod`, … | + +GHCR still publishes under the historical `baseintelligence/base` package +path. Image **contents** are this Cortex tree; the registry path is a pin, +not the product name. + +## On-chain and public hostnames + +- Live gateway hostname `chain.joinbase.ai` is operator DNS, not a GitHub + brand string. Do not retarget it from a docs-only PR. +- Subnet netuid and on-chain names stay whatever the chain already uses. + +## Public miner repos (other GitHub repositories) + +Miner-facing documentation still lives in the historical public repos until +those orgs publish a CortexLM mirror: + +- [`BaseIntelligence/design-challenge`](https://github.com/BaseIntelligence/design-challenge) +- [`BaseIntelligence/prism`](https://github.com/BaseIntelligence/prism) + +This monorepo’s operator mirror is [`docs/external-miner/`](external-miner/). +Do not rewrite those URLs here unless the public repos actually move. + +HuggingFace / GitHub top-model defaults (`BaseIntelligence/top-prism-architecture`, +`BaseIntelligence/prism` `top-model/`) are publish targets configured in +compose. Changing the default without a matching Hub/GitHub move breaks +top-model publish. + +## Postgres + +`design_rating` is a **live table** (see `crates/design-db`, +`crates/design-store-pg`). It is unrelated to the removed unused crate +`crates/design-rating`. Do not drop the table or its accessors as part of +branding work. + +## Historical docs + +`docs/evidence/` and `docs/spikes/` may still say BASE / BaseIntelligence. +Those paths are non-normative. Do not rewrite them in a cleanup pass +(see [`AGENTS.md`](AGENTS.md)). diff --git a/docs/THREAT_MODEL.md b/docs/THREAT_MODEL.md index 1c4d47043..6abfe01d4 100644 --- a/docs/THREAT_MODEL.md +++ b/docs/THREAT_MODEL.md @@ -1,9 +1,9 @@ -# base threat model +# Cortex threat model This document states the **honest** security claim and the properties we deliberately do **not** claim. Overclaiming is treated as a project failure mode. -Frozen contracts: [`BUNDLE_SPEC.md`](./BUNDLE_SPEC.md), [`AGENT_CHALLENGE.md`](./AGENT_CHALLENGE.md). -Architecture map: [`ARCHITECTURE.md`](./ARCHITECTURE.md). +Frozen contracts: [`BUNDLE_SPEC.md`](./BUNDLE_SPEC.md), [`DESIGN_CHALLENGE.md`](./DESIGN_CHALLENGE.md). +Architecture map: [`ARCHITECTURE.md`](./ARCHITECTURE.md). Prism: [`PRISM.md`](./PRISM.md). CI enforces that §1 matches plan decision **D19** word for word (`cargo run -p xtask -- external-docs-check`).