diff --git a/src/pages/docs/self-hosting.mdx b/src/pages/docs/self-hosting.mdx index dd3c9fbf..e2dd80c7 100644 --- a/src/pages/docs/self-hosting.mdx +++ b/src/pages/docs/self-hosting.mdx @@ -1,49 +1,28 @@ --- -title: "Self-Hosting Future AGI: Deploy on Your Own Infrastructure" -description: "Deploy the full Future AGI platform on your own infrastructure using Docker Compose. Follow the step-by-step guide to get all services running locally." +title: "Self-hosting Future AGI" +description: "Run the entire Future AGI platform on your own infrastructure with Docker Compose. Your traces, datasets, evaluations, and model calls stay inside your network" --- -## About +Future AGI is fully open-source. Self-hosting runs the **entire stack on your own machines**, so all traces, datasets, evaluations, and model calls stay within your network. The backend is Django, the frontend is React + Vite, and the LLM gateway is Go, all deployed together with Docker Compose -Future AGI is fully open-source. Self-hosting runs the entire stack on your machines — all traces, datasets, evaluations, and model calls stay within your network. Backend is Django, frontend is React + Vite, LLM gateway is Go. +## When to self-host -Not sure if you need this? The hosted version at [app.futureagi.com](https://app.futureagi.com) is easier to operate. Self-host when you need **data residency**, **air-gapped environments**, **cost control at scale**, or **deep customization**. +The [**cloud hosted version**](https://app.futureagi.com) is the easiest way to run Future AGI, with nothing to operate. Self-host when you need: -## Quick start +- **Data residency**: keep all data inside your own network +- **Air-gapped environments**: run with no outbound dependencies +- **Cost control at scale**: own the infrastructure +- **Deep customization**: modify the open-source stack to fit your needs -```bash -git clone https://github.com/future-agi/future-agi.git -cd future-agi -cp .env.example .env -docker pull futureagi/future-agi:v1.8.19_base -docker compose up -``` - -First boot builds from source (~10–15 min). After `Application startup complete`: - -| Service | URL | -|---|---| -| Frontend | http://localhost:3000 | -| Backend API | http://localhost:8000 | -| PeerDB UI | http://localhost:3001 — `peerdb` / `peerdb` | - -## Deployment options - -| Option | Status | -|---|---| -| Docker Compose | Available | -| Helm / Kubernetes | Coming soon | -| Air-gapped | Coming soon | +## What you deploy -## Architecture - -21 containers across four layers. +Self-hosting brings up the full platform (around **21 containers, with no external dependencies**) across four layers: ``` Browser └─ frontend (React/nginx) - └─ backend (Django) ──── gateway (Go) ──── OpenAI · Anthropic · Gemini · Bedrock - ├── postgres primary DB + WAL replication + └─ backend (Django) ──── gateway (Go) ──── OpenAI · Anthropic · Gemini · Bedrock + ├── postgres primary database ├── clickhouse analytics store ├── redis cache / pub-sub ├── minio object storage @@ -52,51 +31,31 @@ Browser postgres ──── PeerDB CDC ──── clickhouse (continuous replication) ``` -**Application** — `frontend` · `backend` · `worker` · `gateway` · `serving` · `code-executor` - -**Data** — `postgres` · `clickhouse` · `redis` · `minio` +- **Application**: `frontend`, `backend`, `worker`, `gateway`, `serving`, `code-executor` +- **Data**: `postgres`, `clickhouse`, `redis`, `minio` +- **Workflow**: `temporal` +- **CDC**: PeerDB (continuous Postgres → ClickHouse replication) -**Workflow** — `temporal` +Everything runs on your machines; nothing leaves your network. The full service-by-service breakdown lives in [Configure](/docs/self-hosting/configure) -**CDC (PeerDB)** — `peerdb-catalog` · `peerdb-temporal` · `peerdb-minio` · `peerdb-flow-api` · `peerdb-flow-worker` · `peerdb-flow-snapshot-worker` · `peerdb-server` · `peerdb-ui` · `peerdb-temporal-init` · `peerdb-init` +## Deployment options -| Layer | Service | Purpose | -|---|---|---| -| App | `frontend` | React SPA served by nginx | -| App | `backend` | Django REST + gRPC + WebSocket API | -| App | `worker` | Temporal worker — evals, agent loops, data jobs | -| App | `gateway` | Go LLM proxy — routing, retries, rate limits, logging | -| App | `serving` | Embeddings and small model inference | -| App | `code-executor` | nsjail-sandboxed eval code runner (`privileged: true` required) | -| Data | `postgres` | Primary DB — users, traces, datasets, evals, prompts | -| Data | `clickhouse` | Analytics DB — replicated from Postgres via PeerDB | -| Data | `redis` | Cache, rate limits, WebSocket pub/sub | -| Data | `minio` | S3-compatible object storage (swap for S3 in prod) | -| Workflow | `temporal` | Durable workflow engine — shares main Postgres | -| CDC | PeerDB stack | Continuous Postgres → ClickHouse replication (10 services) | +| Option | Status | +|---|---| +| Docker Compose | Available | +| Helm / Kubernetes | Coming soon | +| Air-gapped | Coming soon | -## Next Steps +## Where to go next - - - Hardware tiers, platform compatibility, ports reference. - - - Setup, deployment modes, day-to-day operations. - - - Full `.env` reference — secrets, ports, flags, keys. - - - LLM gateway providers, PeerDB mirrors, Temporal workers. - - - Create accounts via email or Django shell. + + + System requirements, prerequisites, environment variables, and setup - - Hardening, backups, monitoring, upgrades. + + Fixes for common errors and answers to frequent questions - - Solutions for every known error. + + Get help from the Future AGI team and community diff --git a/src/pages/docs/self-hosting/configuration/environment.mdx b/src/pages/docs/self-hosting/configuration/environment.mdx new file mode 100644 index 00000000..8b499e3f --- /dev/null +++ b/src/pages/docs/self-hosting/configuration/environment.mdx @@ -0,0 +1,130 @@ +--- +title: "Environment Variables" +description: "Full .env reference for self-hosted Future AGI — required secrets, database credentials, runtime flags, LLM provider keys, email, and frontend build-time configuration." +--- + +## Introduction + +Every setting the stack reads at boot comes from a single `.env` file in the repo root. This page is the complete reference, grouped by what each variable does. The stack boots fine with the shipped defaults — the only thing you *must* change before sharing the instance is the `CHANGEME` secrets. + +```bash +cp .env.example .env +``` + + +Doing a local trial? Skip straight to [Install](/docs/self-hosting/install) — the defaults work as-is. Come back here when you're ready to set secrets, add LLM provider keys, or turn on email. + + +## Required Secrets + +Replace every `CHANGEME` in this group before anyone else can reach the instance. Generate each value with the command shown. + +| Variable | Generate with | Used by | +|---|---|---| +| `SECRET_KEY` | `openssl rand -hex 32` | Django sessions, CSRF, password reset | +| `PG_PASSWORD` | `openssl rand -base64 24` | PostgreSQL auth | +| `MINIO_ROOT_PASSWORD` | `openssl rand -base64 24` | MinIO object storage auth | +| `AGENTCC_INTERNAL_API_KEY` | `openssl rand -hex 32` | Backend ↔ gateway shared secret | + + +`PG_PASSWORD` is written to the Postgres volume on **first boot only**. If you change it after the volume exists, authentication fails — see the fix in [Troubleshooting](/docs/self-hosting/troubleshooting). Set it before your first `docker compose up`. + + +## Database Credentials + +| Variable | Default | Notes | +|---|---|---| +| `PG_USER` | `futureagi` | PostgreSQL username | +| `PG_PASSWORD` | `CHANGEME` | **Must change** | +| `PG_DB` | `futureagi` | PostgreSQL database name | +| `MINIO_ROOT_USER` | `futureagi` | MinIO username | +| `MINIO_ROOT_PASSWORD` | `CHANGEME` | **Must change** | +| `CH_USE_REPLICATED_ENGINES` | `false` | `true` only for multi-node ClickHouse | + +## Ports + +Every service port is configurable. The full table — defaults, what each binds to, and exposure scope — lives in [Requirements → Ports and Firewall](/docs/self-hosting/requirements#ports-and-firewall), so you can plan firewall rules in one place. + +## Backend Runtime + +| Variable | Default | Description | +|---|---|---| +| `ENV_TYPE` | `development` | `development` · `staging` · `prod`. Prod mode disables debug output and enables `check --deploy`. | +| `FAST_STARTUP` | `false` | Skip migrations on restart (dev only). Always `false` in production. | +| `GRANIAN_WORKERS` | `1` | ASGI worker processes. Set to your CPU count in production. | +| `GRANIAN_THREADS` | `2` | Threads per worker. | +| `ENABLE_GRPC` | `true` | Enable the gRPC endpoint. | +| `ENABLE_HTTP` | `true` | Enable the HTTP/REST endpoint. | + +## Temporal Worker + +| Variable | Default | Description | +|---|---|---| +| `TEMPORAL_NAMESPACE` | `default` | Temporal namespace. | +| `TEMPORAL_ALL_QUEUES` | `true` | Single worker polls all queues. Set `false` and use the dev overlay for per-queue workers. | +| `TEMPORAL_MAX_CONCURRENT_ACTIVITIES` | `50` | Max concurrent activity tasks. | +| `TEMPORAL_MAX_CONCURRENT_WORKFLOW_TASKS` | `50` | Max concurrent workflow tasks. | + +Tuning guidance lives in [System Configuration → Temporal Workers](/docs/self-hosting/configuration/system#temporal-workers). + +## LLM Gateway + +| Variable | Default | Description | +|---|---|---| +| `AGENTCC_INTERNAL_API_KEY` | `CHANGEME` | **Must change.** The backend authenticates gateway calls with this shared secret. | + +Setting a key here is only half the job — the gateway also needs a `config.yaml` listing the providers it may route to. See [System Configuration → LLM Gateway](/docs/self-hosting/configuration/system#llm-gateway). + +## LLM Provider Keys + +Set a key for each provider you'll use and leave the rest blank. These are read by the gateway via `${VAR}` interpolation in `config.yaml`. + +| Variable | Provider | +|---|---| +| `OPENAI_API_KEY` | OpenAI | +| `ANTHROPIC_API_KEY` | Anthropic | +| `GOOGLE_API_KEY` | Google Gemini | +| `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_REGION` | AWS Bedrock + S3 | + +## Email (Mailgun) + +Email delivery powers self-service sign-up and password reset. Without it, you create users from the Django shell during [Install](/docs/self-hosting/install). Set these to turn on the email flow: + +| Variable | Description | +|---|---| +| `MAILGUN_API_KEY` | Mailgun private API key | +| `MAILGUN_SENDER_DOMAIN` | Verified Mailgun sending domain | +| `DEFAULT_FROM_EMAIL` | `From:` address for outbound email | +| `SERVER_EMAIL` | `From:` address for Django admin error email | + +## Frontend Build-Time + + +These are baked into the JavaScript bundle at Vite build time. Changing them requires a rebuild: `docker compose build frontend`. + + +| Variable | Default | Description | +|---|---|---| +| `VITE_HOST_API` | `http://localhost:8000` | Backend URL as seen by the browser. In production, use your public backend URL. | +| `VITE_ENVIRONMENT` | `development` | Frontend analytics and feature flags. | + +## Optional + +| Variable | Default | Description | +|---|---|---| +| `RECAPTCHA_ENABLED` | `false` | Enable reCAPTCHA on registration. | +| `RECAPTCHA_SECRET_KEY` | — | reCAPTCHA v2/v3 server-side key. | +| `VITE_GOOGLE_SITE_KEY` | — | reCAPTCHA client-side key (requires a frontend rebuild). | +| `FUTURE_AGI_CLOUD_API_KEY` | — | Enterprise-tier Cloud features only. Leave blank for the open-source build. | +| `FUTURE_AGI_CLOUD_API_URL` | `https://api.futureagi.com` | Do not change. | + +## Dive Deeper + + + + Point the LLM gateway at your providers and set up PeerDB mirrors. + + + Harden the instance before exposing it to users. + +