From aa8bb618f9f8350ed4cbe4846d09096ca9ae236d Mon Sep 17 00:00:00 2001 From: Ari Nguyen Date: Thu, 9 Jul 2026 07:21:35 -0700 Subject: [PATCH 1/7] feat(dev): add docker-free local mode (`./dev.sh local`) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Stand up the full backend as host processes with embedded datastores — no Docker daemon required — as a lighter alternative to the container stack. `./dev.sh local` (also `make dev-local` / `python -m scripts.dev_local`) provisions: - Postgres via bundled pgserver (unix socket) and Redis via bundled redislite - a Temporal dev server + UI and the agentex worker (--no-temporal to skip) - a local mongod, required for the full stack (--no-mongo / --lean to skip) - an optional OpenTelemetry collector (--no-otel to skip) then runs migrations, supervises uvicorn + the worker, and tears everything down cleanly on SIGINT/SIGTERM. --lean is a minimal Postgres+Redis+API stack; --ephemeral uses a throwaway data dir. The runner is a small scripts/dev_local package (config / services / supervise / runner) so the pure config/env layer stays testable. App-side changes to make the no-Docker path robust (all no-ops when Mongo is configured, as it always is in Docker/prod): - The Temporal worker no longer crashes when MongoDB is unavailable — the Mongo CRUD adapter tolerates an unset database and errors only on real use, so the worker degrades like the API instead of taking down the stack. - Skip the Mongo connection entirely when MONGODB_URI is unset, removing a ~20s startup hang against the implicit localhost:27017 default. - In local mode the backend rewrites agents' host.docker.internal ACP host to loopback (AGENTEX_ACP_HOST_OVERRIDE), so default-scaffolded agents work without manifest edits. Also fix a frontend dev-server process leak in dev.sh (kill the whole make→npm→next tree and sweep orphans; report status by listening port), correct the MongoDB and OpenTelemetry install commands, and document local mode in README and CLAUDE.md. --- .gitignore | 3 + CLAUDE.md | 21 ++ README.md | 48 ++- agentex/Makefile | 10 + agentex/pyproject.toml | 11 + agentex/scripts/dev_local/__init__.py | 50 +++ agentex/scripts/dev_local/__main__.py | 6 + agentex/scripts/dev_local/config.py | 214 ++++++++++++ agentex/scripts/dev_local/runner.py | 184 ++++++++++ agentex/scripts/dev_local/services.py | 212 ++++++++++++ agentex/scripts/dev_local/supervise.py | 125 +++++++ .../use_cases/agent_api_keys_use_case.py | 6 +- .../domain/use_cases/agents_acp_use_case.py | 29 +- .../activities/healthcheck_activities.py | 4 + agentex/src/utils/acp_url.py | 56 +++ dev.sh | 324 ++++++++++++++++-- uv.lock | 61 ++++ 17 files changed, 1311 insertions(+), 53 deletions(-) create mode 100644 agentex/scripts/dev_local/__init__.py create mode 100644 agentex/scripts/dev_local/__main__.py create mode 100644 agentex/scripts/dev_local/config.py create mode 100644 agentex/scripts/dev_local/runner.py create mode 100644 agentex/scripts/dev_local/services.py create mode 100644 agentex/scripts/dev_local/supervise.py create mode 100644 agentex/src/utils/acp_url.py diff --git a/.gitignore b/.gitignore index 073ee249..f55e4ca9 100644 --- a/.gitignore +++ b/.gitignore @@ -43,6 +43,9 @@ tmp # Dev script logs .dev-logs/ +# Local (no-Docker) dev datastore data +agentex/.dev-local/ + ### PYTHON # Byte-compiled / optimized / DLL files diff --git a/CLAUDE.md b/CLAUDE.md index 4986367e..28b872dd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -55,6 +55,27 @@ Other commands: ./dev.sh restart # Restart all services ``` +Docker-free local mode (host processes + embedded datastores, no Docker): +```bash +./dev.sh local # whole stack without Docker (or: make dev-local) +./dev.sh local --lean # Postgres + Redis + API + MongoDB only +./dev.sh local --mongo-uri # use an external MongoDB instead of a local mongod +``` + +> **MongoDB is required for the full local stack** and is always started — the Temporal +> worker builds Mongo-backed repositories at startup, so a missing/unreachable Mongo +> makes the runner fail fast (with an install message) rather than crash the worker. +> `./dev.sh local` auto-installs `mongod`; `make dev-local` / direct `python -m +> scripts.dev_local` do not, so install `mongod` yourself or pass `--mongo-uri ` to +> point at an external MongoDB. In local mode the Temporal UI is on :8233 (Docker mode +> uses :8080). +> +> Agents register their ACP URL as `host.docker.internal` (for a Docker backend), which +> a host-process backend can't resolve. Local mode sets `AGENTEX_ACP_HOST_OVERRIDE=127.0.0.1` +> and the backend rewrites `host.docker.internal` → that value when dialing agents +> (`src/utils/acp_url.py`, applied in the ACP request path + Temporal healthcheck), so +> default-scaffolded agents work without manifest edits. + **Then in a separate terminal - Agent Development:** ```bash agentex init # Create a new agent diff --git a/README.md b/README.md index bc22fe3d..497ea57d 100644 --- a/README.md +++ b/README.md @@ -105,20 +105,23 @@ To do this, you just need to spin up the [Agentex Server](https://github.com/sca ### Quick Start (Recommended) -Just run one command: +Just run one command — no Docker required: ```bash -./dev.sh +./dev.sh local ``` That's it. This will automatically: - Install Homebrew, uv, Node.js, and agentex-sdk if missing (macOS) - Install all backend and frontend dependencies -- Start all Docker services (Postgres, Redis, MongoDB, Temporal) -- Start the backend API and frontend dev server +- Start the backend API and frontend dev server as host processes +- Provision embedded datastores — bundled Postgres + Redis, an auto-downloaded Temporal + dev server, a local MongoDB, and an optional OTel collector - Wait for everything to be healthy -> **Note:** Make sure Docker Desktop or Rancher Desktop is running before you start. +> Prefer containers? `./dev.sh` runs the same stack with Docker instead (needs Docker +> Desktop or Rancher Desktop) — see [Other commands](#other-commands) below. See +> [Docker-free local mode](#docker-free-local-mode) for `local` flags and details. Once ready: | Service | URL | @@ -126,16 +129,47 @@ Once ready: | Frontend UI | http://localhost:3000 | | Backend API | http://localhost:5003 | | Swagger Docs | http://localhost:5003/swagger | -| Temporal UI | http://localhost:8080 | +| Temporal UI | http://localhost:8233 | -**Other commands:** +> With `./dev.sh` (Docker) the Temporal UI is on http://localhost:8080 instead. + +#### Other commands ```bash +./dev.sh # Start everything with Docker instead (containers; needs Docker Desktop/Rancher) ./dev.sh stop # Stop all services ./dev.sh status # Check service status ./dev.sh logs # View all logs ./dev.sh restart # Restart all services ``` +#### Docker-free local mode + +`./dev.sh local` accepts flags to trim the stack: + +```bash +./dev.sh local # whole stack, no Docker +./dev.sh local --lean # Postgres + Redis + API + MongoDB only (no Temporal/OTel) +./dev.sh local --no-temporal # skip Temporal + the worker +./dev.sh local --mongo-uri # use an external MongoDB instead of a local mongod +``` + +> **MongoDB is required for the full local stack** and is always started. The Temporal +> worker needs it, so `./dev.sh local` auto-installs `mongod` (via Homebrew) and startup +> **fails fast** with an install message if it can't be made available. Point at an +> existing MongoDB with `--mongo-uri ` to skip the local `mongod`. The OTel +> collector is optional; if it's absent the runner continues without telemetry. In local +> mode the Temporal UI is at http://localhost:8233 (not :8080). Same `stop` / `status` / +> `logs` / `restart` commands apply. + +**Connecting agents in local mode.** Agents scaffolded by `agentex init` register their +ACP URL as `http://host.docker.internal:` (so a *Docker* backend can reach an +agent on the host). A host-process backend can't resolve that name, so local mode sets +`AGENTEX_ACP_HOST_OVERRIDE=127.0.0.1` and the backend automatically rewrites +`host.docker.internal` → `127.0.0.1` when dialing agents (and their healthchecks). So a +default-scaffolded agent works with `./dev.sh local` **without editing its manifest**. +(You can still set `local_development.agent.host_address: localhost` in the manifest if +you prefer; both work.) + Then skip ahead to [Create Your First Agent](#create-your-first-agent). --- diff --git a/agentex/Makefile b/agentex/Makefile index b3fb0f0a..58e12a0d 100644 --- a/agentex/Makefile +++ b/agentex/Makefile @@ -41,9 +41,19 @@ dev: install-dev ## Start development server with Docker Compose @echo "🚀 Starting development server with Docker Compose..." docker compose up --build +dev-local: install-dev ## Start development server locally (no Docker) + @echo "🚀 Starting development server locally without Docker..." + @echo "ℹ️ MongoDB is REQUIRED for the full stack. If 'mongod' is not on your PATH," + @echo " install it (brew tap mongodb/brew && brew install mongodb-community) or point" + @echo " at an external one with ARGS=\"--mongo-uri \". (Unlike './dev.sh local'," + @echo " make does not auto-install it.)" + uv sync --group dev --group dev-local + uv run python -m scripts.dev_local $(ARGS) + dev-stop: ## Stop development server @echo "Stopping dev server" docker compose down +# TODO: Add support for stopping local dev server dev-wipe: ## Stop dev server and wipe DB @echo "Stopping dev server and wiping DB" diff --git a/agentex/pyproject.toml b/agentex/pyproject.toml index b1785fa1..0fe3039f 100644 --- a/agentex/pyproject.toml +++ b/agentex/pyproject.toml @@ -43,6 +43,14 @@ dev = [ "vulture>=2.14", "ruff>=0.3.4", ] +dev-local = [ + "pgserver>=0.1.4", + "redislite>=6.2.912183", + # SQLAlchemy's async engine needs greenlet at runtime. SQLAlchemy auto-installs + # it on linux x86_64 (docker/prod) but NOT on macOS arm64, so the local runner + # must pull it explicitly or /readyz and engine teardown fail. + "greenlet>=3.2.3", +] test = [ "pytest>=8.3.3,<9", "pytest-asyncio>=1.0.0,<2", @@ -52,6 +60,9 @@ test = [ "factory-boy>=3.3.0,<4", # for test data factories "greenlet>=3.2.3", "asyncpg>=0.29.0", + # Embedded datastores for dev_local.py tests (no testcontainers/docker needed) + "pgserver>=0.1.4", + "redislite>=6.2.912183", ] [tool.hatch.build.targets.sdist] diff --git a/agentex/scripts/dev_local/__init__.py b/agentex/scripts/dev_local/__init__.py new file mode 100644 index 00000000..c368290c --- /dev/null +++ b/agentex/scripts/dev_local/__init__.py @@ -0,0 +1,50 @@ +"""Run the agentex backend locally as host processes, with no Docker. + +The engine behind `./dev.sh local` / `make dev-local`: it provisions embedded datastores, +runs migrations, and supervises uvicorn (plus a Temporal worker), tearing everything down +on Ctrl-C / SIGTERM. Postgres/Redis/Temporal need no system install (bundled / +auto-downloaded). MongoDB is REQUIRED for the full stack — the Temporal worker builds +Mongo-backed repositories at boot, so a missing/unreachable mongod fails fast instead of +crashing the worker. It is always started; --mongo-uri points at an external MongoDB +instead of launching a local mongod. The OTel collector is optional; --lean turns off +temporal/otel (but keeps MongoDB). + +Modules: `config` (pure, unit-testable), `services` (provisioning), `supervise` +(subprocess plumbing), `runner` (orchestration). +""" + +from scripts.dev_local.config import ( + LOOPBACK, + DevLocalConfig, + build_arg_parser, + build_env, + resolve_config, +) +from scripts.dev_local.runner import main, run +from scripts.dev_local.services import ( + provision_mongo, + provision_otel, + provision_postgres, + provision_redis, + provision_temporal, + teardown_redis, +) +from scripts.dev_local.supervise import run_migrations, wait_for_health + +__all__ = [ + "LOOPBACK", + "DevLocalConfig", + "build_arg_parser", + "resolve_config", + "build_env", + "provision_postgres", + "provision_redis", + "provision_temporal", + "provision_mongo", + "provision_otel", + "teardown_redis", + "run_migrations", + "wait_for_health", + "run", + "main", +] diff --git a/agentex/scripts/dev_local/__main__.py b/agentex/scripts/dev_local/__main__.py new file mode 100644 index 00000000..2934f9e5 --- /dev/null +++ b/agentex/scripts/dev_local/__main__.py @@ -0,0 +1,6 @@ +"""Entry point: `python -m scripts.dev_local` (used by `./dev.sh local` / `make dev-local`).""" + +from scripts.dev_local.runner import main + +if __name__ == "__main__": + main() diff --git a/agentex/scripts/dev_local/config.py b/agentex/scripts/dev_local/config.py new file mode 100644 index 00000000..2181d15e --- /dev/null +++ b/agentex/scripts/dev_local/config.py @@ -0,0 +1,214 @@ +"""Pure config/env for the docker-free local runner — no side effects, unit-testable.""" + +from __future__ import annotations + +import argparse +import os +import tempfile +from dataclasses import dataclass +from pathlib import Path + +# scripts/dev_local/config.py -> agentex is three parents up; the runner launches +# uvicorn/alembic/the worker from there. +AGENTEX_DIR = Path(__file__).resolve().parents[2] + +# Loopback address every local service binds to and is dialed on. A constant, not an +# env var: a dev runner should always bind loopback. (The API server binds 0.0.0.0 on +# purpose — see runner.run; banner URLs say "localhost" for readability.) +LOOPBACK = "127.0.0.1" + + +@dataclass +class DevLocalConfig: + agentex_dir: Path + data_dir: Path + ephemeral: bool + api_port: int + redis_port: int + temporal: bool + temporal_port: int + ui_port: int + mongo_uri: str | None # external override; when None, launch a local mongod + mongo_port: int + otel: bool + otel_port: int + + @property + def pg_data(self) -> Path: + return self.data_dir / "pgdata" + + @property + def redis_dir(self) -> Path: + return self.data_dir / "redis" + + @property + def temporal_db(self) -> Path: + return self.data_dir / "temporal.sqlite" + + @property + def mongo_data(self) -> Path: + return self.data_dir / "mongo" + + +def build_arg_parser() -> argparse.ArgumentParser: + p = argparse.ArgumentParser( + prog="dev_local", + description="Run the full agentex backend locally without Docker.", + ) + # Everything is on by default; --no- opts out. + p.add_argument( + "--temporal", + action=argparse.BooleanOptionalAction, + default=True, + help="Start the Temporal dev server (+ UI) and the agentex worker (default: on).", + ) + p.add_argument( + "--otel", + action=argparse.BooleanOptionalAction, + default=True, + help="Start the OpenTelemetry collector (default: on; optional — skipped with a warning if not installed).", + ) + # --lean and --full are opposite shortcuts and cannot be combined. + stack_group = p.add_mutually_exclusive_group() + stack_group.add_argument( + "--lean", + action="store_true", + help="Only Postgres + Redis + API + MongoDB (turns off temporal and otel). MongoDB is always started — the stack requires it.", + ) + stack_group.add_argument( + "--full", + action="store_true", + help="Run the whole stack (pg + redis + api + temporal + mongo + otel). This is already the default; the flag exists so `--full` is accepted rather than rejected.", + ) + p.add_argument( + "--mongo-uri", + default=None, + help="Use an EXTERNAL MongoDB at this URI instead of launching a local mongod.", + ) + p.add_argument( + "--port", type=int, default=5003, help="Port for the API server (default 5003)." + ) + p.add_argument( + "--redis-port", + type=int, + default=6390, + help="TCP port for embedded Redis (default 6390; avoids a Docker Redis on 6379).", + ) + p.add_argument( + "--temporal-port", + type=int, + default=7233, + help="Port for the Temporal frontend (default 7233).", + ) + p.add_argument( + "--ui-port", + type=int, + default=8233, + help="Port for the Temporal Web UI (default 8233).", + ) + p.add_argument( + "--mongo-port", + type=int, + default=27017, + help="Port for the local mongod (default 27017).", + ) + p.add_argument( + "--otel-port", + type=int, + default=4317, + help="OTLP gRPC port for the collector (default 4317).", + ) + p.add_argument( + "--data-dir", + default=None, + help="Directory for persistent datastore data (default /.dev-local). Ignored with --ephemeral.", + ) + p.add_argument( + "--ephemeral", + action="store_true", + help="Use a throwaway temp data dir wiped on exit (fresh state every run).", + ) + return p + + +def resolve_config( + argv: list[str] | None = None, *, agentex_dir: Path = AGENTEX_DIR +) -> DevLocalConfig: + """Parse argv into a DevLocalConfig. Pure: no filesystem side effects.""" + parser = build_arg_parser() + args = parser.parse_args(argv) + + if args.ephemeral: + # Per-process-unique so a hard-killed run can't leave a pgserver orphan that the + # next --ephemeral run re-attaches to (which would defeat "fresh state every run"). + data_dir = Path(tempfile.gettempdir()) / f"agentex-dev-local-{os.getpid()}" + elif args.data_dir: + data_dir = Path(args.data_dir).expanduser().resolve() + else: + data_dir = agentex_dir / ".dev-local" + + # --lean forces the optional services off; --full is a no-op (they're already on). + # MongoDB is always started — the stack requires it — so --lean does not touch it. + temporal = args.temporal and not args.lean + otel = args.otel and not args.lean + + return DevLocalConfig( + agentex_dir=agentex_dir, + data_dir=data_dir, + ephemeral=args.ephemeral, + api_port=args.port, + redis_port=args.redis_port, + temporal=temporal, + temporal_port=args.temporal_port, + ui_port=args.ui_port, + mongo_uri=args.mongo_uri, + mongo_port=args.mongo_port, + otel=otel, + otel_port=args.otel_port, + ) + + +def build_env( + *, + database_url: str, + redis_url: str, + temporal_address: str | None, + mongo_uri: str | None, + otel_endpoint: str | None, +) -> dict[str, str]: + """Build the environment for the API/worker subprocesses. Pure. + + An optional service's env var is set ONLY when that service was provisioned (and any + inherited value is dropped), so the backend's feature gates see the true state and a + leftover env can't re-enable a service we didn't start. + """ + env = dict(os.environ) + env["ENVIRONMENT"] = "development" + env["DATABASE_URL"] = database_url + env["REDIS_URL"] = redis_url + env["MONGODB_DATABASE_NAME"] = "agentex" + + # Agents register their ACP URL at host.docker.internal (SDK default, for a Docker + # backend); that name doesn't resolve for this host-process backend, so have it dial + # such agents at loopback instead. Only the sentinel host is rewritten — see + # src/utils/acp_url.py. + env["AGENTEX_ACP_HOST_OVERRIDE"] = LOOPBACK + + if mongo_uri: + env["MONGODB_URI"] = mongo_uri + else: + env.pop("MONGODB_URI", None) + + if temporal_address: + env["TEMPORAL_ADDRESS"] = temporal_address + env["AGENTEX_SERVER_TASK_QUEUE"] = "agentex-server" + else: + env.pop("TEMPORAL_ADDRESS", None) + + if otel_endpoint: + env["OTEL_EXPORTER_OTLP_ENDPOINT"] = otel_endpoint + env["OTEL_SERVICE_NAME"] = "agentex-api" + else: + env.pop("OTEL_EXPORTER_OTLP_ENDPOINT", None) + + return env diff --git a/agentex/scripts/dev_local/runner.py b/agentex/scripts/dev_local/runner.py new file mode 100644 index 00000000..fa2d5720 --- /dev/null +++ b/agentex/scripts/dev_local/runner.py @@ -0,0 +1,184 @@ +"""Orchestrate provisioning, migration, the API + worker, and clean teardown.""" + +from __future__ import annotations + +import asyncio +import logging +import signal +import sys + +from scripts.dev_local.config import LOOPBACK, DevLocalConfig, build_env, resolve_config +from scripts.dev_local.services import ( + provision_mongo, + provision_otel, + provision_postgres, + provision_redis, + provision_temporal, + teardown_redis, +) +from scripts.dev_local.supervise import ( + run_migrations, + spawn, + terminate, + wait_for_health, +) + +logger = logging.getLogger(__name__) + + +async def run(cfg: DevLocalConfig) -> int: + if cfg.ephemeral: + cfg.data_dir.mkdir(parents=True, exist_ok=True) + + pg_server = None + redis_server = None + temporal_env = None + mongo_proc: asyncio.subprocess.Process | None = None + otel_proc: asyncio.subprocess.Process | None = None + procs: list[tuple[str, asyncio.subprocess.Process]] = [] + + stop = asyncio.Event() + loop = asyncio.get_running_loop() + for sig in (signal.SIGINT, signal.SIGTERM): + loop.add_signal_handler(sig, stop.set) + + try: + pg_server, database_url = provision_postgres(cfg) + redis_server, redis_url = provision_redis(cfg) + + # MongoDB is required for the full stack (the Temporal worker builds + # Mongo-backed repositories at boot), so it is always provisioned. + mongo_proc, mongo_uri = await provision_mongo(cfg) + + temporal_address: str | None = None + if cfg.temporal: + temporal_env, temporal_address = await provision_temporal(cfg) + + otel_endpoint: str | None = None + if cfg.otel: + otel_proc, otel_endpoint = await provision_otel(cfg) + + env = build_env( + database_url=database_url, + redis_url=redis_url, + temporal_address=temporal_address, + mongo_uri=mongo_uri, + otel_endpoint=otel_endpoint, + ) + + await run_migrations(cfg, env) + + logger.info("Starting API server on http://localhost:%d", cfg.api_port) + # Bind 0.0.0.0 (not LOOPBACK) so the containerized frontend / other hosts can reach it. + api = await spawn( + "api", + [ + sys.executable, + "-m", + "uvicorn", + "src.api.app:app", + "--host", + "0.0.0.0", + "--port", + str(cfg.api_port), + "--reload", + "--reload-dir", + "src", + ], + cwd=cfg.agentex_dir, + env=env, + ) + procs.append(("api", api)) + + if cfg.temporal: + worker = await spawn( + "worker", + [sys.executable, "src/temporal/run_worker.py"], + cwd=cfg.agentex_dir, + env=env, + ) + procs.append(("worker", worker)) + + if await wait_for_health(cfg.api_port): + logger.info("API is up.") + else: + logger.warning( + "API did not pass /healthz within the timeout; check the logs above." + ) + + _print_banner(cfg, mongo_uri, temporal_address, otel_endpoint) + + # Exit when a signal arrives or a core process dies unexpectedly. + waiters = [asyncio.create_task(p.wait()) for _, p in procs] + stop_task = asyncio.create_task(stop.wait()) + await asyncio.wait([stop_task, *waiters], return_when=asyncio.FIRST_COMPLETED) + if not stop.is_set(): + dead = [name for name, p in procs if p.returncode is not None] + logger.error( + "A managed process exited unexpectedly: %s. Shutting down.", + ", ".join(dead) or "?", + ) + return 0 + finally: + # Tear down consumers (worker, api) first, then services in reverse. + for name, proc in reversed(procs): + await terminate(proc, name) + if temporal_env is not None: + logger.info("Stopping Temporal dev server ...") + try: + await temporal_env.shutdown() + except Exception as exc: # noqa: BLE001 + logger.warning("Temporal shutdown raised %s", exc) + if otel_proc is not None: + await terminate(otel_proc, "otel") + if mongo_proc is not None: + await terminate(mongo_proc, "mongo") + if redis_server is not None: + logger.info("Stopping embedded Redis ...") + teardown_redis(redis_server) + if pg_server is not None: + logger.info("Stopping embedded Postgres ...") + try: + pg_server.cleanup() + except Exception as exc: # noqa: BLE001 + logger.warning("pgserver cleanup raised %s", exc) + logger.info("All local services stopped.") + + +def _print_banner( + cfg: DevLocalConfig, + mongo_uri: str | None, + temporal_address: str | None, + otel_endpoint: str | None, +) -> None: + temporal = ( + f"{LOOPBACK}:{cfg.temporal_port} (UI http://localhost:{cfg.ui_port})" + if temporal_address + else "off" + ) + print("\n" + "=" * 52, flush=True) + print(" agentex backend (local, no Docker)", flush=True) + print(f" API: http://localhost:{cfg.api_port}", flush=True) + print(f" Swagger: http://localhost:{cfg.api_port}/swagger", flush=True) + print(" Postgres: embedded (socket)", flush=True) + print(f" Redis: {LOOPBACK}:{cfg.redis_port}", flush=True) + print(f" Mongo: {mongo_uri or 'off'}", flush=True) + print(f" Temporal: {temporal}", flush=True) + print(f" OTel: {otel_endpoint or 'off'}", flush=True) + print(" Ctrl-C to stop everything.", flush=True) + print("=" * 52 + "\n", flush=True) + + +def main() -> None: + logging.basicConfig( + level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s" + ) + cfg = resolve_config() + try: + code = asyncio.run(run(cfg)) + except RuntimeError as exc: + # Fail-fast provisioning errors (e.g. Mongo required but missing) read as a + # single actionable line, not a traceback. + logger.error("%s", exc) + raise SystemExit(1) from None + raise SystemExit(code) diff --git a/agentex/scripts/dev_local/services.py b/agentex/scripts/dev_local/services.py new file mode 100644 index 00000000..78d736fa --- /dev/null +++ b/agentex/scripts/dev_local/services.py @@ -0,0 +1,212 @@ +"""Provision the backing datastores/services as host processes (side effects).""" + +from __future__ import annotations + +import asyncio +import logging +import os +import shutil +import signal +from typing import Any +from urllib.parse import urlsplit + +from scripts.dev_local.config import LOOPBACK, DevLocalConfig +from scripts.dev_local.supervise import spawn, terminate, wait_for_port + +logger = logging.getLogger(__name__) + + +def provision_postgres(cfg: DevLocalConfig) -> tuple[Any, str]: + """Start an embedded Postgres and ensure the 'agentex' database exists. + + The returned socket-form `postgresql://` URL works for both the async app (asyncpg) + and alembic (psycopg2) — adjust_db_url normalizes the scheme for each. + """ + import pgserver + + cfg.pg_data.parent.mkdir( + parents=True, exist_ok=True + ) # get_server makes pgdata itself + logger.info("Provisioning embedded Postgres at %s ...", cfg.pg_data) + server = pgserver.get_server( + cfg.pg_data, cleanup_mode="delete" if cfg.ephemeral else "stop" + ) + + # psql() doesn't stop-on-error and CREATE DATABASE can't be IF NOT EXISTS, so guard. + existing = server.psql("SELECT 1 FROM pg_database WHERE datname = 'agentex';") + if "(1 row)" not in existing: + server.psql("CREATE DATABASE agentex;") + logger.info("Created database 'agentex'.") + + return server, server.get_uri(database="agentex") + + +def provision_redis(cfg: DevLocalConfig) -> tuple[Any, str]: + """Start an embedded real redis-server on a TCP port. Returns (server, url).""" + import redislite + + cfg.redis_dir.mkdir(parents=True, exist_ok=True) + logger.info("Provisioning embedded Redis on %s:%s ...", LOOPBACK, cfg.redis_port) + # port MUST go inside serverconfig — a top-level port= kwarg switches redislite into + # client-only mode and starts no server. + server = redislite.Redis( + str(cfg.redis_dir / "dump.rdb"), + serverconfig={"port": str(cfg.redis_port), "bind": LOOPBACK}, + ) + port = server.config_get("port")["port"] + return server, f"redis://{LOOPBACK}:{port}/0" + + +async def provision_temporal(cfg: DevLocalConfig) -> tuple[Any, str]: + """Start the Temporal dev server (+ Web UI). Returns (env, address).""" + from temporalio.testing import WorkflowEnvironment + + db_file: str | None = None + if not cfg.ephemeral: + cfg.data_dir.mkdir(parents=True, exist_ok=True) + db_file = str(cfg.temporal_db) + + logger.info( + "Starting Temporal dev server on %s:%s (UI :%s) ...", + LOOPBACK, + cfg.temporal_port, + cfg.ui_port, + ) + env = await WorkflowEnvironment.start_local( + ip=LOOPBACK, + port=cfg.temporal_port, + ui=True, + ui_port=cfg.ui_port, + dev_server_database_filename=db_file, + ) + return env, f"{LOOPBACK}:{cfg.temporal_port}" + + +async def provision_mongo( + cfg: DevLocalConfig, +) -> tuple[asyncio.subprocess.Process | None, str | None]: + """Start a local mongod (or use an external URI). Returns (proc, uri). + + MongoDB is REQUIRED for the full stack (the Temporal worker builds Mongo-backed repos + at boot), so this is always called and FAILS FAST if Mongo can't be made available — + returning None would let the worker crash and take the stack down. + """ + if cfg.mongo_uri: + logger.info("Using external MongoDB at %s", cfg.mongo_uri) + # Reachability check only for a plain single-host mongodb://host:port URI. SRV + # (mongodb+srv://) and multi-host / replica-set URIs can't be TCP-probed naively. + parsed = urlsplit(cfg.mongo_uri) + try: + host, port = parsed.hostname, parsed.port + except ValueError: # non-integer port in a multi-host netloc + host, port = None, None + probeable = ( + parsed.scheme == "mongodb" + and host is not None + and port is not None + and "," not in (parsed.netloc or "") + ) + if probeable and not await wait_for_port(None, host, port, timeout=5.0): + raise RuntimeError( + f"External MongoDB at {cfg.mongo_uri} is not reachable. Verify it is " + f"running/accessible, or omit --mongo-uri to launch a local mongod." + ) + return None, cfg.mongo_uri + + mongod = shutil.which("mongod") + if not mongod: + raise RuntimeError( + "mongod not found on PATH, but MongoDB is required for the local stack. " + "Install it (macOS: brew tap mongodb/brew && brew install mongodb-community; " + "Linux: install the mongodb-org package so `mongod` is on PATH), or pass " + "--mongo-uri to point at an external MongoDB." + ) + + cfg.mongo_data.mkdir(parents=True, exist_ok=True) + logger.info("Provisioning MongoDB (mongod) on %s:%s ...", LOOPBACK, cfg.mongo_port) + proc = await spawn( + "mongo", + [ + mongod, + "--dbpath", + str(cfg.mongo_data), + "--port", + str(cfg.mongo_port), + "--bind_ip", + LOOPBACK, + "--quiet", + ], + cwd=cfg.agentex_dir, + env=dict(os.environ), + ) + if await wait_for_port(proc, LOOPBACK, cfg.mongo_port): + return proc, f"mongodb://{LOOPBACK}:{cfg.mongo_port}" + + await terminate(proc, "mongo") + raise RuntimeError( + f"mongod failed to become ready on {LOOPBACK}:{cfg.mongo_port}. Check the port " + f"is free and mongod can write to {cfg.mongo_data}, or pass --mongo-uri to point " + f"at an external MongoDB." + ) + + +async def provision_otel( + cfg: DevLocalConfig, +) -> tuple[asyncio.subprocess.Process | None, str | None]: + """Start the OpenTelemetry collector if present. Returns (proc, otlp_endpoint). + + Optional: there's no bundled/pip collector, so we launch `otelcol-contrib`/`otelcol` + when on PATH and otherwise continue without telemetry (the app is unaffected). + """ + binary = shutil.which("otelcol-contrib") or shutil.which("otelcol") + if not binary: + logger.warning( + "otel collector not found on PATH — skipping telemetry export (the app " + "runs fine without it). To enable it, start via `./dev.sh local` (installs " + "the otelcol-contrib release binary automatically) or install it yourself " + "from https://github.com/open-telemetry/opentelemetry-collector-releases/releases." + ) + return None, None + + config = cfg.agentex_dir / "otel" / "otel-collector-config.yaml" + if not config.exists(): + logger.warning("otel config %s missing — skipping telemetry.", config) + return None, None + + logger.info( + "Provisioning OpenTelemetry collector on %s:%s ...", LOOPBACK, cfg.otel_port + ) + proc = await spawn( + "otel", + [binary, "--config", str(config)], + cwd=cfg.agentex_dir, + env=dict(os.environ), + ) + if await wait_for_port(proc, LOOPBACK, cfg.otel_port, timeout=15.0): + return proc, f"http://{LOOPBACK}:{cfg.otel_port}" + + logger.warning( + "otel collector did not become ready — continuing without telemetry." + ) + await terminate(proc, "otel") + return None, None + + +def teardown_redis(server: Any) -> None: + import redis as redis_pkg + + pid = getattr(server, "pid", None) + try: + server.shutdown( + save=False + ) # NOT shutdown(now=True, force=True) — broken on this stack + except redis_pkg.exceptions.ConnectionError: + pass # expected: server drops the socket while replying + except Exception as exc: # noqa: BLE001 - best-effort teardown + logger.warning("redislite shutdown raised %s; falling back to signal", exc) + # Daemonized server leaks on hard exit; make sure the pid is gone. + if pid: + try: + os.kill(pid, signal.SIGKILL) + except ProcessLookupError: + pass diff --git a/agentex/scripts/dev_local/supervise.py b/agentex/scripts/dev_local/supervise.py new file mode 100644 index 00000000..62bb2c05 --- /dev/null +++ b/agentex/scripts/dev_local/supervise.py @@ -0,0 +1,125 @@ +"""Subprocess supervision: spawn/stream children, wait for readiness, migrate, terminate.""" + +from __future__ import annotations + +import asyncio +import logging +import sys +import urllib.error +import urllib.request +from pathlib import Path + +from scripts.dev_local.config import LOOPBACK, DevLocalConfig + +logger = logging.getLogger(__name__) + + +async def stream_output(proc: asyncio.subprocess.Process, prefix: str) -> None: + """Stream a child's combined stdout/stderr with a prefix until EOF.""" + assert proc.stdout is not None + while True: + line = await proc.stdout.readline() + if not line: + break + print(f"[{prefix}] {line.decode(errors='replace').rstrip()}", flush=True) + + +async def spawn( + name: str, cmd: list[str], cwd: Path, env: dict[str, str] +) -> asyncio.subprocess.Process: + logger.info("Starting %s: %s", name, " ".join(cmd)) + proc = await asyncio.create_subprocess_exec( + *cmd, + cwd=str(cwd), + env=env, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.STDOUT, + ) + asyncio.create_task(stream_output(proc, name)) + return proc + + +async def wait_for_port( + proc: asyncio.subprocess.Process | None, + host: str, + port: int, + timeout: float = 30.0, +) -> bool: + """Wait until a TCP port accepts connections, or the process dies, or timeout. + + The process-alive check avoids a false positive where our launch failed (e.g. the + port was already taken by a leftover) but something else answers on it. + """ + loop = asyncio.get_running_loop() + deadline = loop.time() + timeout + while loop.time() < deadline: + if proc is not None and proc.returncode is not None: + return False + try: + _, writer = await asyncio.open_connection(host, port) + writer.close() + await writer.wait_closed() + return True + except OSError: + await asyncio.sleep(0.5) + return False + + +async def run_migrations(cfg: DevLocalConfig, env: dict[str, str]) -> None: + """Run `alembic upgrade head` from the migrations dir (sync psycopg2 engine).""" + migrations_dir = cfg.agentex_dir / "database" / "migrations" + logger.info("Running database migrations ...") + proc = await asyncio.create_subprocess_exec( + sys.executable, + "-m", + "alembic", + "upgrade", + "head", + cwd=str(migrations_dir), + env=env, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.STDOUT, + ) + await stream_output(proc, "alembic") + rc = await proc.wait() + if rc != 0: + raise RuntimeError(f"alembic upgrade head failed (exit {rc})") + + +def _probe(url: str) -> bool: + try: + with urllib.request.urlopen(url, timeout=2) as resp: # noqa: S310 - localhost only + return resp.status == 200 + except (urllib.error.URLError, OSError): + return False + + +async def wait_for_health(port: int, timeout: float = 90.0) -> bool: + """Poll /healthz (liveness) until 200 or timeout.""" + loop = asyncio.get_running_loop() + deadline = loop.time() + timeout + url = f"http://{LOOPBACK}:{port}/healthz" + while loop.time() < deadline: + if await loop.run_in_executor(None, _probe, url): + return True + await asyncio.sleep(1.0) + return False + + +async def terminate( + proc: asyncio.subprocess.Process, name: str, grace: float = 5.0 +) -> None: + if proc.returncode is not None: + return + logger.info("Stopping %s ...", name) + try: + proc.terminate() + except ProcessLookupError: + return + try: + await asyncio.wait_for(proc.wait(), timeout=grace) + except TimeoutError: + try: + proc.kill() + except ProcessLookupError: + pass diff --git a/agentex/src/domain/use_cases/agent_api_keys_use_case.py b/agentex/src/domain/use_cases/agent_api_keys_use_case.py index faf10a2a..8c361de6 100644 --- a/agentex/src/domain/use_cases/agent_api_keys_use_case.py +++ b/agentex/src/domain/use_cases/agent_api_keys_use_case.py @@ -29,6 +29,7 @@ ) from src.domain.repositories.agent_repository import DAgentRepository from src.domain.services.authorization_service import DAuthorizationService +from src.utils.acp_url import resolve_acp_url from src.utils.ids import orm_id from src.utils.logging import make_logger @@ -304,8 +305,9 @@ async def forward_agent_request( if error_response: return error_response - # Construct the full URL for the agent request - agent_url = f"{agent.acp_url}/{path.lstrip('/')}" + # Construct the full URL for the agent request. resolve_acp_url rewrites a + # host.docker.internal host to the local override in docker-free mode (no-op otherwise). + agent_url = f"{resolve_acp_url(agent.acp_url)}/{path.lstrip('/')}" if request.url.query: agent_url += f"?{request.url.query}" logger.info( diff --git a/agentex/src/domain/use_cases/agents_acp_use_case.py b/agentex/src/domain/use_cases/agents_acp_use_case.py index bc99111a..49e7dfa5 100644 --- a/agentex/src/domain/use_cases/agents_acp_use_case.py +++ b/agentex/src/domain/use_cases/agents_acp_use_case.py @@ -50,6 +50,7 @@ from src.domain.services.authorization_service import DAuthorizationService from src.domain.services.task_message_service import DTaskMessageService from src.domain.services.task_service import DAgentTaskService +from src.utils.acp_url import resolve_acp_url from src.utils.logging import make_logger logger = make_logger(__name__) @@ -323,23 +324,29 @@ async def _resolve_acp_url( agent: AgentEntity, acp_url_override: str | None = None, ) -> str: - """Resolve the ACP URL for an agent, optionally overriding with a specific URL.""" - if acp_url_override: - return acp_url_override + """Resolve the ACP URL for an agent, optionally overriding with a specific URL. - # Resolve through production deployment if available - if agent.production_deployment_id: + The resolved URL is passed through resolve_acp_url so that, in docker-free + local mode, an agent registered at host.docker.internal is dialed at the + host-reachable override address (a no-op in Docker/prod). + """ + raw = acp_url_override + + # Prefer the production deployment's URL when there's no explicit override. + if raw is None and agent.production_deployment_id: deployment = await self.deployment_repo.get( id=agent.production_deployment_id ) - if deployment.acp_url: - return deployment.acp_url + raw = deployment.acp_url + + # Legacy fallback to the agent's own URL. + if raw is None: + raw = agent.acp_url - # Legacy fallback - if agent.acp_url: - return agent.acp_url + if raw is None: + raise ClientError(f"Agent {agent.id} does not have an ACP URL configured") - raise ClientError(f"Agent {agent.id} does not have an ACP URL configured") + return resolve_acp_url(raw) async def handle_rpc_request( self, diff --git a/agentex/src/temporal/activities/healthcheck_activities.py b/agentex/src/temporal/activities/healthcheck_activities.py index 4c2a22c2..0aa3dd44 100644 --- a/agentex/src/temporal/activities/healthcheck_activities.py +++ b/agentex/src/temporal/activities/healthcheck_activities.py @@ -11,6 +11,7 @@ import httpx from src.domain.entities.agents import AgentStatus from src.domain.repositories.agent_repository import AgentRepository +from src.utils.acp_url import resolve_acp_url from src.utils.logging import make_logger from temporalio import activity @@ -54,6 +55,9 @@ async def check_status_activity(self, agent_id: str, acp_url: str) -> bool: Returns: bool: True if the agent is healthy, False otherwise """ + # In docker-free local mode, rewrite host.docker.internal -> the host-reachable + # override so the healthcheck matches how the request path dials the agent. + acp_url = resolve_acp_url(acp_url) logger.info(f"Checking status of agent {agent_id} via {acp_url}") try: response = await self.http_client.get(f"{acp_url}/healthz", timeout=5) diff --git a/agentex/src/utils/acp_url.py b/agentex/src/utils/acp_url.py new file mode 100644 index 00000000..a283ea51 --- /dev/null +++ b/agentex/src/utils/acp_url.py @@ -0,0 +1,56 @@ +"""Resolve an agent's registered ACP URL to one the backend can actually reach. + +Agents register their ACP URL as ``http://host.docker.internal:`` (the SDK +default) so that a **Docker-based** backend can reach an agent process running on +the host. When the backend itself runs directly on the host — i.e. the docker-free +local dev mode (``./dev.sh local`` / ``python -m scripts.dev_local``) — ``host.docker.internal`` +does not resolve, and every ACP call fails with +``[Errno 8] nodename nor servname provided, or not known``. + +The correct address depends on the *backend's* network topology, which only the +backend knows. So the runner (``scripts.dev_local``) sets ``AGENTEX_ACP_HOST_OVERRIDE`` +(to ``127.0.0.1``) and the backend rewrites the Docker-only sentinel host to that +value wherever it dials an agent. When the env var is unset (Docker / staging / +prod), this is a no-op and the stored URL is used verbatim. +""" + +from __future__ import annotations + +import os +from urllib.parse import urlsplit, urlunsplit + +from src.utils.logging import make_logger + +logger = make_logger(__name__) + +# The hostname Docker Desktop injects so a container can reach the host. It does +# not resolve for a process running directly on the host. +_DOCKER_HOST_SENTINEL = "host.docker.internal" + +# Env var the local (no-Docker) runner sets to the host address agents are actually +# reachable at (e.g. 127.0.0.1). Unset everywhere else, making resolution a no-op. +_ACP_HOST_OVERRIDE_ENV = "AGENTEX_ACP_HOST_OVERRIDE" + + +def resolve_acp_url(url: str) -> str: + """Rewrite a ``host.docker.internal`` ACP host to the local override, if set. + + Returns the URL unchanged when the override env var is unset or the URL does + not use the Docker sentinel host, so it is safe to call on every ACP dial. + """ + override = os.environ.get(_ACP_HOST_OVERRIDE_ENV) + if not override or not url: + return url + + parts = urlsplit(url) + if parts.hostname != _DOCKER_HOST_SENTINEL: + return url + + netloc = f"{override}:{parts.port}" if parts.port is not None else override + rewritten = urlunsplit( + (parts.scheme, netloc, parts.path, parts.query, parts.fragment) + ) + logger.info( + "Rewriting ACP host for local (no-Docker) mode: %s -> %s", url, rewritten + ) + return rewritten diff --git a/dev.sh b/dev.sh index 6e3d1a42..0d327726 100755 --- a/dev.sh +++ b/dev.sh @@ -5,7 +5,8 @@ # Delegates to existing Makefiles - this is an orchestration layer, not a replacement # # Usage: -# ./dev.sh Start all services +# ./dev.sh Start all services (Docker backend + frontend) +# ./dev.sh local Start all services WITHOUT Docker (embedded pg/redis, local temporal/mongo/otel) # ./dev.sh setup Install all prerequisites (macOS) # ./dev.sh stop Stop all services # ./dev.sh logs Show logs @@ -18,6 +19,7 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" LOG_DIR="$SCRIPT_DIR/.dev-logs" BACKEND_PID_FILE="$LOG_DIR/backend.pid" FRONTEND_PID_FILE="$LOG_DIR/frontend.pid" +MODE_FILE="$LOG_DIR/mode" # records "docker" or "local" so stop/status know how to act # Colors for output RED='\033[0;31m' @@ -166,7 +168,8 @@ install_prerequisites() { echo "========================================" echo "" echo "You can now run:" - echo " ./dev.sh # Start the development environment" + echo " ./dev.sh # Start the development environment (Docker)" + echo " ./dev.sh local # Start without Docker (embedded services)" echo "" echo "Once running, create your first agent:" echo " agentex init # Create a new agent" @@ -202,17 +205,121 @@ ensure_prerequisites() { needs_install=true fi - # Docker must be running (can't auto-install - user chooses Docker Desktop or Rancher) + if [ "$needs_install" = true ]; then + log_info "Some prerequisites missing - installing automatically..." + install_prerequisites + else + log_success "All prerequisites found" + fi +} + +require_docker() { + # Docker must be running for the Docker-based backend (can't auto-install - + # the user chooses Docker Desktop or Rancher). if ! docker info &> /dev/null; then log_error "Docker daemon is not running. Please start Docker Desktop or Rancher Desktop." + log_info "Tip: run './dev.sh local' to start the backend WITHOUT Docker." exit 1 fi +} - if [ "$needs_install" = true ]; then - log_info "Some prerequisites missing - installing automatically..." - install_prerequisites +# Install the OpenTelemetry Collector (contrib). It is NOT distributed via Homebrew, +# so we fetch the official release binary for this OS/arch. Best-effort: OTel is +# optional, so any failure returns non-zero and the caller just warns. +install_otel_collector() { + local ver="0.156.0" + local os arch + case "$(uname -s)" in + Darwin) os=darwin ;; + Linux) os=linux ;; + *) log_warn "Unsupported OS for otel auto-install."; return 1 ;; + esac + case "$(uname -m)" in + arm64|aarch64) arch=arm64 ;; + x86_64|amd64) arch=amd64 ;; + *) log_warn "Unsupported arch for otel auto-install."; return 1 ;; + esac + + local tarball="otelcol-contrib_${ver}_${os}_${arch}.tar.gz" + local base="https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v${ver}" + + local dest + if [ -w /opt/homebrew/bin ]; then dest=/opt/homebrew/bin + elif [ -w /usr/local/bin ]; then dest=/usr/local/bin + else dest="$HOME/.local/bin"; mkdir -p "$dest"; fi + + log_info "Installing OpenTelemetry collector v${ver} (release binary; not in Homebrew)..." + local tmp; tmp="$(mktemp -d)" + if ! curl -fL -o "$tmp/otel.tar.gz" "${base}/${tarball}"; then + rm -rf "$tmp"; return 1 + fi + # Verify against the release checksums file when available. + if curl -fsL -o "$tmp/checksums.txt" "${base}/otelcol-contrib_${ver}_checksums.txt"; then + local expected actual + expected="$(grep " ${tarball}\$" "$tmp/checksums.txt" | awk '{print $1}')" + actual="$(shasum -a 256 "$tmp/otel.tar.gz" | awk '{print $1}')" + if [ -n "$expected" ] && [ "$expected" != "$actual" ]; then + log_warn "otel collector checksum mismatch; skipping install." + rm -rf "$tmp"; return 1 + fi + fi + if ! tar xzf "$tmp/otel.tar.gz" -C "$tmp" otelcol-contrib; then + rm -rf "$tmp"; return 1 + fi + install -m 0755 "$tmp/otelcol-contrib" "$dest/otelcol-contrib" || { rm -rf "$tmp"; return 1; } + rm -rf "$tmp" + # Make it findable this session if we landed in ~/.local/bin. + case ":$PATH:" in *":$dest:"*) : ;; *) export PATH="$dest:$PATH" ;; esac + log_success "otel collector installed to $dest/otelcol-contrib" +} + +ensure_local_service_binaries() { + # `local` mode runs Postgres/Redis/Temporal with no install (bundled / + # auto-downloaded). MongoDB is REQUIRED for the full stack (the Temporal worker + # needs it) so we always ensure it and FAIL FAST if we can't. The OTel collector is + # genuinely optional, so its install is best-effort. Honors the runner opt-out + # flags forwarded in "$@": --lean/--no-otel skip the OTel install, and an external + # --mongo-uri means the runner won't launch a local mongod (so no local install). + local want_mongo=true want_otel=true + for arg in "$@"; do + case "$arg" in + --no-otel) want_otel=false ;; + --lean) want_otel=false ;; + --mongo-uri|--mongo-uri=*) want_mongo=false ;; + esac + done + + # ----- MongoDB: required (unless pointing at an external --mongo-uri) ----- + if [ "$want_mongo" = true ]; then + if command -v mongod &> /dev/null; then + log_success "mongod already installed" + elif command -v brew &> /dev/null; then + log_info "Installing MongoDB (mongod) via Homebrew (required for local mode)..." + brew tap mongodb/brew &> /dev/null || true + # Modern Homebrew refuses to install from an untrusted tap; trust it first. + brew trust mongodb/brew &> /dev/null || true + if ! brew install mongodb-community; then + log_error "MongoDB install failed, but it is REQUIRED for local mode." + log_error "Install mongod manually, or pass --mongo-uri to use an external MongoDB." + exit 1 + fi + else + log_error "mongod is not installed and Homebrew is unavailable to install it." + log_error "MongoDB is REQUIRED for local mode — install mongod (https://www.mongodb.com/docs/manual/administration/install-community/)," + log_error "or pass --mongo-uri to use an external MongoDB." + exit 1 + fi else - log_success "All prerequisites found" + log_info "Using external MongoDB (--mongo-uri); skipping local mongod install." + fi + + # ----- OTel collector: optional ----- + if [ "$want_otel" = true ]; then + if command -v otelcol-contrib &> /dev/null || command -v otelcol &> /dev/null; then + log_success "otel collector already installed" + else + install_otel_collector || log_warn "otel collector install failed; telemetry will be skipped (the app runs fine without it)." + fi fi } @@ -229,7 +336,7 @@ setup_log_dir() { } start_backend() { - log_info "Starting backend services..." + log_info "Starting backend services (Docker)..." cd "$SCRIPT_DIR/agentex" @@ -237,6 +344,27 @@ start_backend() { make dev > "$LOG_DIR/backend.log" 2>&1 & local pid=$! echo $pid > "$BACKEND_PID_FILE" + echo "docker" > "$MODE_FILE" + + log_success "Backend starting (PID: $pid)" + log_info "Backend logs: tail -f $LOG_DIR/backend.log" + + cd "$SCRIPT_DIR" +} + +start_backend_local() { + log_info "Starting backend services (local, no Docker)..." + + cd "$SCRIPT_DIR/agentex" + + # Run the docker-free runner DIRECTLY via `uv run` (not `make`) so a SIGTERM + # from `./dev.sh stop` is forwarded straight to the Python supervisor, which + # tears down the embedded datastores cleanly. --group dev-local pulls + # pgserver/redislite/greenlet. + uv run --group dev-local python -m scripts.dev_local "$@" > "$LOG_DIR/backend.log" 2>&1 & + local pid=$! + echo $pid > "$BACKEND_PID_FILE" + echo "local" > "$MODE_FILE" log_success "Backend starting (PID: $pid)" log_info "Backend logs: tail -f $LOG_DIR/backend.log" @@ -244,9 +372,33 @@ start_backend() { cd "$SCRIPT_DIR" } +# Kill a PID and all its descendants. The frontend runs as make -> npm -> next, so +# killing only the recorded (make) PID orphans the npm/next children; over repeated +# runs those pile up and hold ports 3000, 3001, 3002, ... This walks the tree. +kill_tree() { + local pid=$1 + [ -z "$pid" ] && return 0 + local child + for child in $(pgrep -P "$pid" 2>/dev/null); do + kill_tree "$child" + done + kill "$pid" 2>/dev/null || true +} + +# Sweep any stray `next dev` server belonging to THIS repo's UI. Scoped by absolute +# path so it never touches another project's dev server, and catches orphans whose +# parent (make/npm) has already exited. +sweep_stray_frontends() { + pkill -f "$SCRIPT_DIR/agentex-ui/node_modules/.bin/next" 2>/dev/null || true +} + start_frontend() { log_info "Starting frontend..." + # Clear any stale frontend from a previous run first, so `next dev` can bind :3000 + # instead of silently climbing to :3001/:3002/... + sweep_stray_frontends + cd "$SCRIPT_DIR/agentex-ui" # Delegate to Makefile (handles deps + dev server) @@ -283,29 +435,54 @@ wait_for_backend() { log_info "Check logs with: ./dev.sh logs" } +# Echo the recorded backend mode ("docker" or "local"), defaulting to docker. +current_mode() { + if [ -f "$MODE_FILE" ]; then cat "$MODE_FILE"; else echo "docker"; fi +} + stop_services() { log_info "Stopping all services..." - # Stop frontend + # Stop frontend: kill the whole make -> npm -> next tree, then sweep any strays + # orphaned by earlier runs (their parent may already be gone). if [ -f "$FRONTEND_PID_FILE" ]; then - local frontend_pid=$(cat "$FRONTEND_PID_FILE") - if kill -0 "$frontend_pid" 2>/dev/null; then - kill "$frontend_pid" 2>/dev/null || true - log_success "Frontend stopped" - fi + kill_tree "$(cat "$FRONTEND_PID_FILE")" rm -f "$FRONTEND_PID_FILE" fi - - # Stop backend (docker compose) - cd "$SCRIPT_DIR/agentex" - docker compose down 2>/dev/null || true - log_success "Backend stopped" - - if [ -f "$BACKEND_PID_FILE" ]; then - rm -f "$BACKEND_PID_FILE" + sweep_stray_frontends + log_success "Frontend stopped" + + # Stop backend according to how it was started. + local mode + mode=$(current_mode) + + if [ "$mode" = "local" ]; then + if [ -f "$BACKEND_PID_FILE" ]; then + local backend_pid + backend_pid=$(cat "$BACKEND_PID_FILE") + if kill -0 "$backend_pid" 2>/dev/null; then + log_info "Stopping local backend (PID: $backend_pid) — tearing down embedded services..." + kill -TERM "$backend_pid" 2>/dev/null || true + local waited=0 + while kill -0 "$backend_pid" 2>/dev/null && [ "$waited" -lt 20 ]; do + sleep 1 + waited=$((waited + 1)) + done + if kill -0 "$backend_pid" 2>/dev/null; then + log_warn "Backend did not stop gracefully; forcing." + kill -9 "$backend_pid" 2>/dev/null || true + fi + fi + fi + log_success "Backend stopped" + else + cd "$SCRIPT_DIR/agentex" + docker compose down 2>/dev/null || true + cd "$SCRIPT_DIR" + log_success "Backend stopped" fi - cd "$SCRIPT_DIR" + rm -f "$BACKEND_PID_FILE" "$MODE_FILE" log_success "All services stopped" } @@ -328,37 +505,55 @@ show_logs() { } show_status() { + local mode + mode=$(current_mode) + echo "" echo "=== Agentex Development Status ===" echo "" # Check backend - echo -n "Backend (Docker): " - if docker compose -f "$SCRIPT_DIR/agentex/docker-compose.yml" ps 2>/dev/null | grep -q "Up"; then - echo -e "${GREEN}Running${NC}" + echo -n "Backend ($mode): " + if [ "$mode" = "local" ]; then + if [ -f "$BACKEND_PID_FILE" ] && kill -0 "$(cat "$BACKEND_PID_FILE")" 2>/dev/null; then + echo -e "${GREEN}Running${NC}" + else + echo -e "${RED}Stopped${NC}" + fi else - echo -e "${RED}Stopped${NC}" + if docker compose -f "$SCRIPT_DIR/agentex/docker-compose.yml" ps 2>/dev/null | grep -q "Up"; then + echo -e "${GREEN}Running${NC}" + else + echo -e "${RED}Stopped${NC}" + fi fi - # Check frontend + # Check frontend — report by what's actually serving :3000, not just the recorded + # PID (make/npm can exit while next keeps serving, and vice versa). echo -n "Frontend (Next.js): " - if [ -f "$FRONTEND_PID_FILE" ] && kill -0 "$(cat "$FRONTEND_PID_FILE")" 2>/dev/null; then + if lsof -nP -iTCP:3000 -sTCP:LISTEN &> /dev/null \ + || { [ -f "$FRONTEND_PID_FILE" ] && kill -0 "$(cat "$FRONTEND_PID_FILE")" 2>/dev/null; }; then echo -e "${GREEN}Running${NC}" else echo -e "${RED}Stopped${NC}" fi + # Temporal UI port differs: local dev server uses 8233, Docker UI uses 8080. + local temporal_ui="http://localhost:8080" + [ "$mode" = "local" ] && temporal_ui="http://localhost:8233" + echo "" echo "=== URLs ===" echo "Frontend: http://localhost:3000" echo "Backend API: http://localhost:5003" echo "Swagger: http://localhost:5003/swagger" - echo "Temporal UI: http://localhost:8080" + echo "Temporal UI: $temporal_ui" echo "" } start_all() { ensure_prerequisites + require_docker check_redis_conflict setup_log_dir @@ -403,11 +598,60 @@ start_all() { echo "" } +start_all_local() { + ensure_prerequisites # no Docker requirement in local mode + ensure_local_service_binaries "$@" # ensure mongod (required) + otel (optional); honors --mongo-uri/--lean + setup_log_dir + + echo "" + echo "========================================" + echo " Starting Agentex Development Env (Local, no Docker) " + echo "========================================" + echo "" + + start_backend_local "$@" + start_frontend + + echo "" + log_info "Services are starting up (embedded Postgres/Redis + Temporal + MongoDB [required] + OTel [optional])..." + echo "" + echo "=== URLs (will be ready shortly) ===" + echo "Frontend: http://localhost:3000" + echo "Backend API: http://localhost:5003" + echo "Swagger: http://localhost:5003/swagger" + echo "Temporal UI: http://localhost:8233" + echo "" + echo "=== Commands ===" + echo "./dev.sh logs - View all logs" + echo "./dev.sh logs backend - View backend logs" + echo "./dev.sh logs frontend - View frontend logs" + echo "./dev.sh status - Check service status" + echo "./dev.sh stop - Stop all services" + echo "" + + wait_for_backend + + echo "" + log_success "Development environment is ready!" + log_info "Open http://localhost:3000 to access the UI" + echo "" + log_info "To create an agent, in a new terminal run:" + echo " agentex init" + echo " cd " + echo " uv venv && source .venv/bin/activate && uv sync" + echo " agentex agents run --manifest manifest.yaml" + echo "" +} + # Main command router case "${1:-start}" in start|"") start_all ;; + local) + shift # drop the 'local' subcommand so only flags reach dev_local.py + start_all_local "$@" + ;; setup) install_prerequisites ;; @@ -421,9 +665,17 @@ case "${1:-start}" in show_status ;; restart) + # Restart in whatever mode was last used. stop_services clears MODE_FILE, + # so capture the mode first. (Local restarts use default flags — the original + # runner flags are not persisted.) + restart_mode=$(current_mode) stop_services sleep 2 - start_all + if [ "$restart_mode" = "local" ]; then + start_all_local + else + start_all + fi ;; help|--help|-h) echo "Agentex Development Script" @@ -431,9 +683,15 @@ case "${1:-start}" in echo "Usage: $0 [command]" echo "" echo "Commands:" - echo " (none) Start everything (auto-installs prerequisites if needed)" + echo " (none) Start everything with Docker (auto-installs prerequisites if needed)" + echo " local Start everything WITHOUT Docker (embedded pg/redis + Temporal + MongoDB + OTel)" + echo " MongoDB is required for the full stack and is auto-installed; OTel is optional." + echo " Flags pass through to the runner, e.g.:" + echo " ./dev.sh local --full # whole stack (default)" + echo " ./dev.sh local --lean # Postgres + Redis + API + MongoDB only" + echo " ./dev.sh local --no-temporal # skip Temporal + worker" echo " setup Just install prerequisites without starting" - echo " stop Stop all services" + echo " stop Stop all services (Docker or local)" echo " restart Restart all services" echo " logs Show logs (logs backend|frontend|all)" echo " status Check service status" diff --git a/uv.lock b/uv.lock index 41c44065..3832d2a3 100644 --- a/uv.lock +++ b/uv.lock @@ -95,14 +95,21 @@ dev = [ { name = "ruff", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, { name = "vulture", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, ] +dev-local = [ + { name = "greenlet", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pgserver", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "redislite", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] test = [ { name = "asyncpg", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, { name = "factory-boy", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, { name = "greenlet", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, { name = "httpx", extra = ["http2"], marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pgserver", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, { name = "pytest", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, { name = "pytest-asyncio", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, { name = "pytest-cov", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "redislite", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, { name = "testcontainers", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, ] @@ -145,14 +152,21 @@ dev = [ { name = "ruff", specifier = ">=0.3.4" }, { name = "vulture", specifier = ">=2.14" }, ] +dev-local = [ + { name = "greenlet", specifier = ">=3.2.3" }, + { name = "pgserver", specifier = ">=0.1.4" }, + { name = "redislite", specifier = ">=6.2.912183" }, +] test = [ { name = "asyncpg", specifier = ">=0.29.0" }, { name = "factory-boy", specifier = ">=3.3.0,<4" }, { name = "greenlet", specifier = ">=3.2.3" }, { name = "httpx", extras = ["http2"], specifier = ">=0.27.0,<0.29" }, + { name = "pgserver", specifier = ">=0.1.4" }, { name = "pytest", specifier = ">=8.3.3,<9" }, { name = "pytest-asyncio", specifier = ">=1.0.0,<2" }, { name = "pytest-cov", specifier = ">=5.0.0,<6" }, + { name = "redislite", specifier = ">=6.2.912183" }, { name = "testcontainers", specifier = ">=4.0.0,<5" }, ] @@ -740,6 +754,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/e4/72/42e900510195b23a56bde950d26a51f8b723846bfcaa0286e90287f0422b/fastapi-0.135.1-py3-none-any.whl", hash = "sha256:46e2fc5745924b7c840f71ddd277382af29ce1cdb7d5eab5bf697e3fb9999c9e", size = 116999, upload-time = "2026-03-01T18:18:30.831Z" }, ] +[[package]] +name = "fasteners" +version = "0.20" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/2d/18/7881a99ba5244bfc82f06017316ffe93217dbbbcfa52b887caa1d4f2a6d3/fasteners-0.20.tar.gz", hash = "sha256:55dce8792a41b56f727ba6e123fcaee77fd87e638a6863cec00007bfea84c8d8", size = 25087, upload-time = "2025-08-11T10:19:37.785Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/51/ac/e5d886f892666d2d1e5cb8c1a41146e1d79ae8896477b1153a21711d3b44/fasteners-0.20-py3-none-any.whl", hash = "sha256:9422c40d1e350e4259f509fb2e608d6bc43c0136f79a00db1b49046029d0b3b7", size = 18702, upload-time = "2025-08-11T10:19:35.716Z" }, +] + [[package]] name = "fastuuid" version = "0.14.0" @@ -1727,6 +1750,21 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/9e/c3/059298687310d527a58bb01f3b1965787ee3b40dce76752eda8b44e9a2c5/pexpect-4.9.0-py2.py3-none-any.whl", hash = "sha256:7236d1e080e4936be2dc3e326cec0af72acf9212a7e1d060210e70a47e253523", size = 63772, upload-time = "2023-11-25T06:56:14.81Z" }, ] +[[package]] +name = "pgserver" +version = "0.1.4" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "fasteners", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "platformdirs", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "psutil", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +wheels = [ + { url = "https://files.pythonhosted.org/packages/92/e3/9f8eea535ab4f2906a9924eccc5fb3a7bcff3e02222fbe338d9c24639750/pgserver-0.1.4-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:dc34f88561b18bc08edd98a84528f99a3720fe713a4e39a4a6210a4d009fe465", size = 10378168, upload-time = "2024-06-08T18:41:40.377Z" }, + { url = "https://files.pythonhosted.org/packages/23/57/94b5f05a23d0fa683c01bfc2d785224057a9eaf0eb00cbfd6da19547012f/pgserver-0.1.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:780fa89f26a960cca0215caf471e70848dd8597bd8ceaeba7faf42170278980c", size = 9822137, upload-time = "2024-06-08T18:41:43.017Z" }, + { url = "https://files.pythonhosted.org/packages/cf/f1/c9d717f66d2e4a27801577e1ae233c25aa88db875c586ac3ebe7d73b6b75/pgserver-0.1.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1a5d07c61d51f2abfef4ef61e2ef5cd014b994f7e09de8d3c140d2cf370e84a8", size = 11266316, upload-time = "2024-06-08T18:41:48.033Z" }, +] + [[package]] name = "platformdirs" version = "4.4.0" @@ -2128,6 +2166,20 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/7f/26/5c5fa0e83c3621db835cfc1f1d789b37e7fa99ed54423b5f519beb931aa7/redis-5.3.1-py3-none-any.whl", hash = "sha256:dc1909bd24669cc31b5f67a039700b16ec30571096c5f1f0d9d2324bff31af97", size = 272833, upload-time = "2025-07-25T08:06:26.317Z" }, ] +[[package]] +name = "redislite" +version = "6.2.912183" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "psutil", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "redis", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "setuptools", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/af/7d/b6fc82af45ed2aa31f6c8b2035df009ff9fd8be01ad8a691155cc167f1f2/redislite-6.2.912183.tar.gz", hash = "sha256:40642495cc53bd5ca3cc186355a7235795ef4b36dd167cb77c7e6837d23e5dd6", size = 2555087, upload-time = "2023-12-01T22:19:28.526Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0b/b9/95ce807b397428704bfc01769848d0caa8417e7d3753785ccc14c3b05097/redislite-6.2.912183-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6610f867c2c1b2022613c093d56cfe8293bdf6e081d2259d62b5f5318487c30e", size = 6071189, upload-time = "2023-12-01T22:22:13.983Z" }, +] + [[package]] name = "referencing" version = "0.36.2" @@ -2303,6 +2355,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/af/12/55f47289a0ae1065e4115bc8018b2f01df0b6560b07bacfc0dcf6c3bdcbe/scale_gp_beta-0.1.0a20-py3-none-any.whl", hash = "sha256:482385ee6c3b912aecf70795948ac45b215a4d19feba60f67d6e10c4312440c6", size = 157906, upload-time = "2025-06-12T14:46:16.344Z" }, ] +[[package]] +name = "setuptools" +version = "83.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/34/26/f5d29e25ffdb535afef2d35cdb55b325298f96debd670da4c325e08d70f4/setuptools-83.0.0.tar.gz", hash = "sha256:025bccbbf0fa05b6192bc64ae1e7b16e001fd6d6d4d5de03c97b1c1ade523bef", size = 1154254, upload-time = "2026-07-04T15:31:22.699Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5d/40/e1e72872c6354b306daef1703549e8e83b4d43cfea356311bf722a043752/setuptools-83.0.0-py3-none-any.whl", hash = "sha256:29b23c360f22f414dc7336bb39178cc7bcbf6021ed2733cde173f09dba19abb3", size = 1008090, upload-time = "2026-07-04T15:31:20.885Z" }, +] + [[package]] name = "shellingham" version = "1.5.4" From fd6b808fb177f1aa45ec25103289f11a42f4d9f9 Mon Sep 17 00:00:00 2001 From: Ari Nguyen Date: Thu, 16 Jul 2026 12:30:00 -0700 Subject: [PATCH 2/7] refactor --- agentex/scripts/dev_local/config.py | 8 ++++++- agentex/scripts/dev_local/services.py | 13 +++++----- dev.sh | 34 +++++++++++++++++++++++++++ 3 files changed, 48 insertions(+), 7 deletions(-) diff --git a/agentex/scripts/dev_local/config.py b/agentex/scripts/dev_local/config.py index 2181d15e..781ea8fe 100644 --- a/agentex/scripts/dev_local/config.py +++ b/agentex/scripts/dev_local/config.py @@ -17,6 +17,12 @@ # purpose — see runner.run; banner URLs say "localhost" for readability.) LOOPBACK = "127.0.0.1" +# Name of both the Postgres and Mongo databases the local stack creates. A constant, not +# an env var: the runner creates the database and hands the matching DATABASE_URL / +# MONGODB_DATABASE_NAME to the app, so the two only need to agree — there's nothing for an +# operator to tune. +DB_NAME = "agentex" + @dataclass class DevLocalConfig: @@ -186,7 +192,7 @@ def build_env( env["ENVIRONMENT"] = "development" env["DATABASE_URL"] = database_url env["REDIS_URL"] = redis_url - env["MONGODB_DATABASE_NAME"] = "agentex" + env["MONGODB_DATABASE_NAME"] = DB_NAME # Agents register their ACP URL at host.docker.internal (SDK default, for a Docker # backend); that name doesn't resolve for this host-process backend, so have it dial diff --git a/agentex/scripts/dev_local/services.py b/agentex/scripts/dev_local/services.py index 78d736fa..d4040665 100644 --- a/agentex/scripts/dev_local/services.py +++ b/agentex/scripts/dev_local/services.py @@ -10,14 +10,14 @@ from typing import Any from urllib.parse import urlsplit -from scripts.dev_local.config import LOOPBACK, DevLocalConfig +from scripts.dev_local.config import DB_NAME, LOOPBACK, DevLocalConfig from scripts.dev_local.supervise import spawn, terminate, wait_for_port logger = logging.getLogger(__name__) def provision_postgres(cfg: DevLocalConfig) -> tuple[Any, str]: - """Start an embedded Postgres and ensure the 'agentex' database exists. + """Start an embedded Postgres and ensure the application database (DB_NAME) exists. The returned socket-form `postgresql://` URL works for both the async app (asyncpg) and alembic (psycopg2) — adjust_db_url normalizes the scheme for each. @@ -33,12 +33,13 @@ def provision_postgres(cfg: DevLocalConfig) -> tuple[Any, str]: ) # psql() doesn't stop-on-error and CREATE DATABASE can't be IF NOT EXISTS, so guard. - existing = server.psql("SELECT 1 FROM pg_database WHERE datname = 'agentex';") + # DB_NAME is a trusted constant (not user input), so interpolating it is safe here. + existing = server.psql(f"SELECT 1 FROM pg_database WHERE datname = '{DB_NAME}';") if "(1 row)" not in existing: - server.psql("CREATE DATABASE agentex;") - logger.info("Created database 'agentex'.") + server.psql(f"CREATE DATABASE {DB_NAME};") + logger.info("Created database '%s'.", DB_NAME) - return server, server.get_uri(database="agentex") + return server, server.get_uri(database=DB_NAME) def provision_redis(cfg: DevLocalConfig) -> tuple[Any, str]: diff --git a/dev.sh b/dev.sh index 0d327726..2d71c96f 100755 --- a/dev.sh +++ b/dev.sh @@ -598,7 +598,41 @@ start_all() { echo "" } +# The dev_local runner accepts only optional flags — never a positional argument. Catch a +# stray word up front (almost always a top-level subcommand mistakenly typed after `local`, +# e.g. `./dev.sh local status`) with a clear hint, instead of forwarding it into the +# backgrounded runner where argparse rejects it and the error is buried in backend.log. +validate_local_args() { + local expect_value=false arg + for arg in "$@"; do + if [ "$expect_value" = true ]; then + expect_value=false # this token is the value for the preceding option + continue + fi + case "$arg" in + # Options that consume the next token as their value. + --mongo-uri|--port|--redis-port|--temporal-port|--ui-port|--mongo-port|--otel-port|--data-dir) + expect_value=true + ;; + -*) + : # some other flag (boolean, --opt=value, -h) — let the runner validate it + ;; + *) + log_error "'$arg' is not a valid option for './dev.sh local'." + case "$arg" in + start|setup|stop|logs|status|restart|help) + log_info "Did you mean './dev.sh $arg'? (start/setup/stop/logs/status/restart/help run on their own, not after 'local'.)" + ;; + esac + log_info "Run './dev.sh help' for local flags (e.g. --lean, --no-temporal, --mongo-uri )." + exit 1 + ;; + esac + done +} + start_all_local() { + validate_local_args "$@" # fail fast on a stray positional before any side effects ensure_prerequisites # no Docker requirement in local mode ensure_local_service_binaries "$@" # ensure mongod (required) + otel (optional); honors --mongo-uri/--lean setup_log_dir From 1269052f06bfd691ab602ce4eab471fd99b663f4 Mon Sep 17 00:00:00 2001 From: Ari Nguyen Date: Thu, 16 Jul 2026 16:57:14 -0700 Subject: [PATCH 3/7] fix comments --- CLAUDE.md | 18 +-- README.md | 34 +++--- agentex/.gitignore | 4 +- agentex/Makefile | 8 +- agentex/pyproject.toml | 4 +- agentex/scripts/dev_local/__main__.py | 6 - .../{dev_local => dev_nodocker}/__init__.py | 14 +-- agentex/scripts/dev_nodocker/__main__.py | 6 + .../{dev_local => dev_nodocker}/config.py | 18 +-- .../{dev_local => dev_nodocker}/runner.py | 17 ++- .../{dev_local => dev_nodocker}/services.py | 16 +-- .../{dev_local => dev_nodocker}/supervise.py | 4 +- .../domain/use_cases/agents_acp_use_case.py | 8 +- agentex/src/utils/acp_url.py | 4 +- dev.sh | 104 +++++++++--------- uv.lock | 4 +- 16 files changed, 140 insertions(+), 129 deletions(-) delete mode 100644 agentex/scripts/dev_local/__main__.py rename agentex/scripts/{dev_local => dev_nodocker}/__init__.py (78%) create mode 100644 agentex/scripts/dev_nodocker/__main__.py rename agentex/scripts/{dev_local => dev_nodocker}/config.py (93%) rename agentex/scripts/{dev_local => dev_nodocker}/runner.py (94%) rename agentex/scripts/{dev_local => dev_nodocker}/services.py (94%) rename agentex/scripts/{dev_local => dev_nodocker}/supervise.py (96%) diff --git a/CLAUDE.md b/CLAUDE.md index 28b872dd..8a2ce21e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -55,23 +55,23 @@ Other commands: ./dev.sh restart # Restart all services ``` -Docker-free local mode (host processes + embedded datastores, no Docker): +Docker-free mode (host processes + embedded datastores, no Docker): ```bash -./dev.sh local # whole stack without Docker (or: make dev-local) -./dev.sh local --lean # Postgres + Redis + API + MongoDB only -./dev.sh local --mongo-uri # use an external MongoDB instead of a local mongod +./dev.sh no-docker # whole stack without Docker (or: make dev-no-docker) +./dev.sh no-docker --lean # Postgres + Redis + API + MongoDB only +./dev.sh no-docker --mongo-uri # use an external MongoDB instead of a local mongod ``` -> **MongoDB is required for the full local stack** and is always started — the Temporal +> **MongoDB is required for the full no-docker stack** and is always started — the Temporal > worker builds Mongo-backed repositories at startup, so a missing/unreachable Mongo > makes the runner fail fast (with an install message) rather than crash the worker. -> `./dev.sh local` auto-installs `mongod`; `make dev-local` / direct `python -m -> scripts.dev_local` do not, so install `mongod` yourself or pass `--mongo-uri ` to -> point at an external MongoDB. In local mode the Temporal UI is on :8233 (Docker mode +> `./dev.sh no-docker` auto-installs `mongod`; `make dev-no-docker` / direct `python -m +> scripts.dev_nodocker` do not, so install `mongod` yourself or pass `--mongo-uri ` to +> point at an external MongoDB. In no-docker mode the Temporal UI is on :8233 (Docker mode > uses :8080). > > Agents register their ACP URL as `host.docker.internal` (for a Docker backend), which -> a host-process backend can't resolve. Local mode sets `AGENTEX_ACP_HOST_OVERRIDE=127.0.0.1` +> a host-process backend can't resolve. No-docker mode sets `AGENTEX_ACP_HOST_OVERRIDE=127.0.0.1` > and the backend rewrites `host.docker.internal` → that value when dialing agents > (`src/utils/acp_url.py`, applied in the ACP request path + Temporal healthcheck), so > default-scaffolded agents work without manifest edits. diff --git a/README.md b/README.md index 497ea57d..1253a6ab 100644 --- a/README.md +++ b/README.md @@ -108,7 +108,7 @@ To do this, you just need to spin up the [Agentex Server](https://github.com/sca Just run one command — no Docker required: ```bash -./dev.sh local +./dev.sh no-docker ``` That's it. This will automatically: @@ -119,9 +119,9 @@ That's it. This will automatically: dev server, a local MongoDB, and an optional OTel collector - Wait for everything to be healthy -> Prefer containers? `./dev.sh` runs the same stack with Docker instead (needs Docker -> Desktop or Rancher Desktop) — see [Other commands](#other-commands) below. See -> [Docker-free local mode](#docker-free-local-mode) for `local` flags and details. +> Prefer containers? `./dev.sh` (or `./dev.sh docker`) runs the same stack with Docker +> instead (needs Docker Desktop or Rancher Desktop) — see [Other commands](#other-commands) +> below. See [Docker-free mode](#docker-free-mode) for `no-docker` flags and details. Once ready: | Service | URL | @@ -142,31 +142,31 @@ Once ready: ./dev.sh restart # Restart all services ``` -#### Docker-free local mode +#### Docker-free mode -`./dev.sh local` accepts flags to trim the stack: +`./dev.sh no-docker` accepts flags to trim the stack: ```bash -./dev.sh local # whole stack, no Docker -./dev.sh local --lean # Postgres + Redis + API + MongoDB only (no Temporal/OTel) -./dev.sh local --no-temporal # skip Temporal + the worker -./dev.sh local --mongo-uri # use an external MongoDB instead of a local mongod +./dev.sh no-docker # whole stack, no Docker +./dev.sh no-docker --lean # Postgres + Redis + API + MongoDB only (no Temporal/OTel) +./dev.sh no-docker --no-temporal # skip Temporal + the worker +./dev.sh no-docker --mongo-uri # use an external MongoDB instead of a local mongod ``` -> **MongoDB is required for the full local stack** and is always started. The Temporal -> worker needs it, so `./dev.sh local` auto-installs `mongod` (via Homebrew) and startup +> **MongoDB is required for the full no-docker stack** and is always started. The Temporal +> worker needs it, so `./dev.sh no-docker` auto-installs `mongod` (via Homebrew) and startup > **fails fast** with an install message if it can't be made available. Point at an > existing MongoDB with `--mongo-uri ` to skip the local `mongod`. The OTel -> collector is optional; if it's absent the runner continues without telemetry. In local -> mode the Temporal UI is at http://localhost:8233 (not :8080). Same `stop` / `status` / +> collector is optional; if it's absent the runner continues without telemetry. In +> no-docker mode the Temporal UI is at http://localhost:8233 (not :8080). Same `stop` / `status` / > `logs` / `restart` commands apply. -**Connecting agents in local mode.** Agents scaffolded by `agentex init` register their +**Connecting agents in no-docker mode.** Agents scaffolded by `agentex init` register their ACP URL as `http://host.docker.internal:` (so a *Docker* backend can reach an -agent on the host). A host-process backend can't resolve that name, so local mode sets +agent on the host). A host-process backend can't resolve that name, so no-docker mode sets `AGENTEX_ACP_HOST_OVERRIDE=127.0.0.1` and the backend automatically rewrites `host.docker.internal` → `127.0.0.1` when dialing agents (and their healthchecks). So a -default-scaffolded agent works with `./dev.sh local` **without editing its manifest**. +default-scaffolded agent works with `./dev.sh no-docker` **without editing its manifest**. (You can still set `local_development.agent.host_address: localhost` in the manifest if you prefer; both work.) diff --git a/agentex/.gitignore b/agentex/.gitignore index 2293a6fa..7293a930 100644 --- a/agentex/.gitignore +++ b/agentex/.gitignore @@ -46,4 +46,6 @@ next-env.d.ts .codeartifact-pip-conf -docs/site/ \ No newline at end of file +docs/site/ + +.dev-nodocker/ \ No newline at end of file diff --git a/agentex/Makefile b/agentex/Makefile index 58e12a0d..c66b5a5e 100644 --- a/agentex/Makefile +++ b/agentex/Makefile @@ -41,14 +41,14 @@ dev: install-dev ## Start development server with Docker Compose @echo "🚀 Starting development server with Docker Compose..." docker compose up --build -dev-local: install-dev ## Start development server locally (no Docker) +dev-no-docker: install-dev ## Start development server locally (no Docker) @echo "🚀 Starting development server locally without Docker..." @echo "ℹ️ MongoDB is REQUIRED for the full stack. If 'mongod' is not on your PATH," @echo " install it (brew tap mongodb/brew && brew install mongodb-community) or point" - @echo " at an external one with ARGS=\"--mongo-uri \". (Unlike './dev.sh local'," + @echo " at an external one with ARGS=\"--mongo-uri \". (Unlike './dev.sh no-docker'," @echo " make does not auto-install it.)" - uv sync --group dev --group dev-local - uv run python -m scripts.dev_local $(ARGS) + uv sync --group dev --group dev-no-docker + uv run python -m scripts.dev_nodocker $(ARGS) dev-stop: ## Stop development server @echo "Stopping dev server" diff --git a/agentex/pyproject.toml b/agentex/pyproject.toml index 0fe3039f..cc068176 100644 --- a/agentex/pyproject.toml +++ b/agentex/pyproject.toml @@ -43,7 +43,7 @@ dev = [ "vulture>=2.14", "ruff>=0.3.4", ] -dev-local = [ +dev-no-docker = [ "pgserver>=0.1.4", "redislite>=6.2.912183", # SQLAlchemy's async engine needs greenlet at runtime. SQLAlchemy auto-installs @@ -60,7 +60,7 @@ test = [ "factory-boy>=3.3.0,<4", # for test data factories "greenlet>=3.2.3", "asyncpg>=0.29.0", - # Embedded datastores for dev_local.py tests (no testcontainers/docker needed) + # Embedded datastores for dev_nodocker tests (no testcontainers/docker needed) "pgserver>=0.1.4", "redislite>=6.2.912183", ] diff --git a/agentex/scripts/dev_local/__main__.py b/agentex/scripts/dev_local/__main__.py deleted file mode 100644 index 2934f9e5..00000000 --- a/agentex/scripts/dev_local/__main__.py +++ /dev/null @@ -1,6 +0,0 @@ -"""Entry point: `python -m scripts.dev_local` (used by `./dev.sh local` / `make dev-local`).""" - -from scripts.dev_local.runner import main - -if __name__ == "__main__": - main() diff --git a/agentex/scripts/dev_local/__init__.py b/agentex/scripts/dev_nodocker/__init__.py similarity index 78% rename from agentex/scripts/dev_local/__init__.py rename to agentex/scripts/dev_nodocker/__init__.py index c368290c..5770016d 100644 --- a/agentex/scripts/dev_local/__init__.py +++ b/agentex/scripts/dev_nodocker/__init__.py @@ -1,6 +1,6 @@ """Run the agentex backend locally as host processes, with no Docker. -The engine behind `./dev.sh local` / `make dev-local`: it provisions embedded datastores, +The engine behind `./dev.sh no-docker` / `make dev-no-docker`: it provisions embedded datastores, runs migrations, and supervises uvicorn (plus a Temporal worker), tearing everything down on Ctrl-C / SIGTERM. Postgres/Redis/Temporal need no system install (bundled / auto-downloaded). MongoDB is REQUIRED for the full stack — the Temporal worker builds @@ -13,15 +13,15 @@ (subprocess plumbing), `runner` (orchestration). """ -from scripts.dev_local.config import ( +from scripts.dev_nodocker.config import ( LOOPBACK, - DevLocalConfig, + DevNoDockerConfig, build_arg_parser, build_env, resolve_config, ) -from scripts.dev_local.runner import main, run -from scripts.dev_local.services import ( +from scripts.dev_nodocker.runner import main, run +from scripts.dev_nodocker.services import ( provision_mongo, provision_otel, provision_postgres, @@ -29,11 +29,11 @@ provision_temporal, teardown_redis, ) -from scripts.dev_local.supervise import run_migrations, wait_for_health +from scripts.dev_nodocker.supervise import run_migrations, wait_for_health __all__ = [ "LOOPBACK", - "DevLocalConfig", + "DevNoDockerConfig", "build_arg_parser", "resolve_config", "build_env", diff --git a/agentex/scripts/dev_nodocker/__main__.py b/agentex/scripts/dev_nodocker/__main__.py new file mode 100644 index 00000000..8c6828d5 --- /dev/null +++ b/agentex/scripts/dev_nodocker/__main__.py @@ -0,0 +1,6 @@ +"""Entry point: `python -m scripts.dev_nodocker` (used by `./dev.sh no-docker` / `make dev-no-docker`).""" + +from scripts.dev_nodocker.runner import main + +if __name__ == "__main__": + main() diff --git a/agentex/scripts/dev_local/config.py b/agentex/scripts/dev_nodocker/config.py similarity index 93% rename from agentex/scripts/dev_local/config.py rename to agentex/scripts/dev_nodocker/config.py index 781ea8fe..b45fae8f 100644 --- a/agentex/scripts/dev_local/config.py +++ b/agentex/scripts/dev_nodocker/config.py @@ -8,7 +8,7 @@ from dataclasses import dataclass from pathlib import Path -# scripts/dev_local/config.py -> agentex is three parents up; the runner launches +# scripts/dev_nodocker/config.py -> agentex is three parents up; the runner launches # uvicorn/alembic/the worker from there. AGENTEX_DIR = Path(__file__).resolve().parents[2] @@ -25,7 +25,7 @@ @dataclass -class DevLocalConfig: +class DevNoDockerConfig: agentex_dir: Path data_dir: Path ephemeral: bool @@ -58,7 +58,7 @@ def mongo_data(self) -> Path: def build_arg_parser() -> argparse.ArgumentParser: p = argparse.ArgumentParser( - prog="dev_local", + prog="dev_nodocker", description="Run the full agentex backend locally without Docker.", ) # Everything is on by default; --no- opts out. @@ -127,7 +127,7 @@ def build_arg_parser() -> argparse.ArgumentParser: p.add_argument( "--data-dir", default=None, - help="Directory for persistent datastore data (default /.dev-local). Ignored with --ephemeral.", + help="Directory for persistent datastore data (default /.dev-nodocker). Ignored with --ephemeral.", ) p.add_argument( "--ephemeral", @@ -139,26 +139,26 @@ def build_arg_parser() -> argparse.ArgumentParser: def resolve_config( argv: list[str] | None = None, *, agentex_dir: Path = AGENTEX_DIR -) -> DevLocalConfig: - """Parse argv into a DevLocalConfig. Pure: no filesystem side effects.""" +) -> DevNoDockerConfig: + """Parse argv into a DevNoDockerConfig. Pure: no filesystem side effects.""" parser = build_arg_parser() args = parser.parse_args(argv) if args.ephemeral: # Per-process-unique so a hard-killed run can't leave a pgserver orphan that the # next --ephemeral run re-attaches to (which would defeat "fresh state every run"). - data_dir = Path(tempfile.gettempdir()) / f"agentex-dev-local-{os.getpid()}" + data_dir = Path(tempfile.gettempdir()) / f"agentex-dev-nodocker-{os.getpid()}" elif args.data_dir: data_dir = Path(args.data_dir).expanduser().resolve() else: - data_dir = agentex_dir / ".dev-local" + data_dir = agentex_dir / ".dev-nodocker" # --lean forces the optional services off; --full is a no-op (they're already on). # MongoDB is always started — the stack requires it — so --lean does not touch it. temporal = args.temporal and not args.lean otel = args.otel and not args.lean - return DevLocalConfig( + return DevNoDockerConfig( agentex_dir=agentex_dir, data_dir=data_dir, ephemeral=args.ephemeral, diff --git a/agentex/scripts/dev_local/runner.py b/agentex/scripts/dev_nodocker/runner.py similarity index 94% rename from agentex/scripts/dev_local/runner.py rename to agentex/scripts/dev_nodocker/runner.py index fa2d5720..1738ae5a 100644 --- a/agentex/scripts/dev_local/runner.py +++ b/agentex/scripts/dev_nodocker/runner.py @@ -7,8 +7,13 @@ import signal import sys -from scripts.dev_local.config import LOOPBACK, DevLocalConfig, build_env, resolve_config -from scripts.dev_local.services import ( +from scripts.dev_nodocker.config import ( + LOOPBACK, + DevNoDockerConfig, + build_env, + resolve_config, +) +from scripts.dev_nodocker.services import ( provision_mongo, provision_otel, provision_postgres, @@ -16,7 +21,7 @@ provision_temporal, teardown_redis, ) -from scripts.dev_local.supervise import ( +from scripts.dev_nodocker.supervise import ( run_migrations, spawn, terminate, @@ -26,7 +31,7 @@ logger = logging.getLogger(__name__) -async def run(cfg: DevLocalConfig) -> int: +async def run(cfg: DevNoDockerConfig) -> int: if cfg.ephemeral: cfg.data_dir.mkdir(parents=True, exist_ok=True) @@ -146,7 +151,7 @@ async def run(cfg: DevLocalConfig) -> int: def _print_banner( - cfg: DevLocalConfig, + cfg: DevNoDockerConfig, mongo_uri: str | None, temporal_address: str | None, otel_endpoint: str | None, @@ -157,7 +162,7 @@ def _print_banner( else "off" ) print("\n" + "=" * 52, flush=True) - print(" agentex backend (local, no Docker)", flush=True) + print(" agentex backend (no Docker)", flush=True) print(f" API: http://localhost:{cfg.api_port}", flush=True) print(f" Swagger: http://localhost:{cfg.api_port}/swagger", flush=True) print(" Postgres: embedded (socket)", flush=True) diff --git a/agentex/scripts/dev_local/services.py b/agentex/scripts/dev_nodocker/services.py similarity index 94% rename from agentex/scripts/dev_local/services.py rename to agentex/scripts/dev_nodocker/services.py index d4040665..03e1579e 100644 --- a/agentex/scripts/dev_local/services.py +++ b/agentex/scripts/dev_nodocker/services.py @@ -10,13 +10,13 @@ from typing import Any from urllib.parse import urlsplit -from scripts.dev_local.config import DB_NAME, LOOPBACK, DevLocalConfig -from scripts.dev_local.supervise import spawn, terminate, wait_for_port +from scripts.dev_nodocker.config import DB_NAME, LOOPBACK, DevNoDockerConfig +from scripts.dev_nodocker.supervise import spawn, terminate, wait_for_port logger = logging.getLogger(__name__) -def provision_postgres(cfg: DevLocalConfig) -> tuple[Any, str]: +def provision_postgres(cfg: DevNoDockerConfig) -> tuple[Any, str]: """Start an embedded Postgres and ensure the application database (DB_NAME) exists. The returned socket-form `postgresql://` URL works for both the async app (asyncpg) @@ -42,7 +42,7 @@ def provision_postgres(cfg: DevLocalConfig) -> tuple[Any, str]: return server, server.get_uri(database=DB_NAME) -def provision_redis(cfg: DevLocalConfig) -> tuple[Any, str]: +def provision_redis(cfg: DevNoDockerConfig) -> tuple[Any, str]: """Start an embedded real redis-server on a TCP port. Returns (server, url).""" import redislite @@ -58,7 +58,7 @@ def provision_redis(cfg: DevLocalConfig) -> tuple[Any, str]: return server, f"redis://{LOOPBACK}:{port}/0" -async def provision_temporal(cfg: DevLocalConfig) -> tuple[Any, str]: +async def provision_temporal(cfg: DevNoDockerConfig) -> tuple[Any, str]: """Start the Temporal dev server (+ Web UI). Returns (env, address).""" from temporalio.testing import WorkflowEnvironment @@ -84,7 +84,7 @@ async def provision_temporal(cfg: DevLocalConfig) -> tuple[Any, str]: async def provision_mongo( - cfg: DevLocalConfig, + cfg: DevNoDockerConfig, ) -> tuple[asyncio.subprocess.Process | None, str | None]: """Start a local mongod (or use an external URI). Returns (proc, uri). @@ -152,7 +152,7 @@ async def provision_mongo( async def provision_otel( - cfg: DevLocalConfig, + cfg: DevNoDockerConfig, ) -> tuple[asyncio.subprocess.Process | None, str | None]: """Start the OpenTelemetry collector if present. Returns (proc, otlp_endpoint). @@ -163,7 +163,7 @@ async def provision_otel( if not binary: logger.warning( "otel collector not found on PATH — skipping telemetry export (the app " - "runs fine without it). To enable it, start via `./dev.sh local` (installs " + "runs fine without it). To enable it, start via `./dev.sh no-docker` (installs " "the otelcol-contrib release binary automatically) or install it yourself " "from https://github.com/open-telemetry/opentelemetry-collector-releases/releases." ) diff --git a/agentex/scripts/dev_local/supervise.py b/agentex/scripts/dev_nodocker/supervise.py similarity index 96% rename from agentex/scripts/dev_local/supervise.py rename to agentex/scripts/dev_nodocker/supervise.py index 62bb2c05..90111db2 100644 --- a/agentex/scripts/dev_local/supervise.py +++ b/agentex/scripts/dev_nodocker/supervise.py @@ -9,7 +9,7 @@ import urllib.request from pathlib import Path -from scripts.dev_local.config import LOOPBACK, DevLocalConfig +from scripts.dev_nodocker.config import LOOPBACK, DevNoDockerConfig logger = logging.getLogger(__name__) @@ -65,7 +65,7 @@ async def wait_for_port( return False -async def run_migrations(cfg: DevLocalConfig, env: dict[str, str]) -> None: +async def run_migrations(cfg: DevNoDockerConfig, env: dict[str, str]) -> None: """Run `alembic upgrade head` from the migrations dir (sync psycopg2 engine).""" migrations_dir = cfg.agentex_dir / "database" / "migrations" logger.info("Running database migrations ...") diff --git a/agentex/src/domain/use_cases/agents_acp_use_case.py b/agentex/src/domain/use_cases/agents_acp_use_case.py index 49e7dfa5..704fdc6a 100644 --- a/agentex/src/domain/use_cases/agents_acp_use_case.py +++ b/agentex/src/domain/use_cases/agents_acp_use_case.py @@ -333,17 +333,19 @@ async def _resolve_acp_url( raw = acp_url_override # Prefer the production deployment's URL when there's no explicit override. - if raw is None and agent.production_deployment_id: + # Use truthiness (not `is None`) so an empty-string acp_url falls through + # to the agent's own URL instead of being dialed as "". + if not raw and agent.production_deployment_id: deployment = await self.deployment_repo.get( id=agent.production_deployment_id ) raw = deployment.acp_url # Legacy fallback to the agent's own URL. - if raw is None: + if not raw: raw = agent.acp_url - if raw is None: + if not raw: raise ClientError(f"Agent {agent.id} does not have an ACP URL configured") return resolve_acp_url(raw) diff --git a/agentex/src/utils/acp_url.py b/agentex/src/utils/acp_url.py index a283ea51..a192942a 100644 --- a/agentex/src/utils/acp_url.py +++ b/agentex/src/utils/acp_url.py @@ -3,12 +3,12 @@ Agents register their ACP URL as ``http://host.docker.internal:`` (the SDK default) so that a **Docker-based** backend can reach an agent process running on the host. When the backend itself runs directly on the host — i.e. the docker-free -local dev mode (``./dev.sh local`` / ``python -m scripts.dev_local``) — ``host.docker.internal`` +local dev mode (``./dev.sh no-docker`` / ``python -m scripts.dev_nodocker``) — ``host.docker.internal`` does not resolve, and every ACP call fails with ``[Errno 8] nodename nor servname provided, or not known``. The correct address depends on the *backend's* network topology, which only the -backend knows. So the runner (``scripts.dev_local``) sets ``AGENTEX_ACP_HOST_OVERRIDE`` +backend knows. So the runner (``scripts.dev_nodocker``) sets ``AGENTEX_ACP_HOST_OVERRIDE`` (to ``127.0.0.1``) and the backend rewrites the Docker-only sentinel host to that value wherever it dials an agent. When the env var is unset (Docker / staging / prod), this is a no-op and the stored URL is used verbatim. diff --git a/dev.sh b/dev.sh index 2d71c96f..271fb3c9 100755 --- a/dev.sh +++ b/dev.sh @@ -5,12 +5,13 @@ # Delegates to existing Makefiles - this is an orchestration layer, not a replacement # # Usage: -# ./dev.sh Start all services (Docker backend + frontend) -# ./dev.sh local Start all services WITHOUT Docker (embedded pg/redis, local temporal/mongo/otel) -# ./dev.sh setup Install all prerequisites (macOS) -# ./dev.sh stop Stop all services -# ./dev.sh logs Show logs -# ./dev.sh status Check status of services +# ./dev.sh Start all services (Docker backend + frontend) +# ./dev.sh docker Same as above (explicit Docker mode) +# ./dev.sh no-docker Start all services WITHOUT Docker (embedded pg/redis, local temporal/mongo/otel) +# ./dev.sh setup Install all prerequisites (macOS) +# ./dev.sh stop Stop all services +# ./dev.sh logs Show logs +# ./dev.sh status Check status of services # set -e @@ -19,7 +20,7 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" LOG_DIR="$SCRIPT_DIR/.dev-logs" BACKEND_PID_FILE="$LOG_DIR/backend.pid" FRONTEND_PID_FILE="$LOG_DIR/frontend.pid" -MODE_FILE="$LOG_DIR/mode" # records "docker" or "local" so stop/status know how to act +MODE_FILE="$LOG_DIR/mode" # records "docker" or "nodocker" so stop/status know how to act # Colors for output RED='\033[0;31m' @@ -169,7 +170,7 @@ install_prerequisites() { echo "" echo "You can now run:" echo " ./dev.sh # Start the development environment (Docker)" - echo " ./dev.sh local # Start without Docker (embedded services)" + echo " ./dev.sh no-docker # Start without Docker (embedded services)" echo "" echo "Once running, create your first agent:" echo " agentex init # Create a new agent" @@ -218,7 +219,7 @@ require_docker() { # the user chooses Docker Desktop or Rancher). if ! docker info &> /dev/null; then log_error "Docker daemon is not running. Please start Docker Desktop or Rancher Desktop." - log_info "Tip: run './dev.sh local' to start the backend WITHOUT Docker." + log_info "Tip: run './dev.sh no-docker' to start the backend WITHOUT Docker." exit 1 fi } @@ -273,7 +274,7 @@ install_otel_collector() { log_success "otel collector installed to $dest/otelcol-contrib" } -ensure_local_service_binaries() { +ensure_nodocker_service_binaries() { # `local` mode runs Postgres/Redis/Temporal with no install (bundled / # auto-downloaded). MongoDB is REQUIRED for the full stack (the Temporal worker # needs it) so we always ensure it and FAIL FAST if we can't. The OTel collector is @@ -352,19 +353,19 @@ start_backend() { cd "$SCRIPT_DIR" } -start_backend_local() { - log_info "Starting backend services (local, no Docker)..." +start_backend_nodocker() { + log_info "Starting backend services (no Docker)..." cd "$SCRIPT_DIR/agentex" # Run the docker-free runner DIRECTLY via `uv run` (not `make`) so a SIGTERM # from `./dev.sh stop` is forwarded straight to the Python supervisor, which - # tears down the embedded datastores cleanly. --group dev-local pulls + # tears down the embedded datastores cleanly. --group dev-no-docker pulls # pgserver/redislite/greenlet. - uv run --group dev-local python -m scripts.dev_local "$@" > "$LOG_DIR/backend.log" 2>&1 & + uv run --group dev-no-docker python -m scripts.dev_nodocker "$@" > "$LOG_DIR/backend.log" 2>&1 & local pid=$! echo $pid > "$BACKEND_PID_FILE" - echo "local" > "$MODE_FILE" + echo "nodocker" > "$MODE_FILE" log_success "Backend starting (PID: $pid)" log_info "Backend logs: tail -f $LOG_DIR/backend.log" @@ -435,7 +436,7 @@ wait_for_backend() { log_info "Check logs with: ./dev.sh logs" } -# Echo the recorded backend mode ("docker" or "local"), defaulting to docker. +# Echo the recorded backend mode ("docker" or "nodocker"), defaulting to docker. current_mode() { if [ -f "$MODE_FILE" ]; then cat "$MODE_FILE"; else echo "docker"; fi } @@ -456,7 +457,7 @@ stop_services() { local mode mode=$(current_mode) - if [ "$mode" = "local" ]; then + if [ "$mode" = "nodocker" ]; then if [ -f "$BACKEND_PID_FILE" ]; then local backend_pid backend_pid=$(cat "$BACKEND_PID_FILE") @@ -514,7 +515,7 @@ show_status() { # Check backend echo -n "Backend ($mode): " - if [ "$mode" = "local" ]; then + if [ "$mode" = "nodocker" ]; then if [ -f "$BACKEND_PID_FILE" ] && kill -0 "$(cat "$BACKEND_PID_FILE")" 2>/dev/null; then echo -e "${GREEN}Running${NC}" else @@ -540,7 +541,7 @@ show_status() { # Temporal UI port differs: local dev server uses 8233, Docker UI uses 8080. local temporal_ui="http://localhost:8080" - [ "$mode" = "local" ] && temporal_ui="http://localhost:8233" + [ "$mode" = "nodocker" ] && temporal_ui="http://localhost:8233" echo "" echo "=== URLs ===" @@ -598,11 +599,11 @@ start_all() { echo "" } -# The dev_local runner accepts only optional flags — never a positional argument. Catch a -# stray word up front (almost always a top-level subcommand mistakenly typed after `local`, -# e.g. `./dev.sh local status`) with a clear hint, instead of forwarding it into the +# The dev_nodocker runner accepts only optional flags — never a positional argument. Catch a +# stray word up front (almost always a top-level subcommand mistakenly typed after `no-docker`, +# e.g. `./dev.sh no-docker status`) with a clear hint, instead of forwarding it into the # backgrounded runner where argparse rejects it and the error is buried in backend.log. -validate_local_args() { +validate_nodocker_args() { local expect_value=false arg for arg in "$@"; do if [ "$expect_value" = true ]; then @@ -618,32 +619,32 @@ validate_local_args() { : # some other flag (boolean, --opt=value, -h) — let the runner validate it ;; *) - log_error "'$arg' is not a valid option for './dev.sh local'." + log_error "'$arg' is not a valid option for './dev.sh no-docker'." case "$arg" in - start|setup|stop|logs|status|restart|help) - log_info "Did you mean './dev.sh $arg'? (start/setup/stop/logs/status/restart/help run on their own, not after 'local'.)" + start|docker|setup|stop|logs|status|restart|help) + log_info "Did you mean './dev.sh $arg'? (start/docker/setup/stop/logs/status/restart/help run on their own, not after 'no-docker'.)" ;; esac - log_info "Run './dev.sh help' for local flags (e.g. --lean, --no-temporal, --mongo-uri )." + log_info "Run './dev.sh help' for no-docker flags (e.g. --lean, --no-temporal, --mongo-uri )." exit 1 ;; esac done } -start_all_local() { - validate_local_args "$@" # fail fast on a stray positional before any side effects - ensure_prerequisites # no Docker requirement in local mode - ensure_local_service_binaries "$@" # ensure mongod (required) + otel (optional); honors --mongo-uri/--lean +start_all_nodocker() { + validate_nodocker_args "$@" # fail fast on a stray positional before any side effects + ensure_prerequisites # no Docker requirement in no-docker mode + ensure_nodocker_service_binaries "$@" # ensure mongod (required) + otel (optional); honors --mongo-uri/--lean setup_log_dir echo "" echo "========================================" - echo " Starting Agentex Development Env (Local, no Docker) " + echo " Starting Agentex Development Env (no Docker) " echo "========================================" echo "" - start_backend_local "$@" + start_backend_nodocker "$@" start_frontend echo "" @@ -679,12 +680,12 @@ start_all_local() { # Main command router case "${1:-start}" in - start|"") + start|docker|"") start_all ;; - local) - shift # drop the 'local' subcommand so only flags reach dev_local.py - start_all_local "$@" + no-docker) + shift # drop the 'no-docker' subcommand so only flags reach dev_nodocker + start_all_nodocker "$@" ;; setup) install_prerequisites @@ -705,8 +706,8 @@ case "${1:-start}" in restart_mode=$(current_mode) stop_services sleep 2 - if [ "$restart_mode" = "local" ]; then - start_all_local + if [ "$restart_mode" = "nodocker" ]; then + start_all_nodocker else start_all fi @@ -717,18 +718,19 @@ case "${1:-start}" in echo "Usage: $0 [command]" echo "" echo "Commands:" - echo " (none) Start everything with Docker (auto-installs prerequisites if needed)" - echo " local Start everything WITHOUT Docker (embedded pg/redis + Temporal + MongoDB + OTel)" - echo " MongoDB is required for the full stack and is auto-installed; OTel is optional." - echo " Flags pass through to the runner, e.g.:" - echo " ./dev.sh local --full # whole stack (default)" - echo " ./dev.sh local --lean # Postgres + Redis + API + MongoDB only" - echo " ./dev.sh local --no-temporal # skip Temporal + worker" - echo " setup Just install prerequisites without starting" - echo " stop Stop all services (Docker or local)" - echo " restart Restart all services" - echo " logs Show logs (logs backend|frontend|all)" - echo " status Check service status" + echo " (none) Start everything with Docker (auto-installs prerequisites if needed)" + echo " docker Start everything with Docker (explicit; same as no command)" + echo " no-docker Start everything WITHOUT Docker (embedded pg/redis + Temporal + MongoDB + OTel)" + echo " MongoDB is required for the full stack and is auto-installed; OTel is optional." + echo " Flags pass through to the runner, e.g.:" + echo " ./dev.sh no-docker --full # whole stack (default)" + echo " ./dev.sh no-docker --lean # Postgres + Redis + API + MongoDB only" + echo " ./dev.sh no-docker --no-temporal # skip Temporal + worker" + echo " setup Just install prerequisites without starting" + echo " stop Stop all services (Docker or no-docker)" + echo " restart Restart all services" + echo " logs Show logs (logs backend|frontend|all)" + echo " status Check service status" echo "" ;; *) diff --git a/uv.lock b/uv.lock index 3832d2a3..20b7b849 100644 --- a/uv.lock +++ b/uv.lock @@ -95,7 +95,7 @@ dev = [ { name = "ruff", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, { name = "vulture", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, ] -dev-local = [ +dev-no-docker = [ { name = "greenlet", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, { name = "pgserver", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, { name = "redislite", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, @@ -152,7 +152,7 @@ dev = [ { name = "ruff", specifier = ">=0.3.4" }, { name = "vulture", specifier = ">=2.14" }, ] -dev-local = [ +dev-no-docker = [ { name = "greenlet", specifier = ">=3.2.3" }, { name = "pgserver", specifier = ">=0.1.4" }, { name = "redislite", specifier = ">=6.2.912183" }, From 9c253b80147c0a0a7f5ba084a9cfb54389a79bb7 Mon Sep 17 00:00:00 2001 From: Ari Nguyen Date: Thu, 23 Jul 2026 16:14:20 -0700 Subject: [PATCH 4/7] fix commnet --- agentex/scripts/dev_nodocker/runner.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/agentex/scripts/dev_nodocker/runner.py b/agentex/scripts/dev_nodocker/runner.py index 1738ae5a..9f360527 100644 --- a/agentex/scripts/dev_nodocker/runner.py +++ b/agentex/scripts/dev_nodocker/runner.py @@ -123,6 +123,9 @@ async def run(cfg: DevNoDockerConfig) -> int: "A managed process exited unexpectedly: %s. Shutting down.", ", ".join(dead) or "?", ) + # Non-zero so callers ($?, make dev-no-docker, CI) detect the crash; + # a signal-driven shutdown (stop set) is the clean path and returns 0. + return 1 return 0 finally: # Tear down consumers (worker, api) first, then services in reverse. From 84323346c1f8307f7a93d835d31aeccdb64d6705 Mon Sep 17 00:00:00 2001 From: Ari Nguyen Date: Thu, 30 Jul 2026 15:51:27 -0700 Subject: [PATCH 5/7] fix(dev): align no-docker default ports with Docker mode Default the no-docker embedded Redis to 6379 (was 6390) and the Temporal UI to 8080 (was 8233) so both modes expose the same URLs and quick start needs no port flags. Users can still pass --redis-port / --ui-port if a Docker service already holds a port. Also drop the now-unnecessary per-mode Temporal UI branching in dev.sh and add a README note pointing users to wait for the readiness message before starting an agent. --- CLAUDE.md | 5 +++-- README.md | 14 ++++++++------ agentex/scripts/dev_nodocker/config.py | 8 ++++---- dev.sh | 8 ++------ 4 files changed, 17 insertions(+), 18 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 8a2ce21e..44a3e4af 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -67,8 +67,9 @@ Docker-free mode (host processes + embedded datastores, no Docker): > makes the runner fail fast (with an install message) rather than crash the worker. > `./dev.sh no-docker` auto-installs `mongod`; `make dev-no-docker` / direct `python -m > scripts.dev_nodocker` do not, so install `mongod` yourself or pass `--mongo-uri ` to -> point at an external MongoDB. In no-docker mode the Temporal UI is on :8233 (Docker mode -> uses :8080). +> point at an external MongoDB. No-docker port defaults match Docker mode (Temporal UI on +> :8080), so no port flags are needed for quick start; pass `--redis-port` / `--ui-port` / +> etc. if a Docker service already holds a port. > > Agents register their ACP URL as `host.docker.internal` (for a Docker backend), which > a host-process backend can't resolve. No-docker mode sets `AGENTEX_ACP_HOST_OVERRIDE=127.0.0.1` diff --git a/README.md b/README.md index 1253a6ab..99989f8b 100644 --- a/README.md +++ b/README.md @@ -111,6 +111,10 @@ Just run one command — no Docker required: ./dev.sh no-docker ``` +> Once you see `[SUCCESS] Development environment is ready!` in the terminal, the stack is +> up and waiting. Leave this terminal running and move on to +> [Create Your First Agent](#create-your-first-agent) in a new terminal. + That's it. This will automatically: - Install Homebrew, uv, Node.js, and agentex-sdk if missing (macOS) - Install all backend and frontend dependencies @@ -129,9 +133,7 @@ Once ready: | Frontend UI | http://localhost:3000 | | Backend API | http://localhost:5003 | | Swagger Docs | http://localhost:5003/swagger | -| Temporal UI | http://localhost:8233 | - -> With `./dev.sh` (Docker) the Temporal UI is on http://localhost:8080 instead. +| Temporal UI | http://localhost:8080 | #### Other commands ```bash @@ -157,9 +159,9 @@ Once ready: > worker needs it, so `./dev.sh no-docker` auto-installs `mongod` (via Homebrew) and startup > **fails fast** with an install message if it can't be made available. Point at an > existing MongoDB with `--mongo-uri ` to skip the local `mongod`. The OTel -> collector is optional; if it's absent the runner continues without telemetry. In -> no-docker mode the Temporal UI is at http://localhost:8233 (not :8080). Same `stop` / `status` / -> `logs` / `restart` commands apply. +> collector is optional; if it's absent the runner continues without telemetry. Ports +> match Docker mode (Temporal UI at http://localhost:8080), so the same URLs work either +> way. Same `stop` / `status` / `logs` / `restart` commands apply. **Connecting agents in no-docker mode.** Agents scaffolded by `agentex init` register their ACP URL as `http://host.docker.internal:` (so a *Docker* backend can reach an diff --git a/agentex/scripts/dev_nodocker/config.py b/agentex/scripts/dev_nodocker/config.py index b45fae8f..895b9e79 100644 --- a/agentex/scripts/dev_nodocker/config.py +++ b/agentex/scripts/dev_nodocker/config.py @@ -97,8 +97,8 @@ def build_arg_parser() -> argparse.ArgumentParser: p.add_argument( "--redis-port", type=int, - default=6390, - help="TCP port for embedded Redis (default 6390; avoids a Docker Redis on 6379).", + default=6379, + help="TCP port for embedded Redis (default 6379, matching Docker mode so quick start needs no port flag; pass a free port if a Docker Redis already holds 6379).", ) p.add_argument( "--temporal-port", @@ -109,8 +109,8 @@ def build_arg_parser() -> argparse.ArgumentParser: p.add_argument( "--ui-port", type=int, - default=8233, - help="Port for the Temporal Web UI (default 8233).", + default=8080, + help="Port for the Temporal Web UI (default 8080, matching Docker mode so the same URL works in both; the Temporal dev server's own default is 8233).", ) p.add_argument( "--mongo-port", diff --git a/dev.sh b/dev.sh index 271fb3c9..4f7bb28e 100755 --- a/dev.sh +++ b/dev.sh @@ -539,16 +539,12 @@ show_status() { echo -e "${RED}Stopped${NC}" fi - # Temporal UI port differs: local dev server uses 8233, Docker UI uses 8080. - local temporal_ui="http://localhost:8080" - [ "$mode" = "nodocker" ] && temporal_ui="http://localhost:8233" - echo "" echo "=== URLs ===" echo "Frontend: http://localhost:3000" echo "Backend API: http://localhost:5003" echo "Swagger: http://localhost:5003/swagger" - echo "Temporal UI: $temporal_ui" + echo "Temporal UI: http://localhost:8080" echo "" } @@ -654,7 +650,7 @@ start_all_nodocker() { echo "Frontend: http://localhost:3000" echo "Backend API: http://localhost:5003" echo "Swagger: http://localhost:5003/swagger" - echo "Temporal UI: http://localhost:8233" + echo "Temporal UI: http://localhost:8080" echo "" echo "=== Commands ===" echo "./dev.sh logs - View all logs" From a183b1b4d092282fb9fb4b67691ddc06046599fb Mon Sep 17 00:00:00 2001 From: Ari Nguyen Date: Thu, 30 Jul 2026 15:58:12 -0700 Subject: [PATCH 6/7] docs(windows): document WSL2 (docker-free) vs Docker setup choice Add a "Choosing a Setup" section to WINDOWS.md: run docker-free local mode (./dev.sh no-docker) inside WSL2 when it's available, and fall back to the Docker-based PowerShell flow when WSL2 isn't (locked-down or managed environments). Includes a WSL2 quick-start subsection. Expand the README's Windows pointer to surface the same decision from the main entry point. --- README.md | 2 +- WINDOWS.md | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 99989f8b..35f9b024 100644 --- a/README.md +++ b/README.md @@ -46,7 +46,7 @@ Here is what we will build together in this README. We'll start with a Hello Wor https://github.com/user-attachments/assets/9badad0d-f939-4243-ba39-68cafdae0078 -> **Windows Users**: Please see [WINDOWS.md](WINDOWS.md) for a complete Windows-specific guide with PowerShell commands and troubleshooting tips. +> **Windows Users**: Please see [WINDOWS.md](WINDOWS.md) for a complete Windows-specific guide. If WSL2 is available, run the docker-free local mode (`./dev.sh no-docker`) inside WSL2; otherwise (no WSL2 / locked-down environments) use the Docker-based flow with the PowerShell `build.ps1` scripts. ### Prerequisites diff --git a/WINDOWS.md b/WINDOWS.md index d2c83045..7a6eb296 100644 --- a/WINDOWS.md +++ b/WINDOWS.md @@ -2,6 +2,54 @@ This guide explains how to use Agentex on Windows. All functionality available through Makefiles is also available via PowerShell scripts. +## Choosing a Setup + +There are two supported ways to run Agentex on Windows. Pick based on whether WSL2 is +available in your environment: + +| Your environment | Recommended setup | +|---|---| +| **Windows with WSL2 available** | Docker-free local mode (`./dev.sh no-docker`) **inside WSL2** | +| **Windows without WSL2 / locked-down or managed environments** | Docker-based local dev via PowerShell (`.\build.ps1 dev`) | + +**Why WSL2 → Docker-free.** The docker-free runner spawns host processes and embedded +datastores (Postgres, Redis, MongoDB, a Temporal dev server) and relies on POSIX process +and networking behavior. It runs cleanly inside a WSL2 Linux distribution and follows the +same `./dev.sh` workflow documented in the [main README](README.md) — no Docker Desktop or +container runtime needed. + +**Why no WSL2 → Docker.** Where WSL2 isn't permitted (some corporate/managed machines) or +you'd rather stay in native Windows tooling, use the Docker-based flow with the PowerShell +`build.ps1` scripts below. This is the fully native-Windows path and is what the rest of +this guide covers. + +### Docker-free local mode inside WSL2 + +If WSL2 is available, this is the simplest path. + +1. Install WSL2 with a Linux distribution (from an elevated PowerShell): + + ```powershell + wsl --install + ``` + + Reboot if prompted, then launch your distribution (e.g. Ubuntu) and clone the repo + **inside the WSL2 filesystem** (e.g. under `~/`, not `/mnt/c/...`) for correct file + permissions and performance. + +2. From the repo root inside WSL2, run the docker-free stack exactly as on macOS/Linux: + + ```bash + ./dev.sh no-docker # whole stack, no Docker + ./dev.sh no-docker --lean # Postgres + Redis + API + MongoDB only + ``` + + See the [main README](README.md) for the full no-docker walkthrough, port defaults, and + agent-connection notes. All `stop` / `status` / `logs` / `restart` commands work the same. + +> Everything from **Prerequisites** onward in this guide covers the **Docker-based** +> native-Windows path. Use it if WSL2 isn't available to you. + ## Prerequisites ### Required Software From e34f35706b0b961743e944c31c9376a2c837401f Mon Sep 17 00:00:00 2001 From: Ari Nguyen Date: Fri, 31 Jul 2026 15:57:20 -0700 Subject: [PATCH 7/7] feat(dev): prompt to free ports held by a leftover run before starting A previous dev stack whose supervisor died without reaping its children leaves processes bound to the API/datastore ports. The next launch then fails to bind a managed process and tears the whole stack back down, with the real EADDRINUSE buried in a background log. Add a preflight port-conflict check to both start paths that lists any process already listening on a needed port (with PID + command) and offers to kill it before launching: - TTY: prompts [y/N] to kill and continue, else aborts with a clear hint. - DEV_KILL_PORTS=1: kills without prompting (restart/CI). - Non-interactive without the opt-in: warns and continues (unchanged). Kills gracefully via kill_tree (TERM, then force-KILL survivors) so a supervisor and its datastore children go down together. The no-docker check honors the runner's port-override flags; Postgres is skipped there since it uses a Unix socket. --- dev.sh | 124 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 124 insertions(+) diff --git a/dev.sh b/dev.sh index 4f7bb28e..c9fc9d95 100755 --- a/dev.sh +++ b/dev.sh @@ -332,6 +332,128 @@ check_redis_conflict() { fi } +# Preflight: surface any process already LISTENing on a port the stack needs, and +# offer to kill it before we launch. Without this, a leftover process from a previous +# run (whose supervisor died without reaping its children — see EADDRINUSE crashes in +# backend.log) makes a managed process fail to bind, which tears the whole launch back +# down with the real cause buried in a background log. +# +# Args: "port:label" pairs (label is just for the human-readable report). +# Behavior: +# - No conflicts -> return 0, silent. +# - TTY -> list offenders, prompt [y/N] to kill and continue. +# - DEV_KILL_PORTS=1 -> kill offenders without prompting (useful for restart/CI). +# - Non-interactive, no opt -> warn and continue (preserves old behavior; launch may fail). +preflight_port_check() { + local pairs=("$@") + local conflicts=() # "port|pid|cmd" + local pids_to_kill="" # space-separated, deduped + + local pair port pids pid cmd + for pair in "${pairs[@]}"; do + port="${pair%%:*}" + pids=$(lsof -nP -iTCP:"$port" -sTCP:LISTEN -t 2>/dev/null | sort -u || true) + [ -z "$pids" ] && continue + for pid in $pids; do + cmd=$(ps -p "$pid" -o comm= 2>/dev/null | awk -F/ '{print $NF}') + conflicts+=("$port|$pid|${cmd:-?}") + case " $pids_to_kill " in *" $pid "*) : ;; *) pids_to_kill="$pids_to_kill $pid" ;; esac + done + done + + [ ${#conflicts[@]} -eq 0 ] && return 0 + + log_warn "Some ports the dev stack needs are already in use (likely a leftover run):" + local c + for c in "${conflicts[@]}"; do + IFS='|' read -r port pid cmd <<< "$c" + printf " port %-6s -> PID %-7s (%s)\n" "$port" "$pid" "$cmd" + done + + local do_kill=false + if [ "${DEV_KILL_PORTS:-}" = "1" ]; then + do_kill=true + log_info "DEV_KILL_PORTS=1 set — killing the above processes." + elif [ -t 0 ]; then + local answer + read -r -p "$(echo -e "${YELLOW}[WARN]${NC} Kill these processes and continue? [y/N] ")" answer + case "$answer" in + [yY]|[yY][eE][sS]) do_kill=true ;; + esac + else + log_warn "Not an interactive shell; leaving them running. Set DEV_KILL_PORTS=1 to auto-kill." + log_warn "The launch may fail to bind these ports." + return 0 + fi + + if [ "$do_kill" != true ]; then + log_error "Ports still in use. Stop the offending processes (or run './dev.sh stop') and retry." + exit 1 + fi + + # Graceful TERM, then force-KILL anything that survives. kill_tree walks children so + # a supervisor and its datastore children all go down together. + for pid in $pids_to_kill; do + kill_tree "$pid" + done + sleep 1 + local still="" + for pid in $pids_to_kill; do + if kill -0 "$pid" 2>/dev/null; then + kill -9 "$pid" 2>/dev/null || true + still="$still $pid" + fi + done + [ -n "$still" ] && sleep 1 + log_success "Freed conflicting ports." +} + +# Build the "port:label" list for Docker mode and run the preflight check. +preflight_ports_docker() { + preflight_port_check \ + "5003:API" "5432:Postgres" "5433:Temporal-Postgres" \ + "6379:Redis" "27017:Mongo" "7233:Temporal" "8080:Temporal-UI" +} + +# Build the "port:label" list for no-docker mode, honoring any port-override flags the +# runner accepts, then run the preflight check. (Postgres is an embedded UNIX socket in +# no-docker mode, so it has no TCP port to check.) +preflight_ports_nodocker() { + local api=5003 redis=6379 temporal=7233 ui=8080 mongo=27017 otel=4317 + local expect="" arg + for arg in "$@"; do + if [ -n "$expect" ]; then + case "$expect" in + port) api="$arg" ;; + redis-port) redis="$arg" ;; + temporal-port) temporal="$arg" ;; + ui-port) ui="$arg" ;; + mongo-port) mongo="$arg" ;; + otel-port) otel="$arg" ;; + esac + expect="" + continue + fi + case "$arg" in + --port) expect=port ;; + --redis-port) expect=redis-port ;; + --temporal-port) expect=temporal-port ;; + --ui-port) expect=ui-port ;; + --mongo-port) expect=mongo-port ;; + --otel-port) expect=otel-port ;; + --port=*) api="${arg#*=}" ;; + --redis-port=*) redis="${arg#*=}" ;; + --temporal-port=*) temporal="${arg#*=}" ;; + --ui-port=*) ui="${arg#*=}" ;; + --mongo-port=*) mongo="${arg#*=}" ;; + --otel-port=*) otel="${arg#*=}" ;; + esac + done + preflight_port_check \ + "$api:API" "$redis:Redis" "$temporal:Temporal" \ + "$ui:Temporal-UI" "$mongo:Mongo" "$otel:OTel" +} + setup_log_dir() { mkdir -p "$LOG_DIR" } @@ -552,6 +674,7 @@ start_all() { ensure_prerequisites require_docker check_redis_conflict + preflight_ports_docker # offer to free any port a leftover run is holding setup_log_dir echo "" @@ -632,6 +755,7 @@ start_all_nodocker() { validate_nodocker_args "$@" # fail fast on a stray positional before any side effects ensure_prerequisites # no Docker requirement in no-docker mode ensure_nodocker_service_binaries "$@" # ensure mongod (required) + otel (optional); honors --mongo-uri/--lean + preflight_ports_nodocker "$@" # offer to free any port a leftover run is holding (honors port flags) setup_log_dir echo ""