diff --git a/.claude/skills/mendix/run-local.md b/.claude/skills/mendix/run-local.md index 34f57b5dd..94bb817b3 100644 --- a/.claude/skills/mendix/run-local.md +++ b/.claude/skills/mendix/run-local.md @@ -107,7 +107,9 @@ Launch `run --local` as the **sole** command in its invocation (don't chain a tr | Flag | Default | Purpose | |------|---------|---------| -| `--local` | — | Required; run without Docker | +| `--local` | — | Required; run without Docker (implied by `--hub`) | +| `--hub` | — | Expose the app in a browser at a tunnel-hub URL (see below) | +| `--hub-secret` | — | Shared auth (`user:pass`) matching the hub's `--secret` | | `--watch` | off | Rebuild + hot-apply on each change | | `--ensure-db` | off | Provision local Postgres + app database if missing | | `--setup` | off | Cache MxBuild+runtime + ensure DB, then exit (SessionStart bring-up) | @@ -147,6 +149,46 @@ mxcli run --local -p app.mpr --watch --screenshot reuse the session, so pages behind login render authenticated. Best-effort: an anonymous app with no login form proceeds unauthenticated. +## External browser preview (`--hub`) + +`--hub ` exposes the running app in a **browser at a public URL** without the app +leaving this machine and without committing — for reviewing work-in-progress from a +phone/tablet, or from an egress-only environment like Claude Code on the web. The app +stays here; a **chisel reverse tunnel** dials *out* to a hub over 443 and the hub proxies +browser requests back down it. Nothing is pushed — only live HTTP — and everything rides +one 443 connection, so it works through an egress-only proxy. + +**You run your own hub — there is no hosted service.** Stand up `mxcli tunnel-hub` once on +a host you control (a small VPS with a domain), then point apps at it. + +```bash +# on your VPS: *.example.com + hub.example.com -> this host, inbound 80+443 open +mxcli tunnel-hub --domain example.com --secret alice:s3cret + +# where the app runs: +mxcli run --hub https://hub.example.com --hub-secret alice:s3cret -p app.mpr +# -> registers and prints e.g. "Preview available at https://app.example.com" +``` + +The hub is **multi-tenant**: it fronts many previews at per-preview subdomains +(`-.example.com`; `main`/`master` collapses to ``) with a +sortable overview at `https://hub.example.com/`. Each `run --hub` self-registers: + +- **Project** and **branch** auto-detect (`.mpr` name + git); override with + `--hub-project` / `--hub-branch`. `--hub-worktree` distinguishes worktrees of one branch. +- **`--hub-prefix`** namespaces the hostname (org/solution/team/env) → + `--`; **`--hub-solution`** groups a solution's apps in the overview. +- The overview shows availability (a reaped/idle container goes **stale**), sortable by + last-used / registered / project. Re-registering keeps a **stable URL**. +- `--hub` **implies `--local`**, boots the runtime with `ApplicationRootUrl` set to the + assigned URL (so the SPA/`originURI` work), and the tunnel reconnects forever. Combine + with `--watch` for the full loop: edit here → hot-apply → refresh the browser. + +**Hub setup:** wildcard `*.example.com` A record (+ `hub.example.com`) → the VPS; inbound +80+443 open (per-subdomain Let's Encrypt on demand). **Security:** this version uses one +shared `--secret` and open registration, so keep the hub to people you trust and don't +expose it publicly (per-tenant auth is a follow-up). + ## Validation checklist - [ ] Project is Mendix 11.x. diff --git a/CLAUDE.md b/CLAUDE.md index 60b4d2d65..efd27ce3b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -524,6 +524,7 @@ Full syntax tables for all MDL statements (microflows, pages, security, navigati - VS Code extension (`vscode-mdl`) with context menu commands (Run/Check/Selection) - Docker build integration (`mxcli docker build`) with PAD patching (Phase 1) - Warm local dev loop (`mxcli run --local [--watch] [--screenshot]`): Docker-free `mxbuild --serve` + standalone runtime, hot `reload_model` for behavioural changes and restart+DDL for structural ones (chosen from the serve build's `restartRequired`). Bundles the browser client (`web/dist/` via mxbuild's rollup runner, which the serve Deploy target skips) so Mendix 11.x apps render in a browser. `--watch` keeps an incremental rollup bundler hot (CHOKIDAR_USEPOLLING for container fs; ~3-4s page re-bundle, skipped for model-only edits) and watches only model source (`.mpr`+`mprcontents/`). `--ensure-db` provisions the local Postgres + app database if missing; `--setup` does the non-blocking prerequisites (cache mxbuild+runtime, ensure DB) and exits — `mxcli init` wires it into a Claude Code SessionStart hook so a fresh/reaped web session self-bootstraps, and `docs-site/src/tools/bootstrap-prompt.md` is the empty-repo seed prompt. `--screenshot` captures a Playwright PNG each change (pixel-perfect page loop), with `--screenshot-url` deep links (repeatable for multi-page sets, one PNG per page) and `--screenshot-user`/`--screenshot-password` form login (session saved as Playwright storage state, reused via `screenshot --load-storage`). See `docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md` +- External browser preview (`mxcli run --hub ` + `mxcli tunnel-hub`): the app stays local and reverse-tunnels out over a single 443 connection (embedded chisel) to a static relay, so it is reachable in a browser at a public URL — works from egress-only environments (Claude Code web), verified live through the session's MITM egress proxy. `run --hub` implies `--local`, boots the runtime with `ApplicationRootUrl` set to the assigned URL (so the SPA/`originURI` work under the public origin), resolves the control proxy honouring `NO_PROXY`, and retries forever. `mxcli tunnel-hub --domain ` is the **multi-tenant** relay: a registry keyed by prefix/project/solution/branch/worktree (stable URLs on reconnect) fronts many previews at per-subdomain hosts (`[prefix-]project[-branch].`; main collapses to the project) over one 443 with per-subdomain autocert, a registration API (`/api/register|status|deregister|backends`), and a sortable availability overview at `hub./`. Client identity flags: `--hub-prefix`/`--hub-project`/`--hub-solution`/`--hub-branch`/`--hub-worktree` (project + branch auto-detected). Package: `cmd/mxcli/tunnelhub/`. See `docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md` (slices 3–4) - OQL query execution against running runtime (`mxcli oql`) - Business event services (SHOW/DESCRIBE/CREATE/DROP) - Project settings (SHOW/DESCRIBE/ALTER) diff --git a/README.md b/README.md index 338dcc365..8f0038199 100644 --- a/README.md +++ b/README.md @@ -140,6 +140,14 @@ mxcli run --local -p app.mpr --watch --screenshot # hot-reload + auto screensh `mxcli run --local` keeps `mxbuild --serve` and a standalone runtime hot: a page/microflow edit is hot-applied in seconds (a hot `reload_model`, or a restart + DDL for entity changes), and `--screenshot` captures each page with Playwright. See **[Local Dev Loop](https://mendixlabs.github.io/mxcli/tools/run-local.html)**. +Add `--hub` to preview the running app **in a browser at a public URL** — the app stays local and reverse-tunnels out over a single 443 connection, so it works even from egress-only environments (e.g. Claude Code on the web): + +```bash +mxcli run --hub https://hub.example.com -p app.mpr # -> a shareable preview URL +``` + +`mxcli tunnel-hub --domain example.com` is the static relay you run once on a small VPS; it can front many previews at per-subdomain hosts across projects, solutions, branches, and worktrees, with a sortable availability overview at `hub.example.com/`. See **[Local Dev Loop → External browser preview](https://mendixlabs.github.io/mxcli/tools/run-local.html)**. + ### Existing project For an existing Mendix project, use `mxcli init` to add AI tooling and a Dev Container: diff --git a/cmd/mxcli/cmd_run.go b/cmd/mxcli/cmd_run.go index 66ecafb15..b50e52215 100644 --- a/cmd/mxcli/cmd_run.go +++ b/cmd/mxcli/cmd_run.go @@ -32,13 +32,32 @@ Requirements: already exist. Defaults: 127.0.0.1:5432, user 'mendix', db from the project name. Override with --db-host/--db-name/--db-user/--db-password. +With --hub, the running app is exposed in a browser at a public URL through an +mxcli tunnel-hub, without leaving this machine: a chisel client reverse-tunnels +the local app out over 443, and the runtime boots with ApplicationRootUrl set to +the hub URL so the app works under that origin. --hub implies --local. + Examples: mxcli run --local -p app.mpr mxcli run --local -p app.mpr --watch mxcli run --local -p app.mpr --app-port 8081 --db-name myapp + mxcli run --hub https://hub.example.com -p app.mpr # browser preview + mxcli run --hub https://hub.example.com --hub-secret u:pass -p app.mpr --watch `, Run: func(cmd *cobra.Command, args []string) { local, _ := cmd.Flags().GetBool("local") + hub, _ := cmd.Flags().GetString("hub") + hubSecret, _ := cmd.Flags().GetString("hub-secret") + hubPrefix, _ := cmd.Flags().GetString("hub-prefix") + hubProject, _ := cmd.Flags().GetString("hub-project") + hubSolution, _ := cmd.Flags().GetString("hub-solution") + hubBranch, _ := cmd.Flags().GetString("hub-branch") + hubWorktree, _ := cmd.Flags().GetString("hub-worktree") + // --hub is a cross-cutting ingress and implies the local serving path (the + // only serving mode wired today; a future PAD path will accept --hub too). + if hub != "" { + local = true + } if !local { fmt.Fprintln(os.Stderr, "Error: only --local is supported for now (use 'mxcli docker run' for the container workflow)") os.Exit(1) @@ -67,6 +86,13 @@ Examples: opts := docker.LocalRunOptions{ ProjectPath: projectPath, + Hub: hub, + HubSecret: hubSecret, + HubPrefix: hubPrefix, + HubProject: hubProject, + HubSolution: hubSolution, + HubBranch: hubBranch, + HubWorktree: hubWorktree, AppPort: appPort, AdminPort: adminPort, ServePort: servePort, @@ -97,6 +123,13 @@ Examples: func init() { runCmd.Flags().Bool("local", false, "Run locally without Docker (warm serve + standalone runtime)") + runCmd.Flags().String("hub", "", "Expose the running app in a browser via your own mxcli tunnel-hub URL (e.g. https://hub.example.com). Implies --local; the app stays local and is reverse-tunnelled out") + runCmd.Flags().String("hub-secret", "", "Shared auth secret for --hub (\"user:pass\"), matching the hub's --secret") + runCmd.Flags().String("hub-prefix", "", "Optional subdomain prefix on the hub (org/solution/team/env): --") + runCmd.Flags().String("hub-project", "", "Project name for the hub subdomain + overview (default: the .mpr name)") + runCmd.Flags().String("hub-solution", "", "Solution name to group this app under in the hub overview (multi-app solutions)") + runCmd.Flags().String("hub-branch", "", "Branch for the hub subdomain + overview (default: the git branch)") + runCmd.Flags().String("hub-worktree", "", "Worktree label to distinguish multiple worktrees of one branch") runCmd.Flags().Bool("watch", false, "Rebuild and hot-apply on every project change") runCmd.Flags().Bool("ensure-db", false, "Provision the local Postgres + app database if missing (fresh-session bootstrap)") runCmd.Flags().Bool("setup", false, "Prepare prerequisites (cache MxBuild+runtime, ensure DB) and exit without booting — for a SessionStart hook") diff --git a/cmd/mxcli/cmd_tunnelhub.go b/cmd/mxcli/cmd_tunnelhub.go new file mode 100644 index 000000000..0fced4aec --- /dev/null +++ b/cmd/mxcli/cmd_tunnelhub.go @@ -0,0 +1,107 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "context" + "fmt" + "os" + "os/signal" + "path/filepath" + "syscall" + + "github.com/mendixlabs/mxcli/cmd/mxcli/tunnelhub" + "github.com/spf13/cobra" +) + +// tunnelHubCmd runs the multi-tenant ingress relay. It fronts many locally-running +// Mendix apps — across projects, solutions, branches, and worktrees — each at its +// own . over a single 443 connection, with a registration API +// and an admin overview at the hub host. Apps stay in their own (possibly +// egress-only) environments and reverse-tunnel out; nothing is pushed here. +var tunnelHubCmd = &cobra.Command{ + Use: "tunnel-hub", + Short: "Multi-tenant ingress relay: front many locally-running mxcli apps at per-preview subdomains", + Long: `Run the static ingress relay that exposes locally-running Mendix apps +(started elsewhere with 'mxcli run --hub ') in a browser. + +Each app self-registers and is served at its own subdomain +(-., or -- with --hub-prefix); +the hub host (hub.) serves the registration API, the admin overview, and +the chisel control connection. Everything rides one 443 connection, so apps in +egress-only environments (e.g. Claude Code on the web) can reverse-tunnel out. + +You run your own hub — there is no hosted service. Stand it up on a host you +control (a small VPS with a domain). + +DNS: point a wildcard '*.' A record (and 'hub.') at this host. +TLS is issued per subdomain via Let's Encrypt on demand (needs inbound 80+443). + +Security: set --secret and keep the hub to people you trust — this version uses a +single shared secret and open registration, so anyone with it can register a +preview (per-tenant auth is a follow-up). Don't expose it to the public. + +Example (on your own VPS; *.example.com -> this host, ports 80+443 open): + mxcli tunnel-hub --domain example.com --secret alice:s3cret + +Then, in each app's environment: + mxcli run --hub https://hub.example.com --hub-secret alice:s3cret \ + --hub-solution CustomerPortal -p app.mpr +`, + Run: func(cmd *cobra.Command, args []string) { + domain, _ := cmd.Flags().GetString("domain") + hubHost, _ := cmd.Flags().GetString("hub-host") + secret, _ := cmd.Flags().GetString("secret") + httpsPort, _ := cmd.Flags().GetInt("port") + httpPort, _ := cmd.Flags().GetInt("http-port") + certCache, _ := cmd.Flags().GetString("cert-cache") + + if domain == "" { + fmt.Fprintln(os.Stderr, "Error: --domain is required (the wildcard base, e.g. example.com)") + os.Exit(1) + } + if certCache == "" { + home, _ := os.UserHomeDir() + certCache = filepath.Join(home, ".mxcli", "hub-certs") + } + + reg := tunnelhub.NewRegistry(tunnelhub.RegistryOptions{Domain: domain}) + srv, err := tunnelhub.NewServer(tunnelhub.ServerOptions{ + Domain: domain, + HubHost: hubHost, + Registry: reg, + TunnelAuth: secret, + RegisterSecret: secret, + CertCacheDir: certCache, + }) + if err != nil { + fmt.Fprintf(os.Stderr, "Error: configuring tunnel-hub: %v\n", err) + os.Exit(1) + } + + host := hubHost + if host == "" { + host = "hub." + domain + } + fmt.Printf("mxcli tunnel-hub: serving *.%s (control/admin at https://%s) on :%d\n", domain, host, httpsPort) + fmt.Printf(" admin overview: https://%s/\n", host) + + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) + defer stop() + + if err := srv.Start(ctx, fmt.Sprintf(":%d", httpsPort), fmt.Sprintf(":%d", httpPort)); err != nil { + fmt.Fprintf(os.Stderr, "Error: tunnel-hub: %v\n", err) + os.Exit(1) + } + }, +} + +func init() { + tunnelHubCmd.Flags().String("domain", "", "Wildcard base domain you control, e.g. example.com (previews served at .)") + tunnelHubCmd.Flags().String("hub-host", "", "Control/admin host (default hub.)") + tunnelHubCmd.Flags().String("secret", "", "Shared secret (\"user:pass\") apps present via --hub-secret; empty = open") + tunnelHubCmd.Flags().Int("port", 443, "HTTPS port to listen on") + tunnelHubCmd.Flags().Int("http-port", 80, "HTTP port for ACME challenges + http->https redirect") + tunnelHubCmd.Flags().String("cert-cache", "", "Directory for Let's Encrypt certificates (default ~/.mxcli/hub-certs)") + rootCmd.AddCommand(tunnelHubCmd) +} diff --git a/cmd/mxcli/docker/hubclient.go b/cmd/mxcli/docker/hubclient.go new file mode 100644 index 000000000..c6483329a --- /dev/null +++ b/cmd/mxcli/docker/hubclient.go @@ -0,0 +1,195 @@ +// SPDX-License-Identifier: Apache-2.0 + +package docker + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "os/exec" + "path/filepath" + "strings" + "time" +) + +// HubMeta identifies a preview to the hub: it drives the subdomain and the +// overview grouping. Blank fields are auto-detected where possible. +type HubMeta struct { + Prefix string // optional hostname namespace (org/solution/team/env) + Project string // default: the .mpr basename + Solution string // optional grouping for multi-app solutions + Branch string // default: the project's git branch + Worktree string // optional; distinguishes worktrees of one branch +} + +// HubRegistration is the result of registering with a hub: everything the client +// needs to tunnel and to advertise the app under its public URL. +type HubRegistration struct { + URL string // assigned public URL -> ApplicationRootUrl + Subdomain string // "" in single-app fallback + ControlURL string // where the chisel client connects + ReversePort int // reverse port to tunnel to + Token string // heartbeat/deregister auth ("" in fallback) + TunnelAuth string // chisel auth to use + HeartbeatInterval time.Duration // 0 in fallback + MultiTenant bool // false when we fell back to a slice-1 single-app hub + + hubURL string +} + +type registerResponse struct { + Subdomain string `json:"subdomain"` + URL string `json:"url"` + ReversePort int `json:"reversePort"` + ControlURL string `json:"controlUrl"` + Token string `json:"token"` + TunnelAuth string `json:"tunnelAuth"` + HeartbeatIntervalSec int `json:"heartbeatIntervalSec"` +} + +// RegisterWithHub registers a preview with a multi-tenant hub. If the hub has no +// registration API (a slice-1 single-app hub), it falls back to serving directly +// at the hub URL on the default reverse port. The HTTP client uses the standard +// proxy environment (honouring NO_PROXY), so an external hub goes through the +// egress proxy. +func RegisterWithHub(hubURL, secret string, meta HubMeta, appPort int) (*HubRegistration, error) { + body, _ := json.Marshal(map[string]any{ + "prefix": meta.Prefix, + "project": meta.Project, + "solution": meta.Solution, + "branch": meta.Branch, + "worktree": meta.Worktree, + "appPort": appPort, + }) + req, err := http.NewRequest(http.MethodPost, strings.TrimRight(hubURL, "/")+"/api/register", bytes.NewReader(body)) + if err != nil { + return nil, err + } + req.Header.Set("Content-Type", "application/json") + if secret != "" { + req.Header.Set("X-Hub-Secret", secret) + } + resp, err := http.DefaultClient.Do(req) + if err != nil { + return nil, fmt.Errorf("contacting hub: %w", err) + } + defer resp.Body.Close() + + switch resp.StatusCode { + case http.StatusOK: + var rr registerResponse + if err := json.NewDecoder(io.LimitReader(resp.Body, 1<<16)).Decode(&rr); err != nil { + return nil, fmt.Errorf("decoding hub response: %w", err) + } + return &HubRegistration{ + URL: rr.URL, + Subdomain: rr.Subdomain, + ControlURL: rr.ControlURL, + ReversePort: rr.ReversePort, + Token: rr.Token, + TunnelAuth: rr.TunnelAuth, + HeartbeatInterval: time.Duration(rr.HeartbeatIntervalSec) * time.Second, + MultiTenant: true, + hubURL: hubURL, + }, nil + case http.StatusNotFound: + // No registration API — a single-app hub. Serve directly at the hub URL. + return &HubRegistration{ + URL: strings.TrimRight(hubURL, "/"), + ControlURL: hubURL, + ReversePort: DefaultHubBackendPort, + TunnelAuth: secret, + MultiTenant: false, + hubURL: hubURL, + }, nil + default: + msg, _ := io.ReadAll(io.LimitReader(resp.Body, 4096)) + return nil, fmt.Errorf("hub registration failed (HTTP %d): %s", resp.StatusCode, strings.TrimSpace(string(msg))) + } +} + +// Heartbeat periodically pings the hub so the preview shows as available, and +// deregisters on Stop. No-op for a single-app (fallback) registration. +type Heartbeat struct { + cancel context.CancelFunc + done chan struct{} + reg *HubRegistration +} + +// StartHeartbeat begins heartbeating (only for a multi-tenant registration). +func StartHeartbeat(reg *HubRegistration) *Heartbeat { + if !reg.MultiTenant || reg.HeartbeatInterval <= 0 { + return &Heartbeat{reg: reg} + } + ctx, cancel := context.WithCancel(context.Background()) + h := &Heartbeat{cancel: cancel, done: make(chan struct{}), reg: reg} + go func() { + defer close(h.done) + t := time.NewTicker(reg.HeartbeatInterval) + defer t.Stop() + for { + select { + case <-ctx.Done(): + return + case <-t.C: + postToken(ctx, reg.hubURL+"/api/status", reg.Token) + } + } + }() + return h +} + +// Stop ends heartbeating and best-effort deregisters the preview from the hub. +func (h *Heartbeat) Stop() { + if h.cancel != nil { + h.cancel() + <-h.done + } + if h.reg != nil && h.reg.MultiTenant && h.reg.Token != "" { + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + postToken(ctx, h.reg.hubURL+"/api/deregister", h.reg.Token) + } +} + +func postToken(ctx context.Context, url, token string) { + req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, nil) + if err != nil { + return + } + req.Header.Set("Authorization", "Bearer "+token) + if resp, err := http.DefaultClient.Do(req); err == nil { + _ = resp.Body.Close() + } +} + +// DetectHubMeta fills a HubMeta from the project path + git, with explicit +// overrides taking precedence. +func DetectHubMeta(projectPath string, override HubMeta) HubMeta { + m := override + if m.Project == "" { + base := filepath.Base(projectPath) + m.Project = strings.TrimSuffix(base, filepath.Ext(base)) + } + if m.Branch == "" { + m.Branch = gitBranch(filepath.Dir(projectPath)) + } + return m +} + +// gitBranch returns the current branch of the repo containing dir, or "". +func gitBranch(dir string) string { + cmd := exec.Command("git", "-C", dir, "rev-parse", "--abbrev-ref", "HEAD") + out, err := cmd.Output() + if err != nil { + return "" + } + b := strings.TrimSpace(string(out)) + if b == "HEAD" { // detached + return "" + } + return b +} diff --git a/cmd/mxcli/docker/localboot.go b/cmd/mxcli/docker/localboot.go index 276f0e481..0df5974cd 100644 --- a/cmd/mxcli/docker/localboot.go +++ b/cmd/mxcli/docker/localboot.go @@ -60,6 +60,12 @@ type LocalRuntimeOptions struct { ListenAddr string // DTAPMode is D/A/T/P (default "D"). DTAPMode string + // ApplicationRootUrl, when set, is the public URL the app is reached at + // (e.g. https://hub.example.com). Mendix uses it for absolute-URL generation + // and to accept requests whose Host differs from the listen address — needed + // when the app is served through an external tunnel/reverse proxy rather than + // localhost. Empty for a plain local run. + ApplicationRootUrl string // DB is the database the runtime connects to. DB DBConfig // ReadyTimeout bounds how long StartLocalRuntime waits for the admin API @@ -139,7 +145,7 @@ func runtimeConfigParams(o LocalRuntimeOptions, constants map[string]string) map if constants == nil { constants = map[string]string{} } - return map[string]any{ + params := map[string]any{ "BasePath": o.DeployDir, "RuntimePath": o.runtimeDir(), "DTAPMode": o.DTAPMode, @@ -150,6 +156,12 @@ func runtimeConfigParams(o LocalRuntimeOptions, constants map[string]string) map "DatabasePassword": o.DB.Password, "MicroflowConstants": constants, } + // Only set ApplicationRootUrl when the app is served behind an external URL; + // on a plain local run the runtime defaults it from the listen address. + if o.ApplicationRootUrl != "" { + params["ApplicationRootUrl"] = o.ApplicationRootUrl + } + return params } // deploymentConfig mirrors the parts of /model/config.json mxbuild diff --git a/cmd/mxcli/docker/localboot_test.go b/cmd/mxcli/docker/localboot_test.go index 712d68bda..21250ea6c 100644 --- a/cmd/mxcli/docker/localboot_test.go +++ b/cmd/mxcli/docker/localboot_test.go @@ -119,6 +119,20 @@ func TestRuntimeConfigParams_NilConstants(t *testing.T) { } } +func TestRuntimeConfigParams_ApplicationRootUrl(t *testing.T) { + // Absent by default (plain local run): the runtime derives it from the listen + // address, so we must not pin it. + if _, ok := runtimeConfigParams(testLocalOpts(), nil)["ApplicationRootUrl"]; ok { + t.Error("ApplicationRootUrl must be absent when not set") + } + // Present when serving behind a hub, so the SPA works under the public origin. + o := testLocalOpts() + o.ApplicationRootUrl = "https://hub.example.com" + if got := runtimeConfigParams(o, nil)["ApplicationRootUrl"]; got != "https://hub.example.com" { + t.Errorf("ApplicationRootUrl = %v, want https://hub.example.com", got) + } +} + func TestReadDeploymentConstants(t *testing.T) { dir := t.TempDir() modelDir := filepath.Join(dir, "model") diff --git a/cmd/mxcli/docker/runlocal.go b/cmd/mxcli/docker/runlocal.go index e7feb546d..915bab54a 100644 --- a/cmd/mxcli/docker/runlocal.go +++ b/cmd/mxcli/docker/runlocal.go @@ -40,6 +40,22 @@ type LocalRunOptions struct { AdminPass string // DB is the Postgres the runtime connects to (devcontainer defaults applied). DB DBConfig + // Hub, when set, is the URL of an mxcli tunnel-hub (e.g. https://hub.example.com). + // The app stays running here; a chisel client reverse-tunnels it out to the hub + // so it is reachable in a browser at the hub URL. Implies a local run. The + // runtime boots with ApplicationRootUrl = Hub so the SPA works under that origin. + Hub string + // HubSecret is the shared auth secret for the hub ("user:pass"), matching the + // hub's --secret. Optional but recommended. + HubSecret string + // Hub identity (multi-tenant hub): these drive the assigned subdomain and the + // hub overview grouping. Blank Project/Branch are auto-detected from the .mpr + // name and git. + HubPrefix string // optional hostname namespace (org/solution/team/env) + HubProject string // override the auto-detected project name + HubSolution string // grouping for multi-app solutions + HubBranch string // override the auto-detected git branch + HubWorktree string // distinguish worktrees of one branch // Watch keeps running, rebuilding+applying on every project change. Watch bool // EnsureDB provisions the local Postgres + app database if missing (otherwise @@ -420,17 +436,38 @@ func RunLocal(opts LocalRunOptions) error { } } + // 5c. With --hub, register with the hub first (before boot) so we know the + // assigned public URL — the runtime must boot with ApplicationRootUrl set to it + // (else the SPA/originURI misbehave across origins). A multi-tenant hub hands + // back a per-preview subdomain + reverse port; a single-app hub falls back to + // the hub URL itself. + var hubReg *HubRegistration + appRootURL := "" + if opts.Hub != "" { + meta := DetectHubMeta(opts.ProjectPath, HubMeta{ + Prefix: opts.HubPrefix, Project: opts.HubProject, Solution: opts.HubSolution, + Branch: opts.HubBranch, Worktree: opts.HubWorktree, + }) + fmt.Fprintf(w, "Registering with hub %s...\n", opts.Hub) + hubReg, err = RegisterWithHub(opts.Hub, opts.HubSecret, meta, opts.AppPort) + if err != nil { + return fmt.Errorf("hub registration: %w", err) + } + appRootURL = hubReg.URL + } + // 6. Boot the runtime against the fresh deployment. rt, err := StartLocalRuntime(LocalRuntimeOptions{ DeployDir: opts.DeployDir, InstallPath: installPath, // JavaHome left empty: StartLocalRuntime resolves JDK 21. - AppPort: opts.AppPort, - AdminPort: opts.AdminPort, - AdminPass: opts.AdminPass, - DB: opts.DB, - Stdout: w, - Stderr: stderr, + AppPort: opts.AppPort, + AdminPort: opts.AdminPort, + AdminPass: opts.AdminPass, + ApplicationRootUrl: appRootURL, + DB: opts.DB, + Stdout: w, + Stderr: stderr, }) if err != nil { return err @@ -439,6 +476,29 @@ func RunLocal(opts LocalRunOptions) error { fmt.Fprintf(w, "\nApp is running at %s\n", rt.AppURL()) + // 6a. With --hub, open a reverse tunnel so the app is reachable in a browser at + // its public URL, and heartbeat so it shows as available in the hub overview. + // The app stays here; only live HTTP flows through the tunnel (nothing pushed). + if hubReg != nil { + tunnel, err := StartTunnel(TunnelOptions{ + HubURL: hubReg.ControlURL, + LocalPort: opts.AppPort, + RemotePort: hubReg.ReversePort, + Secret: hubReg.TunnelAuth, + PublicURL: hubReg.URL, + Stdout: w, + }) + if err != nil { + return fmt.Errorf("starting hub tunnel: %w", err) + } + defer tunnel.Stop() + + hb := StartHeartbeat(hubReg) + defer hb.Stop() + + fmt.Fprintf(w, "Preview available at %s\n", hubReg.URL) + } + // 6b. If screenshot auth was requested, log in once and reuse the session for // every screenshot (pages behind login render authenticated). if opts.Screenshot && opts.ScreenshotUser != "" { diff --git a/cmd/mxcli/docker/tunnel.go b/cmd/mxcli/docker/tunnel.go new file mode 100644 index 000000000..0e7034b4d --- /dev/null +++ b/cmd/mxcli/docker/tunnel.go @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: Apache-2.0 + +package docker + +import ( + "context" + "fmt" + "io" + "net/url" + "os" + "strings" + "time" + + chclient "github.com/jpillora/chisel/client" + "golang.org/x/net/http/httpproxy" +) + +// DefaultHubBackendPort is the port an mxcli tunnel-hub proxies public requests +// to (its chisel server's --backend), and the reverse port the client tunnels +// into. The two must agree; both default to 9000. +const DefaultHubBackendPort = 9000 + +// TunnelOptions configures an outbound chisel reverse tunnel from a locally +// running app to an mxcli tunnel-hub, so the app is reachable in a browser at +// the hub's public URL. The app never leaves this machine — only live HTTP flows +// through the tunnel. +type TunnelOptions struct { + // HubURL is the tunnel-hub base URL, e.g. https://hub.example.com. The chisel + // control connection dials it over 443. + HubURL string + // LocalPort is the local app port to expose (e.g. 8080). + LocalPort int + // RemotePort is the hub's chisel-server reverse port, which the hub proxies + // public traffic to (must match the hub's --backend port). Default 9000. + RemotePort int + // Secret is the shared chisel auth ("user:pass"), matching the hub's --secret. + // Optional but recommended. + Secret string + // Proxy is the outbound HTTP CONNECT proxy the control connection dials + // through. In a Claude Code web session egress is proxy-only, so this must be + // set; it defaults from HTTPS_PROXY/https_proxy in the environment. chisel does + // not read the proxy env itself, so we pass it explicitly. + Proxy string + // PublicURL is the browser-facing URL the app is served at (an assigned + // subdomain on a multi-tenant hub). Defaults to HubURL when empty. + PublicURL string + // Stdout receives progress messages (default os.Stdout). + Stdout io.Writer +} + +// Tunnel is a running reverse tunnel to a hub. +type Tunnel struct { + client *chclient.Client + cancel context.CancelFunc + publicURL string +} + +func (o *TunnelOptions) applyDefaults() { + if o.RemotePort == 0 { + o.RemotePort = DefaultHubBackendPort + } + if o.Proxy == "" { + o.Proxy = proxyForURL(o.HubURL) + } + if o.Stdout == nil { + o.Stdout = os.Stdout + } +} + +// proxyForURL resolves the outbound HTTP proxy for hubURL from the standard +// proxy environment (HTTPS_PROXY etc.), honouring NO_PROXY — so an external hub +// goes through the egress proxy while a loopback or allow-listed hub connects +// directly. chisel does not consult the proxy env itself, so we do it here. +func proxyForURL(hubURL string) string { + u, err := url.Parse(hubURL) + if err != nil || u.Host == "" { + return "" + } + p, err := httpproxy.FromEnvironment().ProxyFunc()(u) + if err != nil || p == nil { + return "" + } + return p.String() +} + +// StartTunnel opens the reverse tunnel and returns once the chisel client has +// started connecting (it retries in the background until the process exits). +// Call Stop to tear it down. +func StartTunnel(o TunnelOptions) (*Tunnel, error) { + o.applyDefaults() + if o.HubURL == "" { + return nil, fmt.Errorf("hub URL is required") + } + if o.LocalPort == 0 { + return nil, fmt.Errorf("local app port is required") + } + + // R::127.0.0.1: — the hub's chisel server listens on + // and forwards to this app's port (the app binds 127.0.0.1). + remote := fmt.Sprintf("R:%d:127.0.0.1:%d", o.RemotePort, o.LocalPort) + cfg := &chclient.Config{ + Server: o.HubURL, + Proxy: o.Proxy, + Auth: o.Secret, + Remotes: []string{remote}, + KeepAlive: 25 * time.Second, + MaxRetryCount: -1, // retry forever: survive a hub restart or network blip + MaxRetryInterval: 30 * time.Second, + } + c, err := chclient.NewClient(cfg) + if err != nil { + return nil, fmt.Errorf("configuring tunnel client: %w", err) + } + + ctx, cancel := context.WithCancel(context.Background()) + if err := c.Start(ctx); err != nil { + cancel() + return nil, fmt.Errorf("starting tunnel: %w", err) + } + + public := o.PublicURL + if public == "" { + public = o.HubURL + } + t := &Tunnel{client: c, cancel: cancel, publicURL: strings.TrimRight(public, "/")} + via := "" + if o.Proxy != "" { + via = " (via proxy)" + } + fmt.Fprintf(o.Stdout, "Tunnel: exposing local :%d at %s%s\n", o.LocalPort, t.publicURL, via) + return t, nil +} + +// PublicURL is the browser-reachable URL the app is served at through the hub. +func (t *Tunnel) PublicURL() string { return t.publicURL } + +// Stop tears down the tunnel. +func (t *Tunnel) Stop() { + if t == nil { + return + } + if t.cancel != nil { + t.cancel() + } + if t.client != nil { + _ = t.client.Close() + } +} diff --git a/cmd/mxcli/docker/tunnel_test.go b/cmd/mxcli/docker/tunnel_test.go new file mode 100644 index 000000000..09cd49582 --- /dev/null +++ b/cmd/mxcli/docker/tunnel_test.go @@ -0,0 +1,135 @@ +// SPDX-License-Identifier: Apache-2.0 + +package docker + +import ( + "fmt" + "io" + "net" + "net/http" + "net/http/httptest" + "net/url" + "strconv" + "testing" + "time" + + chserver "github.com/jpillora/chisel/server" +) + +// proxyForURL must route an external hub through the egress proxy but connect to +// a loopback / NO_PROXY hub directly — otherwise a local hub (or an allow-listed +// one) would be forced through a proxy that refuses it. +func TestProxyForURL(t *testing.T) { + t.Setenv("HTTPS_PROXY", "http://127.0.0.1:33451") + t.Setenv("HTTP_PROXY", "http://127.0.0.1:33451") + t.Setenv("NO_PROXY", "127.0.0.1,localhost,.internal.example") + + cases := []struct { + url string + want string + }{ + {"https://hub.example.com", "http://127.0.0.1:33451"}, + {"http://127.0.0.1:9500", ""}, // loopback → NO_PROXY + {"https://relay.internal.example", ""}, // suffix match in NO_PROXY + {"", ""}, // no URL + {"://bad", ""}, // unparseable + } + for _, c := range cases { + if got := proxyForURL(c.url); got != c.want { + t.Errorf("proxyForURL(%q) = %q, want %q", c.url, got, c.want) + } + } +} + +// StartTunnel must reverse-tunnel a local port out to a hub so requests to the +// hub's reverse port reach the local app. This exercises the real embedded +// chisel client + server end to end, in-process (no external binary). +func TestTunnelRoundTrip(t *testing.T) { + // The "local app" the tunnel exposes. + backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + fmt.Fprintf(w, "hello from %s", r.Host) + })) + defer backend.Close() + backendPort := mustPort(t, backend.URL) + + // The hub: a chisel reverse server on a free port. + hubPort := freePort(t) + srv, err := chserver.NewServer(&chserver.Config{Reverse: true}) + if err != nil { + t.Fatalf("NewServer: %v", err) + } + if err := srv.Start("127.0.0.1", strconv.Itoa(hubPort)); err != nil { + t.Fatalf("hub Start: %v", err) + } + defer srv.Close() + + // The reverse port the hub will open and forward to the backend. + remotePort := freePort(t) + + tun, err := StartTunnel(TunnelOptions{ + HubURL: "http://127.0.0.1:" + strconv.Itoa(hubPort), + LocalPort: backendPort, + RemotePort: remotePort, + Proxy: "", // loopback: no proxy + Stdout: io.Discard, + }) + if err != nil { + t.Fatalf("StartTunnel: %v", err) + } + defer tun.Stop() + + if tun.PublicURL() != "http://127.0.0.1:"+strconv.Itoa(hubPort) { + t.Errorf("PublicURL = %q", tun.PublicURL()) + } + + // The client connects + opens the reverse listener asynchronously; poll it. + reverseURL := fmt.Sprintf("http://127.0.0.1:%d/", remotePort) + body := pollGet(t, reverseURL, 10*time.Second) + if body == "" { + t.Fatalf("no response through tunnel at %s", reverseURL) + } + // The request reached the backend through the tunnel. + if want := "hello from "; len(body) < len(want) || body[:len(want)] != want { + t.Errorf("through-tunnel body = %q, want prefix %q", body, want) + } +} + +func mustPort(t *testing.T, rawURL string) int { + t.Helper() + u, err := url.Parse(rawURL) + if err != nil { + t.Fatalf("parse %q: %v", rawURL, err) + } + p, err := strconv.Atoi(u.Port()) + if err != nil { + t.Fatalf("port of %q: %v", rawURL, err) + } + return p +} + +func freePort(t *testing.T) int { + t.Helper() + l, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatalf("free port: %v", err) + } + defer l.Close() + return l.Addr().(*net.TCPAddr).Port +} + +func pollGet(t *testing.T, url string, timeout time.Duration) string { + t.Helper() + deadline := time.Now().Add(timeout) + for time.Now().Before(deadline) { + resp, err := http.Get(url) + if err == nil { + b, _ := io.ReadAll(resp.Body) + resp.Body.Close() + if resp.StatusCode == http.StatusOK { + return string(b) + } + } + time.Sleep(150 * time.Millisecond) + } + return "" +} diff --git a/cmd/mxcli/tunnelhub/admin.go b/cmd/mxcli/tunnelhub/admin.go new file mode 100644 index 000000000..73f14980c --- /dev/null +++ b/cmd/mxcli/tunnelhub/admin.go @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import "net/http" + +// NewAdmin returns the handler for the hub's admin overview page. The page is +// self-contained (inline CSS/JS, no external assets) and refreshes itself from +// GET /api/backends. +func NewAdmin(_ *Registry) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/" { + http.NotFound(w, r) + return + } + w.Header().Set("Content-Type", "text/html; charset=utf-8") + _, _ = w.Write([]byte(adminHTML)) + }) +} + +const adminHTML = ` + + +mxcli tunnel-hub — previews + + +
+

mxcli tunnel-hub

+ … + +
+
+ + + + + + + + + + + + + +
StatusSolutionProjectBranchURLRegisteredLast seenLast usedUptime
+ +
+ +` diff --git a/cmd/mxcli/tunnelhub/api.go b/cmd/mxcli/tunnelhub/api.go new file mode 100644 index 000000000..c0a614d62 --- /dev/null +++ b/cmd/mxcli/tunnelhub/api.go @@ -0,0 +1,144 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import ( + "encoding/json" + "net/http" + "strings" +) + +// APIOptions configures the registration API. +type APIOptions struct { + Registry *Registry + // ControlURL is where the client points its chisel control connection + // (e.g. https://hub.example.com). Returned in the registration response. + ControlURL string + // TunnelAuth is the shared chisel auth ("user:pass") the client must use, if + // the hub's tunnel server requires one. Returned to the client. + TunnelAuth string + // RegisterSecret, if set, gates /api/register: the client must send a matching + // X-Hub-Secret header (from --hub-secret). Empty means open registration. + RegisterSecret string + // HeartbeatIntervalSec is how often the client should heartbeat (default 20). + HeartbeatIntervalSec int +} + +// API serves the hub's registration + query endpoints over the registry. +type API struct { + opts APIOptions +} + +// RegisterResponse is returned to `mxcli run --hub` after registration. +type RegisterResponse struct { + Subdomain string `json:"subdomain"` + URL string `json:"url"` + ReversePort int `json:"reversePort"` + ControlURL string `json:"controlUrl"` + Token string `json:"token"` + TunnelAuth string `json:"tunnelAuth,omitempty"` + HeartbeatIntervalSec int `json:"heartbeatIntervalSec"` +} + +// NewAPI builds the API handler set. +func NewAPI(o APIOptions) *API { + if o.HeartbeatIntervalSec == 0 { + o.HeartbeatIntervalSec = 20 + } + return &API{opts: o} +} + +// Mount registers the API routes on mux under /api/. +func (a *API) Mount(mux *http.ServeMux) { + mux.HandleFunc("/api/register", a.handleRegister) + mux.HandleFunc("/api/status", a.handleStatus) + mux.HandleFunc("/api/deregister", a.handleDeregister) + mux.HandleFunc("/api/backends", a.handleBackends) +} + +func (a *API) handleRegister(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodPost { + http.Error(w, "method not allowed", http.StatusMethodNotAllowed) + return + } + if a.opts.RegisterSecret != "" && r.Header.Get("X-Hub-Secret") != a.opts.RegisterSecret { + http.Error(w, "invalid or missing hub secret", http.StatusUnauthorized) + return + } + var req RegisterRequest + if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<16)).Decode(&req); err != nil { + http.Error(w, "invalid request body: "+err.Error(), http.StatusBadRequest) + return + } + if strings.TrimSpace(req.Project) == "" { + http.Error(w, "project is required", http.StatusBadRequest) + return + } + b, err := a.opts.Registry.Register(req) + if err != nil { + http.Error(w, err.Error(), http.StatusServiceUnavailable) + return + } + host := b.Subdomain + if d := a.opts.Registry.domain; d != "" { + host = b.Subdomain + "." + d + } + writeJSON(w, http.StatusOK, RegisterResponse{ + Subdomain: b.Subdomain, + URL: "https://" + host, + ReversePort: b.ReversePort, + ControlURL: a.opts.ControlURL, + Token: b.ID, + TunnelAuth: a.opts.TunnelAuth, + HeartbeatIntervalSec: a.opts.HeartbeatIntervalSec, + }) +} + +func (a *API) handleStatus(w http.ResponseWriter, r *http.Request) { + a.byToken(w, r, func(token string) { + if !a.opts.Registry.Heartbeat(token) { + http.Error(w, "unknown token", http.StatusNotFound) + return + } + w.WriteHeader(http.StatusNoContent) + }) +} + +func (a *API) handleDeregister(w http.ResponseWriter, r *http.Request) { + a.byToken(w, r, func(token string) { + a.opts.Registry.Deregister(token) + w.WriteHeader(http.StatusNoContent) + }) +} + +// byToken extracts the bearer token (Authorization: Bearer or ?token=) and +// invokes fn. POST only. +func (a *API) byToken(w http.ResponseWriter, r *http.Request, fn func(token string)) { + if r.Method != http.MethodPost { + http.Error(w, "method not allowed", http.StatusMethodNotAllowed) + return + } + token := bearerToken(r) + if token == "" { + http.Error(w, "missing token", http.StatusUnauthorized) + return + } + fn(token) +} + +func (a *API) handleBackends(w http.ResponseWriter, r *http.Request) { + writeJSON(w, http.StatusOK, a.opts.Registry.List(r.URL.Query().Get("sort"))) +} + +func bearerToken(r *http.Request) string { + if h := r.Header.Get("Authorization"); strings.HasPrefix(h, "Bearer ") { + return strings.TrimSpace(strings.TrimPrefix(h, "Bearer ")) + } + return strings.TrimSpace(r.URL.Query().Get("token")) +} + +func writeJSON(w http.ResponseWriter, status int, v any) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(status) + _ = json.NewEncoder(w).Encode(v) +} diff --git a/cmd/mxcli/tunnelhub/api_test.go b/cmd/mxcli/tunnelhub/api_test.go new file mode 100644 index 000000000..380e9fcba --- /dev/null +++ b/cmd/mxcli/tunnelhub/api_test.go @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import ( + "encoding/json" + "net/http" + "net/http/httptest" + "strings" + "testing" + "time" +) + +func newTestAPI(t *testing.T, secret string) (*API, *Registry) { + t.Helper() + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + reg := newTestRegistry(clk) + api := NewAPI(APIOptions{ + Registry: reg, + ControlURL: "https://hub.example.com", + RegisterSecret: secret, + }) + return api, reg +} + +func doJSON(t *testing.T, api *API, method, path, body string, headers map[string]string) *httptest.ResponseRecorder { + t.Helper() + mux := http.NewServeMux() + api.Mount(mux) + req := httptest.NewRequest(method, path, strings.NewReader(body)) + for k, v := range headers { + req.Header.Set(k, v) + } + rec := httptest.NewRecorder() + mux.ServeHTTP(rec, req) + return rec +} + +func TestAPI_RegisterAndBackends(t *testing.T) { + api, _ := newTestAPI(t, "") + rec := doJSON(t, api, http.MethodPost, "/api/register", + `{"project":"MyApp","solution":"Sol","branch":"feature/x","appPort":8080}`, nil) + if rec.Code != http.StatusOK { + t.Fatalf("register status = %d, body %s", rec.Code, rec.Body) + } + var resp RegisterResponse + if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil { + t.Fatal(err) + } + if resp.URL != "https://myapp-feature-x.example.com" { + t.Errorf("url = %q", resp.URL) + } + if resp.ReversePort == 0 || resp.Token == "" || resp.ControlURL != "https://hub.example.com" { + t.Errorf("incomplete response: %+v", resp) + } + + // backends lists it. + lb := doJSON(t, api, http.MethodGet, "/api/backends?sort=project", "", nil) + var views []BackendView + if err := json.Unmarshal(lb.Body.Bytes(), &views); err != nil { + t.Fatal(err) + } + if len(views) != 1 || views[0].Project != "MyApp" || views[0].Availability != Available { + t.Errorf("backends = %+v", views) + } +} + +func TestAPI_RegisterRequiresProject(t *testing.T) { + api, _ := newTestAPI(t, "") + rec := doJSON(t, api, http.MethodPost, "/api/register", `{"branch":"main"}`, nil) + if rec.Code != http.StatusBadRequest { + t.Errorf("status = %d, want 400", rec.Code) + } +} + +func TestAPI_RegisterSecretGate(t *testing.T) { + api, _ := newTestAPI(t, "s3cret") + // missing secret -> 401 + if rec := doJSON(t, api, http.MethodPost, "/api/register", `{"project":"A"}`, nil); rec.Code != http.StatusUnauthorized { + t.Errorf("no secret: status = %d, want 401", rec.Code) + } + // correct secret -> 200 + rec := doJSON(t, api, http.MethodPost, "/api/register", `{"project":"A"}`, + map[string]string{"X-Hub-Secret": "s3cret"}) + if rec.Code != http.StatusOK { + t.Errorf("with secret: status = %d, want 200", rec.Code) + } +} + +func TestAPI_HeartbeatAndDeregister(t *testing.T) { + api, reg := newTestAPI(t, "") + var resp RegisterResponse + rec := doJSON(t, api, http.MethodPost, "/api/register", `{"project":"A","branch":"main"}`, nil) + _ = json.Unmarshal(rec.Body.Bytes(), &resp) + + // heartbeat with the token succeeds. + hb := doJSON(t, api, http.MethodPost, "/api/status", "", + map[string]string{"Authorization": "Bearer " + resp.Token}) + if hb.Code != http.StatusNoContent { + t.Errorf("heartbeat status = %d, want 204", hb.Code) + } + // bad token -> 404. + bad := doJSON(t, api, http.MethodPost, "/api/status", "", + map[string]string{"Authorization": "Bearer nope"}) + if bad.Code != http.StatusNotFound { + t.Errorf("bad-token heartbeat status = %d, want 404", bad.Code) + } + // missing token -> 401. + if none := doJSON(t, api, http.MethodPost, "/api/status", "", nil); none.Code != http.StatusUnauthorized { + t.Errorf("no-token heartbeat status = %d, want 401", none.Code) + } + // deregister removes it. + dr := doJSON(t, api, http.MethodPost, "/api/deregister", "", + map[string]string{"Authorization": "Bearer " + resp.Token}) + if dr.Code != http.StatusNoContent { + t.Errorf("deregister status = %d, want 204", dr.Code) + } + if len(reg.List("used")) != 0 { + t.Error("backend should be gone after deregister") + } +} + +func TestAPI_RegisterMethodNotAllowed(t *testing.T) { + api, _ := newTestAPI(t, "") + if rec := doJSON(t, api, http.MethodGet, "/api/register", "", nil); rec.Code != http.StatusMethodNotAllowed { + t.Errorf("GET register status = %d, want 405", rec.Code) + } +} diff --git a/cmd/mxcli/tunnelhub/integration_test.go b/cmd/mxcli/tunnelhub/integration_test.go new file mode 100644 index 000000000..3e6653a77 --- /dev/null +++ b/cmd/mxcli/tunnelhub/integration_test.go @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import ( + "context" + "fmt" + "net" + "net/http" + "net/http/httptest" + "strconv" + "testing" + "time" + + chclient "github.com/jpillora/chisel/client" +) + +func freePort(t *testing.T) int { + t.Helper() + l, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatalf("free port: %v", err) + } + defer l.Close() + return l.Addr().(*net.TCPAddr).Port +} + +func mustPort(t *testing.T, rawURL string) int { + t.Helper() + _, portStr, err := net.SplitHostPort(rawURL[len("http://"):]) + if err != nil { + t.Fatalf("split %q: %v", rawURL, err) + } + p, err := strconv.Atoi(portStr) + if err != nil { + t.Fatalf("port %q: %v", portStr, err) + } + return p +} + +// End-to-end, in-process: a client reverse-tunnels a backend to the hub's +// embedded chisel server, then a request to that backend's subdomain routes +// through the tunnel and returns the backend's response. Exercises registration +// -> subdomain routing -> chisel reverse tunnel -> app. +func TestFront_ProxiesThroughTunnel(t *testing.T) { + // The "app" being previewed. It echoes the Host it sees, so we can assert the + // front preserves the public host rather than the loopback backend host. + backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + fmt.Fprintf(w, "TUNNELED host=%s", r.Host) + })) + defer backend.Close() + backendPort := mustPort(t, backend.URL) + + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + reg := NewRegistry(RegistryOptions{Domain: "example.com", PortBase: freePort(t), PortCount: 20, Now: clk.now}) + chiselPort := freePort(t) + srv, err := NewServer(ServerOptions{ + Domain: "example.com", + Registry: reg, + ChiselAddr: "127.0.0.1:" + strconv.Itoa(chiselPort), + CertCacheDir: t.TempDir(), + }) + if err != nil { + t.Fatalf("NewServer: %v", err) + } + // Start just the embedded chisel server (Start() would also bind TLS). + if err := srv.chisel.Start("127.0.0.1", strconv.Itoa(chiselPort)); err != nil { + t.Fatalf("chisel start: %v", err) + } + defer srv.chisel.Close() + + // Register the preview -> assigned reverse port. + b, err := reg.Register(RegisterRequest{Project: "App", Branch: "main", AppPort: backendPort}) + if err != nil { + t.Fatalf("Register: %v", err) + } + + // The client tunnels the backend to the assigned reverse port. + client, err := chclient.NewClient(&chclient.Config{ + Server: "http://127.0.0.1:" + strconv.Itoa(chiselPort), + Remotes: []string{fmt.Sprintf("R:%d:127.0.0.1:%d", b.ReversePort, backendPort)}, + }) + if err != nil { + t.Fatalf("client: %v", err) + } + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + if err := client.Start(ctx); err != nil { + t.Fatalf("client start: %v", err) + } + defer client.Close() + + // A request to app.example.com must route through the tunnel to the backend. + deadline := time.Now().Add(10 * time.Second) + var body string + for time.Now().Before(deadline) { + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req("app.example.com", "/")) + if rec.Code == http.StatusOK { + body = rec.Body.String() + break + } + time.Sleep(150 * time.Millisecond) + } + if body == "" { + t.Fatal("no successful response through the tunnel within timeout") + } + if want := "TUNNELED host=app.example.com"; body != want { + t.Errorf("through-tunnel body = %q, want %q (public Host must be preserved)", body, want) + } +} diff --git a/cmd/mxcli/tunnelhub/registry.go b/cmd/mxcli/tunnelhub/registry.go new file mode 100644 index 000000000..f2debb32d --- /dev/null +++ b/cmd/mxcli/tunnelhub/registry.go @@ -0,0 +1,319 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import ( + "crypto/rand" + "encoding/hex" + "fmt" + "sort" + "strings" + "sync" + "time" +) + +// Availability is a backend's liveness, derived from heartbeat freshness. +type Availability string + +const ( + // Available: the container heartbeat is fresh — the app should be reachable. + Available Availability = "available" + // Stale: no recent heartbeat (e.g. a Claude Code web container reaped on idle), + // but not yet expired — shown so the user can see it went away. + Stale Availability = "stale" +) + +// Backend is one registered preview: a locally-running app reverse-tunnelled to +// the hub and served at its subdomain. +type Backend struct { + ID string `json:"id"` // opaque token (auth for heartbeat/deregister + chisel) + Prefix string `json:"prefix"` // optional hostname namespace (org/solution/team/env) + Project string `json:"project"` // e.g. the .mpr name + Solution string `json:"solution"` // optional grouping for multi-app solutions + Branch string `json:"branch"` // git branch + Worktree string `json:"worktree"` // optional, distinguishes worktrees of one branch + Subdomain string `json:"subdomain"` + ReversePort int `json:"reversePort"` + AppPort int `json:"appPort"` + + RegisteredAt time.Time `json:"registeredAt"` + LastSeenAt time.Time `json:"lastSeenAt"` // last heartbeat + LastUsedAt time.Time `json:"lastUsedAt"` // last browser request to the subdomain +} + +// identity is the stable key for a preview across reconnects: same prefix + +// project + branch + worktree + solution re-registers to the same slot (stable URL). +func (b *Backend) identity() string { + return strings.Join([]string{b.Prefix, b.Solution, b.Project, b.Branch, b.Worktree}, "\x00") +} + +// BackendView is a Backend plus derived fields, for the API/admin page. +type BackendView struct { + Backend + URL string `json:"url"` + Availability Availability `json:"availability"` + UptimeSec int64 `json:"uptimeSec"` +} + +// RegisterRequest is the registration payload from `mxcli run --hub`. +type RegisterRequest struct { + Prefix string `json:"prefix"` + Project string `json:"project"` + Solution string `json:"solution"` + Branch string `json:"branch"` + Worktree string `json:"worktree"` + AppPort int `json:"appPort"` +} + +// Registry is the in-memory store of registered backends. All methods are safe +// for concurrent use. +type Registry struct { + mu sync.Mutex + byID map[string]*Backend + bySubdomain map[string]*Backend + byIdentity map[string]*Backend + usedPorts map[int]bool + + domain string // e.g. "example.com" + portBase int // first reverse port to allocate + portCount int // number of reverse ports available + staleFor time.Duration + expireFor time.Duration + now func() time.Time +} + +// RegistryOptions configures a Registry. Zero values get sensible defaults. +type RegistryOptions struct { + Domain string + PortBase int + PortCount int + StaleFor time.Duration // no heartbeat within this -> Stale (default 45s) + ExpireFor time.Duration // no heartbeat within this -> removed (default 10m) + Now func() time.Time +} + +// NewRegistry creates an empty registry. +func NewRegistry(o RegistryOptions) *Registry { + if o.PortBase == 0 { + o.PortBase = 9001 + } + if o.PortCount == 0 { + o.PortCount = 200 + } + if o.StaleFor == 0 { + o.StaleFor = 45 * time.Second + } + if o.ExpireFor == 0 { + o.ExpireFor = 10 * time.Minute + } + if o.Now == nil { + o.Now = time.Now + } + return &Registry{ + byID: map[string]*Backend{}, + bySubdomain: map[string]*Backend{}, + byIdentity: map[string]*Backend{}, + usedPorts: map[int]bool{}, + domain: o.Domain, + portBase: o.PortBase, + portCount: o.PortCount, + staleFor: o.StaleFor, + expireFor: o.ExpireFor, + now: o.Now, + } +} + +// Register allocates (or refreshes) a backend for the request and returns it. A +// re-registration with the same identity (project/branch/worktree/solution) +// returns the existing slot with a fresh heartbeat, so URLs are stable across +// reconnects. +func (r *Registry) Register(req RegisterRequest) (*Backend, error) { + r.mu.Lock() + defer r.mu.Unlock() + r.reapLocked() + + now := r.now() + b := &Backend{ + Prefix: strings.TrimSpace(req.Prefix), + Project: strings.TrimSpace(req.Project), + Solution: strings.TrimSpace(req.Solution), + Branch: strings.TrimSpace(req.Branch), + Worktree: strings.TrimSpace(req.Worktree), + AppPort: req.AppPort, + } + if existing, ok := r.byIdentity[b.identity()]; ok { + existing.LastSeenAt = now + existing.AppPort = req.AppPort + return existing, nil + } + + port, err := r.allocPortLocked() + if err != nil { + return nil, err + } + b.ID = newToken() + b.Subdomain = r.allocSubdomainLocked(b.Prefix, b.Project, b.Branch, b.Worktree) + b.ReversePort = port + b.RegisteredAt = now + b.LastSeenAt = now + b.LastUsedAt = time.Time{} + + r.byID[b.ID] = b + r.bySubdomain[b.Subdomain] = b + r.byIdentity[b.identity()] = b + r.usedPorts[port] = true + return b, nil +} + +// Heartbeat refreshes a backend's liveness by token. +func (r *Registry) Heartbeat(id string) bool { + r.mu.Lock() + defer r.mu.Unlock() + b, ok := r.byID[id] + if !ok { + return false + } + b.LastSeenAt = r.now() + return true +} + +// TouchUsed records a browser request to a subdomain (updates LastUsedAt). +func (r *Registry) TouchUsed(subdomain string) { + r.mu.Lock() + defer r.mu.Unlock() + if b, ok := r.bySubdomain[subdomain]; ok { + b.LastUsedAt = r.now() + } +} + +// Deregister removes a backend by token. +func (r *Registry) Deregister(id string) bool { + r.mu.Lock() + defer r.mu.Unlock() + b, ok := r.byID[id] + if !ok { + return false + } + r.removeLocked(b) + return true +} + +// LookupSubdomain returns the backend serving a subdomain. +func (r *Registry) LookupSubdomain(subdomain string) (*Backend, bool) { + r.mu.Lock() + defer r.mu.Unlock() + b, ok := r.bySubdomain[subdomain] + if !ok { + return nil, false + } + cp := *b + return &cp, true +} + +// List returns a snapshot of all backends as views, sorted by the given key +// ("used", "registered", "project"; default "used"), most-recent/first. +func (r *Registry) List(sortKey string) []BackendView { + r.mu.Lock() + defer r.mu.Unlock() + r.reapLocked() + + out := make([]BackendView, 0, len(r.byID)) + for _, b := range r.byID { + out = append(out, r.viewLocked(b)) + } + sortViews(out, sortKey) + return out +} + +// viewLocked builds the derived view for a backend. +func (r *Registry) viewLocked(b *Backend) BackendView { + host := b.Subdomain + if r.domain != "" { + host = b.Subdomain + "." + r.domain + } + av := Available + if r.now().Sub(b.LastSeenAt) > r.staleFor { + av = Stale + } + return BackendView{ + Backend: *b, + URL: "https://" + host, + Availability: av, + UptimeSec: int64(r.now().Sub(b.RegisteredAt).Seconds()), + } +} + +// reapLocked removes backends whose heartbeat is older than expireFor. +func (r *Registry) reapLocked() { + cutoff := r.now().Add(-r.expireFor) + for _, b := range r.byID { + if b.LastSeenAt.Before(cutoff) { + r.removeLocked(b) + } + } +} + +func (r *Registry) removeLocked(b *Backend) { + delete(r.byID, b.ID) + delete(r.bySubdomain, b.Subdomain) + delete(r.byIdentity, b.identity()) + delete(r.usedPorts, b.ReversePort) +} + +// allocPortLocked returns a free reverse port from the configured range. +func (r *Registry) allocPortLocked() (int, error) { + for p := r.portBase; p < r.portBase+r.portCount; p++ { + if !r.usedPorts[p] { + return p, nil + } + } + return 0, fmt.Errorf("no free reverse port (all %d in use)", r.portCount) +} + +// allocSubdomainLocked returns a unique subdomain slug, disambiguating a +// collision with the worktree name then a numeric suffix. +func (r *Registry) allocSubdomainLocked(prefix, project, branch, worktree string) string { + base := baseSlug(prefix, project, branch) + if _, taken := r.bySubdomain[base]; !taken { + return base + } + if wt := slugify(worktree); wt != "" { + cand := truncateLabel(base + "-" + wt) + if _, taken := r.bySubdomain[cand]; !taken { + return cand + } + } + for i := 2; ; i++ { + cand := truncateLabel(fmt.Sprintf("%s-%d", base, i)) + if _, taken := r.bySubdomain[cand]; !taken { + return cand + } + } +} + +func newToken() string { + var b [16]byte + _, _ = rand.Read(b[:]) + return hex.EncodeToString(b[:]) +} + +// sortViews orders views in place by key, newest first for time keys. +func sortViews(v []BackendView, key string) { + less := map[string]func(a, b BackendView) bool{ + "registered": func(a, b BackendView) bool { return a.RegisteredAt.After(b.RegisteredAt) }, + "project": func(a, b BackendView) bool { + if a.Solution != b.Solution { + return a.Solution < b.Solution + } + if a.Project != b.Project { + return a.Project < b.Project + } + return a.Branch < b.Branch + }, + "used": func(a, b BackendView) bool { return a.LastUsedAt.After(b.LastUsedAt) }, + }[key] + if less == nil { + less = func(a, b BackendView) bool { return a.LastUsedAt.After(b.LastUsedAt) } + } + sort.SliceStable(v, func(i, j int) bool { return less(v[i], v[j]) }) +} diff --git a/cmd/mxcli/tunnelhub/registry_test.go b/cmd/mxcli/tunnelhub/registry_test.go new file mode 100644 index 000000000..1d1a3ff0c --- /dev/null +++ b/cmd/mxcli/tunnelhub/registry_test.go @@ -0,0 +1,184 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import ( + "testing" + "time" +) + +// fakeClock is a controllable time source for deterministic tests. +type fakeClock struct{ t time.Time } + +func (c *fakeClock) now() time.Time { return c.t } +func (c *fakeClock) add(d time.Duration) { c.t = c.t.Add(d) } + +func newTestRegistry(clk *fakeClock) *Registry { + return NewRegistry(RegistryOptions{ + Domain: "example.com", + PortBase: 9001, + PortCount: 5, + StaleFor: 45 * time.Second, + ExpireFor: 10 * time.Minute, + Now: clk.now, + }) +} + +func TestRegister_AllocatesSubdomainAndPort(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := newTestRegistry(clk) + + b, err := r.Register(RegisterRequest{Project: "MyApp", Branch: "feature/x", AppPort: 8080}) + if err != nil { + t.Fatalf("Register: %v", err) + } + if b.Subdomain != "myapp-feature-x" { + t.Errorf("subdomain = %q, want myapp-feature-x", b.Subdomain) + } + if b.ReversePort != 9001 { + t.Errorf("reversePort = %d, want 9001", b.ReversePort) + } + if b.ID == "" { + t.Error("token must be set") + } + // main branch collapses to just the project. + m, _ := r.Register(RegisterRequest{Project: "MyApp", Branch: "main"}) + if m.Subdomain != "myapp" { + t.Errorf("main-branch subdomain = %q, want myapp", m.Subdomain) + } +} + +func TestRegister_Prefix(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := newTestRegistry(clk) + + // prefix namespaces the hostname: -[-]. + a, _ := r.Register(RegisterRequest{Prefix: "AcmeCorp", Project: "Portal", Branch: "feature/x"}) + if a.Subdomain != "acmecorp-portal-feature-x" { + t.Errorf("prefixed subdomain = %q, want acmecorp-portal-feature-x", a.Subdomain) + } + // main branch with a prefix -> -. + b, _ := r.Register(RegisterRequest{Prefix: "AcmeCorp", Project: "Portal", Branch: "main"}) + if b.Subdomain != "acmecorp-portal" { + t.Errorf("prefixed main subdomain = %q, want acmecorp-portal", b.Subdomain) + } + // same project, different prefix -> distinct slot (prefix is part of identity). + c, _ := r.Register(RegisterRequest{Prefix: "Other", Project: "Portal", Branch: "main"}) + if c.Subdomain != "other-portal" || c.ID == b.ID { + t.Errorf("different prefix should be a distinct backend: %q id=%s vs %s", c.Subdomain, c.ID, b.ID) + } +} + +func TestRegister_SameIdentityIsStable(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := newTestRegistry(clk) + + a, _ := r.Register(RegisterRequest{Project: "App", Branch: "main", AppPort: 8080}) + clk.add(30 * time.Second) + b, _ := r.Register(RegisterRequest{Project: "App", Branch: "main", AppPort: 8080}) + + if a.ID != b.ID || a.Subdomain != b.Subdomain || a.ReversePort != b.ReversePort { + t.Errorf("re-register changed the slot: %+v vs %+v", a, b) + } + if !b.LastSeenAt.Equal(clk.t) { + t.Error("re-register should refresh the heartbeat") + } + if got := r.List("used"); len(got) != 1 { + t.Errorf("want 1 backend after re-register, got %d", len(got)) + } +} + +func TestRegister_CollisionDisambiguation(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := newTestRegistry(clk) + + // Same project+branch, different worktree -> worktree disambiguates. + a, _ := r.Register(RegisterRequest{Project: "App", Branch: "dev", Worktree: "wt-a"}) + b, _ := r.Register(RegisterRequest{Project: "App", Branch: "dev", Worktree: "wt-b"}) + if a.Subdomain != "app-dev" { + t.Errorf("first subdomain = %q, want app-dev", a.Subdomain) + } + if b.Subdomain != "app-dev-wt-b" { + t.Errorf("second subdomain = %q, want app-dev-wt-b", b.Subdomain) + } + // A third with no distinguishing worktree -> numeric suffix. + c, _ := r.Register(RegisterRequest{Project: "App", Branch: "dev"}) + if c.Subdomain != "app-dev-2" { + t.Errorf("third subdomain = %q, want app-dev-2", c.Subdomain) + } +} + +func TestAvailability_StaleAfterNoHeartbeat(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := newTestRegistry(clk) + b, _ := r.Register(RegisterRequest{Project: "App", Branch: "main"}) + + if av := r.List("used")[0].Availability; av != Available { + t.Errorf("fresh backend = %q, want available", av) + } + clk.add(60 * time.Second) // past StaleFor (45s), before ExpireFor + if av := r.List("used")[0].Availability; av != Stale { + t.Errorf("after 60s = %q, want stale", av) + } + // A heartbeat brings it back. + r.Heartbeat(b.ID) + if av := r.List("used")[0].Availability; av != Available { + t.Errorf("after heartbeat = %q, want available", av) + } +} + +func TestReap_RemovesExpiredAndFreesPort(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := newTestRegistry(clk) + first, _ := r.Register(RegisterRequest{Project: "App", Branch: "main"}) + + clk.add(11 * time.Minute) // past ExpireFor + // List triggers a reap. + if got := r.List("used"); len(got) != 0 { + t.Fatalf("expired backend should be reaped, got %d", len(got)) + } + // Its port is freed and reusable. + next, _ := r.Register(RegisterRequest{Project: "Other", Branch: "main"}) + if next.ReversePort != first.ReversePort { + t.Errorf("freed port %d should be reused, got %d", first.ReversePort, next.ReversePort) + } +} + +func TestList_Sorting(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := newTestRegistry(clk) + + a, _ := r.Register(RegisterRequest{Project: "Beta", Solution: "S", Branch: "main"}) + clk.add(time.Second) + b, _ := r.Register(RegisterRequest{Project: "Alpha", Solution: "S", Branch: "main"}) + + // used: Beta (a) is touched last -> most recently used -> first. + r.TouchUsed(b.Subdomain) + clk.add(time.Second) + r.TouchUsed(a.Subdomain) + if got := r.List("used"); got[0].Project != "Beta" { + t.Errorf("used sort: first = %q, want Beta", got[0].Project) + } + // registered: newest first -> Alpha. + if got := r.List("registered"); got[0].Project != "Alpha" { + t.Errorf("registered sort: first = %q, want Alpha", got[0].Project) + } + // project: alphabetical -> Alpha. + if got := r.List("project"); got[0].Project != "Alpha" { + t.Errorf("project sort: first = %q, want Alpha", got[0].Project) + } +} + +func TestPortExhaustion(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := NewRegistry(RegistryOptions{Domain: "example.com", PortBase: 9001, PortCount: 2, Now: clk.now}) + if _, err := r.Register(RegisterRequest{Project: "A", Branch: "main"}); err != nil { + t.Fatal(err) + } + if _, err := r.Register(RegisterRequest{Project: "B", Branch: "main"}); err != nil { + t.Fatal(err) + } + if _, err := r.Register(RegisterRequest{Project: "C", Branch: "main"}); err == nil { + t.Error("expected port-exhaustion error on the 3rd registration") + } +} diff --git a/cmd/mxcli/tunnelhub/server.go b/cmd/mxcli/tunnelhub/server.go new file mode 100644 index 000000000..4b4bfa2fa --- /dev/null +++ b/cmd/mxcli/tunnelhub/server.go @@ -0,0 +1,291 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import ( + "context" + "fmt" + "html" + "net/http" + "net/http/httputil" + "net/url" + "strings" + "time" + + chserver "github.com/jpillora/chisel/server" + "golang.org/x/crypto/acme/autocert" +) + +// ctxKey is the type for request-context values the front handler passes to the +// app reverse proxy. +type ctxKey int + +const ( + targetPortKey ctxKey = iota + publicHostKey +) + +// ServerOptions configures the multi-tenant hub front. +type ServerOptions struct { + // Domain is the wildcard base, e.g. "example.com". App previews are served at + // .. + Domain string + // HubHost is the control/admin/API host (default "hub."+Domain). Clients dial + // their chisel control connection here and the admin page lives here. + HubHost string + // Registry is the shared backend store. + Registry *Registry + // TunnelAuth is the shared chisel auth ("user:pass"); empty disables auth. + TunnelAuth string + // RegisterSecret optionally gates /api/register (matched against X-Hub-Secret). + RegisterSecret string + // CertCacheDir is the autocert certificate cache directory. + CertCacheDir string + // chiselAddr is the internal address the embedded chisel control server binds + // (default 127.0.0.1:8100). Not public — the front proxies the WS here. + ChiselAddr string +} + +// Server is the running multi-tenant hub: one embedded chisel reverse server +// (fanning in all client tunnels) behind a single-443 TLS front that routes by +// Host — the hub host to the admin/API/chisel-control, each preview subdomain to +// its tunnel. +type Server struct { + opts ServerOptions + reg *Registry + chisel *chserver.Server + manager *autocert.Manager + http *http.Server + apiMux *http.ServeMux + admin http.Handler + + chiselProxy *httputil.ReverseProxy // -> internal chisel control (WS) + appProxy *httputil.ReverseProxy // -> 127.0.0.1: (per request) +} + +// NewServer wires the registry, API, admin page, embedded chisel server, and the +// TLS front. Call Start to listen. +func NewServer(o ServerOptions) (*Server, error) { + if o.Domain == "" { + return nil, fmt.Errorf("Domain is required") + } + if o.HubHost == "" { + o.HubHost = "hub." + o.Domain + } + if o.ChiselAddr == "" { + o.ChiselAddr = "127.0.0.1:8100" + } + if o.Registry == nil { + return nil, fmt.Errorf("Registry is required") + } + + chisel, err := chserver.NewServer(&chserver.Config{Reverse: true, Auth: o.TunnelAuth}) + if err != nil { + return nil, fmt.Errorf("chisel server: %w", err) + } + + api := NewAPI(APIOptions{ + Registry: o.Registry, + ControlURL: "https://" + o.HubHost, + TunnelAuth: o.TunnelAuth, + RegisterSecret: o.RegisterSecret, + }) + apiMux := http.NewServeMux() + api.Mount(apiMux) + + s := &Server{ + opts: o, + reg: o.Registry, + chisel: chisel, + apiMux: apiMux, + admin: NewAdmin(o.Registry), + } + + // Front proxies: chisel control (WS) to the internal chisel server, and app + // traffic to the per-request reverse port (Host preserved as the public host). + chiselURL := &url.URL{Scheme: "http", Host: o.ChiselAddr} + s.chiselProxy = httputil.NewSingleHostReverseProxy(chiselURL) + + s.appProxy = &httputil.ReverseProxy{ + Director: func(req *http.Request) { + port, _ := req.Context().Value(targetPortKey).(int) + host, _ := req.Context().Value(publicHostKey).(string) + req.URL.Scheme = "http" + req.URL.Host = fmt.Sprintf("127.0.0.1:%d", port) + if host != "" { + req.Host = host // the app sees its real public host, not the loopback + } + }, + ErrorHandler: func(w http.ResponseWriter, _ *http.Request, _ error) { + // The subdomain is registered but nothing answers on its reverse port — + // the client's tunnel is down (e.g. the container was reaped). + writeOfflinePage(w) + }, + } + + // autocert: issue a cert per host, but only for the hub host or a currently + // registered subdomain — so a request for a random subdomain can't drive cert + // issuance. + s.manager = &autocert.Manager{ + Prompt: autocert.AcceptTOS, + Cache: autocert.DirCache(o.CertCacheDir), + HostPolicy: s.hostPolicy, + } + return s, nil +} + +// hostPolicy allows the hub host and any registered preview subdomain. +func (s *Server) hostPolicy(_ context.Context, host string) error { + if host == s.opts.HubHost { + return nil + } + if sub, ok := s.subOf(host); ok { + if _, found := s.reg.LookupSubdomain(sub); found { + return nil + } + } + return fmt.Errorf("host %q is not the hub or a registered preview", host) +} + +// subOf returns the subdomain label of host under Domain (e.g. "app" from +// "app.example.com"), and whether host is under Domain. +func (s *Server) subOf(host string) (string, bool) { + suffix := "." + s.opts.Domain + if !strings.HasSuffix(host, suffix) { + return "", false + } + sub := strings.TrimSuffix(host, suffix) + if sub == "" || strings.Contains(sub, ".") { + return "", false // only single-label subdomains + } + return sub, true +} + +// ServeHTTP routes by Host: the hub host serves chisel control (WS upgrade), +// the API, and the admin page; a preview subdomain proxies to its tunnel. +func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { + host := stripPort(r.Host) + switch { + case host == s.opts.HubHost: + if isWebSocketUpgrade(r) { + s.chiselProxy.ServeHTTP(w, r) // chisel client control connection + return + } + if strings.HasPrefix(r.URL.Path, "/api/") { + s.apiMux.ServeHTTP(w, r) + return + } + s.admin.ServeHTTP(w, r) + default: + sub, ok := s.subOf(host) + if !ok { + http.Error(w, "unknown host", http.StatusMisdirectedRequest) + return + } + b, found := s.reg.LookupSubdomain(sub) + if !found { + writeNoSuchPreview(w, host) + return + } + s.reg.TouchUsed(sub) + ctx := context.WithValue(r.Context(), targetPortKey, b.ReversePort) + ctx = context.WithValue(ctx, publicHostKey, host) + s.appProxy.ServeHTTP(w, r.WithContext(ctx)) + } +} + +// Start binds the internal chisel server and the public TLS front (443), plus an +// HTTP :80 listener for ACME challenges and http->https redirects. It blocks +// until ctx is cancelled. +func (s *Server) Start(ctx context.Context, httpsAddr, httpAddr string) error { + if err := s.chisel.Start("127.0.0.1", portOf(s.opts.ChiselAddr)); err != nil { + return fmt.Errorf("starting chisel: %w", err) + } + + s.http = &http.Server{ + Addr: httpsAddr, + Handler: s, + TLSConfig: s.manager.TLSConfig(), + // WebSocket/long-poll: no write timeout; bound only the header read. + ReadHeaderTimeout: 20 * time.Second, + } + // :80 serves ACME HTTP-01 + redirects everything else to https. + httpSrv := &http.Server{Addr: httpAddr, Handler: s.manager.HTTPHandler(redirectToHTTPS()), ReadHeaderTimeout: 10 * time.Second} + go func() { _ = httpSrv.ListenAndServe() }() + + // Periodically trigger a reap so expired backends free their ports even when + // nobody loads the admin page (List reaps as a side effect). + reaper := time.NewTicker(30 * time.Second) + defer reaper.Stop() + go func() { + for { + select { + case <-ctx.Done(): + return + case <-reaper.C: + s.reg.List("") + } + } + }() + + errc := make(chan error, 1) + go func() { errc <- s.http.ListenAndServeTLS("", "") }() + + select { + case <-ctx.Done(): + shutCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + _ = s.http.Shutdown(shutCtx) + _ = httpSrv.Shutdown(shutCtx) + _ = s.chisel.Close() + return nil + case err := <-errc: + _ = httpSrv.Close() + _ = s.chisel.Close() + return err + } +} + +func redirectToHTTPS() http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, "https://"+stripPort(r.Host)+r.URL.RequestURI(), http.StatusMovedPermanently) + }) +} + +func isWebSocketUpgrade(r *http.Request) bool { + return strings.EqualFold(r.Header.Get("Upgrade"), "websocket") && + strings.Contains(strings.ToLower(r.Header.Get("Connection")), "upgrade") +} + +func stripPort(hostport string) string { + if i := strings.LastIndexByte(hostport, ':'); i >= 0 && !strings.Contains(hostport[i:], "]") { + return hostport[:i] + } + return hostport +} + +func portOf(addr string) string { + if i := strings.LastIndexByte(addr, ':'); i >= 0 { + return addr[i+1:] + } + return addr +} + +func writeNoSuchPreview(w http.ResponseWriter, host string) { + w.Header().Set("Content-Type", "text/html; charset=utf-8") + w.WriteHeader(http.StatusNotFound) + fmt.Fprintf(w, `No such preview`+ + ``+ + `

No preview here

Nothing is registered for %s. `+ + `It may have ended, or the name is wrong.

`, html.EscapeString(host)) +} + +func writeOfflinePage(w http.ResponseWriter) { + w.Header().Set("Content-Type", "text/html; charset=utf-8") + w.WriteHeader(http.StatusBadGateway) + fmt.Fprint(w, `Preview offline`+ + ``+ + `

Preview offline

This preview is registered but its tunnel isn't `+ + `connected right now — the dev container may be asleep or reaped. It will `+ + `return when the app is running again.

`) +} diff --git a/cmd/mxcli/tunnelhub/server_test.go b/cmd/mxcli/tunnelhub/server_test.go new file mode 100644 index 000000000..7561d1c00 --- /dev/null +++ b/cmd/mxcli/tunnelhub/server_test.go @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import ( + "net/http" + "net/http/httptest" + "strings" + "testing" + "time" +) + +func newTestServer(t *testing.T) (*Server, *Registry) { + t.Helper() + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + reg := newTestRegistry(clk) + srv, err := NewServer(ServerOptions{ + Domain: "example.com", + Registry: reg, + CertCacheDir: t.TempDir(), + }) + if err != nil { + t.Fatalf("NewServer: %v", err) + } + return srv, reg +} + +func req(host, target string) *http.Request { + r := httptest.NewRequest(http.MethodGet, "http://"+host+target, nil) + r.Host = host + return r +} + +func TestFront_HubHostServesAdmin(t *testing.T) { + srv, _ := newTestServer(t) + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req("hub.example.com", "/")) + if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), "mxcli tunnel-hub") { + t.Errorf("admin page: code=%d, body has hub title=%v", rec.Code, strings.Contains(rec.Body.String(), "mxcli tunnel-hub")) + } +} + +func TestFront_HubHostAPI(t *testing.T) { + srv, reg := newTestServer(t) + reg.Register(RegisterRequest{Project: "App", Branch: "main"}) + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req("hub.example.com", "/api/backends")) + if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), `"project":"App"`) { + t.Errorf("api/backends: code=%d body=%s", rec.Code, rec.Body) + } +} + +func TestFront_UnknownSubdomain(t *testing.T) { + srv, _ := newTestServer(t) + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req("ghost.example.com", "/")) + if rec.Code != http.StatusNotFound || !strings.Contains(rec.Body.String(), "No preview") { + t.Errorf("unknown subdomain: code=%d body=%s", rec.Code, rec.Body) + } +} + +func TestFront_RegisteredButOffline(t *testing.T) { + srv, reg := newTestServer(t) + b, _ := reg.Register(RegisterRequest{Project: "App", Branch: "main"}) // subdomain "app" + + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req("app.example.com", "/")) + // Registered but its reverse port has no tunnel -> the proxy error handler + // returns the offline page. + if rec.Code != http.StatusBadGateway || !strings.Contains(rec.Body.String(), "offline") { + t.Errorf("offline preview: code=%d body=%s", rec.Code, rec.Body) + } + // The request still counted as usage. + if v, ok := reg.LookupSubdomain(b.Subdomain); !ok || v.LastUsedAt.IsZero() { + t.Error("a request to a preview should update LastUsedAt") + } +} + +func TestSubOf(t *testing.T) { + srv, _ := newTestServer(t) + cases := []struct { + host string + sub string + ok bool + }{ + {"app.example.com", "app", true}, + {"my-app-feat.example.com", "my-app-feat", true}, + {"hub.example.com", "hub", true}, // subOf is purely structural; routing handles hub separately + {"a.b.example.com", "", false}, // multi-label not allowed + {"example.com", "", false}, + {"evil.com", "", false}, + } + for _, c := range cases { + sub, ok := srv.subOf(c.host) + if sub != c.sub || ok != c.ok { + t.Errorf("subOf(%q) = (%q,%v), want (%q,%v)", c.host, sub, ok, c.sub, c.ok) + } + } +} + +func TestStripPort(t *testing.T) { + for in, want := range map[string]string{ + "app.example.com": "app.example.com", + "app.example.com:443": "app.example.com", + "hub.example.com:8443": "hub.example.com", + } { + if got := stripPort(in); got != want { + t.Errorf("stripPort(%q) = %q, want %q", in, got, want) + } + } +} diff --git a/cmd/mxcli/tunnelhub/slug.go b/cmd/mxcli/tunnelhub/slug.go new file mode 100644 index 000000000..4e4982e2e --- /dev/null +++ b/cmd/mxcli/tunnelhub/slug.go @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Package tunnelhub implements the multi-tenant mxcli tunnel-hub: a static +// ingress relay that fronts many locally-running Mendix apps (across projects, +// solutions, branches, and worktrees) at per-preview subdomains over a single +// 443 connection, with a registration API and an admin overview. +package tunnelhub + +import ( + "regexp" + "strings" +) + +// maxLabelLen is the DNS label limit; a subdomain slug must fit in one label. +const maxLabelLen = 63 + +var nonLabel = regexp.MustCompile(`[^a-z0-9-]+`) + +// slugify reduces s to a DNS-label-safe fragment: lowercase, only [a-z0-9-], +// runs collapsed, no leading/trailing dashes. +func slugify(s string) string { + s = strings.ToLower(strings.TrimSpace(s)) + s = strings.NewReplacer("/", "-", "_", "-", ".", "-", " ", "-").Replace(s) + s = nonLabel.ReplaceAllString(s, "-") + for strings.Contains(s, "--") { + s = strings.ReplaceAll(s, "--", "-") + } + return strings.Trim(s, "-") +} + +// baseSlug builds the preferred subdomain slug from a registration: +// [prefix-]project[-branch]. The optional prefix namespaces the hostname +// (organization, solution, team, env — whatever the client passes). The main / +// master branch is dropped, so a project's primary preview is the clean +// [prefix-]project; other branches append the branch. Solution is a grouping +// dimension in the overview, not part of the hostname (unless passed as prefix). +func baseSlug(prefix, project, branch string) string { + var parts []string + if pre := slugify(prefix); pre != "" { + parts = append(parts, pre) + } + if p := slugify(project); p != "" { + parts = append(parts, p) + } + if b := slugify(branch); b != "" && b != "main" && b != "master" { + parts = append(parts, b) + } + if len(parts) == 0 { + return "app" + } + return truncateLabel(strings.Join(parts, "-")) +} + +// truncateLabel bounds a slug to a single DNS label, trimming any dash the cut +// leaves at the end. +func truncateLabel(s string) string { + if len(s) <= maxLabelLen { + return s + } + return strings.TrimRight(s[:maxLabelLen], "-") +} diff --git a/docs-site/src/tools/run-local.md b/docs-site/src/tools/run-local.md index 0fd95f999..977bc584d 100644 --- a/docs-site/src/tools/run-local.md +++ b/docs-site/src/tools/run-local.md @@ -61,7 +61,9 @@ so structural changes need a restart; behavioural changes do not. | Flag | Default | Purpose | |------|---------|---------| -| `--local` | — | Required; run without Docker | +| `--local` | — | Required; run without Docker (implied by `--hub`) | +| `--hub` | — | Expose the running app in a browser at a tunnel-hub URL (see [External browser preview](#external-browser-preview---hub)) | +| `--hub-secret` | — | Shared auth (`user:pass`) matching the hub's `--secret` | | `--watch` | off | Rebuild + hot-apply on every project change | | `--ensure-db` | off | Provision local Postgres + the app database if missing (fresh-session bootstrap) | | `--setup` | off | Prepare prerequisites (cache MxBuild+runtime, ensure DB) and exit without booting — for a SessionStart hook | @@ -76,6 +78,48 @@ so structural changes need a restart; behavioural changes do not. | `--screenshot-url` | app root | Page to shoot: full URL, or a path relative to the app root (e.g. `/p/customers`). Repeat for a multi-page set. | | `--screenshot-user` / `--screenshot-password` | — | Log in once (Mendix form auth) and reuse the session, so pages behind login render authenticated | +## External browser preview (`--hub`) + +`--hub ` makes the running app reachable **in a browser at a public URL** — without +the app leaving this machine and without committing. It's for reviewing work-in-progress +from a phone or tablet, or from an egress-only environment such as Claude Code on the web. +The app stays local and a **chisel reverse tunnel** dials *out* to a hub over 443; the hub +proxies browser requests back down the tunnel. Nothing is pushed — only live HTTP — and +because everything rides a single 443 connection, it works even from an egress-only proxy. + +**You run your own hub — there is no hosted service.** Stand up `mxcli tunnel-hub` once on +a host you control (a small VPS with a domain), then point apps at it. + +```bash +# on your VPS: *.example.com + hub.example.com -> this host, inbound 80+443 open +mxcli tunnel-hub --domain example.com --secret alice:s3cret + +# where the app runs: +mxcli run --hub https://hub.example.com --hub-secret alice:s3cret -p app.mpr +# -> registers and prints e.g. "Preview available at https://app.example.com" +``` + +The hub is **multi-tenant**: it fronts many previews at per-preview subdomains +(`-.example.com`; `main`/`master` collapses to ``) — across +projects, solutions, branches, and worktrees — with a sortable overview at +`https://hub.example.com/`. Each `run --hub` self-registers: + +- **Project** and **branch** auto-detect from the `.mpr` name and git; override with + `--hub-project`/`--hub-branch`, and `--hub-worktree` separates worktrees of one branch. +- **`--hub-prefix`** namespaces the hostname (org/solution/team/env) → + `--`; **`--hub-solution`** groups a solution's apps in the overview. +- The overview shows availability — a reaped/idle container turns **stale** — and sorts by + last-used, registered, or project. Re-registering keeps a **stable URL**. +- `--hub` **implies `--local`**, boots the runtime with `ApplicationRootUrl` set to the + assigned URL (so the SPA and `originURI` cookie work), and the tunnel reconnects forever. + Combine with `--watch` for the full remote loop: edit here → hot-apply → refresh the tab. + +**Hub setup:** a wildcard `*.example.com` A record (and `hub.example.com`) pointed at the +VPS; inbound 80 + 443 open — a Let's Encrypt cert is issued per subdomain on demand. + +**Security:** this version uses a single shared `--secret` with open registration, so keep +the hub to people you trust and don't expose it publicly (per-tenant auth is a follow-up). + ## The change signal `--watch` watches two **source** trees and rebuilds when either changes: diff --git a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md index acbe24d20..edd18c286 100644 --- a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md +++ b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md @@ -303,7 +303,7 @@ mxcli dev push --to https://-9000.app.github.dev --secret $TOKEN -p app.mp | **A. Push built deployment artifact over 443** (recommended) | ✅ | robust; runtime in a legit host; ~1–2 s + network | | B. Push MDL/model, build in the Preview container | ✅ | keeps a synced `.mpr` in the Preview container; heavier transfer for MPR v2 | | C. Git branch, Codespace pulls | ❌ | requires a commit — **rejected**, violates the core requirement | -| D. Live tunnel (app stays in DEV, static relay forwards) | ✅ | **viable — chisel verified** (see § Scaling). Cleanest dev loop: hot-reload stays local, no artifact sync, and the relay is a generic static component. Caveat: the DEV runtime is ephemeral. **Preferred for the multi-container dev loop; A remains the durable/persistent-preview fallback.** | +| D. Live tunnel (app stays in DEV, static relay forwards) | ✅ | **verified end-to-end 2026-07-23** — a real Mendix app in a Claude Code web container rendered in a browser via a Scaleway VPS relay over 443 (see § Scaling → *Verified end-to-end*). Cleanest dev loop: hot-reload stays local, no artifact sync, relay is a generic static component. Caveat: the DEV runtime is ephemeral. **Confirmed preferred for the multi-container dev loop; A remains the durable/persistent-preview fallback.** | ## Provisioning: from new Claude Code Web project to a running testable app @@ -420,6 +420,68 @@ failed only because it pins its own roots). The one hop not provable without a l host: the WebSocket upgrade through Codespaces' `app.github.dev` proxy — a **VPS relay removes that unknown** and is cleaner AUP-wise for a pure relay. +### Verified end-to-end on real infrastructure (2026-07-23) + +The load-bearing unknown (Open Q9 — does chisel's WebSocket tunnel survive the Claude +Code **web** session's mandatory egress proxy?) is now **closed, positive**, tested +live from a Claude Code web container against a **Scaleway VPS** relay on the +registrant's own domain (`hub.example.com`): + +- **chisel client connected out through the egress proxy:** + `Connecting to wss://hub.example.com:443 via http://127.0.0.1:33451 … Connected`. + The web session's proxy re-terminates TLS (MITM) and its README lists "WebSocket + upgrades" under *not supported* — but chisel's upgrade **passed** on this path. The + egress itself is open (arbitrary external hosts reachable); the proxy is not an + allowlist wall for this use. +- **Full path proven:** a browser hitting the Scaleway relay on 443 rendered the + **homepage of a blank Mendix 11.12.1 app running inside the Claude container** + (`mxcli run --local` on `:8080`), reached over the reverse tunnel. Nothing was + pushed to the relay — only live HTTP flows through the tunnel (the app never leaves + the DEV container). This validates **Option D** (below) as the model for Claude Code + web, not just an option on paper. +- The artifact-push coupling (Option A) was **also** verified locally the same session + (an independent Deploy build swapped into a running runtime + reload) — it remains + the durable/persistent-preview fallback, but Option D is confirmed preferred. + +**Command shape — `--hub` is a cross-cutting *ingress*, not a sub-feature of the local +loop.** The external preview is a **shared `--hub ` flag**, **not** a new `mxcli +dev` command (that name never shipped — see the slice-1 naming note above). Crucially it +is **orthogonal to how the app is served**: the same flag attaches to the warm path +(`mxcli run --local --hub `) today and to the **PAD / container path +(`mxcli docker run --hub `) later**, both backed by **one internal +tunnel/registration package**, so `tunnel-hub` fronts local and PAD runtimes +identically. `mxcli run` already reserves non-`--local` modes — it errors today with an +explicit pointer to `mxcli docker run` — which is exactly the seam this slots into. Keep +`--local` as the mode selector (PAD is a real sibling mode), but let **`--hub` imply the +local path** so bare `mxcli run --hub ` works for the common case. The only +serving-path-specific piece is the `ApplicationRootUrl`/subdomain wiring below (each path +owns its own runtime boot); the tunnel itself is generic. + +**Implementation gotchas found (must be encoded in `tunnel-hub` and the shared `--hub` +path):** + +1. **chisel's default server port is `8080`, not `443`.** `--tls-domain` requires the + server on 443 (`-p 443`) or Let's Encrypt never provisions and 443 resets mid-TLS. +2. **The chisel client must be handed the proxy explicitly** — `--proxy $HTTPS_PROXY` + (a hand-rolled Go dialer does not auto-read the env; without it the client tries a + direct connection the sandbox blocks). `run --local --hub` must read `HTTPS_PROXY` + from the environment and pass it through. +3. **DNS + port 80 are the only real setup blockers:** the relay's subdomain needs an + A record, and inbound **80** must be open for the ACME challenge (in addition to + 443). Egress from the sandbox is otherwise unrestricted. +4. **`ApplicationRootUrl` + subdomain-over-443 is required for a *correct* preview.** + Reaching the runtime as `hub:8099` while it booted with + `ApplicationRootUrl=localhost:8080` loads the index + client bundle, but the SPA's + session/XAS calls and the `originURI` cookie misbehave across the origin mismatch. + `run --local --hub` must boot the runtime with `ApplicationRootUrl` = the assigned + `https://.example.com`, and the hub must serve each preview over **443 on a + subdomain**, not a raw port. + +**Reference relay stood up for the test:** stock +`chisel server --reverse --tls-domain hub.example.com -p 443` on a Scaleway instance, +`hub.example.com` A → the instance IP, inbound 80+443 open. `mxcli tunnel-hub` is this +plus the registration API, wildcard-subdomain routing, and the admin overview. + ### `mxcli tunnel-hub` (static) + `mxcli dev --hub` (dynamic) - **`mxcli tunnel-hub`** — the always-on static container. Embeds a chisel server @@ -537,8 +599,8 @@ builds on the previous. |---|-------|----------|------------|-------------| | 1 | **Warm local loop** — shipped as `mxcli run --local [--watch]` (serve daemon + M2EE admin client + `restartRequired` branching; + client bundling & Playwright screenshots) | Docker-free ~1 s edit→test loop, locally | nothing new | ✅ **shipped** | | 2 | **Provisioning** — `run --local --ensure-db` (DB) + `run --local --setup` (non-blocking bring-up) + `mxcli init` SessionStart hook + bootstrap prompt template | a fresh Claude Code Web session comes up testable; iPad-native start | slice 1 | ✅ **shipped** | -| 3 | **Single-app external preview** — `mxcli dev serve` + chisel client → one static relay + `ApplicationRootUrl` wiring | a shareable live preview URL (the iPad two-container flow) | slice 1 | medium / medium — the `app.github.dev` WebSocket hop is unverified (a VPS relay avoids it) | -| 4 | **Tunnel hub** — `mxcli tunnel-hub` + `mxcli dev --hub` + registration API + admin overview | many dev containers behind one ingress; fleet overview + per-container change lists | slice 3 | large / higher — a product in its own right, with a multi-tenant auth surface | +| 3 | **Single-app external preview** — `mxcli run --hub ` (embedded chisel client, proxy honouring `NO_PROXY`, `ApplicationRootUrl` boot wiring) + `mxcli tunnel-hub` (embedded chisel server, autocert, single-443 `--backend`) | a shareable live preview URL (the iPad two-container flow) | slice 1 | ✅ **shipped** (2026-07-23) — code + in-process tunnel test + local end-to-end boot; Mendix renders through the hub's Host-rewriting backend, no shim needed. External E2E against the Scaleway hub is the remaining confirmation | +| 4 | **Tunnel hub** — `mxcli tunnel-hub` (multi-tenant) + `mxcli run --hub` registration + admin overview | many dev containers behind one ingress; per-preview subdomains across projects/solutions/branches/worktrees; sortable overview with availability | slice 3 | ✅ **built** (2026-07-23) — registry + registration API + single-443 host-routing front with per-subdomain autocert + sortable admin page + client registration; unit + in-process tunnel integration tests. External E2E against the Scaleway hub (wildcard DNS) is the remaining confirmation; deeper multi-tenant auth (per-container tokens, admin auth) is a follow-on | Recommended sequencing: **1 → 2** delivers the complete solo dev experience (Scenario A plus provisioning) with no external moving parts, and can ship first. **3** adds external @@ -608,10 +670,13 @@ should front slice 1's build on every iteration, so most errors never reach even 8. **Change-summary source & cadence.** `mxcli diff-local` (git baseline) vs a running MDL-statement log — which is the better per-container "changes" feed for the admin page, and how often to push it (every reload vs periodic)? -9. **WebSocket through `app.github.dev` unverified.** chisel's reverse tunnel is proven - through the sandbox egress + MITM, but not through a live Codespace forward. Verify - on a real Codespace, or default the hub to a VPS (also the cleaner AUP posture for a - pure relay). +9. **WebSocket through the egress proxy — RESOLVED (2026-07-23).** chisel's reverse + tunnel is now **verified live** out through the Claude Code **web** session's + mandatory MITM egress proxy to a **Scaleway VPS** relay on 443, with a real Mendix + app rendering through it (see § Scaling → *Verified end-to-end*). The hub defaults to + a VPS (not a Codespace), which also sidesteps the never-tested `app.github.dev` hop + and is the cleaner AUP posture. Remaining sub-item: confirm the same over a Codespace + forward *if* that host is ever wanted — not needed for the VPS path. 10. **Fleet lifecycle.** TTL/heartbeat-timeout cleanup of dead containers, port/subdomain reclamation, and what the admin page shows for a container whose sandbox was reaped (the ephemeral-runtime reality observed this session). diff --git a/go.mod b/go.mod index c7ca28503..ab948e40c 100644 --- a/go.mod +++ b/go.mod @@ -13,6 +13,7 @@ require ( github.com/chzyer/readline v1.5.1 github.com/fsnotify/fsnotify v1.10.1 github.com/jackc/pgx/v5 v5.9.2 + github.com/jpillora/chisel v1.11.8 github.com/mattn/go-runewidth v0.0.24 github.com/microsoft/go-mssqldb v1.10.0 github.com/pmezard/go-difflib v1.0.0 @@ -27,6 +28,8 @@ require ( go.mongodb.org/mongo-driver/v2 v2.6.0 go.starlark.net v0.0.0-20260102030733-3fee463870c9 go.uber.org/zap v1.28.0 + golang.org/x/crypto v0.52.0 + golang.org/x/net v0.55.0 golang.org/x/sync v0.21.0 golang.org/x/term v0.43.0 gopkg.in/yaml.v3 v3.0.1 @@ -34,6 +37,8 @@ require ( ) require ( + github.com/andrew-d/go-termutil v0.0.0-20150726205930-009166a695a2 // indirect + github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 // indirect github.com/atotto/clipboard v0.1.4 // indirect github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect github.com/charmbracelet/colorprofile v0.4.1 // indirect @@ -49,10 +54,15 @@ require ( github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9 // indirect github.com/golang-sql/sqlexp v0.1.0 // indirect github.com/google/uuid v1.6.0 // indirect + github.com/gorilla/websocket v1.5.3 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/jackc/pgpassfile v1.0.0 // indirect github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect github.com/jackc/puddle/v2 v2.2.2 // indirect + github.com/jpillora/ansi v1.0.3 // indirect + github.com/jpillora/backoff v1.0.0 // indirect + github.com/jpillora/requestlog v1.0.0 // indirect + github.com/jpillora/sizestr v1.0.0 // indirect github.com/lucasb-eyer/go-colorful v1.3.0 // indirect github.com/mattn/go-isatty v0.0.20 // indirect github.com/mattn/go-localereader v0.0.1 // indirect @@ -66,10 +76,10 @@ require ( github.com/segmentio/asm v1.1.3 // indirect github.com/segmentio/encoding v0.3.4 // indirect github.com/shopspring/decimal v1.4.0 // indirect + github.com/tomasen/realip v0.0.0-20180522021738-f0c99a92ddce // indirect github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect go.lsp.dev/pkg v0.0.0-20210717090340-384b27a52fb2 // indirect go.uber.org/multierr v1.10.0 // indirect - golang.org/x/crypto v0.52.0 // indirect golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 // indirect golang.org/x/sys v0.45.0 // indirect golang.org/x/text v0.39.0 // indirect diff --git a/go.sum b/go.sum index cd91f4f4a..8be85ee4c 100644 --- a/go.sum +++ b/go.sum @@ -18,8 +18,12 @@ github.com/alecthomas/chroma/v2 v2.26.1 h1:2X21EdxGZNv5GF9mG5u+uzc02GCFyGxbcBm3G github.com/alecthomas/chroma/v2 v2.26.1/go.mod h1:lxhRRa9H4hPmRLOOdYga4zkQIQjq3dtrrdwQeCfu78Y= github.com/alecthomas/repr v0.5.2 h1:SU73FTI9D1P5UNtvseffFSGmdNci/O6RsqzeXJtP0Qs= github.com/alecthomas/repr v0.5.2/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4= +github.com/andrew-d/go-termutil v0.0.0-20150726205930-009166a695a2 h1:axBiC50cNZOs7ygH5BgQp4N+aYrZ2DNpWZ1KG3VOSOM= +github.com/andrew-d/go-termutil v0.0.0-20150726205930-009166a695a2/go.mod h1:jnzFpU88PccN/tPPhCpnNU8mZphvKxYM9lLNkd8e+os= github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYWrPrQ= github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmOABFgjdkM7Nw= +github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio= +github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs= github.com/atotto/clipboard v0.1.4 h1:EH0zSVneZPSuFR11BlR9YppQTVDbh5+16AmcJi4g1z4= github.com/atotto/clipboard v0.1.4/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn0Yu86PYI= github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k= @@ -77,6 +81,8 @@ github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17k github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg= +github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM= @@ -91,6 +97,16 @@ github.com/jackc/pgx/v5 v5.9.2 h1:3ZhOzMWnR4yJ+RW1XImIPsD1aNSz4T4fyP7zlQb56hw= github.com/jackc/pgx/v5 v5.9.2/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4= github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo= github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4= +github.com/jpillora/ansi v1.0.3 h1:nn4Jzti0EmRfDxm7JtEs5LzCbNwd5sv+0aE+LdS9/ZQ= +github.com/jpillora/ansi v1.0.3/go.mod h1:D2tT+6uzJvN1nBVQILYWkIdq7zG+b5gcFN5WI/VyjMY= +github.com/jpillora/backoff v1.0.0 h1:uvFg412JmmHBHw7iwprIxkPMI+sGQ4kzOWsMeHnm2EA= +github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= +github.com/jpillora/chisel v1.11.8 h1:pf2ufs2kfytmSWkZoeEgSs5pGrn9ODL2UaZRiR/meyk= +github.com/jpillora/chisel v1.11.8/go.mod h1:efqW+iikEM4E7CxQSR1LW1oyErlZtWXC/R81C/Fzk2w= +github.com/jpillora/requestlog v1.0.0 h1:bg++eJ74T7DYL3DlIpiwknrtfdUA9oP/M4fL+PpqnyA= +github.com/jpillora/requestlog v1.0.0/go.mod h1:HTWQb7QfDc2jtHnWe2XEIEeJB7gJPnVdpNn52HXPvy8= +github.com/jpillora/sizestr v1.0.0 h1:4tr0FLxs1Mtq3TnsLDV+GYUWG7Q26a6s+tV5Zfw2ygw= +github.com/jpillora/sizestr v1.0.0/go.mod h1:bUhLv4ctkknatr6gR42qPxirmd5+ds1u7mzD+MZ33f0= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0= github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk= @@ -150,6 +166,8 @@ github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81P github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +github.com/tomasen/realip v0.0.0-20180522021738-f0c99a92ddce h1:fb190+cK2Xz/dvi9Hv8eCYJYvIGUTN2/KLq1pT6CjEc= +github.com/tomasen/realip v0.0.0-20180522021738-f0c99a92ddce/go.mod h1:o8v6yHRoik09Xen7gje4m9ERNah1d1PPsVq1VEx9vE4= github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no= github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM= go.lsp.dev/jsonrpc2 v0.10.0 h1:Pr/YcXJoEOTMc/b6OTmcR1DPJ3mSWl/SWiU1Cct6VmI= @@ -180,8 +198,8 @@ golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 h1:mgKeJMpvi0yx/sU5GsxQ7p6s2 golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546/go.mod h1:j/pmGrbnkbPtQfxEe5D0VQhZC6qKbfKifgD0oM7sR70= golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ= golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0= -golang.org/x/net v0.54.0 h1:2zJIZAxAHV/OHCDTCOHAYehQzLfSXuf/5SoL/Dv6w/w= -golang.org/x/net v0.54.0/go.mod h1:Sj4oj8jK6XmHpBZU/zWHw3BV3abl4Kvi+Ut7cQcY+cQ= +golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8= +golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww= golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM= golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=