From 6e87da04c966c612f75ecf0e4fabe9ef35a7fb53 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 23 Jul 2026 05:43:16 +0000 Subject: [PATCH 01/12] docs(warm-loop): record live verification of the external-preview tunnel MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verified end-to-end from a Claude Code web container: chisel's WebSocket reverse tunnel connects OUT through the session's mandatory MITM egress proxy to a Scaleway VPS relay on 443, and a real Mendix app running in the container renders in a browser via that relay. Closes the proposal's load-bearing open question (WebSocket-through-proxy) positive, and confirms the live-tunnel model (Option D) as preferred for Claude Code web — the app stays in the DEV container, the relay is static, nothing is pushed. Captures the implementation gotchas for the future tunnel-hub / dev --hub build: chisel default port is 8080 not 443 (-p 443 for --tls-domain); the client must be handed --proxy $HTTPS_PROXY explicitly; DNS + inbound 80 (ACME) are the only setup blockers; and ApplicationRootUrl must be set to the public subdomain (served over 443, not a raw port) for a correct SPA preview. Updates Open Q9 (resolved), the Option D coupling row, and the slice-3 risk note accordingly. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .../PROPOSAL_mxcli_dev_warm_loop.md | 79 +++++++++++++++++-- 1 file changed, 72 insertions(+), 7 deletions(-) diff --git a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md index acbe24d20..cb09fec53 100644 --- a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md +++ b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md @@ -303,7 +303,7 @@ mxcli dev push --to https://-9000.app.github.dev --secret $TOKEN -p app.mp | **A. Push built deployment artifact over 443** (recommended) | ✅ | robust; runtime in a legit host; ~1–2 s + network | | B. Push MDL/model, build in the Preview container | ✅ | keeps a synced `.mpr` in the Preview container; heavier transfer for MPR v2 | | C. Git branch, Codespace pulls | ❌ | requires a commit — **rejected**, violates the core requirement | -| D. Live tunnel (app stays in DEV, static relay forwards) | ✅ | **viable — chisel verified** (see § Scaling). Cleanest dev loop: hot-reload stays local, no artifact sync, and the relay is a generic static component. Caveat: the DEV runtime is ephemeral. **Preferred for the multi-container dev loop; A remains the durable/persistent-preview fallback.** | +| D. Live tunnel (app stays in DEV, static relay forwards) | ✅ | **verified end-to-end 2026-07-23** — a real Mendix app in a Claude Code web container rendered in a browser via a Scaleway VPS relay over 443 (see § Scaling → *Verified end-to-end*). Cleanest dev loop: hot-reload stays local, no artifact sync, relay is a generic static component. Caveat: the DEV runtime is ephemeral. **Confirmed preferred for the multi-container dev loop; A remains the durable/persistent-preview fallback.** | ## Provisioning: from new Claude Code Web project to a running testable app @@ -420,6 +420,68 @@ failed only because it pins its own roots). The one hop not provable without a l host: the WebSocket upgrade through Codespaces' `app.github.dev` proxy — a **VPS relay removes that unknown** and is cleaner AUP-wise for a pure relay. +### Verified end-to-end on real infrastructure (2026-07-23) + +The load-bearing unknown (Open Q9 — does chisel's WebSocket tunnel survive the Claude +Code **web** session's mandatory egress proxy?) is now **closed, positive**, tested +live from a Claude Code web container against a **Scaleway VPS** relay on the +registrant's own domain (`hub.mxcli.org`): + +- **chisel client connected out through the egress proxy:** + `Connecting to wss://hub.mxcli.org:443 via http://127.0.0.1:33451 … Connected`. + The web session's proxy re-terminates TLS (MITM) and its README lists "WebSocket + upgrades" under *not supported* — but chisel's upgrade **passed** on this path. The + egress itself is open (arbitrary external hosts reachable); the proxy is not an + allowlist wall for this use. +- **Full path proven:** a browser hitting the Scaleway relay on 443 rendered the + **homepage of a blank Mendix 11.12.1 app running inside the Claude container** + (`mxcli run --local` on `:8080`), reached over the reverse tunnel. Nothing was + pushed to the relay — only live HTTP flows through the tunnel (the app never leaves + the DEV container). This validates **Option D** (below) as the model for Claude Code + web, not just an option on paper. +- The artifact-push coupling (Option A) was **also** verified locally the same session + (an independent Deploy build swapped into a running runtime + reload) — it remains + the durable/persistent-preview fallback, but Option D is confirmed preferred. + +**Command shape — `--hub` is a cross-cutting *ingress*, not a sub-feature of the local +loop.** The external preview is a **shared `--hub ` flag**, **not** a new `mxcli +dev` command (that name never shipped — see the slice-1 naming note above). Crucially it +is **orthogonal to how the app is served**: the same flag attaches to the warm path +(`mxcli run --local --hub `) today and to the **PAD / container path +(`mxcli docker run --hub `) later**, both backed by **one internal +tunnel/registration package**, so `tunnel-hub` fronts local and PAD runtimes +identically. `mxcli run` already reserves non-`--local` modes — it errors today with an +explicit pointer to `mxcli docker run` — which is exactly the seam this slots into. Keep +`--local` as the mode selector (PAD is a real sibling mode), but let **`--hub` imply the +local path** so bare `mxcli run --hub ` works for the common case. The only +serving-path-specific piece is the `ApplicationRootUrl`/subdomain wiring below (each path +owns its own runtime boot); the tunnel itself is generic. + +**Implementation gotchas found (must be encoded in `tunnel-hub` and the shared `--hub` +path):** + +1. **chisel's default server port is `8080`, not `443`.** `--tls-domain` requires the + server on 443 (`-p 443`) or Let's Encrypt never provisions and 443 resets mid-TLS. +2. **The chisel client must be handed the proxy explicitly** — `--proxy $HTTPS_PROXY` + (a hand-rolled Go dialer does not auto-read the env; without it the client tries a + direct connection the sandbox blocks). `run --local --hub` must read `HTTPS_PROXY` + from the environment and pass it through. +3. **DNS + port 80 are the only real setup blockers:** the relay's subdomain needs an + A record, and inbound **80** must be open for the ACME challenge (in addition to + 443). Egress from the sandbox is otherwise unrestricted. +4. **`ApplicationRootUrl` + subdomain-over-443 is required for a *correct* preview.** + Reaching the runtime as `hub:8099` while it booted with + `ApplicationRootUrl=localhost:8080` loads the index + client bundle, but the SPA's + session/XAS calls and the `originURI` cookie misbehave across the origin mismatch. + `run --local --hub` must boot the runtime with `ApplicationRootUrl` = the assigned + `https://.mxcli.org`, and the hub must serve each preview over **443 on a + subdomain**, not a raw port. + +**Reference relay stood up for the test:** stock +`chisel server --reverse --tls-domain hub.mxcli.org -p 443` on a Scaleway instance, +`hub.mxcli.org` A → the instance IP, inbound 80+443 open. `mxcli tunnel-hub` is this +plus the registration API, wildcard-subdomain routing, and the admin overview. + ### `mxcli tunnel-hub` (static) + `mxcli dev --hub` (dynamic) - **`mxcli tunnel-hub`** — the always-on static container. Embeds a chisel server @@ -537,8 +599,8 @@ builds on the previous. |---|-------|----------|------------|-------------| | 1 | **Warm local loop** — shipped as `mxcli run --local [--watch]` (serve daemon + M2EE admin client + `restartRequired` branching; + client bundling & Playwright screenshots) | Docker-free ~1 s edit→test loop, locally | nothing new | ✅ **shipped** | | 2 | **Provisioning** — `run --local --ensure-db` (DB) + `run --local --setup` (non-blocking bring-up) + `mxcli init` SessionStart hook + bootstrap prompt template | a fresh Claude Code Web session comes up testable; iPad-native start | slice 1 | ✅ **shipped** | -| 3 | **Single-app external preview** — `mxcli dev serve` + chisel client → one static relay + `ApplicationRootUrl` wiring | a shareable live preview URL (the iPad two-container flow) | slice 1 | medium / medium — the `app.github.dev` WebSocket hop is unverified (a VPS relay avoids it) | -| 4 | **Tunnel hub** — `mxcli tunnel-hub` + `mxcli dev --hub` + registration API + admin overview | many dev containers behind one ingress; fleet overview + per-container change lists | slice 3 | large / higher — a product in its own right, with a multi-tenant auth surface | +| 3 | **Single-app external preview** — `mxcli run --local --hub ` (chisel client + `--proxy $HTTPS_PROXY`) → static VPS relay + `ApplicationRootUrl`-on-subdomain wiring | a shareable live preview URL (the iPad two-container flow) | slice 1 | medium / **low** — transport verified end-to-end 2026-07-23 (§ Scaling); remaining work is the `ApplicationRootUrl`/subdomain wiring, not the tunnel | +| 4 | **Tunnel hub** — `mxcli tunnel-hub` + `mxcli run --local --hub` + registration API + admin overview | many dev containers behind one ingress; fleet overview + per-container change lists | slice 3 | large / higher — a product in its own right, with a multi-tenant auth surface | Recommended sequencing: **1 → 2** delivers the complete solo dev experience (Scenario A plus provisioning) with no external moving parts, and can ship first. **3** adds external @@ -608,10 +670,13 @@ should front slice 1's build on every iteration, so most errors never reach even 8. **Change-summary source & cadence.** `mxcli diff-local` (git baseline) vs a running MDL-statement log — which is the better per-container "changes" feed for the admin page, and how often to push it (every reload vs periodic)? -9. **WebSocket through `app.github.dev` unverified.** chisel's reverse tunnel is proven - through the sandbox egress + MITM, but not through a live Codespace forward. Verify - on a real Codespace, or default the hub to a VPS (also the cleaner AUP posture for a - pure relay). +9. **WebSocket through the egress proxy — RESOLVED (2026-07-23).** chisel's reverse + tunnel is now **verified live** out through the Claude Code **web** session's + mandatory MITM egress proxy to a **Scaleway VPS** relay on 443, with a real Mendix + app rendering through it (see § Scaling → *Verified end-to-end*). The hub defaults to + a VPS (not a Codespace), which also sidesteps the never-tested `app.github.dev` hop + and is the cleaner AUP posture. Remaining sub-item: confirm the same over a Codespace + forward *if* that host is ever wanted — not needed for the VPS path. 10. **Fleet lifecycle.** TTL/heartbeat-timeout cleanup of dead containers, port/subdomain reclamation, and what the admin page shows for a container whose sandbox was reaped (the ephemeral-runtime reality observed this session). From d018ff4d6336d7c679a8c8fc3bba2841bfe0190c Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 23 Jul 2026 06:22:44 +0000 Subject: [PATCH 02/12] feat(run): external browser preview via a tunnel-hub (mxcli run --hub / mxcli tunnel-hub) Slice 3 of the warm-loop proposal: a locally-running Mendix app is made reachable in a browser at a public URL, without leaving the machine and without committing. The app stays in its (possibly egress-only) environment and reverse- tunnels out over 443 to a static hub; only live HTTP flows through the tunnel. - mxcli run --hub [--hub-secret user:pass]: a cross-cutting ingress flag that implies --local. Boots the runtime with ApplicationRootUrl set to the hub URL (so the SPA/originURI work under that origin) and opens an embedded chisel reverse tunnel from the app port out to the hub. The proxy for the control connection is resolved from the environment honouring NO_PROXY, so an external hub goes through the egress proxy while a loopback hub connects directly. - mxcli tunnel-hub --domain [--secret ...]: the static relay. Embeds a chisel reverse server with automatic Let's Encrypt (or --tls-cert/--tls-key) and proxies non-tunnel HTTP down the tunnel to the app, so everything rides a single 443 connection (works from egress-only environments like Claude Code on the web). Single-app for this slice; multi-tenant registration + subdomains are a follow-on. The chisel WebSocket transport was verified live out through the Claude Code web egress proxy to a real VPS relay; here the embedded client/server, the ApplicationRootUrl boot wiring, and a full run --hub -> hub -> app round trip are covered by an in-process tunnel test plus a local end-to-end boot. Mendix renders byte-identically through the hub's Host-rewriting backend, so no Host-preserving shim is needed. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- cmd/mxcli/cmd_run.go | 18 ++++ cmd/mxcli/cmd_tunnelhub.go | 125 ++++++++++++++++++++++++++ cmd/mxcli/docker/localboot.go | 14 ++- cmd/mxcli/docker/localboot_test.go | 14 +++ cmd/mxcli/docker/runlocal.go | 42 +++++++-- cmd/mxcli/docker/tunnel.go | 140 +++++++++++++++++++++++++++++ cmd/mxcli/docker/tunnel_test.go | 135 ++++++++++++++++++++++++++++ go.mod | 10 +++ go.sum | 18 ++++ 9 files changed, 508 insertions(+), 8 deletions(-) create mode 100644 cmd/mxcli/cmd_tunnelhub.go create mode 100644 cmd/mxcli/docker/tunnel.go create mode 100644 cmd/mxcli/docker/tunnel_test.go diff --git a/cmd/mxcli/cmd_run.go b/cmd/mxcli/cmd_run.go index 66ecafb15..cf8ed28ca 100644 --- a/cmd/mxcli/cmd_run.go +++ b/cmd/mxcli/cmd_run.go @@ -32,13 +32,27 @@ Requirements: already exist. Defaults: 127.0.0.1:5432, user 'mendix', db from the project name. Override with --db-host/--db-name/--db-user/--db-password. +With --hub, the running app is exposed in a browser at a public URL through an +mxcli tunnel-hub, without leaving this machine: a chisel client reverse-tunnels +the local app out over 443, and the runtime boots with ApplicationRootUrl set to +the hub URL so the app works under that origin. --hub implies --local. + Examples: mxcli run --local -p app.mpr mxcli run --local -p app.mpr --watch mxcli run --local -p app.mpr --app-port 8081 --db-name myapp + mxcli run --hub https://hub.mxcli.org -p app.mpr # browser preview + mxcli run --hub https://hub.mxcli.org --hub-secret u:pass -p app.mpr --watch `, Run: func(cmd *cobra.Command, args []string) { local, _ := cmd.Flags().GetBool("local") + hub, _ := cmd.Flags().GetString("hub") + hubSecret, _ := cmd.Flags().GetString("hub-secret") + // --hub is a cross-cutting ingress and implies the local serving path (the + // only serving mode wired today; a future PAD path will accept --hub too). + if hub != "" { + local = true + } if !local { fmt.Fprintln(os.Stderr, "Error: only --local is supported for now (use 'mxcli docker run' for the container workflow)") os.Exit(1) @@ -67,6 +81,8 @@ Examples: opts := docker.LocalRunOptions{ ProjectPath: projectPath, + Hub: hub, + HubSecret: hubSecret, AppPort: appPort, AdminPort: adminPort, ServePort: servePort, @@ -97,6 +113,8 @@ Examples: func init() { runCmd.Flags().Bool("local", false, "Run locally without Docker (warm serve + standalone runtime)") + runCmd.Flags().String("hub", "", "Expose the running app in a browser via an mxcli tunnel-hub URL (e.g. https://hub.mxcli.org). Implies --local; the app stays local and is reverse-tunnelled out") + runCmd.Flags().String("hub-secret", "", "Shared auth secret for --hub (\"user:pass\"), matching the hub's --secret") runCmd.Flags().Bool("watch", false, "Rebuild and hot-apply on every project change") runCmd.Flags().Bool("ensure-db", false, "Provision the local Postgres + app database if missing (fresh-session bootstrap)") runCmd.Flags().Bool("setup", false, "Prepare prerequisites (cache MxBuild+runtime, ensure DB) and exit without booting — for a SessionStart hook") diff --git a/cmd/mxcli/cmd_tunnelhub.go b/cmd/mxcli/cmd_tunnelhub.go new file mode 100644 index 000000000..ed67647c4 --- /dev/null +++ b/cmd/mxcli/cmd_tunnelhub.go @@ -0,0 +1,125 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "context" + "fmt" + "os" + "os/signal" + "strconv" + "syscall" + + chserver "github.com/jpillora/chisel/server" + "github.com/mendixlabs/mxcli/cmd/mxcli/docker" + "github.com/spf13/cobra" +) + +// tunnelHubCmd is the static ingress relay. It runs on a host with a public IP +// and domain (e.g. a small VPS) and fronts a locally-running mxcli app: the app +// stays in its own (possibly egress-only) environment and reverse-tunnels out to +// this hub over 443; browsers reach it at the hub's URL. Nothing is pushed here — +// only live HTTP flows through the tunnel. +// +// Slice 1 fronts a single app. Multi-tenant registration (a /register API that +// allocates per-container subdomains + tokens) and the admin overview are a +// follow-on (see PROPOSAL_mxcli_dev_warm_loop.md § Scaling). +var tunnelHubCmd = &cobra.Command{ + Use: "tunnel-hub", + Short: "Static ingress relay that fronts a locally-running mxcli app at a public URL", + Long: `Run a static ingress relay that exposes a locally-running Mendix app +(started elsewhere with 'mxcli run --hub ') in a browser at this host's +public URL. + +It embeds a chisel reverse-tunnel server: the app's environment dials in over 443 +and reverse-tunnels its local port here; every non-tunnel HTTP request to this +host is proxied down that tunnel to the app. Everything rides a single 443 +connection, so it works from egress-only environments (e.g. Claude Code on the +web). + +TLS: pass --domain for automatic Let's Encrypt (the host must be reachable on 80 +and 443 for the ACME challenge), or --tls-cert/--tls-key for an existing +certificate. + +Example (on a VPS with hub.mxcli.org -> this host, ports 80+443 open): + mxcli tunnel-hub --domain hub.mxcli.org --secret myuser:mypass + +Then, in the app's environment: + mxcli run --hub https://hub.mxcli.org --hub-secret myuser:mypass -p app.mpr +`, + Run: func(cmd *cobra.Command, args []string) { + domain, _ := cmd.Flags().GetString("domain") + secret, _ := cmd.Flags().GetString("secret") + port, _ := cmd.Flags().GetInt("port") + backendPort, _ := cmd.Flags().GetInt("backend-port") + tlsKey, _ := cmd.Flags().GetString("tls-key") + tlsCert, _ := cmd.Flags().GetString("tls-cert") + host, _ := cmd.Flags().GetString("host") + + hasCert := tlsKey != "" && tlsCert != "" + if domain == "" && !hasCert { + fmt.Fprintln(os.Stderr, "Error: --domain (automatic Let's Encrypt) or both --tls-cert and --tls-key are required") + os.Exit(1) + } + + cfg := &chserver.Config{ + Reverse: true, + Auth: secret, + // Proxy is chisel's HTTP backend: non-tunnel requests are reverse-proxied + // here, which is the reverse-tunnel listener the app dials into. + Proxy: fmt.Sprintf("http://127.0.0.1:%d", backendPort), + TLS: chserver.TLSConfig{ + Key: tlsKey, + Cert: tlsCert, + }, + } + if domain != "" { + cfg.TLS.Domains = []string{domain} + } + + srv, err := chserver.NewServer(cfg) + if err != nil { + fmt.Fprintf(os.Stderr, "Error: configuring tunnel-hub: %v\n", err) + os.Exit(1) + } + + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) + defer stop() + + if domain != "" { + fmt.Printf("mxcli tunnel-hub: fronting https://%s on %s:%d (reverse port %d)\n", domain, host, port, backendPort) + fmt.Printf(" app connects with: mxcli run --hub https://%s%s -p app.mpr\n", domain, secretHint(secret)) + } else { + fmt.Printf("mxcli tunnel-hub: listening on %s:%d (reverse port %d)\n", host, port, backendPort) + } + + if err := srv.StartContext(ctx, host, strconv.Itoa(port)); err != nil { + fmt.Fprintf(os.Stderr, "Error: starting tunnel-hub: %v\n", err) + os.Exit(1) + } + if err := srv.Wait(); err != nil { + fmt.Fprintf(os.Stderr, "Error: tunnel-hub stopped: %v\n", err) + os.Exit(1) + } + }, +} + +// secretHint renders " --hub-secret " for the copy-paste hint, or "" when +// no secret is set. +func secretHint(secret string) string { + if secret == "" { + return "" + } + return " --hub-secret " + secret +} + +func init() { + tunnelHubCmd.Flags().String("domain", "", "Domain for automatic Let's Encrypt TLS (e.g. hub.mxcli.org); host must be reachable on 80+443") + tunnelHubCmd.Flags().String("secret", "", "Shared auth secret (\"user:pass\") the app must present via --hub-secret") + tunnelHubCmd.Flags().Int("port", 443, "Public port to listen on") + tunnelHubCmd.Flags().Int("backend-port", docker.DefaultHubBackendPort, "Reverse-tunnel port the app dials into (must match the app side; default 9000)") + tunnelHubCmd.Flags().String("tls-cert", "", "TLS certificate file (instead of --domain autocert)") + tunnelHubCmd.Flags().String("tls-key", "", "TLS key file (instead of --domain autocert)") + tunnelHubCmd.Flags().String("host", "0.0.0.0", "Address to bind") + rootCmd.AddCommand(tunnelHubCmd) +} diff --git a/cmd/mxcli/docker/localboot.go b/cmd/mxcli/docker/localboot.go index 276f0e481..dbfd9b7db 100644 --- a/cmd/mxcli/docker/localboot.go +++ b/cmd/mxcli/docker/localboot.go @@ -60,6 +60,12 @@ type LocalRuntimeOptions struct { ListenAddr string // DTAPMode is D/A/T/P (default "D"). DTAPMode string + // ApplicationRootUrl, when set, is the public URL the app is reached at + // (e.g. https://hub.mxcli.org). Mendix uses it for absolute-URL generation + // and to accept requests whose Host differs from the listen address — needed + // when the app is served through an external tunnel/reverse proxy rather than + // localhost. Empty for a plain local run. + ApplicationRootUrl string // DB is the database the runtime connects to. DB DBConfig // ReadyTimeout bounds how long StartLocalRuntime waits for the admin API @@ -139,7 +145,7 @@ func runtimeConfigParams(o LocalRuntimeOptions, constants map[string]string) map if constants == nil { constants = map[string]string{} } - return map[string]any{ + params := map[string]any{ "BasePath": o.DeployDir, "RuntimePath": o.runtimeDir(), "DTAPMode": o.DTAPMode, @@ -150,6 +156,12 @@ func runtimeConfigParams(o LocalRuntimeOptions, constants map[string]string) map "DatabasePassword": o.DB.Password, "MicroflowConstants": constants, } + // Only set ApplicationRootUrl when the app is served behind an external URL; + // on a plain local run the runtime defaults it from the listen address. + if o.ApplicationRootUrl != "" { + params["ApplicationRootUrl"] = o.ApplicationRootUrl + } + return params } // deploymentConfig mirrors the parts of /model/config.json mxbuild diff --git a/cmd/mxcli/docker/localboot_test.go b/cmd/mxcli/docker/localboot_test.go index 712d68bda..5b282f01a 100644 --- a/cmd/mxcli/docker/localboot_test.go +++ b/cmd/mxcli/docker/localboot_test.go @@ -119,6 +119,20 @@ func TestRuntimeConfigParams_NilConstants(t *testing.T) { } } +func TestRuntimeConfigParams_ApplicationRootUrl(t *testing.T) { + // Absent by default (plain local run): the runtime derives it from the listen + // address, so we must not pin it. + if _, ok := runtimeConfigParams(testLocalOpts(), nil)["ApplicationRootUrl"]; ok { + t.Error("ApplicationRootUrl must be absent when not set") + } + // Present when serving behind a hub, so the SPA works under the public origin. + o := testLocalOpts() + o.ApplicationRootUrl = "https://hub.mxcli.org" + if got := runtimeConfigParams(o, nil)["ApplicationRootUrl"]; got != "https://hub.mxcli.org" { + t.Errorf("ApplicationRootUrl = %v, want https://hub.mxcli.org", got) + } +} + func TestReadDeploymentConstants(t *testing.T) { dir := t.TempDir() modelDir := filepath.Join(dir, "model") diff --git a/cmd/mxcli/docker/runlocal.go b/cmd/mxcli/docker/runlocal.go index e7feb546d..4b345fd58 100644 --- a/cmd/mxcli/docker/runlocal.go +++ b/cmd/mxcli/docker/runlocal.go @@ -40,6 +40,14 @@ type LocalRunOptions struct { AdminPass string // DB is the Postgres the runtime connects to (devcontainer defaults applied). DB DBConfig + // Hub, when set, is the URL of an mxcli tunnel-hub (e.g. https://hub.mxcli.org). + // The app stays running here; a chisel client reverse-tunnels it out to the hub + // so it is reachable in a browser at the hub URL. Implies a local run. The + // runtime boots with ApplicationRootUrl = Hub so the SPA works under that origin. + Hub string + // HubSecret is the shared auth secret for the hub ("user:pass"), matching the + // hub's --secret. Optional but recommended. + HubSecret string // Watch keeps running, rebuilding+applying on every project change. Watch bool // EnsureDB provisions the local Postgres + app database if missing (otherwise @@ -420,17 +428,20 @@ func RunLocal(opts LocalRunOptions) error { } } - // 6. Boot the runtime against the fresh deployment. + // 6. Boot the runtime against the fresh deployment. With --hub, the app is + // reached at the hub's public URL, so the runtime must boot with + // ApplicationRootUrl set to it (else the SPA/originURI misbehave across origins). rt, err := StartLocalRuntime(LocalRuntimeOptions{ DeployDir: opts.DeployDir, InstallPath: installPath, // JavaHome left empty: StartLocalRuntime resolves JDK 21. - AppPort: opts.AppPort, - AdminPort: opts.AdminPort, - AdminPass: opts.AdminPass, - DB: opts.DB, - Stdout: w, - Stderr: stderr, + AppPort: opts.AppPort, + AdminPort: opts.AdminPort, + AdminPass: opts.AdminPass, + ApplicationRootUrl: opts.Hub, + DB: opts.DB, + Stdout: w, + Stderr: stderr, }) if err != nil { return err @@ -439,6 +450,23 @@ func RunLocal(opts LocalRunOptions) error { fmt.Fprintf(w, "\nApp is running at %s\n", rt.AppURL()) + // 6a. With --hub, open a reverse tunnel so the app is reachable in a browser at + // the hub's public URL. The app stays here; only live HTTP flows through the + // tunnel (nothing is pushed to the hub). + if opts.Hub != "" { + tunnel, err := StartTunnel(TunnelOptions{ + HubURL: opts.Hub, + LocalPort: opts.AppPort, + Secret: opts.HubSecret, + Stdout: w, + }) + if err != nil { + return fmt.Errorf("starting hub tunnel: %w", err) + } + defer tunnel.Stop() + fmt.Fprintf(w, "Preview available at %s\n", tunnel.PublicURL()) + } + // 6b. If screenshot auth was requested, log in once and reuse the session for // every screenshot (pages behind login render authenticated). if opts.Screenshot && opts.ScreenshotUser != "" { diff --git a/cmd/mxcli/docker/tunnel.go b/cmd/mxcli/docker/tunnel.go new file mode 100644 index 000000000..56b590cd1 --- /dev/null +++ b/cmd/mxcli/docker/tunnel.go @@ -0,0 +1,140 @@ +// SPDX-License-Identifier: Apache-2.0 + +package docker + +import ( + "context" + "fmt" + "io" + "net/url" + "os" + "strings" + "time" + + chclient "github.com/jpillora/chisel/client" + "golang.org/x/net/http/httpproxy" +) + +// DefaultHubBackendPort is the port an mxcli tunnel-hub proxies public requests +// to (its chisel server's --backend), and the reverse port the client tunnels +// into. The two must agree; both default to 9000. +const DefaultHubBackendPort = 9000 + +// TunnelOptions configures an outbound chisel reverse tunnel from a locally +// running app to an mxcli tunnel-hub, so the app is reachable in a browser at +// the hub's public URL. The app never leaves this machine — only live HTTP flows +// through the tunnel. +type TunnelOptions struct { + // HubURL is the tunnel-hub base URL, e.g. https://hub.mxcli.org. The chisel + // control connection dials it over 443. + HubURL string + // LocalPort is the local app port to expose (e.g. 8080). + LocalPort int + // RemotePort is the hub's chisel-server reverse port, which the hub proxies + // public traffic to (must match the hub's --backend port). Default 9000. + RemotePort int + // Secret is the shared chisel auth ("user:pass"), matching the hub's --secret. + // Optional but recommended. + Secret string + // Proxy is the outbound HTTP CONNECT proxy the control connection dials + // through. In a Claude Code web session egress is proxy-only, so this must be + // set; it defaults from HTTPS_PROXY/https_proxy in the environment. chisel does + // not read the proxy env itself, so we pass it explicitly. + Proxy string + // Stdout receives progress messages (default os.Stdout). + Stdout io.Writer +} + +// Tunnel is a running reverse tunnel to a hub. +type Tunnel struct { + client *chclient.Client + cancel context.CancelFunc + publicURL string +} + +func (o *TunnelOptions) applyDefaults() { + if o.RemotePort == 0 { + o.RemotePort = DefaultHubBackendPort + } + if o.Proxy == "" { + o.Proxy = proxyForURL(o.HubURL) + } + if o.Stdout == nil { + o.Stdout = os.Stdout + } +} + +// proxyForURL resolves the outbound HTTP proxy for hubURL from the standard +// proxy environment (HTTPS_PROXY etc.), honouring NO_PROXY — so an external hub +// goes through the egress proxy while a loopback or allow-listed hub connects +// directly. chisel does not consult the proxy env itself, so we do it here. +func proxyForURL(hubURL string) string { + u, err := url.Parse(hubURL) + if err != nil || u.Host == "" { + return "" + } + p, err := httpproxy.FromEnvironment().ProxyFunc()(u) + if err != nil || p == nil { + return "" + } + return p.String() +} + +// StartTunnel opens the reverse tunnel and returns once the chisel client has +// started connecting (it retries in the background until the process exits). +// Call Stop to tear it down. +func StartTunnel(o TunnelOptions) (*Tunnel, error) { + o.applyDefaults() + if o.HubURL == "" { + return nil, fmt.Errorf("hub URL is required") + } + if o.LocalPort == 0 { + return nil, fmt.Errorf("local app port is required") + } + + // R::127.0.0.1: — the hub's chisel server listens on + // and forwards to this app's port (the app binds 127.0.0.1). + remote := fmt.Sprintf("R:%d:127.0.0.1:%d", o.RemotePort, o.LocalPort) + cfg := &chclient.Config{ + Server: o.HubURL, + Proxy: o.Proxy, + Auth: o.Secret, + Remotes: []string{remote}, + KeepAlive: 25 * time.Second, + MaxRetryInterval: 30 * time.Second, + } + c, err := chclient.NewClient(cfg) + if err != nil { + return nil, fmt.Errorf("configuring tunnel client: %w", err) + } + + ctx, cancel := context.WithCancel(context.Background()) + if err := c.Start(ctx); err != nil { + cancel() + return nil, fmt.Errorf("starting tunnel: %w", err) + } + + t := &Tunnel{client: c, cancel: cancel, publicURL: strings.TrimRight(o.HubURL, "/")} + if o.Proxy != "" { + fmt.Fprintf(o.Stdout, "Tunnel: exposing local :%d at %s (via proxy)\n", o.LocalPort, t.publicURL) + } else { + fmt.Fprintf(o.Stdout, "Tunnel: exposing local :%d at %s\n", o.LocalPort, t.publicURL) + } + return t, nil +} + +// PublicURL is the browser-reachable URL the app is served at through the hub. +func (t *Tunnel) PublicURL() string { return t.publicURL } + +// Stop tears down the tunnel. +func (t *Tunnel) Stop() { + if t == nil { + return + } + if t.cancel != nil { + t.cancel() + } + if t.client != nil { + _ = t.client.Close() + } +} diff --git a/cmd/mxcli/docker/tunnel_test.go b/cmd/mxcli/docker/tunnel_test.go new file mode 100644 index 000000000..13c360aaf --- /dev/null +++ b/cmd/mxcli/docker/tunnel_test.go @@ -0,0 +1,135 @@ +// SPDX-License-Identifier: Apache-2.0 + +package docker + +import ( + "fmt" + "io" + "net" + "net/http" + "net/http/httptest" + "net/url" + "strconv" + "testing" + "time" + + chserver "github.com/jpillora/chisel/server" +) + +// proxyForURL must route an external hub through the egress proxy but connect to +// a loopback / NO_PROXY hub directly — otherwise a local hub (or an allow-listed +// one) would be forced through a proxy that refuses it. +func TestProxyForURL(t *testing.T) { + t.Setenv("HTTPS_PROXY", "http://127.0.0.1:33451") + t.Setenv("HTTP_PROXY", "http://127.0.0.1:33451") + t.Setenv("NO_PROXY", "127.0.0.1,localhost,.internal.example") + + cases := []struct { + url string + want string + }{ + {"https://hub.mxcli.org", "http://127.0.0.1:33451"}, + {"http://127.0.0.1:9500", ""}, // loopback → NO_PROXY + {"https://relay.internal.example", ""}, // suffix match in NO_PROXY + {"", ""}, // no URL + {"://bad", ""}, // unparseable + } + for _, c := range cases { + if got := proxyForURL(c.url); got != c.want { + t.Errorf("proxyForURL(%q) = %q, want %q", c.url, got, c.want) + } + } +} + +// StartTunnel must reverse-tunnel a local port out to a hub so requests to the +// hub's reverse port reach the local app. This exercises the real embedded +// chisel client + server end to end, in-process (no external binary). +func TestTunnelRoundTrip(t *testing.T) { + // The "local app" the tunnel exposes. + backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + fmt.Fprintf(w, "hello from %s", r.Host) + })) + defer backend.Close() + backendPort := mustPort(t, backend.URL) + + // The hub: a chisel reverse server on a free port. + hubPort := freePort(t) + srv, err := chserver.NewServer(&chserver.Config{Reverse: true}) + if err != nil { + t.Fatalf("NewServer: %v", err) + } + if err := srv.Start("127.0.0.1", strconv.Itoa(hubPort)); err != nil { + t.Fatalf("hub Start: %v", err) + } + defer srv.Close() + + // The reverse port the hub will open and forward to the backend. + remotePort := freePort(t) + + tun, err := StartTunnel(TunnelOptions{ + HubURL: "http://127.0.0.1:" + strconv.Itoa(hubPort), + LocalPort: backendPort, + RemotePort: remotePort, + Proxy: "", // loopback: no proxy + Stdout: io.Discard, + }) + if err != nil { + t.Fatalf("StartTunnel: %v", err) + } + defer tun.Stop() + + if tun.PublicURL() != "http://127.0.0.1:"+strconv.Itoa(hubPort) { + t.Errorf("PublicURL = %q", tun.PublicURL()) + } + + // The client connects + opens the reverse listener asynchronously; poll it. + reverseURL := fmt.Sprintf("http://127.0.0.1:%d/", remotePort) + body := pollGet(t, reverseURL, 10*time.Second) + if body == "" { + t.Fatalf("no response through tunnel at %s", reverseURL) + } + // The request reached the backend through the tunnel. + if want := "hello from "; len(body) < len(want) || body[:len(want)] != want { + t.Errorf("through-tunnel body = %q, want prefix %q", body, want) + } +} + +func mustPort(t *testing.T, rawURL string) int { + t.Helper() + u, err := url.Parse(rawURL) + if err != nil { + t.Fatalf("parse %q: %v", rawURL, err) + } + p, err := strconv.Atoi(u.Port()) + if err != nil { + t.Fatalf("port of %q: %v", rawURL, err) + } + return p +} + +func freePort(t *testing.T) int { + t.Helper() + l, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatalf("free port: %v", err) + } + defer l.Close() + return l.Addr().(*net.TCPAddr).Port +} + +func pollGet(t *testing.T, url string, timeout time.Duration) string { + t.Helper() + deadline := time.Now().Add(timeout) + for time.Now().Before(deadline) { + resp, err := http.Get(url) + if err == nil { + b, _ := io.ReadAll(resp.Body) + resp.Body.Close() + if resp.StatusCode == http.StatusOK { + return string(b) + } + } + time.Sleep(150 * time.Millisecond) + } + return "" +} diff --git a/go.mod b/go.mod index c7ca28503..d29ee4c2c 100644 --- a/go.mod +++ b/go.mod @@ -13,6 +13,7 @@ require ( github.com/chzyer/readline v1.5.1 github.com/fsnotify/fsnotify v1.10.1 github.com/jackc/pgx/v5 v5.9.2 + github.com/jpillora/chisel v1.11.8 github.com/mattn/go-runewidth v0.0.24 github.com/microsoft/go-mssqldb v1.10.0 github.com/pmezard/go-difflib v1.0.0 @@ -27,6 +28,7 @@ require ( go.mongodb.org/mongo-driver/v2 v2.6.0 go.starlark.net v0.0.0-20260102030733-3fee463870c9 go.uber.org/zap v1.28.0 + golang.org/x/net v0.54.0 golang.org/x/sync v0.21.0 golang.org/x/term v0.43.0 gopkg.in/yaml.v3 v3.0.1 @@ -34,6 +36,8 @@ require ( ) require ( + github.com/andrew-d/go-termutil v0.0.0-20150726205930-009166a695a2 // indirect + github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 // indirect github.com/atotto/clipboard v0.1.4 // indirect github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect github.com/charmbracelet/colorprofile v0.4.1 // indirect @@ -49,10 +53,15 @@ require ( github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9 // indirect github.com/golang-sql/sqlexp v0.1.0 // indirect github.com/google/uuid v1.6.0 // indirect + github.com/gorilla/websocket v1.5.3 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/jackc/pgpassfile v1.0.0 // indirect github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect github.com/jackc/puddle/v2 v2.2.2 // indirect + github.com/jpillora/ansi v1.0.3 // indirect + github.com/jpillora/backoff v1.0.0 // indirect + github.com/jpillora/requestlog v1.0.0 // indirect + github.com/jpillora/sizestr v1.0.0 // indirect github.com/lucasb-eyer/go-colorful v1.3.0 // indirect github.com/mattn/go-isatty v0.0.20 // indirect github.com/mattn/go-localereader v0.0.1 // indirect @@ -66,6 +75,7 @@ require ( github.com/segmentio/asm v1.1.3 // indirect github.com/segmentio/encoding v0.3.4 // indirect github.com/shopspring/decimal v1.4.0 // indirect + github.com/tomasen/realip v0.0.0-20180522021738-f0c99a92ddce // indirect github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect go.lsp.dev/pkg v0.0.0-20210717090340-384b27a52fb2 // indirect go.uber.org/multierr v1.10.0 // indirect diff --git a/go.sum b/go.sum index cd91f4f4a..eb4ec15c4 100644 --- a/go.sum +++ b/go.sum @@ -18,8 +18,12 @@ github.com/alecthomas/chroma/v2 v2.26.1 h1:2X21EdxGZNv5GF9mG5u+uzc02GCFyGxbcBm3G github.com/alecthomas/chroma/v2 v2.26.1/go.mod h1:lxhRRa9H4hPmRLOOdYga4zkQIQjq3dtrrdwQeCfu78Y= github.com/alecthomas/repr v0.5.2 h1:SU73FTI9D1P5UNtvseffFSGmdNci/O6RsqzeXJtP0Qs= github.com/alecthomas/repr v0.5.2/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4= +github.com/andrew-d/go-termutil v0.0.0-20150726205930-009166a695a2 h1:axBiC50cNZOs7ygH5BgQp4N+aYrZ2DNpWZ1KG3VOSOM= +github.com/andrew-d/go-termutil v0.0.0-20150726205930-009166a695a2/go.mod h1:jnzFpU88PccN/tPPhCpnNU8mZphvKxYM9lLNkd8e+os= github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYWrPrQ= github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmOABFgjdkM7Nw= +github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio= +github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs= github.com/atotto/clipboard v0.1.4 h1:EH0zSVneZPSuFR11BlR9YppQTVDbh5+16AmcJi4g1z4= github.com/atotto/clipboard v0.1.4/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn0Yu86PYI= github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k= @@ -77,6 +81,8 @@ github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17k github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg= +github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM= @@ -91,6 +97,16 @@ github.com/jackc/pgx/v5 v5.9.2 h1:3ZhOzMWnR4yJ+RW1XImIPsD1aNSz4T4fyP7zlQb56hw= github.com/jackc/pgx/v5 v5.9.2/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4= github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo= github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4= +github.com/jpillora/ansi v1.0.3 h1:nn4Jzti0EmRfDxm7JtEs5LzCbNwd5sv+0aE+LdS9/ZQ= +github.com/jpillora/ansi v1.0.3/go.mod h1:D2tT+6uzJvN1nBVQILYWkIdq7zG+b5gcFN5WI/VyjMY= +github.com/jpillora/backoff v1.0.0 h1:uvFg412JmmHBHw7iwprIxkPMI+sGQ4kzOWsMeHnm2EA= +github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= +github.com/jpillora/chisel v1.11.8 h1:pf2ufs2kfytmSWkZoeEgSs5pGrn9ODL2UaZRiR/meyk= +github.com/jpillora/chisel v1.11.8/go.mod h1:efqW+iikEM4E7CxQSR1LW1oyErlZtWXC/R81C/Fzk2w= +github.com/jpillora/requestlog v1.0.0 h1:bg++eJ74T7DYL3DlIpiwknrtfdUA9oP/M4fL+PpqnyA= +github.com/jpillora/requestlog v1.0.0/go.mod h1:HTWQb7QfDc2jtHnWe2XEIEeJB7gJPnVdpNn52HXPvy8= +github.com/jpillora/sizestr v1.0.0 h1:4tr0FLxs1Mtq3TnsLDV+GYUWG7Q26a6s+tV5Zfw2ygw= +github.com/jpillora/sizestr v1.0.0/go.mod h1:bUhLv4ctkknatr6gR42qPxirmd5+ds1u7mzD+MZ33f0= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0= github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk= @@ -150,6 +166,8 @@ github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81P github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +github.com/tomasen/realip v0.0.0-20180522021738-f0c99a92ddce h1:fb190+cK2Xz/dvi9Hv8eCYJYvIGUTN2/KLq1pT6CjEc= +github.com/tomasen/realip v0.0.0-20180522021738-f0c99a92ddce/go.mod h1:o8v6yHRoik09Xen7gje4m9ERNah1d1PPsVq1VEx9vE4= github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no= github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM= go.lsp.dev/jsonrpc2 v0.10.0 h1:Pr/YcXJoEOTMc/b6OTmcR1DPJ3mSWl/SWiU1Cct6VmI= From 8535f0a5ad592fd1d095baff085061a750d2e018 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 23 Jul 2026 06:23:57 +0000 Subject: [PATCH 03/12] docs(run-local): document --hub external preview + mxcli tunnel-hub Add the --hub/--hub-secret flags and an "External browser preview" section to the run-local skill and the docs-site page, and mark slice 3 shipped in the warm-loop proposal (code + in-process tunnel test + local end-to-end; external E2E against the Scaleway hub pending). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .claude/skills/mendix/run-local.md | 31 +++++++++++++++- docs-site/src/tools/run-local.md | 36 ++++++++++++++++++- .../PROPOSAL_mxcli_dev_warm_loop.md | 2 +- 3 files changed, 66 insertions(+), 3 deletions(-) diff --git a/.claude/skills/mendix/run-local.md b/.claude/skills/mendix/run-local.md index 34f57b5dd..ef58ce1d6 100644 --- a/.claude/skills/mendix/run-local.md +++ b/.claude/skills/mendix/run-local.md @@ -107,7 +107,9 @@ Launch `run --local` as the **sole** command in its invocation (don't chain a tr | Flag | Default | Purpose | |------|---------|---------| -| `--local` | — | Required; run without Docker | +| `--local` | — | Required; run without Docker (implied by `--hub`) | +| `--hub` | — | Expose the app in a browser at a tunnel-hub URL (see below) | +| `--hub-secret` | — | Shared auth (`user:pass`) matching the hub's `--secret` | | `--watch` | off | Rebuild + hot-apply on each change | | `--ensure-db` | off | Provision local Postgres + app database if missing | | `--setup` | off | Cache MxBuild+runtime + ensure DB, then exit (SessionStart bring-up) | @@ -147,6 +149,33 @@ mxcli run --local -p app.mpr --watch --screenshot reuse the session, so pages behind login render authenticated. Best-effort: an anonymous app with no login form proceeds unauthenticated. +## External browser preview (`--hub`) + +`--hub ` exposes the running app in a **browser at a public URL** without the app +leaving this machine and without committing — for reviewing work-in-progress from a +phone/tablet, or from an egress-only environment like Claude Code on the web. + +```bash +# on a small VPS with a public IP + domain (hub.mxcli.org -> it, ports 80+443 open): +mxcli tunnel-hub --domain hub.mxcli.org --secret alice:s3cret + +# here, where the app runs: +mxcli run --hub https://hub.mxcli.org --hub-secret alice:s3cret -p app.mpr +# -> boots the app locally and prints "Preview available at https://hub.mxcli.org" +``` + +- The app stays here; a **chisel reverse tunnel** dials *out* to the hub over 443, and + the hub proxies browser requests back down it. Nothing is pushed — only live HTTP. + Everything rides one 443 connection, so it works through an egress-only proxy. +- `--hub` **implies `--local`** and boots the runtime with `ApplicationRootUrl` set to + the hub URL, so the SPA/`originURI` work under the public origin. +- Combine with `--watch` for the full loop: edit here → hot-apply → refresh the browser. +- The control connection honours `NO_PROXY`: an external hub goes through the egress + proxy, a loopback hub connects directly. + +`mxcli tunnel-hub` is the static relay (one small VPS fronts your previews). It uses +automatic Let's Encrypt for `--domain` (needs inbound 80+443), or `--tls-cert`/`--tls-key`. + ## Validation checklist - [ ] Project is Mendix 11.x. diff --git a/docs-site/src/tools/run-local.md b/docs-site/src/tools/run-local.md index 0fd95f999..7fa631c9f 100644 --- a/docs-site/src/tools/run-local.md +++ b/docs-site/src/tools/run-local.md @@ -61,7 +61,9 @@ so structural changes need a restart; behavioural changes do not. | Flag | Default | Purpose | |------|---------|---------| -| `--local` | — | Required; run without Docker | +| `--local` | — | Required; run without Docker (implied by `--hub`) | +| `--hub` | — | Expose the running app in a browser at a tunnel-hub URL (see [External browser preview](#external-browser-preview---hub)) | +| `--hub-secret` | — | Shared auth (`user:pass`) matching the hub's `--secret` | | `--watch` | off | Rebuild + hot-apply on every project change | | `--ensure-db` | off | Provision local Postgres + the app database if missing (fresh-session bootstrap) | | `--setup` | off | Prepare prerequisites (cache MxBuild+runtime, ensure DB) and exit without booting — for a SessionStart hook | @@ -76,6 +78,38 @@ so structural changes need a restart; behavioural changes do not. | `--screenshot-url` | app root | Page to shoot: full URL, or a path relative to the app root (e.g. `/p/customers`). Repeat for a multi-page set. | | `--screenshot-user` / `--screenshot-password` | — | Log in once (Mendix form auth) and reuse the session, so pages behind login render authenticated | +## External browser preview (`--hub`) + +`--hub ` makes the running app reachable **in a browser at a public URL** — without +the app leaving this machine and without committing. It's for reviewing work-in-progress +from a phone or tablet, or from an egress-only environment such as Claude Code on the web. + +```bash +# on a small VPS with a public IP + domain (hub.mxcli.org -> it, inbound 80+443 open): +mxcli tunnel-hub --domain hub.mxcli.org --secret alice:s3cret + +# where the app runs (here): +mxcli run --hub https://hub.mxcli.org --hub-secret alice:s3cret -p app.mpr +# -> "Preview available at https://hub.mxcli.org" +``` + +How it works: the app stays local and a **chisel reverse tunnel** dials *out* to the hub +over 443; the hub proxies browser requests back down the tunnel. Nothing is pushed — only +live HTTP flows. Because everything rides a single 443 connection, it works even from an +egress-only environment. + +- `--hub` **implies `--local`**, and boots the runtime with `ApplicationRootUrl` set to + the hub URL so the SPA and `originURI` cookie work under the public origin. +- Combine with `--watch` for the full remote loop: edit here → hot-apply → refresh the + browser tab. +- The control connection honours `NO_PROXY` — an external hub goes through the egress + proxy, a loopback hub connects directly. + +`mxcli tunnel-hub` is the static relay: run it once on a small VPS to front your previews. +TLS is automatic via Let's Encrypt for `--domain` (needs inbound 80 + 443), or supply +`--tls-cert`/`--tls-key`. This slice fronts a single app; multi-tenant registration and +per-preview subdomains are planned. + ## The change signal `--watch` watches two **source** trees and rebuilds when either changes: diff --git a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md index cb09fec53..21b4bfe23 100644 --- a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md +++ b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md @@ -599,7 +599,7 @@ builds on the previous. |---|-------|----------|------------|-------------| | 1 | **Warm local loop** — shipped as `mxcli run --local [--watch]` (serve daemon + M2EE admin client + `restartRequired` branching; + client bundling & Playwright screenshots) | Docker-free ~1 s edit→test loop, locally | nothing new | ✅ **shipped** | | 2 | **Provisioning** — `run --local --ensure-db` (DB) + `run --local --setup` (non-blocking bring-up) + `mxcli init` SessionStart hook + bootstrap prompt template | a fresh Claude Code Web session comes up testable; iPad-native start | slice 1 | ✅ **shipped** | -| 3 | **Single-app external preview** — `mxcli run --local --hub ` (chisel client + `--proxy $HTTPS_PROXY`) → static VPS relay + `ApplicationRootUrl`-on-subdomain wiring | a shareable live preview URL (the iPad two-container flow) | slice 1 | medium / **low** — transport verified end-to-end 2026-07-23 (§ Scaling); remaining work is the `ApplicationRootUrl`/subdomain wiring, not the tunnel | +| 3 | **Single-app external preview** — `mxcli run --hub ` (embedded chisel client, proxy honouring `NO_PROXY`, `ApplicationRootUrl` boot wiring) + `mxcli tunnel-hub` (embedded chisel server, autocert, single-443 `--backend`) | a shareable live preview URL (the iPad two-container flow) | slice 1 | ✅ **shipped** (2026-07-23) — code + in-process tunnel test + local end-to-end boot; Mendix renders through the hub's Host-rewriting backend, no shim needed. External E2E against the Scaleway hub is the remaining confirmation | | 4 | **Tunnel hub** — `mxcli tunnel-hub` + `mxcli run --local --hub` + registration API + admin overview | many dev containers behind one ingress; fleet overview + per-container change lists | slice 3 | large / higher — a product in its own right, with a multi-tenant auth surface | Recommended sequencing: **1 → 2** delivers the complete solo dev experience (Scenario A From 8aea8a4122d3da62ac89af31d53aaf26e05dfacf Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 23 Jul 2026 07:04:46 +0000 Subject: [PATCH 04/12] feat(tunnel-hub): multi-tenant registry + registration API (slice 4 foundation) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The data plane for the multi-tenant hub: a Registry of registered previews keyed by identity (solution/project/branch/worktree) so reconnects keep a stable URL, with subdomain-slug allocation (project-branch; main collapses to the project; collisions disambiguated by worktree then a numeric suffix), reverse-port allocation from a range, heartbeat-based availability (available/stale) with TTL reaping, and deterministic sorting by last-used / registered / project+solution. Plus the registration API over it: POST /api/register (returns subdomain, url, reverse port, control URL, token, heartbeat interval), POST /api/status (heartbeat by bearer token), POST /api/deregister, GET /api/backends (for the admin page), with an optional shared-secret gate on registration. Foundation only — the 443 front (host routing + dynamic TLS + chisel control), the admin overview page, and the run --hub registration client follow. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- cmd/mxcli/tunnelhub/api.go | 144 ++++++++++++ cmd/mxcli/tunnelhub/api_test.go | 128 +++++++++++ cmd/mxcli/tunnelhub/registry.go | 316 +++++++++++++++++++++++++++ cmd/mxcli/tunnelhub/registry_test.go | 163 ++++++++++++++ cmd/mxcli/tunnelhub/slug.go | 57 +++++ 5 files changed, 808 insertions(+) create mode 100644 cmd/mxcli/tunnelhub/api.go create mode 100644 cmd/mxcli/tunnelhub/api_test.go create mode 100644 cmd/mxcli/tunnelhub/registry.go create mode 100644 cmd/mxcli/tunnelhub/registry_test.go create mode 100644 cmd/mxcli/tunnelhub/slug.go diff --git a/cmd/mxcli/tunnelhub/api.go b/cmd/mxcli/tunnelhub/api.go new file mode 100644 index 000000000..781b5d67b --- /dev/null +++ b/cmd/mxcli/tunnelhub/api.go @@ -0,0 +1,144 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import ( + "encoding/json" + "net/http" + "strings" +) + +// APIOptions configures the registration API. +type APIOptions struct { + Registry *Registry + // ControlURL is where the client points its chisel control connection + // (e.g. https://hub.mxcli.org). Returned in the registration response. + ControlURL string + // TunnelAuth is the shared chisel auth ("user:pass") the client must use, if + // the hub's tunnel server requires one. Returned to the client. + TunnelAuth string + // RegisterSecret, if set, gates /api/register: the client must send a matching + // X-Hub-Secret header (from --hub-secret). Empty means open registration. + RegisterSecret string + // HeartbeatIntervalSec is how often the client should heartbeat (default 20). + HeartbeatIntervalSec int +} + +// API serves the hub's registration + query endpoints over the registry. +type API struct { + opts APIOptions +} + +// RegisterResponse is returned to `mxcli run --hub` after registration. +type RegisterResponse struct { + Subdomain string `json:"subdomain"` + URL string `json:"url"` + ReversePort int `json:"reversePort"` + ControlURL string `json:"controlUrl"` + Token string `json:"token"` + TunnelAuth string `json:"tunnelAuth,omitempty"` + HeartbeatIntervalSec int `json:"heartbeatIntervalSec"` +} + +// NewAPI builds the API handler set. +func NewAPI(o APIOptions) *API { + if o.HeartbeatIntervalSec == 0 { + o.HeartbeatIntervalSec = 20 + } + return &API{opts: o} +} + +// Mount registers the API routes on mux under /api/. +func (a *API) Mount(mux *http.ServeMux) { + mux.HandleFunc("/api/register", a.handleRegister) + mux.HandleFunc("/api/status", a.handleStatus) + mux.HandleFunc("/api/deregister", a.handleDeregister) + mux.HandleFunc("/api/backends", a.handleBackends) +} + +func (a *API) handleRegister(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodPost { + http.Error(w, "method not allowed", http.StatusMethodNotAllowed) + return + } + if a.opts.RegisterSecret != "" && r.Header.Get("X-Hub-Secret") != a.opts.RegisterSecret { + http.Error(w, "invalid or missing hub secret", http.StatusUnauthorized) + return + } + var req RegisterRequest + if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<16)).Decode(&req); err != nil { + http.Error(w, "invalid request body: "+err.Error(), http.StatusBadRequest) + return + } + if strings.TrimSpace(req.Project) == "" { + http.Error(w, "project is required", http.StatusBadRequest) + return + } + b, err := a.opts.Registry.Register(req) + if err != nil { + http.Error(w, err.Error(), http.StatusServiceUnavailable) + return + } + host := b.Subdomain + if d := a.opts.Registry.domain; d != "" { + host = b.Subdomain + "." + d + } + writeJSON(w, http.StatusOK, RegisterResponse{ + Subdomain: b.Subdomain, + URL: "https://" + host, + ReversePort: b.ReversePort, + ControlURL: a.opts.ControlURL, + Token: b.ID, + TunnelAuth: a.opts.TunnelAuth, + HeartbeatIntervalSec: a.opts.HeartbeatIntervalSec, + }) +} + +func (a *API) handleStatus(w http.ResponseWriter, r *http.Request) { + a.byToken(w, r, func(token string) { + if !a.opts.Registry.Heartbeat(token) { + http.Error(w, "unknown token", http.StatusNotFound) + return + } + w.WriteHeader(http.StatusNoContent) + }) +} + +func (a *API) handleDeregister(w http.ResponseWriter, r *http.Request) { + a.byToken(w, r, func(token string) { + a.opts.Registry.Deregister(token) + w.WriteHeader(http.StatusNoContent) + }) +} + +// byToken extracts the bearer token (Authorization: Bearer or ?token=) and +// invokes fn. POST only. +func (a *API) byToken(w http.ResponseWriter, r *http.Request, fn func(token string)) { + if r.Method != http.MethodPost { + http.Error(w, "method not allowed", http.StatusMethodNotAllowed) + return + } + token := bearerToken(r) + if token == "" { + http.Error(w, "missing token", http.StatusUnauthorized) + return + } + fn(token) +} + +func (a *API) handleBackends(w http.ResponseWriter, r *http.Request) { + writeJSON(w, http.StatusOK, a.opts.Registry.List(r.URL.Query().Get("sort"))) +} + +func bearerToken(r *http.Request) string { + if h := r.Header.Get("Authorization"); strings.HasPrefix(h, "Bearer ") { + return strings.TrimSpace(strings.TrimPrefix(h, "Bearer ")) + } + return strings.TrimSpace(r.URL.Query().Get("token")) +} + +func writeJSON(w http.ResponseWriter, status int, v any) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(status) + _ = json.NewEncoder(w).Encode(v) +} diff --git a/cmd/mxcli/tunnelhub/api_test.go b/cmd/mxcli/tunnelhub/api_test.go new file mode 100644 index 000000000..d3576790a --- /dev/null +++ b/cmd/mxcli/tunnelhub/api_test.go @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import ( + "encoding/json" + "net/http" + "net/http/httptest" + "strings" + "testing" + "time" +) + +func newTestAPI(t *testing.T, secret string) (*API, *Registry) { + t.Helper() + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + reg := newTestRegistry(clk) + api := NewAPI(APIOptions{ + Registry: reg, + ControlURL: "https://hub.mxcli.org", + RegisterSecret: secret, + }) + return api, reg +} + +func doJSON(t *testing.T, api *API, method, path, body string, headers map[string]string) *httptest.ResponseRecorder { + t.Helper() + mux := http.NewServeMux() + api.Mount(mux) + req := httptest.NewRequest(method, path, strings.NewReader(body)) + for k, v := range headers { + req.Header.Set(k, v) + } + rec := httptest.NewRecorder() + mux.ServeHTTP(rec, req) + return rec +} + +func TestAPI_RegisterAndBackends(t *testing.T) { + api, _ := newTestAPI(t, "") + rec := doJSON(t, api, http.MethodPost, "/api/register", + `{"project":"MyApp","solution":"Sol","branch":"feature/x","appPort":8080}`, nil) + if rec.Code != http.StatusOK { + t.Fatalf("register status = %d, body %s", rec.Code, rec.Body) + } + var resp RegisterResponse + if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil { + t.Fatal(err) + } + if resp.URL != "https://myapp-feature-x.mxcli.org" { + t.Errorf("url = %q", resp.URL) + } + if resp.ReversePort == 0 || resp.Token == "" || resp.ControlURL != "https://hub.mxcli.org" { + t.Errorf("incomplete response: %+v", resp) + } + + // backends lists it. + lb := doJSON(t, api, http.MethodGet, "/api/backends?sort=project", "", nil) + var views []BackendView + if err := json.Unmarshal(lb.Body.Bytes(), &views); err != nil { + t.Fatal(err) + } + if len(views) != 1 || views[0].Project != "MyApp" || views[0].Availability != Available { + t.Errorf("backends = %+v", views) + } +} + +func TestAPI_RegisterRequiresProject(t *testing.T) { + api, _ := newTestAPI(t, "") + rec := doJSON(t, api, http.MethodPost, "/api/register", `{"branch":"main"}`, nil) + if rec.Code != http.StatusBadRequest { + t.Errorf("status = %d, want 400", rec.Code) + } +} + +func TestAPI_RegisterSecretGate(t *testing.T) { + api, _ := newTestAPI(t, "s3cret") + // missing secret -> 401 + if rec := doJSON(t, api, http.MethodPost, "/api/register", `{"project":"A"}`, nil); rec.Code != http.StatusUnauthorized { + t.Errorf("no secret: status = %d, want 401", rec.Code) + } + // correct secret -> 200 + rec := doJSON(t, api, http.MethodPost, "/api/register", `{"project":"A"}`, + map[string]string{"X-Hub-Secret": "s3cret"}) + if rec.Code != http.StatusOK { + t.Errorf("with secret: status = %d, want 200", rec.Code) + } +} + +func TestAPI_HeartbeatAndDeregister(t *testing.T) { + api, reg := newTestAPI(t, "") + var resp RegisterResponse + rec := doJSON(t, api, http.MethodPost, "/api/register", `{"project":"A","branch":"main"}`, nil) + _ = json.Unmarshal(rec.Body.Bytes(), &resp) + + // heartbeat with the token succeeds. + hb := doJSON(t, api, http.MethodPost, "/api/status", "", + map[string]string{"Authorization": "Bearer " + resp.Token}) + if hb.Code != http.StatusNoContent { + t.Errorf("heartbeat status = %d, want 204", hb.Code) + } + // bad token -> 404. + bad := doJSON(t, api, http.MethodPost, "/api/status", "", + map[string]string{"Authorization": "Bearer nope"}) + if bad.Code != http.StatusNotFound { + t.Errorf("bad-token heartbeat status = %d, want 404", bad.Code) + } + // missing token -> 401. + if none := doJSON(t, api, http.MethodPost, "/api/status", "", nil); none.Code != http.StatusUnauthorized { + t.Errorf("no-token heartbeat status = %d, want 401", none.Code) + } + // deregister removes it. + dr := doJSON(t, api, http.MethodPost, "/api/deregister", "", + map[string]string{"Authorization": "Bearer " + resp.Token}) + if dr.Code != http.StatusNoContent { + t.Errorf("deregister status = %d, want 204", dr.Code) + } + if len(reg.List("used")) != 0 { + t.Error("backend should be gone after deregister") + } +} + +func TestAPI_RegisterMethodNotAllowed(t *testing.T) { + api, _ := newTestAPI(t, "") + if rec := doJSON(t, api, http.MethodGet, "/api/register", "", nil); rec.Code != http.StatusMethodNotAllowed { + t.Errorf("GET register status = %d, want 405", rec.Code) + } +} diff --git a/cmd/mxcli/tunnelhub/registry.go b/cmd/mxcli/tunnelhub/registry.go new file mode 100644 index 000000000..966de6a50 --- /dev/null +++ b/cmd/mxcli/tunnelhub/registry.go @@ -0,0 +1,316 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import ( + "crypto/rand" + "encoding/hex" + "fmt" + "sort" + "strings" + "sync" + "time" +) + +// Availability is a backend's liveness, derived from heartbeat freshness. +type Availability string + +const ( + // Available: the container heartbeat is fresh — the app should be reachable. + Available Availability = "available" + // Stale: no recent heartbeat (e.g. a Claude Code web container reaped on idle), + // but not yet expired — shown so the user can see it went away. + Stale Availability = "stale" +) + +// Backend is one registered preview: a locally-running app reverse-tunnelled to +// the hub and served at its subdomain. +type Backend struct { + ID string `json:"id"` // opaque token (auth for heartbeat/deregister + chisel) + Project string `json:"project"` // e.g. the .mpr name + Solution string `json:"solution"` // optional grouping for multi-app solutions + Branch string `json:"branch"` // git branch + Worktree string `json:"worktree"` // optional, distinguishes worktrees of one branch + Subdomain string `json:"subdomain"` + ReversePort int `json:"reversePort"` + AppPort int `json:"appPort"` + + RegisteredAt time.Time `json:"registeredAt"` + LastSeenAt time.Time `json:"lastSeenAt"` // last heartbeat + LastUsedAt time.Time `json:"lastUsedAt"` // last browser request to the subdomain +} + +// identity is the stable key for a preview across reconnects: same project + +// branch + worktree + solution re-registers to the same slot (stable URL). +func (b *Backend) identity() string { + return strings.Join([]string{b.Solution, b.Project, b.Branch, b.Worktree}, "\x00") +} + +// BackendView is a Backend plus derived fields, for the API/admin page. +type BackendView struct { + Backend + URL string `json:"url"` + Availability Availability `json:"availability"` + UptimeSec int64 `json:"uptimeSec"` +} + +// RegisterRequest is the registration payload from `mxcli run --hub`. +type RegisterRequest struct { + Project string `json:"project"` + Solution string `json:"solution"` + Branch string `json:"branch"` + Worktree string `json:"worktree"` + AppPort int `json:"appPort"` +} + +// Registry is the in-memory store of registered backends. All methods are safe +// for concurrent use. +type Registry struct { + mu sync.Mutex + byID map[string]*Backend + bySubdomain map[string]*Backend + byIdentity map[string]*Backend + usedPorts map[int]bool + + domain string // e.g. "mxcli.org" + portBase int // first reverse port to allocate + portCount int // number of reverse ports available + staleFor time.Duration + expireFor time.Duration + now func() time.Time +} + +// RegistryOptions configures a Registry. Zero values get sensible defaults. +type RegistryOptions struct { + Domain string + PortBase int + PortCount int + StaleFor time.Duration // no heartbeat within this -> Stale (default 45s) + ExpireFor time.Duration // no heartbeat within this -> removed (default 10m) + Now func() time.Time +} + +// NewRegistry creates an empty registry. +func NewRegistry(o RegistryOptions) *Registry { + if o.PortBase == 0 { + o.PortBase = 9001 + } + if o.PortCount == 0 { + o.PortCount = 200 + } + if o.StaleFor == 0 { + o.StaleFor = 45 * time.Second + } + if o.ExpireFor == 0 { + o.ExpireFor = 10 * time.Minute + } + if o.Now == nil { + o.Now = time.Now + } + return &Registry{ + byID: map[string]*Backend{}, + bySubdomain: map[string]*Backend{}, + byIdentity: map[string]*Backend{}, + usedPorts: map[int]bool{}, + domain: o.Domain, + portBase: o.PortBase, + portCount: o.PortCount, + staleFor: o.StaleFor, + expireFor: o.ExpireFor, + now: o.Now, + } +} + +// Register allocates (or refreshes) a backend for the request and returns it. A +// re-registration with the same identity (project/branch/worktree/solution) +// returns the existing slot with a fresh heartbeat, so URLs are stable across +// reconnects. +func (r *Registry) Register(req RegisterRequest) (*Backend, error) { + r.mu.Lock() + defer r.mu.Unlock() + r.reapLocked() + + now := r.now() + b := &Backend{ + Project: strings.TrimSpace(req.Project), + Solution: strings.TrimSpace(req.Solution), + Branch: strings.TrimSpace(req.Branch), + Worktree: strings.TrimSpace(req.Worktree), + AppPort: req.AppPort, + } + if existing, ok := r.byIdentity[b.identity()]; ok { + existing.LastSeenAt = now + existing.AppPort = req.AppPort + return existing, nil + } + + port, err := r.allocPortLocked() + if err != nil { + return nil, err + } + b.ID = newToken() + b.Subdomain = r.allocSubdomainLocked(b.Project, b.Branch, b.Worktree) + b.ReversePort = port + b.RegisteredAt = now + b.LastSeenAt = now + b.LastUsedAt = time.Time{} + + r.byID[b.ID] = b + r.bySubdomain[b.Subdomain] = b + r.byIdentity[b.identity()] = b + r.usedPorts[port] = true + return b, nil +} + +// Heartbeat refreshes a backend's liveness by token. +func (r *Registry) Heartbeat(id string) bool { + r.mu.Lock() + defer r.mu.Unlock() + b, ok := r.byID[id] + if !ok { + return false + } + b.LastSeenAt = r.now() + return true +} + +// TouchUsed records a browser request to a subdomain (updates LastUsedAt). +func (r *Registry) TouchUsed(subdomain string) { + r.mu.Lock() + defer r.mu.Unlock() + if b, ok := r.bySubdomain[subdomain]; ok { + b.LastUsedAt = r.now() + } +} + +// Deregister removes a backend by token. +func (r *Registry) Deregister(id string) bool { + r.mu.Lock() + defer r.mu.Unlock() + b, ok := r.byID[id] + if !ok { + return false + } + r.removeLocked(b) + return true +} + +// LookupSubdomain returns the backend serving a subdomain. +func (r *Registry) LookupSubdomain(subdomain string) (*Backend, bool) { + r.mu.Lock() + defer r.mu.Unlock() + b, ok := r.bySubdomain[subdomain] + if !ok { + return nil, false + } + cp := *b + return &cp, true +} + +// List returns a snapshot of all backends as views, sorted by the given key +// ("used", "registered", "project"; default "used"), most-recent/first. +func (r *Registry) List(sortKey string) []BackendView { + r.mu.Lock() + defer r.mu.Unlock() + r.reapLocked() + + out := make([]BackendView, 0, len(r.byID)) + for _, b := range r.byID { + out = append(out, r.viewLocked(b)) + } + sortViews(out, sortKey) + return out +} + +// viewLocked builds the derived view for a backend. +func (r *Registry) viewLocked(b *Backend) BackendView { + host := b.Subdomain + if r.domain != "" { + host = b.Subdomain + "." + r.domain + } + av := Available + if r.now().Sub(b.LastSeenAt) > r.staleFor { + av = Stale + } + return BackendView{ + Backend: *b, + URL: "https://" + host, + Availability: av, + UptimeSec: int64(r.now().Sub(b.RegisteredAt).Seconds()), + } +} + +// reapLocked removes backends whose heartbeat is older than expireFor. +func (r *Registry) reapLocked() { + cutoff := r.now().Add(-r.expireFor) + for _, b := range r.byID { + if b.LastSeenAt.Before(cutoff) { + r.removeLocked(b) + } + } +} + +func (r *Registry) removeLocked(b *Backend) { + delete(r.byID, b.ID) + delete(r.bySubdomain, b.Subdomain) + delete(r.byIdentity, b.identity()) + delete(r.usedPorts, b.ReversePort) +} + +// allocPortLocked returns a free reverse port from the configured range. +func (r *Registry) allocPortLocked() (int, error) { + for p := r.portBase; p < r.portBase+r.portCount; p++ { + if !r.usedPorts[p] { + return p, nil + } + } + return 0, fmt.Errorf("no free reverse port (all %d in use)", r.portCount) +} + +// allocSubdomainLocked returns a unique subdomain slug, disambiguating a +// collision with the worktree name then a numeric suffix. +func (r *Registry) allocSubdomainLocked(project, branch, worktree string) string { + base := baseSlug(project, branch) + if _, taken := r.bySubdomain[base]; !taken { + return base + } + if wt := slugify(worktree); wt != "" { + cand := truncateLabel(base + "-" + wt) + if _, taken := r.bySubdomain[cand]; !taken { + return cand + } + } + for i := 2; ; i++ { + cand := truncateLabel(fmt.Sprintf("%s-%d", base, i)) + if _, taken := r.bySubdomain[cand]; !taken { + return cand + } + } +} + +func newToken() string { + var b [16]byte + _, _ = rand.Read(b[:]) + return hex.EncodeToString(b[:]) +} + +// sortViews orders views in place by key, newest first for time keys. +func sortViews(v []BackendView, key string) { + less := map[string]func(a, b BackendView) bool{ + "registered": func(a, b BackendView) bool { return a.RegisteredAt.After(b.RegisteredAt) }, + "project": func(a, b BackendView) bool { + if a.Solution != b.Solution { + return a.Solution < b.Solution + } + if a.Project != b.Project { + return a.Project < b.Project + } + return a.Branch < b.Branch + }, + "used": func(a, b BackendView) bool { return a.LastUsedAt.After(b.LastUsedAt) }, + }[key] + if less == nil { + less = func(a, b BackendView) bool { return a.LastUsedAt.After(b.LastUsedAt) } + } + sort.SliceStable(v, func(i, j int) bool { return less(v[i], v[j]) }) +} diff --git a/cmd/mxcli/tunnelhub/registry_test.go b/cmd/mxcli/tunnelhub/registry_test.go new file mode 100644 index 000000000..064f1c748 --- /dev/null +++ b/cmd/mxcli/tunnelhub/registry_test.go @@ -0,0 +1,163 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import ( + "testing" + "time" +) + +// fakeClock is a controllable time source for deterministic tests. +type fakeClock struct{ t time.Time } + +func (c *fakeClock) now() time.Time { return c.t } +func (c *fakeClock) add(d time.Duration) { c.t = c.t.Add(d) } + +func newTestRegistry(clk *fakeClock) *Registry { + return NewRegistry(RegistryOptions{ + Domain: "mxcli.org", + PortBase: 9001, + PortCount: 5, + StaleFor: 45 * time.Second, + ExpireFor: 10 * time.Minute, + Now: clk.now, + }) +} + +func TestRegister_AllocatesSubdomainAndPort(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := newTestRegistry(clk) + + b, err := r.Register(RegisterRequest{Project: "MyApp", Branch: "feature/x", AppPort: 8080}) + if err != nil { + t.Fatalf("Register: %v", err) + } + if b.Subdomain != "myapp-feature-x" { + t.Errorf("subdomain = %q, want myapp-feature-x", b.Subdomain) + } + if b.ReversePort != 9001 { + t.Errorf("reversePort = %d, want 9001", b.ReversePort) + } + if b.ID == "" { + t.Error("token must be set") + } + // main branch collapses to just the project. + m, _ := r.Register(RegisterRequest{Project: "MyApp", Branch: "main"}) + if m.Subdomain != "myapp" { + t.Errorf("main-branch subdomain = %q, want myapp", m.Subdomain) + } +} + +func TestRegister_SameIdentityIsStable(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := newTestRegistry(clk) + + a, _ := r.Register(RegisterRequest{Project: "App", Branch: "main", AppPort: 8080}) + clk.add(30 * time.Second) + b, _ := r.Register(RegisterRequest{Project: "App", Branch: "main", AppPort: 8080}) + + if a.ID != b.ID || a.Subdomain != b.Subdomain || a.ReversePort != b.ReversePort { + t.Errorf("re-register changed the slot: %+v vs %+v", a, b) + } + if !b.LastSeenAt.Equal(clk.t) { + t.Error("re-register should refresh the heartbeat") + } + if got := r.List("used"); len(got) != 1 { + t.Errorf("want 1 backend after re-register, got %d", len(got)) + } +} + +func TestRegister_CollisionDisambiguation(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := newTestRegistry(clk) + + // Same project+branch, different worktree -> worktree disambiguates. + a, _ := r.Register(RegisterRequest{Project: "App", Branch: "dev", Worktree: "wt-a"}) + b, _ := r.Register(RegisterRequest{Project: "App", Branch: "dev", Worktree: "wt-b"}) + if a.Subdomain != "app-dev" { + t.Errorf("first subdomain = %q, want app-dev", a.Subdomain) + } + if b.Subdomain != "app-dev-wt-b" { + t.Errorf("second subdomain = %q, want app-dev-wt-b", b.Subdomain) + } + // A third with no distinguishing worktree -> numeric suffix. + c, _ := r.Register(RegisterRequest{Project: "App", Branch: "dev"}) + if c.Subdomain != "app-dev-2" { + t.Errorf("third subdomain = %q, want app-dev-2", c.Subdomain) + } +} + +func TestAvailability_StaleAfterNoHeartbeat(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := newTestRegistry(clk) + b, _ := r.Register(RegisterRequest{Project: "App", Branch: "main"}) + + if av := r.List("used")[0].Availability; av != Available { + t.Errorf("fresh backend = %q, want available", av) + } + clk.add(60 * time.Second) // past StaleFor (45s), before ExpireFor + if av := r.List("used")[0].Availability; av != Stale { + t.Errorf("after 60s = %q, want stale", av) + } + // A heartbeat brings it back. + r.Heartbeat(b.ID) + if av := r.List("used")[0].Availability; av != Available { + t.Errorf("after heartbeat = %q, want available", av) + } +} + +func TestReap_RemovesExpiredAndFreesPort(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := newTestRegistry(clk) + first, _ := r.Register(RegisterRequest{Project: "App", Branch: "main"}) + + clk.add(11 * time.Minute) // past ExpireFor + // List triggers a reap. + if got := r.List("used"); len(got) != 0 { + t.Fatalf("expired backend should be reaped, got %d", len(got)) + } + // Its port is freed and reusable. + next, _ := r.Register(RegisterRequest{Project: "Other", Branch: "main"}) + if next.ReversePort != first.ReversePort { + t.Errorf("freed port %d should be reused, got %d", first.ReversePort, next.ReversePort) + } +} + +func TestList_Sorting(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := newTestRegistry(clk) + + a, _ := r.Register(RegisterRequest{Project: "Beta", Solution: "S", Branch: "main"}) + clk.add(time.Second) + b, _ := r.Register(RegisterRequest{Project: "Alpha", Solution: "S", Branch: "main"}) + + // used: Beta (a) is touched last -> most recently used -> first. + r.TouchUsed(b.Subdomain) + clk.add(time.Second) + r.TouchUsed(a.Subdomain) + if got := r.List("used"); got[0].Project != "Beta" { + t.Errorf("used sort: first = %q, want Beta", got[0].Project) + } + // registered: newest first -> Alpha. + if got := r.List("registered"); got[0].Project != "Alpha" { + t.Errorf("registered sort: first = %q, want Alpha", got[0].Project) + } + // project: alphabetical -> Alpha. + if got := r.List("project"); got[0].Project != "Alpha" { + t.Errorf("project sort: first = %q, want Alpha", got[0].Project) + } +} + +func TestPortExhaustion(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := NewRegistry(RegistryOptions{Domain: "mxcli.org", PortBase: 9001, PortCount: 2, Now: clk.now}) + if _, err := r.Register(RegisterRequest{Project: "A", Branch: "main"}); err != nil { + t.Fatal(err) + } + if _, err := r.Register(RegisterRequest{Project: "B", Branch: "main"}); err != nil { + t.Fatal(err) + } + if _, err := r.Register(RegisterRequest{Project: "C", Branch: "main"}); err == nil { + t.Error("expected port-exhaustion error on the 3rd registration") + } +} diff --git a/cmd/mxcli/tunnelhub/slug.go b/cmd/mxcli/tunnelhub/slug.go new file mode 100644 index 000000000..b714f5c96 --- /dev/null +++ b/cmd/mxcli/tunnelhub/slug.go @@ -0,0 +1,57 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Package tunnelhub implements the multi-tenant mxcli tunnel-hub: a static +// ingress relay that fronts many locally-running Mendix apps (across projects, +// solutions, branches, and worktrees) at per-preview subdomains over a single +// 443 connection, with a registration API and an admin overview. +package tunnelhub + +import ( + "regexp" + "strings" +) + +// maxLabelLen is the DNS label limit; a subdomain slug must fit in one label. +const maxLabelLen = 63 + +var nonLabel = regexp.MustCompile(`[^a-z0-9-]+`) + +// slugify reduces s to a DNS-label-safe fragment: lowercase, only [a-z0-9-], +// runs collapsed, no leading/trailing dashes. +func slugify(s string) string { + s = strings.ToLower(strings.TrimSpace(s)) + s = strings.NewReplacer("/", "-", "_", "-", ".", "-", " ", "-").Replace(s) + s = nonLabel.ReplaceAllString(s, "-") + for strings.Contains(s, "--") { + s = strings.ReplaceAll(s, "--", "-") + } + return strings.Trim(s, "-") +} + +// baseSlug builds the preferred subdomain slug from a registration. The main / +// master branch collapses to just the project (so a project's primary preview is +// project.mxcli.org); other branches append the branch. Solution is a grouping +// dimension in the overview, not part of the hostname. +func baseSlug(project, branch string) string { + p := slugify(project) + b := slugify(branch) + switch { + case p == "" && b == "": + return "app" + case p == "": + return truncateLabel(b) + case b == "" || b == "main" || b == "master": + return truncateLabel(p) + default: + return truncateLabel(p + "-" + b) + } +} + +// truncateLabel bounds a slug to a single DNS label, trimming any dash the cut +// leaves at the end. +func truncateLabel(s string) string { + if len(s) <= maxLabelLen { + return s + } + return strings.TrimRight(s[:maxLabelLen], "-") +} From 1fa2680cfd231760c6719078025c90c120c070ea Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 23 Jul 2026 07:17:29 +0000 Subject: [PATCH 05/12] feat(tunnel-hub): single-443 front (host routing + per-subdomain TLS) + admin page MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The serving plane for the multi-tenant hub, plus an optional hostname prefix. - server.go: one TLS front on 443 that routes by Host — the hub host serves the chisel control connection (WebSocket upgrade), the registration API, and the admin page; each . reverse-proxies to that preview's chisel reverse port with the public Host preserved. Certificates are issued per host via autocert, gated by a HostPolicy that only allows the hub host or a currently registered subdomain (so random subdomains can't drive issuance). A :80 listener serves ACME HTTP-01 + redirects to https. Friendly pages for unknown / offline previews. One embedded chisel reverse server fans in all client tunnels. - admin.go: a self-contained (no external assets) overview page that polls /api/backends and renders a sortable table — status (available/stale dot), solution, project, branch, clickable URL, registered / last-seen / last-used, uptime — sortable by any column, auto-refreshing. - Optional prefix: [prefix-]project[-branch] namespaces the hostname (organization / solution / team / env), passed through the registry + API and part of a backend's stable identity. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- cmd/mxcli/tunnelhub/admin.go | 128 +++++++++++++ cmd/mxcli/tunnelhub/registry.go | 15 +- cmd/mxcli/tunnelhub/registry_test.go | 21 ++ cmd/mxcli/tunnelhub/server.go | 276 +++++++++++++++++++++++++++ cmd/mxcli/tunnelhub/server_test.go | 111 +++++++++++ cmd/mxcli/tunnelhub/slug.go | 34 ++-- 6 files changed, 564 insertions(+), 21 deletions(-) create mode 100644 cmd/mxcli/tunnelhub/admin.go create mode 100644 cmd/mxcli/tunnelhub/server.go create mode 100644 cmd/mxcli/tunnelhub/server_test.go diff --git a/cmd/mxcli/tunnelhub/admin.go b/cmd/mxcli/tunnelhub/admin.go new file mode 100644 index 000000000..73f14980c --- /dev/null +++ b/cmd/mxcli/tunnelhub/admin.go @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import "net/http" + +// NewAdmin returns the handler for the hub's admin overview page. The page is +// self-contained (inline CSS/JS, no external assets) and refreshes itself from +// GET /api/backends. +func NewAdmin(_ *Registry) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/" { + http.NotFound(w, r) + return + } + w.Header().Set("Content-Type", "text/html; charset=utf-8") + _, _ = w.Write([]byte(adminHTML)) + }) +} + +const adminHTML = ` + + +mxcli tunnel-hub — previews + + +
+

mxcli tunnel-hub

+ … + +
+
+ + + + + + + + + + + + + +
StatusSolutionProjectBranchURLRegisteredLast seenLast usedUptime
+ +
+ +` diff --git a/cmd/mxcli/tunnelhub/registry.go b/cmd/mxcli/tunnelhub/registry.go index 966de6a50..d4aa54688 100644 --- a/cmd/mxcli/tunnelhub/registry.go +++ b/cmd/mxcli/tunnelhub/registry.go @@ -27,6 +27,7 @@ const ( // the hub and served at its subdomain. type Backend struct { ID string `json:"id"` // opaque token (auth for heartbeat/deregister + chisel) + Prefix string `json:"prefix"` // optional hostname namespace (org/solution/team/env) Project string `json:"project"` // e.g. the .mpr name Solution string `json:"solution"` // optional grouping for multi-app solutions Branch string `json:"branch"` // git branch @@ -40,10 +41,10 @@ type Backend struct { LastUsedAt time.Time `json:"lastUsedAt"` // last browser request to the subdomain } -// identity is the stable key for a preview across reconnects: same project + -// branch + worktree + solution re-registers to the same slot (stable URL). +// identity is the stable key for a preview across reconnects: same prefix + +// project + branch + worktree + solution re-registers to the same slot (stable URL). func (b *Backend) identity() string { - return strings.Join([]string{b.Solution, b.Project, b.Branch, b.Worktree}, "\x00") + return strings.Join([]string{b.Prefix, b.Solution, b.Project, b.Branch, b.Worktree}, "\x00") } // BackendView is a Backend plus derived fields, for the API/admin page. @@ -56,6 +57,7 @@ type BackendView struct { // RegisterRequest is the registration payload from `mxcli run --hub`. type RegisterRequest struct { + Prefix string `json:"prefix"` Project string `json:"project"` Solution string `json:"solution"` Branch string `json:"branch"` @@ -132,6 +134,7 @@ func (r *Registry) Register(req RegisterRequest) (*Backend, error) { now := r.now() b := &Backend{ + Prefix: strings.TrimSpace(req.Prefix), Project: strings.TrimSpace(req.Project), Solution: strings.TrimSpace(req.Solution), Branch: strings.TrimSpace(req.Branch), @@ -149,7 +152,7 @@ func (r *Registry) Register(req RegisterRequest) (*Backend, error) { return nil, err } b.ID = newToken() - b.Subdomain = r.allocSubdomainLocked(b.Project, b.Branch, b.Worktree) + b.Subdomain = r.allocSubdomainLocked(b.Prefix, b.Project, b.Branch, b.Worktree) b.ReversePort = port b.RegisteredAt = now b.LastSeenAt = now @@ -269,8 +272,8 @@ func (r *Registry) allocPortLocked() (int, error) { // allocSubdomainLocked returns a unique subdomain slug, disambiguating a // collision with the worktree name then a numeric suffix. -func (r *Registry) allocSubdomainLocked(project, branch, worktree string) string { - base := baseSlug(project, branch) +func (r *Registry) allocSubdomainLocked(prefix, project, branch, worktree string) string { + base := baseSlug(prefix, project, branch) if _, taken := r.bySubdomain[base]; !taken { return base } diff --git a/cmd/mxcli/tunnelhub/registry_test.go b/cmd/mxcli/tunnelhub/registry_test.go index 064f1c748..38e0d5eb1 100644 --- a/cmd/mxcli/tunnelhub/registry_test.go +++ b/cmd/mxcli/tunnelhub/registry_test.go @@ -48,6 +48,27 @@ func TestRegister_AllocatesSubdomainAndPort(t *testing.T) { } } +func TestRegister_Prefix(t *testing.T) { + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + r := newTestRegistry(clk) + + // prefix namespaces the hostname: -[-]. + a, _ := r.Register(RegisterRequest{Prefix: "AcmeCorp", Project: "Portal", Branch: "feature/x"}) + if a.Subdomain != "acmecorp-portal-feature-x" { + t.Errorf("prefixed subdomain = %q, want acmecorp-portal-feature-x", a.Subdomain) + } + // main branch with a prefix -> -. + b, _ := r.Register(RegisterRequest{Prefix: "AcmeCorp", Project: "Portal", Branch: "main"}) + if b.Subdomain != "acmecorp-portal" { + t.Errorf("prefixed main subdomain = %q, want acmecorp-portal", b.Subdomain) + } + // same project, different prefix -> distinct slot (prefix is part of identity). + c, _ := r.Register(RegisterRequest{Prefix: "Other", Project: "Portal", Branch: "main"}) + if c.Subdomain != "other-portal" || c.ID == b.ID { + t.Errorf("different prefix should be a distinct backend: %q id=%s vs %s", c.Subdomain, c.ID, b.ID) + } +} + func TestRegister_SameIdentityIsStable(t *testing.T) { clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} r := newTestRegistry(clk) diff --git a/cmd/mxcli/tunnelhub/server.go b/cmd/mxcli/tunnelhub/server.go new file mode 100644 index 000000000..3eb6161ef --- /dev/null +++ b/cmd/mxcli/tunnelhub/server.go @@ -0,0 +1,276 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import ( + "context" + "fmt" + "html" + "net/http" + "net/http/httputil" + "net/url" + "strings" + "time" + + chserver "github.com/jpillora/chisel/server" + "golang.org/x/crypto/acme/autocert" +) + +// ctxKey is the type for request-context values the front handler passes to the +// app reverse proxy. +type ctxKey int + +const ( + targetPortKey ctxKey = iota + publicHostKey +) + +// ServerOptions configures the multi-tenant hub front. +type ServerOptions struct { + // Domain is the wildcard base, e.g. "mxcli.org". App previews are served at + // .. + Domain string + // HubHost is the control/admin/API host (default "hub."+Domain). Clients dial + // their chisel control connection here and the admin page lives here. + HubHost string + // Registry is the shared backend store. + Registry *Registry + // TunnelAuth is the shared chisel auth ("user:pass"); empty disables auth. + TunnelAuth string + // RegisterSecret optionally gates /api/register (matched against X-Hub-Secret). + RegisterSecret string + // CertCacheDir is the autocert certificate cache directory. + CertCacheDir string + // chiselAddr is the internal address the embedded chisel control server binds + // (default 127.0.0.1:8100). Not public — the front proxies the WS here. + ChiselAddr string +} + +// Server is the running multi-tenant hub: one embedded chisel reverse server +// (fanning in all client tunnels) behind a single-443 TLS front that routes by +// Host — the hub host to the admin/API/chisel-control, each preview subdomain to +// its tunnel. +type Server struct { + opts ServerOptions + reg *Registry + chisel *chserver.Server + manager *autocert.Manager + http *http.Server + apiMux *http.ServeMux + admin http.Handler + + chiselProxy *httputil.ReverseProxy // -> internal chisel control (WS) + appProxy *httputil.ReverseProxy // -> 127.0.0.1: (per request) +} + +// NewServer wires the registry, API, admin page, embedded chisel server, and the +// TLS front. Call Start to listen. +func NewServer(o ServerOptions) (*Server, error) { + if o.Domain == "" { + return nil, fmt.Errorf("Domain is required") + } + if o.HubHost == "" { + o.HubHost = "hub." + o.Domain + } + if o.ChiselAddr == "" { + o.ChiselAddr = "127.0.0.1:8100" + } + if o.Registry == nil { + return nil, fmt.Errorf("Registry is required") + } + + chisel, err := chserver.NewServer(&chserver.Config{Reverse: true, Auth: o.TunnelAuth}) + if err != nil { + return nil, fmt.Errorf("chisel server: %w", err) + } + + api := NewAPI(APIOptions{ + Registry: o.Registry, + ControlURL: "https://" + o.HubHost, + TunnelAuth: o.TunnelAuth, + RegisterSecret: o.RegisterSecret, + }) + apiMux := http.NewServeMux() + api.Mount(apiMux) + + s := &Server{ + opts: o, + reg: o.Registry, + chisel: chisel, + apiMux: apiMux, + admin: NewAdmin(o.Registry), + } + + // Front proxies: chisel control (WS) to the internal chisel server, and app + // traffic to the per-request reverse port (Host preserved as the public host). + chiselURL := &url.URL{Scheme: "http", Host: o.ChiselAddr} + s.chiselProxy = httputil.NewSingleHostReverseProxy(chiselURL) + + s.appProxy = &httputil.ReverseProxy{ + Director: func(req *http.Request) { + port, _ := req.Context().Value(targetPortKey).(int) + host, _ := req.Context().Value(publicHostKey).(string) + req.URL.Scheme = "http" + req.URL.Host = fmt.Sprintf("127.0.0.1:%d", port) + if host != "" { + req.Host = host // the app sees its real public host, not the loopback + } + }, + ErrorHandler: func(w http.ResponseWriter, _ *http.Request, _ error) { + // The subdomain is registered but nothing answers on its reverse port — + // the client's tunnel is down (e.g. the container was reaped). + writeOfflinePage(w) + }, + } + + // autocert: issue a cert per host, but only for the hub host or a currently + // registered subdomain — so a request for a random subdomain can't drive cert + // issuance. + s.manager = &autocert.Manager{ + Prompt: autocert.AcceptTOS, + Cache: autocert.DirCache(o.CertCacheDir), + HostPolicy: s.hostPolicy, + } + return s, nil +} + +// hostPolicy allows the hub host and any registered preview subdomain. +func (s *Server) hostPolicy(_ context.Context, host string) error { + if host == s.opts.HubHost { + return nil + } + if sub, ok := s.subOf(host); ok { + if _, found := s.reg.LookupSubdomain(sub); found { + return nil + } + } + return fmt.Errorf("host %q is not the hub or a registered preview", host) +} + +// subOf returns the subdomain label of host under Domain (e.g. "app" from +// "app.mxcli.org"), and whether host is under Domain. +func (s *Server) subOf(host string) (string, bool) { + suffix := "." + s.opts.Domain + if !strings.HasSuffix(host, suffix) { + return "", false + } + sub := strings.TrimSuffix(host, suffix) + if sub == "" || strings.Contains(sub, ".") { + return "", false // only single-label subdomains + } + return sub, true +} + +// ServeHTTP routes by Host: the hub host serves chisel control (WS upgrade), +// the API, and the admin page; a preview subdomain proxies to its tunnel. +func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { + host := stripPort(r.Host) + switch { + case host == s.opts.HubHost: + if isWebSocketUpgrade(r) { + s.chiselProxy.ServeHTTP(w, r) // chisel client control connection + return + } + if strings.HasPrefix(r.URL.Path, "/api/") { + s.apiMux.ServeHTTP(w, r) + return + } + s.admin.ServeHTTP(w, r) + default: + sub, ok := s.subOf(host) + if !ok { + http.Error(w, "unknown host", http.StatusMisdirectedRequest) + return + } + b, found := s.reg.LookupSubdomain(sub) + if !found { + writeNoSuchPreview(w, host) + return + } + s.reg.TouchUsed(sub) + ctx := context.WithValue(r.Context(), targetPortKey, b.ReversePort) + ctx = context.WithValue(ctx, publicHostKey, host) + s.appProxy.ServeHTTP(w, r.WithContext(ctx)) + } +} + +// Start binds the internal chisel server and the public TLS front (443), plus an +// HTTP :80 listener for ACME challenges and http->https redirects. It blocks +// until ctx is cancelled. +func (s *Server) Start(ctx context.Context, httpsAddr, httpAddr string) error { + if err := s.chisel.Start("127.0.0.1", portOf(s.opts.ChiselAddr)); err != nil { + return fmt.Errorf("starting chisel: %w", err) + } + + s.http = &http.Server{ + Addr: httpsAddr, + Handler: s, + TLSConfig: s.manager.TLSConfig(), + // WebSocket/long-poll: no write timeout; bound only the header read. + ReadHeaderTimeout: 20 * time.Second, + } + // :80 serves ACME HTTP-01 + redirects everything else to https. + httpSrv := &http.Server{Addr: httpAddr, Handler: s.manager.HTTPHandler(redirectToHTTPS()), ReadHeaderTimeout: 10 * time.Second} + go func() { _ = httpSrv.ListenAndServe() }() + + errc := make(chan error, 1) + go func() { errc <- s.http.ListenAndServeTLS("", "") }() + + select { + case <-ctx.Done(): + shutCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + _ = s.http.Shutdown(shutCtx) + _ = httpSrv.Shutdown(shutCtx) + _ = s.chisel.Close() + return nil + case err := <-errc: + _ = httpSrv.Close() + _ = s.chisel.Close() + return err + } +} + +func redirectToHTTPS() http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, "https://"+stripPort(r.Host)+r.URL.RequestURI(), http.StatusMovedPermanently) + }) +} + +func isWebSocketUpgrade(r *http.Request) bool { + return strings.EqualFold(r.Header.Get("Upgrade"), "websocket") && + strings.Contains(strings.ToLower(r.Header.Get("Connection")), "upgrade") +} + +func stripPort(hostport string) string { + if i := strings.LastIndexByte(hostport, ':'); i >= 0 && !strings.Contains(hostport[i:], "]") { + return hostport[:i] + } + return hostport +} + +func portOf(addr string) string { + if i := strings.LastIndexByte(addr, ':'); i >= 0 { + return addr[i+1:] + } + return addr +} + +func writeNoSuchPreview(w http.ResponseWriter, host string) { + w.Header().Set("Content-Type", "text/html; charset=utf-8") + w.WriteHeader(http.StatusNotFound) + fmt.Fprintf(w, `No such preview`+ + ``+ + `

No preview here

Nothing is registered for %s. `+ + `It may have ended, or the name is wrong.

`, html.EscapeString(host)) +} + +func writeOfflinePage(w http.ResponseWriter) { + w.Header().Set("Content-Type", "text/html; charset=utf-8") + w.WriteHeader(http.StatusBadGateway) + fmt.Fprint(w, `Preview offline`+ + ``+ + `

Preview offline

This preview is registered but its tunnel isn't `+ + `connected right now — the dev container may be asleep or reaped. It will `+ + `return when the app is running again.

`) +} diff --git a/cmd/mxcli/tunnelhub/server_test.go b/cmd/mxcli/tunnelhub/server_test.go new file mode 100644 index 000000000..a30c9a3f0 --- /dev/null +++ b/cmd/mxcli/tunnelhub/server_test.go @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import ( + "net/http" + "net/http/httptest" + "strings" + "testing" + "time" +) + +func newTestServer(t *testing.T) (*Server, *Registry) { + t.Helper() + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + reg := newTestRegistry(clk) + srv, err := NewServer(ServerOptions{ + Domain: "mxcli.org", + Registry: reg, + CertCacheDir: t.TempDir(), + }) + if err != nil { + t.Fatalf("NewServer: %v", err) + } + return srv, reg +} + +func req(host, target string) *http.Request { + r := httptest.NewRequest(http.MethodGet, "http://"+host+target, nil) + r.Host = host + return r +} + +func TestFront_HubHostServesAdmin(t *testing.T) { + srv, _ := newTestServer(t) + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req("hub.mxcli.org", "/")) + if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), "mxcli tunnel-hub") { + t.Errorf("admin page: code=%d, body has hub title=%v", rec.Code, strings.Contains(rec.Body.String(), "mxcli tunnel-hub")) + } +} + +func TestFront_HubHostAPI(t *testing.T) { + srv, reg := newTestServer(t) + reg.Register(RegisterRequest{Project: "App", Branch: "main"}) + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req("hub.mxcli.org", "/api/backends")) + if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), `"project":"App"`) { + t.Errorf("api/backends: code=%d body=%s", rec.Code, rec.Body) + } +} + +func TestFront_UnknownSubdomain(t *testing.T) { + srv, _ := newTestServer(t) + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req("ghost.mxcli.org", "/")) + if rec.Code != http.StatusNotFound || !strings.Contains(rec.Body.String(), "No preview") { + t.Errorf("unknown subdomain: code=%d body=%s", rec.Code, rec.Body) + } +} + +func TestFront_RegisteredButOffline(t *testing.T) { + srv, reg := newTestServer(t) + b, _ := reg.Register(RegisterRequest{Project: "App", Branch: "main"}) // subdomain "app" + + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req("app.mxcli.org", "/")) + // Registered but its reverse port has no tunnel -> the proxy error handler + // returns the offline page. + if rec.Code != http.StatusBadGateway || !strings.Contains(rec.Body.String(), "offline") { + t.Errorf("offline preview: code=%d body=%s", rec.Code, rec.Body) + } + // The request still counted as usage. + if v, ok := reg.LookupSubdomain(b.Subdomain); !ok || v.LastUsedAt.IsZero() { + t.Error("a request to a preview should update LastUsedAt") + } +} + +func TestSubOf(t *testing.T) { + srv, _ := newTestServer(t) + cases := []struct { + host string + sub string + ok bool + }{ + {"app.mxcli.org", "app", true}, + {"my-app-feat.mxcli.org", "my-app-feat", true}, + {"hub.mxcli.org", "hub", true}, // subOf is purely structural; routing handles hub separately + {"a.b.mxcli.org", "", false}, // multi-label not allowed + {"mxcli.org", "", false}, + {"evil.com", "", false}, + } + for _, c := range cases { + sub, ok := srv.subOf(c.host) + if sub != c.sub || ok != c.ok { + t.Errorf("subOf(%q) = (%q,%v), want (%q,%v)", c.host, sub, ok, c.sub, c.ok) + } + } +} + +func TestStripPort(t *testing.T) { + for in, want := range map[string]string{ + "app.mxcli.org": "app.mxcli.org", + "app.mxcli.org:443": "app.mxcli.org", + "hub.mxcli.org:8443": "hub.mxcli.org", + } { + if got := stripPort(in); got != want { + t.Errorf("stripPort(%q) = %q, want %q", in, got, want) + } + } +} diff --git a/cmd/mxcli/tunnelhub/slug.go b/cmd/mxcli/tunnelhub/slug.go index b714f5c96..4e4982e2e 100644 --- a/cmd/mxcli/tunnelhub/slug.go +++ b/cmd/mxcli/tunnelhub/slug.go @@ -28,23 +28,27 @@ func slugify(s string) string { return strings.Trim(s, "-") } -// baseSlug builds the preferred subdomain slug from a registration. The main / -// master branch collapses to just the project (so a project's primary preview is -// project.mxcli.org); other branches append the branch. Solution is a grouping -// dimension in the overview, not part of the hostname. -func baseSlug(project, branch string) string { - p := slugify(project) - b := slugify(branch) - switch { - case p == "" && b == "": +// baseSlug builds the preferred subdomain slug from a registration: +// [prefix-]project[-branch]. The optional prefix namespaces the hostname +// (organization, solution, team, env — whatever the client passes). The main / +// master branch is dropped, so a project's primary preview is the clean +// [prefix-]project; other branches append the branch. Solution is a grouping +// dimension in the overview, not part of the hostname (unless passed as prefix). +func baseSlug(prefix, project, branch string) string { + var parts []string + if pre := slugify(prefix); pre != "" { + parts = append(parts, pre) + } + if p := slugify(project); p != "" { + parts = append(parts, p) + } + if b := slugify(branch); b != "" && b != "main" && b != "master" { + parts = append(parts, b) + } + if len(parts) == 0 { return "app" - case p == "": - return truncateLabel(b) - case b == "" || b == "main" || b == "master": - return truncateLabel(p) - default: - return truncateLabel(p + "-" + b) } + return truncateLabel(strings.Join(parts, "-")) } // truncateLabel bounds a slug to a single DNS label, trimming any dash the cut From bd1c9e73fbecd1a05e01667e7ed4ccd4cd0c766b Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 23 Jul 2026 07:23:57 +0000 Subject: [PATCH 06/12] feat(run,tunnel-hub): multi-tenant client registration + wire the hub command MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes slice 4 end to end. `mxcli run --hub` now registers with the hub before booting: it auto-detects project (.mpr name) and branch (git), takes --hub-prefix/--hub-project/--hub-solution/--hub-branch/--hub-worktree overrides, POSTs /api/register, and uses the assigned subdomain URL as ApplicationRootUrl, tunnels to the assigned reverse port, and heartbeats (deregistering on exit). It falls back to the slice-1 single-app behaviour when the hub has no registration API, so both hub types keep working. `mxcli tunnel-hub` now runs the multi-tenant server (registry + registration API + admin overview + single-443 host-routing front with per-subdomain autocert), keyed off --domain (the wildcard base) with the control/admin at hub.. A background reaper frees ports for expired backends. An in-process integration test drives the whole path — register a preview, tunnel a backend to its reverse port with a real embedded chisel client, and assert a request to . routes through the tunnel with the public Host preserved. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- cmd/mxcli/cmd_run.go | 15 ++ cmd/mxcli/cmd_tunnelhub.go | 135 +++++++--------- cmd/mxcli/docker/hubclient.go | 195 ++++++++++++++++++++++++ cmd/mxcli/docker/runlocal.go | 56 +++++-- cmd/mxcli/docker/tunnel.go | 15 +- cmd/mxcli/tunnelhub/integration_test.go | 111 ++++++++++++++ cmd/mxcli/tunnelhub/server.go | 15 ++ 7 files changed, 446 insertions(+), 96 deletions(-) create mode 100644 cmd/mxcli/docker/hubclient.go create mode 100644 cmd/mxcli/tunnelhub/integration_test.go diff --git a/cmd/mxcli/cmd_run.go b/cmd/mxcli/cmd_run.go index cf8ed28ca..52d18dafa 100644 --- a/cmd/mxcli/cmd_run.go +++ b/cmd/mxcli/cmd_run.go @@ -48,6 +48,11 @@ Examples: local, _ := cmd.Flags().GetBool("local") hub, _ := cmd.Flags().GetString("hub") hubSecret, _ := cmd.Flags().GetString("hub-secret") + hubPrefix, _ := cmd.Flags().GetString("hub-prefix") + hubProject, _ := cmd.Flags().GetString("hub-project") + hubSolution, _ := cmd.Flags().GetString("hub-solution") + hubBranch, _ := cmd.Flags().GetString("hub-branch") + hubWorktree, _ := cmd.Flags().GetString("hub-worktree") // --hub is a cross-cutting ingress and implies the local serving path (the // only serving mode wired today; a future PAD path will accept --hub too). if hub != "" { @@ -83,6 +88,11 @@ Examples: ProjectPath: projectPath, Hub: hub, HubSecret: hubSecret, + HubPrefix: hubPrefix, + HubProject: hubProject, + HubSolution: hubSolution, + HubBranch: hubBranch, + HubWorktree: hubWorktree, AppPort: appPort, AdminPort: adminPort, ServePort: servePort, @@ -115,6 +125,11 @@ func init() { runCmd.Flags().Bool("local", false, "Run locally without Docker (warm serve + standalone runtime)") runCmd.Flags().String("hub", "", "Expose the running app in a browser via an mxcli tunnel-hub URL (e.g. https://hub.mxcli.org). Implies --local; the app stays local and is reverse-tunnelled out") runCmd.Flags().String("hub-secret", "", "Shared auth secret for --hub (\"user:pass\"), matching the hub's --secret") + runCmd.Flags().String("hub-prefix", "", "Optional subdomain prefix on the hub (org/solution/team/env): --") + runCmd.Flags().String("hub-project", "", "Project name for the hub subdomain + overview (default: the .mpr name)") + runCmd.Flags().String("hub-solution", "", "Solution name to group this app under in the hub overview (multi-app solutions)") + runCmd.Flags().String("hub-branch", "", "Branch for the hub subdomain + overview (default: the git branch)") + runCmd.Flags().String("hub-worktree", "", "Worktree label to distinguish multiple worktrees of one branch") runCmd.Flags().Bool("watch", false, "Rebuild and hot-apply on every project change") runCmd.Flags().Bool("ensure-db", false, "Provision the local Postgres + app database if missing (fresh-session bootstrap)") runCmd.Flags().Bool("setup", false, "Prepare prerequisites (cache MxBuild+runtime, ensure DB) and exit without booting — for a SessionStart hook") diff --git a/cmd/mxcli/cmd_tunnelhub.go b/cmd/mxcli/cmd_tunnelhub.go index ed67647c4..c82782d90 100644 --- a/cmd/mxcli/cmd_tunnelhub.go +++ b/cmd/mxcli/cmd_tunnelhub.go @@ -7,119 +7,94 @@ import ( "fmt" "os" "os/signal" - "strconv" + "path/filepath" "syscall" - chserver "github.com/jpillora/chisel/server" - "github.com/mendixlabs/mxcli/cmd/mxcli/docker" + "github.com/mendixlabs/mxcli/cmd/mxcli/tunnelhub" "github.com/spf13/cobra" ) -// tunnelHubCmd is the static ingress relay. It runs on a host with a public IP -// and domain (e.g. a small VPS) and fronts a locally-running mxcli app: the app -// stays in its own (possibly egress-only) environment and reverse-tunnels out to -// this hub over 443; browsers reach it at the hub's URL. Nothing is pushed here — -// only live HTTP flows through the tunnel. -// -// Slice 1 fronts a single app. Multi-tenant registration (a /register API that -// allocates per-container subdomains + tokens) and the admin overview are a -// follow-on (see PROPOSAL_mxcli_dev_warm_loop.md § Scaling). +// tunnelHubCmd runs the multi-tenant ingress relay. It fronts many locally-running +// Mendix apps — across projects, solutions, branches, and worktrees — each at its +// own . over a single 443 connection, with a registration API +// and an admin overview at the hub host. Apps stay in their own (possibly +// egress-only) environments and reverse-tunnel out; nothing is pushed here. var tunnelHubCmd = &cobra.Command{ Use: "tunnel-hub", - Short: "Static ingress relay that fronts a locally-running mxcli app at a public URL", - Long: `Run a static ingress relay that exposes a locally-running Mendix app -(started elsewhere with 'mxcli run --hub ') in a browser at this host's -public URL. + Short: "Multi-tenant ingress relay: front many locally-running mxcli apps at per-preview subdomains", + Long: `Run the static ingress relay that exposes locally-running Mendix apps +(started elsewhere with 'mxcli run --hub ') in a browser. -It embeds a chisel reverse-tunnel server: the app's environment dials in over 443 -and reverse-tunnels its local port here; every non-tunnel HTTP request to this -host is proxied down that tunnel to the app. Everything rides a single 443 -connection, so it works from egress-only environments (e.g. Claude Code on the -web). +Each app self-registers and is served at its own subdomain +(-., or -- with --hub-prefix); +the hub host (hub.) serves the registration API, the admin overview, and +the chisel control connection. Everything rides one 443 connection, so apps in +egress-only environments (e.g. Claude Code on the web) can reverse-tunnel out. -TLS: pass --domain for automatic Let's Encrypt (the host must be reachable on 80 -and 443 for the ACME challenge), or --tls-cert/--tls-key for an existing -certificate. +DNS: point a wildcard '*.' A record (and 'hub.') at this host. +TLS is issued per subdomain via Let's Encrypt on demand (needs inbound 80+443). -Example (on a VPS with hub.mxcli.org -> this host, ports 80+443 open): - mxcli tunnel-hub --domain hub.mxcli.org --secret myuser:mypass +Example (on a VPS; *.mxcli.org -> this host, ports 80+443 open): + mxcli tunnel-hub --domain mxcli.org --secret alice:s3cret -Then, in the app's environment: - mxcli run --hub https://hub.mxcli.org --hub-secret myuser:mypass -p app.mpr +Then, in each app's environment: + mxcli run --hub https://hub.mxcli.org --hub-secret alice:s3cret \ + --hub-solution CustomerPortal -p app.mpr `, Run: func(cmd *cobra.Command, args []string) { domain, _ := cmd.Flags().GetString("domain") + hubHost, _ := cmd.Flags().GetString("hub-host") secret, _ := cmd.Flags().GetString("secret") - port, _ := cmd.Flags().GetInt("port") - backendPort, _ := cmd.Flags().GetInt("backend-port") - tlsKey, _ := cmd.Flags().GetString("tls-key") - tlsCert, _ := cmd.Flags().GetString("tls-cert") - host, _ := cmd.Flags().GetString("host") + httpsPort, _ := cmd.Flags().GetInt("port") + httpPort, _ := cmd.Flags().GetInt("http-port") + certCache, _ := cmd.Flags().GetString("cert-cache") - hasCert := tlsKey != "" && tlsCert != "" - if domain == "" && !hasCert { - fmt.Fprintln(os.Stderr, "Error: --domain (automatic Let's Encrypt) or both --tls-cert and --tls-key are required") + if domain == "" { + fmt.Fprintln(os.Stderr, "Error: --domain is required (the wildcard base, e.g. mxcli.org)") os.Exit(1) } - - cfg := &chserver.Config{ - Reverse: true, - Auth: secret, - // Proxy is chisel's HTTP backend: non-tunnel requests are reverse-proxied - // here, which is the reverse-tunnel listener the app dials into. - Proxy: fmt.Sprintf("http://127.0.0.1:%d", backendPort), - TLS: chserver.TLSConfig{ - Key: tlsKey, - Cert: tlsCert, - }, - } - if domain != "" { - cfg.TLS.Domains = []string{domain} + if certCache == "" { + home, _ := os.UserHomeDir() + certCache = filepath.Join(home, ".mxcli", "hub-certs") } - srv, err := chserver.NewServer(cfg) + reg := tunnelhub.NewRegistry(tunnelhub.RegistryOptions{Domain: domain}) + srv, err := tunnelhub.NewServer(tunnelhub.ServerOptions{ + Domain: domain, + HubHost: hubHost, + Registry: reg, + TunnelAuth: secret, + RegisterSecret: secret, + CertCacheDir: certCache, + }) if err != nil { fmt.Fprintf(os.Stderr, "Error: configuring tunnel-hub: %v\n", err) os.Exit(1) } + host := hubHost + if host == "" { + host = "hub." + domain + } + fmt.Printf("mxcli tunnel-hub: serving *.%s (control/admin at https://%s) on :%d\n", domain, host, httpsPort) + fmt.Printf(" admin overview: https://%s/\n", host) + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) defer stop() - if domain != "" { - fmt.Printf("mxcli tunnel-hub: fronting https://%s on %s:%d (reverse port %d)\n", domain, host, port, backendPort) - fmt.Printf(" app connects with: mxcli run --hub https://%s%s -p app.mpr\n", domain, secretHint(secret)) - } else { - fmt.Printf("mxcli tunnel-hub: listening on %s:%d (reverse port %d)\n", host, port, backendPort) - } - - if err := srv.StartContext(ctx, host, strconv.Itoa(port)); err != nil { - fmt.Fprintf(os.Stderr, "Error: starting tunnel-hub: %v\n", err) - os.Exit(1) - } - if err := srv.Wait(); err != nil { - fmt.Fprintf(os.Stderr, "Error: tunnel-hub stopped: %v\n", err) + if err := srv.Start(ctx, fmt.Sprintf(":%d", httpsPort), fmt.Sprintf(":%d", httpPort)); err != nil { + fmt.Fprintf(os.Stderr, "Error: tunnel-hub: %v\n", err) os.Exit(1) } }, } -// secretHint renders " --hub-secret " for the copy-paste hint, or "" when -// no secret is set. -func secretHint(secret string) string { - if secret == "" { - return "" - } - return " --hub-secret " + secret -} - func init() { - tunnelHubCmd.Flags().String("domain", "", "Domain for automatic Let's Encrypt TLS (e.g. hub.mxcli.org); host must be reachable on 80+443") - tunnelHubCmd.Flags().String("secret", "", "Shared auth secret (\"user:pass\") the app must present via --hub-secret") - tunnelHubCmd.Flags().Int("port", 443, "Public port to listen on") - tunnelHubCmd.Flags().Int("backend-port", docker.DefaultHubBackendPort, "Reverse-tunnel port the app dials into (must match the app side; default 9000)") - tunnelHubCmd.Flags().String("tls-cert", "", "TLS certificate file (instead of --domain autocert)") - tunnelHubCmd.Flags().String("tls-key", "", "TLS key file (instead of --domain autocert)") - tunnelHubCmd.Flags().String("host", "0.0.0.0", "Address to bind") + tunnelHubCmd.Flags().String("domain", "", "Wildcard base domain, e.g. mxcli.org (previews served at .)") + tunnelHubCmd.Flags().String("hub-host", "", "Control/admin host (default hub.)") + tunnelHubCmd.Flags().String("secret", "", "Shared secret (\"user:pass\") apps present via --hub-secret; empty = open") + tunnelHubCmd.Flags().Int("port", 443, "HTTPS port to listen on") + tunnelHubCmd.Flags().Int("http-port", 80, "HTTP port for ACME challenges + http->https redirect") + tunnelHubCmd.Flags().String("cert-cache", "", "Directory for Let's Encrypt certificates (default ~/.mxcli/hub-certs)") rootCmd.AddCommand(tunnelHubCmd) } diff --git a/cmd/mxcli/docker/hubclient.go b/cmd/mxcli/docker/hubclient.go new file mode 100644 index 000000000..c6483329a --- /dev/null +++ b/cmd/mxcli/docker/hubclient.go @@ -0,0 +1,195 @@ +// SPDX-License-Identifier: Apache-2.0 + +package docker + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "os/exec" + "path/filepath" + "strings" + "time" +) + +// HubMeta identifies a preview to the hub: it drives the subdomain and the +// overview grouping. Blank fields are auto-detected where possible. +type HubMeta struct { + Prefix string // optional hostname namespace (org/solution/team/env) + Project string // default: the .mpr basename + Solution string // optional grouping for multi-app solutions + Branch string // default: the project's git branch + Worktree string // optional; distinguishes worktrees of one branch +} + +// HubRegistration is the result of registering with a hub: everything the client +// needs to tunnel and to advertise the app under its public URL. +type HubRegistration struct { + URL string // assigned public URL -> ApplicationRootUrl + Subdomain string // "" in single-app fallback + ControlURL string // where the chisel client connects + ReversePort int // reverse port to tunnel to + Token string // heartbeat/deregister auth ("" in fallback) + TunnelAuth string // chisel auth to use + HeartbeatInterval time.Duration // 0 in fallback + MultiTenant bool // false when we fell back to a slice-1 single-app hub + + hubURL string +} + +type registerResponse struct { + Subdomain string `json:"subdomain"` + URL string `json:"url"` + ReversePort int `json:"reversePort"` + ControlURL string `json:"controlUrl"` + Token string `json:"token"` + TunnelAuth string `json:"tunnelAuth"` + HeartbeatIntervalSec int `json:"heartbeatIntervalSec"` +} + +// RegisterWithHub registers a preview with a multi-tenant hub. If the hub has no +// registration API (a slice-1 single-app hub), it falls back to serving directly +// at the hub URL on the default reverse port. The HTTP client uses the standard +// proxy environment (honouring NO_PROXY), so an external hub goes through the +// egress proxy. +func RegisterWithHub(hubURL, secret string, meta HubMeta, appPort int) (*HubRegistration, error) { + body, _ := json.Marshal(map[string]any{ + "prefix": meta.Prefix, + "project": meta.Project, + "solution": meta.Solution, + "branch": meta.Branch, + "worktree": meta.Worktree, + "appPort": appPort, + }) + req, err := http.NewRequest(http.MethodPost, strings.TrimRight(hubURL, "/")+"/api/register", bytes.NewReader(body)) + if err != nil { + return nil, err + } + req.Header.Set("Content-Type", "application/json") + if secret != "" { + req.Header.Set("X-Hub-Secret", secret) + } + resp, err := http.DefaultClient.Do(req) + if err != nil { + return nil, fmt.Errorf("contacting hub: %w", err) + } + defer resp.Body.Close() + + switch resp.StatusCode { + case http.StatusOK: + var rr registerResponse + if err := json.NewDecoder(io.LimitReader(resp.Body, 1<<16)).Decode(&rr); err != nil { + return nil, fmt.Errorf("decoding hub response: %w", err) + } + return &HubRegistration{ + URL: rr.URL, + Subdomain: rr.Subdomain, + ControlURL: rr.ControlURL, + ReversePort: rr.ReversePort, + Token: rr.Token, + TunnelAuth: rr.TunnelAuth, + HeartbeatInterval: time.Duration(rr.HeartbeatIntervalSec) * time.Second, + MultiTenant: true, + hubURL: hubURL, + }, nil + case http.StatusNotFound: + // No registration API — a single-app hub. Serve directly at the hub URL. + return &HubRegistration{ + URL: strings.TrimRight(hubURL, "/"), + ControlURL: hubURL, + ReversePort: DefaultHubBackendPort, + TunnelAuth: secret, + MultiTenant: false, + hubURL: hubURL, + }, nil + default: + msg, _ := io.ReadAll(io.LimitReader(resp.Body, 4096)) + return nil, fmt.Errorf("hub registration failed (HTTP %d): %s", resp.StatusCode, strings.TrimSpace(string(msg))) + } +} + +// Heartbeat periodically pings the hub so the preview shows as available, and +// deregisters on Stop. No-op for a single-app (fallback) registration. +type Heartbeat struct { + cancel context.CancelFunc + done chan struct{} + reg *HubRegistration +} + +// StartHeartbeat begins heartbeating (only for a multi-tenant registration). +func StartHeartbeat(reg *HubRegistration) *Heartbeat { + if !reg.MultiTenant || reg.HeartbeatInterval <= 0 { + return &Heartbeat{reg: reg} + } + ctx, cancel := context.WithCancel(context.Background()) + h := &Heartbeat{cancel: cancel, done: make(chan struct{}), reg: reg} + go func() { + defer close(h.done) + t := time.NewTicker(reg.HeartbeatInterval) + defer t.Stop() + for { + select { + case <-ctx.Done(): + return + case <-t.C: + postToken(ctx, reg.hubURL+"/api/status", reg.Token) + } + } + }() + return h +} + +// Stop ends heartbeating and best-effort deregisters the preview from the hub. +func (h *Heartbeat) Stop() { + if h.cancel != nil { + h.cancel() + <-h.done + } + if h.reg != nil && h.reg.MultiTenant && h.reg.Token != "" { + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + postToken(ctx, h.reg.hubURL+"/api/deregister", h.reg.Token) + } +} + +func postToken(ctx context.Context, url, token string) { + req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, nil) + if err != nil { + return + } + req.Header.Set("Authorization", "Bearer "+token) + if resp, err := http.DefaultClient.Do(req); err == nil { + _ = resp.Body.Close() + } +} + +// DetectHubMeta fills a HubMeta from the project path + git, with explicit +// overrides taking precedence. +func DetectHubMeta(projectPath string, override HubMeta) HubMeta { + m := override + if m.Project == "" { + base := filepath.Base(projectPath) + m.Project = strings.TrimSuffix(base, filepath.Ext(base)) + } + if m.Branch == "" { + m.Branch = gitBranch(filepath.Dir(projectPath)) + } + return m +} + +// gitBranch returns the current branch of the repo containing dir, or "". +func gitBranch(dir string) string { + cmd := exec.Command("git", "-C", dir, "rev-parse", "--abbrev-ref", "HEAD") + out, err := cmd.Output() + if err != nil { + return "" + } + b := strings.TrimSpace(string(out)) + if b == "HEAD" { // detached + return "" + } + return b +} diff --git a/cmd/mxcli/docker/runlocal.go b/cmd/mxcli/docker/runlocal.go index 4b345fd58..e12e9c681 100644 --- a/cmd/mxcli/docker/runlocal.go +++ b/cmd/mxcli/docker/runlocal.go @@ -48,6 +48,14 @@ type LocalRunOptions struct { // HubSecret is the shared auth secret for the hub ("user:pass"), matching the // hub's --secret. Optional but recommended. HubSecret string + // Hub identity (multi-tenant hub): these drive the assigned subdomain and the + // hub overview grouping. Blank Project/Branch are auto-detected from the .mpr + // name and git. + HubPrefix string // optional hostname namespace (org/solution/team/env) + HubProject string // override the auto-detected project name + HubSolution string // grouping for multi-app solutions + HubBranch string // override the auto-detected git branch + HubWorktree string // distinguish worktrees of one branch // Watch keeps running, rebuilding+applying on every project change. Watch bool // EnsureDB provisions the local Postgres + app database if missing (otherwise @@ -428,9 +436,27 @@ func RunLocal(opts LocalRunOptions) error { } } - // 6. Boot the runtime against the fresh deployment. With --hub, the app is - // reached at the hub's public URL, so the runtime must boot with - // ApplicationRootUrl set to it (else the SPA/originURI misbehave across origins). + // 5c. With --hub, register with the hub first (before boot) so we know the + // assigned public URL — the runtime must boot with ApplicationRootUrl set to it + // (else the SPA/originURI misbehave across origins). A multi-tenant hub hands + // back a per-preview subdomain + reverse port; a single-app hub falls back to + // the hub URL itself. + var hubReg *HubRegistration + appRootURL := "" + if opts.Hub != "" { + meta := DetectHubMeta(opts.ProjectPath, HubMeta{ + Prefix: opts.HubPrefix, Project: opts.HubProject, Solution: opts.HubSolution, + Branch: opts.HubBranch, Worktree: opts.HubWorktree, + }) + fmt.Fprintf(w, "Registering with hub %s...\n", opts.Hub) + hubReg, err = RegisterWithHub(opts.Hub, opts.HubSecret, meta, opts.AppPort) + if err != nil { + return fmt.Errorf("hub registration: %w", err) + } + appRootURL = hubReg.URL + } + + // 6. Boot the runtime against the fresh deployment. rt, err := StartLocalRuntime(LocalRuntimeOptions{ DeployDir: opts.DeployDir, InstallPath: installPath, @@ -438,7 +464,7 @@ func RunLocal(opts LocalRunOptions) error { AppPort: opts.AppPort, AdminPort: opts.AdminPort, AdminPass: opts.AdminPass, - ApplicationRootUrl: opts.Hub, + ApplicationRootUrl: appRootURL, DB: opts.DB, Stdout: w, Stderr: stderr, @@ -451,20 +477,26 @@ func RunLocal(opts LocalRunOptions) error { fmt.Fprintf(w, "\nApp is running at %s\n", rt.AppURL()) // 6a. With --hub, open a reverse tunnel so the app is reachable in a browser at - // the hub's public URL. The app stays here; only live HTTP flows through the - // tunnel (nothing is pushed to the hub). - if opts.Hub != "" { + // its public URL, and heartbeat so it shows as available in the hub overview. + // The app stays here; only live HTTP flows through the tunnel (nothing pushed). + if hubReg != nil { tunnel, err := StartTunnel(TunnelOptions{ - HubURL: opts.Hub, - LocalPort: opts.AppPort, - Secret: opts.HubSecret, - Stdout: w, + HubURL: hubReg.ControlURL, + LocalPort: opts.AppPort, + RemotePort: hubReg.ReversePort, + Secret: hubReg.TunnelAuth, + PublicURL: hubReg.URL, + Stdout: w, }) if err != nil { return fmt.Errorf("starting hub tunnel: %w", err) } defer tunnel.Stop() - fmt.Fprintf(w, "Preview available at %s\n", tunnel.PublicURL()) + + hb := StartHeartbeat(hubReg) + defer hb.Stop() + + fmt.Fprintf(w, "Preview available at %s\n", hubReg.URL) } // 6b. If screenshot auth was requested, log in once and reuse the session for diff --git a/cmd/mxcli/docker/tunnel.go b/cmd/mxcli/docker/tunnel.go index 56b590cd1..c36cc83bd 100644 --- a/cmd/mxcli/docker/tunnel.go +++ b/cmd/mxcli/docker/tunnel.go @@ -41,6 +41,9 @@ type TunnelOptions struct { // set; it defaults from HTTPS_PROXY/https_proxy in the environment. chisel does // not read the proxy env itself, so we pass it explicitly. Proxy string + // PublicURL is the browser-facing URL the app is served at (an assigned + // subdomain on a multi-tenant hub). Defaults to HubURL when empty. + PublicURL string // Stdout receives progress messages (default os.Stdout). Stdout io.Writer } @@ -114,12 +117,16 @@ func StartTunnel(o TunnelOptions) (*Tunnel, error) { return nil, fmt.Errorf("starting tunnel: %w", err) } - t := &Tunnel{client: c, cancel: cancel, publicURL: strings.TrimRight(o.HubURL, "/")} + public := o.PublicURL + if public == "" { + public = o.HubURL + } + t := &Tunnel{client: c, cancel: cancel, publicURL: strings.TrimRight(public, "/")} + via := "" if o.Proxy != "" { - fmt.Fprintf(o.Stdout, "Tunnel: exposing local :%d at %s (via proxy)\n", o.LocalPort, t.publicURL) - } else { - fmt.Fprintf(o.Stdout, "Tunnel: exposing local :%d at %s\n", o.LocalPort, t.publicURL) + via = " (via proxy)" } + fmt.Fprintf(o.Stdout, "Tunnel: exposing local :%d at %s%s\n", o.LocalPort, t.publicURL, via) return t, nil } diff --git a/cmd/mxcli/tunnelhub/integration_test.go b/cmd/mxcli/tunnelhub/integration_test.go new file mode 100644 index 000000000..a6519d490 --- /dev/null +++ b/cmd/mxcli/tunnelhub/integration_test.go @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: Apache-2.0 + +package tunnelhub + +import ( + "context" + "fmt" + "net" + "net/http" + "net/http/httptest" + "strconv" + "testing" + "time" + + chclient "github.com/jpillora/chisel/client" +) + +func freePort(t *testing.T) int { + t.Helper() + l, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatalf("free port: %v", err) + } + defer l.Close() + return l.Addr().(*net.TCPAddr).Port +} + +func mustPort(t *testing.T, rawURL string) int { + t.Helper() + _, portStr, err := net.SplitHostPort(rawURL[len("http://"):]) + if err != nil { + t.Fatalf("split %q: %v", rawURL, err) + } + p, err := strconv.Atoi(portStr) + if err != nil { + t.Fatalf("port %q: %v", portStr, err) + } + return p +} + +// End-to-end, in-process: a client reverse-tunnels a backend to the hub's +// embedded chisel server, then a request to that backend's subdomain routes +// through the tunnel and returns the backend's response. Exercises registration +// -> subdomain routing -> chisel reverse tunnel -> app. +func TestFront_ProxiesThroughTunnel(t *testing.T) { + // The "app" being previewed. It echoes the Host it sees, so we can assert the + // front preserves the public host rather than the loopback backend host. + backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + fmt.Fprintf(w, "TUNNELED host=%s", r.Host) + })) + defer backend.Close() + backendPort := mustPort(t, backend.URL) + + clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} + reg := NewRegistry(RegistryOptions{Domain: "mxcli.org", PortBase: freePort(t), PortCount: 20, Now: clk.now}) + chiselPort := freePort(t) + srv, err := NewServer(ServerOptions{ + Domain: "mxcli.org", + Registry: reg, + ChiselAddr: "127.0.0.1:" + strconv.Itoa(chiselPort), + CertCacheDir: t.TempDir(), + }) + if err != nil { + t.Fatalf("NewServer: %v", err) + } + // Start just the embedded chisel server (Start() would also bind TLS). + if err := srv.chisel.Start("127.0.0.1", strconv.Itoa(chiselPort)); err != nil { + t.Fatalf("chisel start: %v", err) + } + defer srv.chisel.Close() + + // Register the preview -> assigned reverse port. + b, err := reg.Register(RegisterRequest{Project: "App", Branch: "main", AppPort: backendPort}) + if err != nil { + t.Fatalf("Register: %v", err) + } + + // The client tunnels the backend to the assigned reverse port. + client, err := chclient.NewClient(&chclient.Config{ + Server: "http://127.0.0.1:" + strconv.Itoa(chiselPort), + Remotes: []string{fmt.Sprintf("R:%d:127.0.0.1:%d", b.ReversePort, backendPort)}, + }) + if err != nil { + t.Fatalf("client: %v", err) + } + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + if err := client.Start(ctx); err != nil { + t.Fatalf("client start: %v", err) + } + defer client.Close() + + // A request to app.mxcli.org must route through the tunnel to the backend. + deadline := time.Now().Add(10 * time.Second) + var body string + for time.Now().Before(deadline) { + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req("app.mxcli.org", "/")) + if rec.Code == http.StatusOK { + body = rec.Body.String() + break + } + time.Sleep(150 * time.Millisecond) + } + if body == "" { + t.Fatal("no successful response through the tunnel within timeout") + } + if want := "TUNNELED host=app.mxcli.org"; body != want { + t.Errorf("through-tunnel body = %q, want %q (public Host must be preserved)", body, want) + } +} diff --git a/cmd/mxcli/tunnelhub/server.go b/cmd/mxcli/tunnelhub/server.go index 3eb6161ef..0ef836d98 100644 --- a/cmd/mxcli/tunnelhub/server.go +++ b/cmd/mxcli/tunnelhub/server.go @@ -213,6 +213,21 @@ func (s *Server) Start(ctx context.Context, httpsAddr, httpAddr string) error { httpSrv := &http.Server{Addr: httpAddr, Handler: s.manager.HTTPHandler(redirectToHTTPS()), ReadHeaderTimeout: 10 * time.Second} go func() { _ = httpSrv.ListenAndServe() }() + // Periodically trigger a reap so expired backends free their ports even when + // nobody loads the admin page (List reaps as a side effect). + reaper := time.NewTicker(30 * time.Second) + defer reaper.Stop() + go func() { + for { + select { + case <-ctx.Done(): + return + case <-reaper.C: + s.reg.List("") + } + } + }() + errc := make(chan error, 1) go func() { errc <- s.http.ListenAndServeTLS("", "") }() From b990548fcf766f91a8bdaf72b2330694951f0afd Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 23 Jul 2026 07:25:03 +0000 Subject: [PATCH 07/12] docs(tunnel-hub): document the multi-tenant hub (subdomains, prefix, solution, overview) Update the run-local skill + docs-site with the multi-tenant flags (--hub-prefix/-project/-solution/-branch/-worktree), per-preview subdomains, the sortable availability overview, and stable URLs on re-register. Mark slice 4 built in the warm-loop proposal (external Scaleway E2E pending). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .claude/skills/mendix/run-local.md | 26 +++++++++++++++++-- docs-site/src/tools/run-local.md | 26 ++++++++++++++++--- .../PROPOSAL_mxcli_dev_warm_loop.md | 2 +- 3 files changed, 48 insertions(+), 6 deletions(-) diff --git a/.claude/skills/mendix/run-local.md b/.claude/skills/mendix/run-local.md index ef58ce1d6..89c859ac3 100644 --- a/.claude/skills/mendix/run-local.md +++ b/.claude/skills/mendix/run-local.md @@ -173,8 +173,30 @@ mxcli run --hub https://hub.mxcli.org --hub-secret alice:s3cret -p app.mpr - The control connection honours `NO_PROXY`: an external hub goes through the egress proxy, a loopback hub connects directly. -`mxcli tunnel-hub` is the static relay (one small VPS fronts your previews). It uses -automatic Let's Encrypt for `--domain` (needs inbound 80+443), or `--tls-cert`/`--tls-key`. +### Multi-tenant hub (many previews at once) + +`mxcli tunnel-hub --domain mxcli.org` fronts **many** apps at per-preview subdomains +over one 443 — across projects, solutions, branches, and worktrees — with a sortable +overview at `https://hub.mxcli.org/`. Needs a wildcard `*.mxcli.org` A record; TLS is +issued per subdomain on demand. + +Each `run --hub` self-registers and is served at its own subdomain: + +```bash +mxcli run --hub https://hub.mxcli.org --hub-secret alice:s3cret \ + --hub-solution CustomerPortal --hub-prefix acme -p Web.mpr +# -> https://acme-web-.mxcli.org (main/master collapses to acme-web) +``` + +- **Project** and **branch** are auto-detected (`.mpr` name + git); override with + `--hub-project` / `--hub-branch`. `--hub-worktree` distinguishes worktrees of one branch. +- **`--hub-prefix`** namespaces the hostname (org/solution/team/env); **`--hub-solution`** + groups apps of one solution in the overview. +- The overview shows availability (a reaped/idle container goes **stale**), and is + sortable by last-used / registered / project. Re-registering keeps a **stable URL**. + +`mxcli tunnel-hub` is the static relay (one small VPS fronts your previews). Automatic +Let's Encrypt for `--domain` (needs inbound 80+443). ## Validation checklist diff --git a/docs-site/src/tools/run-local.md b/docs-site/src/tools/run-local.md index 7fa631c9f..a7d766f42 100644 --- a/docs-site/src/tools/run-local.md +++ b/docs-site/src/tools/run-local.md @@ -106,9 +106,29 @@ egress-only environment. proxy, a loopback hub connects directly. `mxcli tunnel-hub` is the static relay: run it once on a small VPS to front your previews. -TLS is automatic via Let's Encrypt for `--domain` (needs inbound 80 + 443), or supply -`--tls-cert`/`--tls-key`. This slice fronts a single app; multi-tenant registration and -per-preview subdomains are planned. +TLS is automatic via Let's Encrypt for `--domain` (needs inbound 80 + 443). + +### Many previews at once (multi-tenant) + +`mxcli tunnel-hub --domain mxcli.org` fronts **many** apps at per-preview subdomains over +one 443 — across projects, solutions, branches, and worktrees — with a sortable overview +at `https://hub.mxcli.org/`. It needs a wildcard `*.mxcli.org` A record; a Let's Encrypt +cert is issued per subdomain on demand. + +Each `run --hub` self-registers and gets its own subdomain: + +```bash +mxcli run --hub https://hub.mxcli.org --hub-secret alice:s3cret \ + --hub-solution CustomerPortal --hub-prefix acme -p Web.mpr +# -> https://acme-web-.mxcli.org (main/master collapses to acme-web) +``` + +- **Project** and **branch** auto-detect from the `.mpr` name and git; override with + `--hub-project`/`--hub-branch`, and `--hub-worktree` separates worktrees of one branch. +- **`--hub-prefix`** namespaces the hostname (org/solution/team/env); **`--hub-solution`** + groups a solution's apps in the overview. +- The overview shows availability — a reaped/idle container turns **stale** — and sorts by + last-used, registered, or project. Re-registering keeps a **stable URL**. ## The change signal diff --git a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md index 21b4bfe23..7ae4a9d28 100644 --- a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md +++ b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md @@ -600,7 +600,7 @@ builds on the previous. | 1 | **Warm local loop** — shipped as `mxcli run --local [--watch]` (serve daemon + M2EE admin client + `restartRequired` branching; + client bundling & Playwright screenshots) | Docker-free ~1 s edit→test loop, locally | nothing new | ✅ **shipped** | | 2 | **Provisioning** — `run --local --ensure-db` (DB) + `run --local --setup` (non-blocking bring-up) + `mxcli init` SessionStart hook + bootstrap prompt template | a fresh Claude Code Web session comes up testable; iPad-native start | slice 1 | ✅ **shipped** | | 3 | **Single-app external preview** — `mxcli run --hub ` (embedded chisel client, proxy honouring `NO_PROXY`, `ApplicationRootUrl` boot wiring) + `mxcli tunnel-hub` (embedded chisel server, autocert, single-443 `--backend`) | a shareable live preview URL (the iPad two-container flow) | slice 1 | ✅ **shipped** (2026-07-23) — code + in-process tunnel test + local end-to-end boot; Mendix renders through the hub's Host-rewriting backend, no shim needed. External E2E against the Scaleway hub is the remaining confirmation | -| 4 | **Tunnel hub** — `mxcli tunnel-hub` + `mxcli run --local --hub` + registration API + admin overview | many dev containers behind one ingress; fleet overview + per-container change lists | slice 3 | large / higher — a product in its own right, with a multi-tenant auth surface | +| 4 | **Tunnel hub** — `mxcli tunnel-hub` (multi-tenant) + `mxcli run --hub` registration + admin overview | many dev containers behind one ingress; per-preview subdomains across projects/solutions/branches/worktrees; sortable overview with availability | slice 3 | ✅ **built** (2026-07-23) — registry + registration API + single-443 host-routing front with per-subdomain autocert + sortable admin page + client registration; unit + in-process tunnel integration tests. External E2E against the Scaleway hub (wildcard DNS) is the remaining confirmation; deeper multi-tenant auth (per-container tokens, admin auth) is a follow-on | Recommended sequencing: **1 → 2** delivers the complete solo dev experience (Scenario A plus provisioning) with no external moving parts, and can ship first. **3** adds external From 0ac599fa217051b1c061d3b222172bd6e747a51e Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 23 Jul 2026 08:00:59 +0000 Subject: [PATCH 08/12] fix(run): hub tunnel now retries forever instead of giving up on first drop chisel's client MaxRetryCount defaulted to 0, so the reverse tunnel quit on the first disconnect it couldn't instantly recover (observed as "client: Give up" after a hub restart). Set MaxRetryCount = -1 so the preview reconnects indefinitely and survives a hub restart or a transient network blip. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- cmd/mxcli/docker/tunnel.go | 1 + 1 file changed, 1 insertion(+) diff --git a/cmd/mxcli/docker/tunnel.go b/cmd/mxcli/docker/tunnel.go index c36cc83bd..3a31c6a43 100644 --- a/cmd/mxcli/docker/tunnel.go +++ b/cmd/mxcli/docker/tunnel.go @@ -104,6 +104,7 @@ func StartTunnel(o TunnelOptions) (*Tunnel, error) { Auth: o.Secret, Remotes: []string{remote}, KeepAlive: 25 * time.Second, + MaxRetryCount: -1, // retry forever: survive a hub restart or network blip MaxRetryInterval: 30 * time.Second, } c, err := chclient.NewClient(cfg) From c33416496116b344a4cb98a19c02e3d9253e0e7c Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 23 Jul 2026 14:08:25 +0000 Subject: [PATCH 09/12] fix(deps): bump golang.org/x/net to v0.55.0 for GO-2026-5026 The tunnel-hub work added a call path (docker.proxyForURL -> httpproxy.Config.ProxyFunc -> idna.Profile.ToASCII) into golang.org/x/net/idna, which govulncheck flags as GO-2026-5026 (failure to reject ASCII-only Punycode-encoded labels) in x/net@v0.54.0. Fixed in v0.55.0. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- go.mod | 4 ++-- go.sum | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/go.mod b/go.mod index d29ee4c2c..ab948e40c 100644 --- a/go.mod +++ b/go.mod @@ -28,7 +28,8 @@ require ( go.mongodb.org/mongo-driver/v2 v2.6.0 go.starlark.net v0.0.0-20260102030733-3fee463870c9 go.uber.org/zap v1.28.0 - golang.org/x/net v0.54.0 + golang.org/x/crypto v0.52.0 + golang.org/x/net v0.55.0 golang.org/x/sync v0.21.0 golang.org/x/term v0.43.0 gopkg.in/yaml.v3 v3.0.1 @@ -79,7 +80,6 @@ require ( github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect go.lsp.dev/pkg v0.0.0-20210717090340-384b27a52fb2 // indirect go.uber.org/multierr v1.10.0 // indirect - golang.org/x/crypto v0.52.0 // indirect golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 // indirect golang.org/x/sys v0.45.0 // indirect golang.org/x/text v0.39.0 // indirect diff --git a/go.sum b/go.sum index eb4ec15c4..8be85ee4c 100644 --- a/go.sum +++ b/go.sum @@ -198,8 +198,8 @@ golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 h1:mgKeJMpvi0yx/sU5GsxQ7p6s2 golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546/go.mod h1:j/pmGrbnkbPtQfxEe5D0VQhZC6qKbfKifgD0oM7sR70= golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ= golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0= -golang.org/x/net v0.54.0 h1:2zJIZAxAHV/OHCDTCOHAYehQzLfSXuf/5SoL/Dv6w/w= -golang.org/x/net v0.54.0/go.mod h1:Sj4oj8jK6XmHpBZU/zWHw3BV3abl4Kvi+Ut7cQcY+cQ= +golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8= +golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww= golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM= golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= From 628e418377d03062a2c63b02150018f58b13cce3 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 23 Jul 2026 14:24:51 +0000 Subject: [PATCH 10/12] docs: cover external preview + tunnel-hub in CLAUDE.md and README The load-bearing project doc (CLAUDE.md Implemented list) and the README described only run --local; add the external browser preview (run --hub + multi-tenant tunnel-hub: subdomains, prefix/solution/branch/worktree, autocert, registration API, availability overview). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- CLAUDE.md | 1 + README.md | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/CLAUDE.md b/CLAUDE.md index 60b4d2d65..efd27ce3b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -524,6 +524,7 @@ Full syntax tables for all MDL statements (microflows, pages, security, navigati - VS Code extension (`vscode-mdl`) with context menu commands (Run/Check/Selection) - Docker build integration (`mxcli docker build`) with PAD patching (Phase 1) - Warm local dev loop (`mxcli run --local [--watch] [--screenshot]`): Docker-free `mxbuild --serve` + standalone runtime, hot `reload_model` for behavioural changes and restart+DDL for structural ones (chosen from the serve build's `restartRequired`). Bundles the browser client (`web/dist/` via mxbuild's rollup runner, which the serve Deploy target skips) so Mendix 11.x apps render in a browser. `--watch` keeps an incremental rollup bundler hot (CHOKIDAR_USEPOLLING for container fs; ~3-4s page re-bundle, skipped for model-only edits) and watches only model source (`.mpr`+`mprcontents/`). `--ensure-db` provisions the local Postgres + app database if missing; `--setup` does the non-blocking prerequisites (cache mxbuild+runtime, ensure DB) and exits — `mxcli init` wires it into a Claude Code SessionStart hook so a fresh/reaped web session self-bootstraps, and `docs-site/src/tools/bootstrap-prompt.md` is the empty-repo seed prompt. `--screenshot` captures a Playwright PNG each change (pixel-perfect page loop), with `--screenshot-url` deep links (repeatable for multi-page sets, one PNG per page) and `--screenshot-user`/`--screenshot-password` form login (session saved as Playwright storage state, reused via `screenshot --load-storage`). See `docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md` +- External browser preview (`mxcli run --hub ` + `mxcli tunnel-hub`): the app stays local and reverse-tunnels out over a single 443 connection (embedded chisel) to a static relay, so it is reachable in a browser at a public URL — works from egress-only environments (Claude Code web), verified live through the session's MITM egress proxy. `run --hub` implies `--local`, boots the runtime with `ApplicationRootUrl` set to the assigned URL (so the SPA/`originURI` work under the public origin), resolves the control proxy honouring `NO_PROXY`, and retries forever. `mxcli tunnel-hub --domain ` is the **multi-tenant** relay: a registry keyed by prefix/project/solution/branch/worktree (stable URLs on reconnect) fronts many previews at per-subdomain hosts (`[prefix-]project[-branch].`; main collapses to the project) over one 443 with per-subdomain autocert, a registration API (`/api/register|status|deregister|backends`), and a sortable availability overview at `hub./`. Client identity flags: `--hub-prefix`/`--hub-project`/`--hub-solution`/`--hub-branch`/`--hub-worktree` (project + branch auto-detected). Package: `cmd/mxcli/tunnelhub/`. See `docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md` (slices 3–4) - OQL query execution against running runtime (`mxcli oql`) - Business event services (SHOW/DESCRIBE/CREATE/DROP) - Project settings (SHOW/DESCRIBE/ALTER) diff --git a/README.md b/README.md index 338dcc365..8f0038199 100644 --- a/README.md +++ b/README.md @@ -140,6 +140,14 @@ mxcli run --local -p app.mpr --watch --screenshot # hot-reload + auto screensh `mxcli run --local` keeps `mxbuild --serve` and a standalone runtime hot: a page/microflow edit is hot-applied in seconds (a hot `reload_model`, or a restart + DDL for entity changes), and `--screenshot` captures each page with Playwright. See **[Local Dev Loop](https://mendixlabs.github.io/mxcli/tools/run-local.html)**. +Add `--hub` to preview the running app **in a browser at a public URL** — the app stays local and reverse-tunnels out over a single 443 connection, so it works even from egress-only environments (e.g. Claude Code on the web): + +```bash +mxcli run --hub https://hub.example.com -p app.mpr # -> a shareable preview URL +``` + +`mxcli tunnel-hub --domain example.com` is the static relay you run once on a small VPS; it can front many previews at per-subdomain hosts across projects, solutions, branches, and worktrees, with a sortable availability overview at `hub.example.com/`. See **[Local Dev Loop → External browser preview](https://mendixlabs.github.io/mxcli/tools/run-local.html)**. + ### Existing project For an existing Mendix project, use `mxcli init` to add AI tooling and a Dev Container: From 86581b1a9e438ab6ed2ddeff706c4eaf339064f6 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 23 Jul 2026 14:32:16 +0000 Subject: [PATCH 11/12] docs: use generic hub domain + make the hub self-hosted (no hosted service) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace the mxcli.org examples in CLI help, the run-local skill, and the docs-site with example.com placeholders, and state plainly that there is no hosted hub — you run your own mxcli tunnel-hub on a host you control. Add a security note (this version uses a single shared secret with open registration, so keep the hub to people you trust; per-tenant auth is a follow-up). Also reconcile the preview docs with the shipped multi-tenant command: the hub is started with --domain (not the hub host) and previews are served at -., with the admin overview at hub. — the old single-app framing ("--domain hub...", "Preview available at hub...") was stale. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .claude/skills/mendix/run-local.md | 65 +++++++++++++----------------- cmd/mxcli/cmd_run.go | 6 +-- cmd/mxcli/cmd_tunnelhub.go | 17 +++++--- docs-site/src/tools/run-local.md | 62 ++++++++++++---------------- 4 files changed, 69 insertions(+), 81 deletions(-) diff --git a/.claude/skills/mendix/run-local.md b/.claude/skills/mendix/run-local.md index 89c859ac3..94bb817b3 100644 --- a/.claude/skills/mendix/run-local.md +++ b/.claude/skills/mendix/run-local.md @@ -153,50 +153,41 @@ mxcli run --local -p app.mpr --watch --screenshot `--hub ` exposes the running app in a **browser at a public URL** without the app leaving this machine and without committing — for reviewing work-in-progress from a -phone/tablet, or from an egress-only environment like Claude Code on the web. +phone/tablet, or from an egress-only environment like Claude Code on the web. The app +stays here; a **chisel reverse tunnel** dials *out* to a hub over 443 and the hub proxies +browser requests back down it. Nothing is pushed — only live HTTP — and everything rides +one 443 connection, so it works through an egress-only proxy. + +**You run your own hub — there is no hosted service.** Stand up `mxcli tunnel-hub` once on +a host you control (a small VPS with a domain), then point apps at it. ```bash -# on a small VPS with a public IP + domain (hub.mxcli.org -> it, ports 80+443 open): -mxcli tunnel-hub --domain hub.mxcli.org --secret alice:s3cret +# on your VPS: *.example.com + hub.example.com -> this host, inbound 80+443 open +mxcli tunnel-hub --domain example.com --secret alice:s3cret -# here, where the app runs: -mxcli run --hub https://hub.mxcli.org --hub-secret alice:s3cret -p app.mpr -# -> boots the app locally and prints "Preview available at https://hub.mxcli.org" +# where the app runs: +mxcli run --hub https://hub.example.com --hub-secret alice:s3cret -p app.mpr +# -> registers and prints e.g. "Preview available at https://app.example.com" ``` -- The app stays here; a **chisel reverse tunnel** dials *out* to the hub over 443, and - the hub proxies browser requests back down it. Nothing is pushed — only live HTTP. - Everything rides one 443 connection, so it works through an egress-only proxy. -- `--hub` **implies `--local`** and boots the runtime with `ApplicationRootUrl` set to - the hub URL, so the SPA/`originURI` work under the public origin. -- Combine with `--watch` for the full loop: edit here → hot-apply → refresh the browser. -- The control connection honours `NO_PROXY`: an external hub goes through the egress - proxy, a loopback hub connects directly. - -### Multi-tenant hub (many previews at once) - -`mxcli tunnel-hub --domain mxcli.org` fronts **many** apps at per-preview subdomains -over one 443 — across projects, solutions, branches, and worktrees — with a sortable -overview at `https://hub.mxcli.org/`. Needs a wildcard `*.mxcli.org` A record; TLS is -issued per subdomain on demand. - -Each `run --hub` self-registers and is served at its own subdomain: +The hub is **multi-tenant**: it fronts many previews at per-preview subdomains +(`-.example.com`; `main`/`master` collapses to ``) with a +sortable overview at `https://hub.example.com/`. Each `run --hub` self-registers: -```bash -mxcli run --hub https://hub.mxcli.org --hub-secret alice:s3cret \ - --hub-solution CustomerPortal --hub-prefix acme -p Web.mpr -# -> https://acme-web-.mxcli.org (main/master collapses to acme-web) -``` - -- **Project** and **branch** are auto-detected (`.mpr` name + git); override with +- **Project** and **branch** auto-detect (`.mpr` name + git); override with `--hub-project` / `--hub-branch`. `--hub-worktree` distinguishes worktrees of one branch. -- **`--hub-prefix`** namespaces the hostname (org/solution/team/env); **`--hub-solution`** - groups apps of one solution in the overview. -- The overview shows availability (a reaped/idle container goes **stale**), and is - sortable by last-used / registered / project. Re-registering keeps a **stable URL**. - -`mxcli tunnel-hub` is the static relay (one small VPS fronts your previews). Automatic -Let's Encrypt for `--domain` (needs inbound 80+443). +- **`--hub-prefix`** namespaces the hostname (org/solution/team/env) → + `--`; **`--hub-solution`** groups a solution's apps in the overview. +- The overview shows availability (a reaped/idle container goes **stale**), sortable by + last-used / registered / project. Re-registering keeps a **stable URL**. +- `--hub` **implies `--local`**, boots the runtime with `ApplicationRootUrl` set to the + assigned URL (so the SPA/`originURI` work), and the tunnel reconnects forever. Combine + with `--watch` for the full loop: edit here → hot-apply → refresh the browser. + +**Hub setup:** wildcard `*.example.com` A record (+ `hub.example.com`) → the VPS; inbound +80+443 open (per-subdomain Let's Encrypt on demand). **Security:** this version uses one +shared `--secret` and open registration, so keep the hub to people you trust and don't +expose it publicly (per-tenant auth is a follow-up). ## Validation checklist diff --git a/cmd/mxcli/cmd_run.go b/cmd/mxcli/cmd_run.go index 52d18dafa..b50e52215 100644 --- a/cmd/mxcli/cmd_run.go +++ b/cmd/mxcli/cmd_run.go @@ -41,8 +41,8 @@ Examples: mxcli run --local -p app.mpr mxcli run --local -p app.mpr --watch mxcli run --local -p app.mpr --app-port 8081 --db-name myapp - mxcli run --hub https://hub.mxcli.org -p app.mpr # browser preview - mxcli run --hub https://hub.mxcli.org --hub-secret u:pass -p app.mpr --watch + mxcli run --hub https://hub.example.com -p app.mpr # browser preview + mxcli run --hub https://hub.example.com --hub-secret u:pass -p app.mpr --watch `, Run: func(cmd *cobra.Command, args []string) { local, _ := cmd.Flags().GetBool("local") @@ -123,7 +123,7 @@ Examples: func init() { runCmd.Flags().Bool("local", false, "Run locally without Docker (warm serve + standalone runtime)") - runCmd.Flags().String("hub", "", "Expose the running app in a browser via an mxcli tunnel-hub URL (e.g. https://hub.mxcli.org). Implies --local; the app stays local and is reverse-tunnelled out") + runCmd.Flags().String("hub", "", "Expose the running app in a browser via your own mxcli tunnel-hub URL (e.g. https://hub.example.com). Implies --local; the app stays local and is reverse-tunnelled out") runCmd.Flags().String("hub-secret", "", "Shared auth secret for --hub (\"user:pass\"), matching the hub's --secret") runCmd.Flags().String("hub-prefix", "", "Optional subdomain prefix on the hub (org/solution/team/env): --") runCmd.Flags().String("hub-project", "", "Project name for the hub subdomain + overview (default: the .mpr name)") diff --git a/cmd/mxcli/cmd_tunnelhub.go b/cmd/mxcli/cmd_tunnelhub.go index c82782d90..0fced4aec 100644 --- a/cmd/mxcli/cmd_tunnelhub.go +++ b/cmd/mxcli/cmd_tunnelhub.go @@ -31,14 +31,21 @@ the hub host (hub.) serves the registration API, the admin overview, and the chisel control connection. Everything rides one 443 connection, so apps in egress-only environments (e.g. Claude Code on the web) can reverse-tunnel out. +You run your own hub — there is no hosted service. Stand it up on a host you +control (a small VPS with a domain). + DNS: point a wildcard '*.' A record (and 'hub.') at this host. TLS is issued per subdomain via Let's Encrypt on demand (needs inbound 80+443). -Example (on a VPS; *.mxcli.org -> this host, ports 80+443 open): - mxcli tunnel-hub --domain mxcli.org --secret alice:s3cret +Security: set --secret and keep the hub to people you trust — this version uses a +single shared secret and open registration, so anyone with it can register a +preview (per-tenant auth is a follow-up). Don't expose it to the public. + +Example (on your own VPS; *.example.com -> this host, ports 80+443 open): + mxcli tunnel-hub --domain example.com --secret alice:s3cret Then, in each app's environment: - mxcli run --hub https://hub.mxcli.org --hub-secret alice:s3cret \ + mxcli run --hub https://hub.example.com --hub-secret alice:s3cret \ --hub-solution CustomerPortal -p app.mpr `, Run: func(cmd *cobra.Command, args []string) { @@ -50,7 +57,7 @@ Then, in each app's environment: certCache, _ := cmd.Flags().GetString("cert-cache") if domain == "" { - fmt.Fprintln(os.Stderr, "Error: --domain is required (the wildcard base, e.g. mxcli.org)") + fmt.Fprintln(os.Stderr, "Error: --domain is required (the wildcard base, e.g. example.com)") os.Exit(1) } if certCache == "" { @@ -90,7 +97,7 @@ Then, in each app's environment: } func init() { - tunnelHubCmd.Flags().String("domain", "", "Wildcard base domain, e.g. mxcli.org (previews served at .)") + tunnelHubCmd.Flags().String("domain", "", "Wildcard base domain you control, e.g. example.com (previews served at .)") tunnelHubCmd.Flags().String("hub-host", "", "Control/admin host (default hub.)") tunnelHubCmd.Flags().String("secret", "", "Shared secret (\"user:pass\") apps present via --hub-secret; empty = open") tunnelHubCmd.Flags().Int("port", 443, "HTTPS port to listen on") diff --git a/docs-site/src/tools/run-local.md b/docs-site/src/tools/run-local.md index a7d766f42..977bc584d 100644 --- a/docs-site/src/tools/run-local.md +++ b/docs-site/src/tools/run-local.md @@ -83,52 +83,42 @@ so structural changes need a restart; behavioural changes do not. `--hub ` makes the running app reachable **in a browser at a public URL** — without the app leaving this machine and without committing. It's for reviewing work-in-progress from a phone or tablet, or from an egress-only environment such as Claude Code on the web. +The app stays local and a **chisel reverse tunnel** dials *out* to a hub over 443; the hub +proxies browser requests back down the tunnel. Nothing is pushed — only live HTTP — and +because everything rides a single 443 connection, it works even from an egress-only proxy. + +**You run your own hub — there is no hosted service.** Stand up `mxcli tunnel-hub` once on +a host you control (a small VPS with a domain), then point apps at it. ```bash -# on a small VPS with a public IP + domain (hub.mxcli.org -> it, inbound 80+443 open): -mxcli tunnel-hub --domain hub.mxcli.org --secret alice:s3cret +# on your VPS: *.example.com + hub.example.com -> this host, inbound 80+443 open +mxcli tunnel-hub --domain example.com --secret alice:s3cret -# where the app runs (here): -mxcli run --hub https://hub.mxcli.org --hub-secret alice:s3cret -p app.mpr -# -> "Preview available at https://hub.mxcli.org" +# where the app runs: +mxcli run --hub https://hub.example.com --hub-secret alice:s3cret -p app.mpr +# -> registers and prints e.g. "Preview available at https://app.example.com" ``` -How it works: the app stays local and a **chisel reverse tunnel** dials *out* to the hub -over 443; the hub proxies browser requests back down the tunnel. Nothing is pushed — only -live HTTP flows. Because everything rides a single 443 connection, it works even from an -egress-only environment. - -- `--hub` **implies `--local`**, and boots the runtime with `ApplicationRootUrl` set to - the hub URL so the SPA and `originURI` cookie work under the public origin. -- Combine with `--watch` for the full remote loop: edit here → hot-apply → refresh the - browser tab. -- The control connection honours `NO_PROXY` — an external hub goes through the egress - proxy, a loopback hub connects directly. - -`mxcli tunnel-hub` is the static relay: run it once on a small VPS to front your previews. -TLS is automatic via Let's Encrypt for `--domain` (needs inbound 80 + 443). - -### Many previews at once (multi-tenant) - -`mxcli tunnel-hub --domain mxcli.org` fronts **many** apps at per-preview subdomains over -one 443 — across projects, solutions, branches, and worktrees — with a sortable overview -at `https://hub.mxcli.org/`. It needs a wildcard `*.mxcli.org` A record; a Let's Encrypt -cert is issued per subdomain on demand. - -Each `run --hub` self-registers and gets its own subdomain: - -```bash -mxcli run --hub https://hub.mxcli.org --hub-secret alice:s3cret \ - --hub-solution CustomerPortal --hub-prefix acme -p Web.mpr -# -> https://acme-web-.mxcli.org (main/master collapses to acme-web) -``` +The hub is **multi-tenant**: it fronts many previews at per-preview subdomains +(`-.example.com`; `main`/`master` collapses to ``) — across +projects, solutions, branches, and worktrees — with a sortable overview at +`https://hub.example.com/`. Each `run --hub` self-registers: - **Project** and **branch** auto-detect from the `.mpr` name and git; override with `--hub-project`/`--hub-branch`, and `--hub-worktree` separates worktrees of one branch. -- **`--hub-prefix`** namespaces the hostname (org/solution/team/env); **`--hub-solution`** - groups a solution's apps in the overview. +- **`--hub-prefix`** namespaces the hostname (org/solution/team/env) → + `--`; **`--hub-solution`** groups a solution's apps in the overview. - The overview shows availability — a reaped/idle container turns **stale** — and sorts by last-used, registered, or project. Re-registering keeps a **stable URL**. +- `--hub` **implies `--local`**, boots the runtime with `ApplicationRootUrl` set to the + assigned URL (so the SPA and `originURI` cookie work), and the tunnel reconnects forever. + Combine with `--watch` for the full remote loop: edit here → hot-apply → refresh the tab. + +**Hub setup:** a wildcard `*.example.com` A record (and `hub.example.com`) pointed at the +VPS; inbound 80 + 443 open — a Let's Encrypt cert is issued per subdomain on demand. + +**Security:** this version uses a single shared `--secret` with open registration, so keep +the hub to people you trust and don't expose it publicly (per-tenant auth is a follow-up). ## The change signal From 97e72b3d45280a66cfe738d95be1e88bf4777b27 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 23 Jul 2026 14:55:29 +0000 Subject: [PATCH 12/12] docs: genericize the real hub host in the proposal + tests (avoid exposure) Replace the remaining mxcli.org references (the proposal's verification record, code comment examples, and test fixtures) with example.com, so a scanner can't find and target the real hub host from the repo. Only hub-domain usage was changed; there was no www.mxcli.org (docs) or repo-link usage to preserve. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- cmd/mxcli/docker/localboot.go | 2 +- cmd/mxcli/docker/localboot_test.go | 6 ++--- cmd/mxcli/docker/runlocal.go | 2 +- cmd/mxcli/docker/tunnel.go | 2 +- cmd/mxcli/docker/tunnel_test.go | 2 +- cmd/mxcli/tunnelhub/api.go | 2 +- cmd/mxcli/tunnelhub/api_test.go | 6 ++--- cmd/mxcli/tunnelhub/integration_test.go | 10 +++---- cmd/mxcli/tunnelhub/registry.go | 2 +- cmd/mxcli/tunnelhub/registry_test.go | 4 +-- cmd/mxcli/tunnelhub/server.go | 4 +-- cmd/mxcli/tunnelhub/server_test.go | 26 +++++++++---------- .../PROPOSAL_mxcli_dev_warm_loop.md | 10 +++---- 13 files changed, 39 insertions(+), 39 deletions(-) diff --git a/cmd/mxcli/docker/localboot.go b/cmd/mxcli/docker/localboot.go index dbfd9b7db..0df5974cd 100644 --- a/cmd/mxcli/docker/localboot.go +++ b/cmd/mxcli/docker/localboot.go @@ -61,7 +61,7 @@ type LocalRuntimeOptions struct { // DTAPMode is D/A/T/P (default "D"). DTAPMode string // ApplicationRootUrl, when set, is the public URL the app is reached at - // (e.g. https://hub.mxcli.org). Mendix uses it for absolute-URL generation + // (e.g. https://hub.example.com). Mendix uses it for absolute-URL generation // and to accept requests whose Host differs from the listen address — needed // when the app is served through an external tunnel/reverse proxy rather than // localhost. Empty for a plain local run. diff --git a/cmd/mxcli/docker/localboot_test.go b/cmd/mxcli/docker/localboot_test.go index 5b282f01a..21250ea6c 100644 --- a/cmd/mxcli/docker/localboot_test.go +++ b/cmd/mxcli/docker/localboot_test.go @@ -127,9 +127,9 @@ func TestRuntimeConfigParams_ApplicationRootUrl(t *testing.T) { } // Present when serving behind a hub, so the SPA works under the public origin. o := testLocalOpts() - o.ApplicationRootUrl = "https://hub.mxcli.org" - if got := runtimeConfigParams(o, nil)["ApplicationRootUrl"]; got != "https://hub.mxcli.org" { - t.Errorf("ApplicationRootUrl = %v, want https://hub.mxcli.org", got) + o.ApplicationRootUrl = "https://hub.example.com" + if got := runtimeConfigParams(o, nil)["ApplicationRootUrl"]; got != "https://hub.example.com" { + t.Errorf("ApplicationRootUrl = %v, want https://hub.example.com", got) } } diff --git a/cmd/mxcli/docker/runlocal.go b/cmd/mxcli/docker/runlocal.go index e12e9c681..915bab54a 100644 --- a/cmd/mxcli/docker/runlocal.go +++ b/cmd/mxcli/docker/runlocal.go @@ -40,7 +40,7 @@ type LocalRunOptions struct { AdminPass string // DB is the Postgres the runtime connects to (devcontainer defaults applied). DB DBConfig - // Hub, when set, is the URL of an mxcli tunnel-hub (e.g. https://hub.mxcli.org). + // Hub, when set, is the URL of an mxcli tunnel-hub (e.g. https://hub.example.com). // The app stays running here; a chisel client reverse-tunnels it out to the hub // so it is reachable in a browser at the hub URL. Implies a local run. The // runtime boots with ApplicationRootUrl = Hub so the SPA works under that origin. diff --git a/cmd/mxcli/docker/tunnel.go b/cmd/mxcli/docker/tunnel.go index 3a31c6a43..0e7034b4d 100644 --- a/cmd/mxcli/docker/tunnel.go +++ b/cmd/mxcli/docker/tunnel.go @@ -25,7 +25,7 @@ const DefaultHubBackendPort = 9000 // the hub's public URL. The app never leaves this machine — only live HTTP flows // through the tunnel. type TunnelOptions struct { - // HubURL is the tunnel-hub base URL, e.g. https://hub.mxcli.org. The chisel + // HubURL is the tunnel-hub base URL, e.g. https://hub.example.com. The chisel // control connection dials it over 443. HubURL string // LocalPort is the local app port to expose (e.g. 8080). diff --git a/cmd/mxcli/docker/tunnel_test.go b/cmd/mxcli/docker/tunnel_test.go index 13c360aaf..09cd49582 100644 --- a/cmd/mxcli/docker/tunnel_test.go +++ b/cmd/mxcli/docker/tunnel_test.go @@ -28,7 +28,7 @@ func TestProxyForURL(t *testing.T) { url string want string }{ - {"https://hub.mxcli.org", "http://127.0.0.1:33451"}, + {"https://hub.example.com", "http://127.0.0.1:33451"}, {"http://127.0.0.1:9500", ""}, // loopback → NO_PROXY {"https://relay.internal.example", ""}, // suffix match in NO_PROXY {"", ""}, // no URL diff --git a/cmd/mxcli/tunnelhub/api.go b/cmd/mxcli/tunnelhub/api.go index 781b5d67b..c0a614d62 100644 --- a/cmd/mxcli/tunnelhub/api.go +++ b/cmd/mxcli/tunnelhub/api.go @@ -12,7 +12,7 @@ import ( type APIOptions struct { Registry *Registry // ControlURL is where the client points its chisel control connection - // (e.g. https://hub.mxcli.org). Returned in the registration response. + // (e.g. https://hub.example.com). Returned in the registration response. ControlURL string // TunnelAuth is the shared chisel auth ("user:pass") the client must use, if // the hub's tunnel server requires one. Returned to the client. diff --git a/cmd/mxcli/tunnelhub/api_test.go b/cmd/mxcli/tunnelhub/api_test.go index d3576790a..380e9fcba 100644 --- a/cmd/mxcli/tunnelhub/api_test.go +++ b/cmd/mxcli/tunnelhub/api_test.go @@ -17,7 +17,7 @@ func newTestAPI(t *testing.T, secret string) (*API, *Registry) { reg := newTestRegistry(clk) api := NewAPI(APIOptions{ Registry: reg, - ControlURL: "https://hub.mxcli.org", + ControlURL: "https://hub.example.com", RegisterSecret: secret, }) return api, reg @@ -47,10 +47,10 @@ func TestAPI_RegisterAndBackends(t *testing.T) { if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil { t.Fatal(err) } - if resp.URL != "https://myapp-feature-x.mxcli.org" { + if resp.URL != "https://myapp-feature-x.example.com" { t.Errorf("url = %q", resp.URL) } - if resp.ReversePort == 0 || resp.Token == "" || resp.ControlURL != "https://hub.mxcli.org" { + if resp.ReversePort == 0 || resp.Token == "" || resp.ControlURL != "https://hub.example.com" { t.Errorf("incomplete response: %+v", resp) } diff --git a/cmd/mxcli/tunnelhub/integration_test.go b/cmd/mxcli/tunnelhub/integration_test.go index a6519d490..3e6653a77 100644 --- a/cmd/mxcli/tunnelhub/integration_test.go +++ b/cmd/mxcli/tunnelhub/integration_test.go @@ -52,10 +52,10 @@ func TestFront_ProxiesThroughTunnel(t *testing.T) { backendPort := mustPort(t, backend.URL) clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} - reg := NewRegistry(RegistryOptions{Domain: "mxcli.org", PortBase: freePort(t), PortCount: 20, Now: clk.now}) + reg := NewRegistry(RegistryOptions{Domain: "example.com", PortBase: freePort(t), PortCount: 20, Now: clk.now}) chiselPort := freePort(t) srv, err := NewServer(ServerOptions{ - Domain: "mxcli.org", + Domain: "example.com", Registry: reg, ChiselAddr: "127.0.0.1:" + strconv.Itoa(chiselPort), CertCacheDir: t.TempDir(), @@ -90,12 +90,12 @@ func TestFront_ProxiesThroughTunnel(t *testing.T) { } defer client.Close() - // A request to app.mxcli.org must route through the tunnel to the backend. + // A request to app.example.com must route through the tunnel to the backend. deadline := time.Now().Add(10 * time.Second) var body string for time.Now().Before(deadline) { rec := httptest.NewRecorder() - srv.ServeHTTP(rec, req("app.mxcli.org", "/")) + srv.ServeHTTP(rec, req("app.example.com", "/")) if rec.Code == http.StatusOK { body = rec.Body.String() break @@ -105,7 +105,7 @@ func TestFront_ProxiesThroughTunnel(t *testing.T) { if body == "" { t.Fatal("no successful response through the tunnel within timeout") } - if want := "TUNNELED host=app.mxcli.org"; body != want { + if want := "TUNNELED host=app.example.com"; body != want { t.Errorf("through-tunnel body = %q, want %q (public Host must be preserved)", body, want) } } diff --git a/cmd/mxcli/tunnelhub/registry.go b/cmd/mxcli/tunnelhub/registry.go index d4aa54688..f2debb32d 100644 --- a/cmd/mxcli/tunnelhub/registry.go +++ b/cmd/mxcli/tunnelhub/registry.go @@ -74,7 +74,7 @@ type Registry struct { byIdentity map[string]*Backend usedPorts map[int]bool - domain string // e.g. "mxcli.org" + domain string // e.g. "example.com" portBase int // first reverse port to allocate portCount int // number of reverse ports available staleFor time.Duration diff --git a/cmd/mxcli/tunnelhub/registry_test.go b/cmd/mxcli/tunnelhub/registry_test.go index 38e0d5eb1..1d1a3ff0c 100644 --- a/cmd/mxcli/tunnelhub/registry_test.go +++ b/cmd/mxcli/tunnelhub/registry_test.go @@ -15,7 +15,7 @@ func (c *fakeClock) add(d time.Duration) { c.t = c.t.Add(d) } func newTestRegistry(clk *fakeClock) *Registry { return NewRegistry(RegistryOptions{ - Domain: "mxcli.org", + Domain: "example.com", PortBase: 9001, PortCount: 5, StaleFor: 45 * time.Second, @@ -171,7 +171,7 @@ func TestList_Sorting(t *testing.T) { func TestPortExhaustion(t *testing.T) { clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} - r := NewRegistry(RegistryOptions{Domain: "mxcli.org", PortBase: 9001, PortCount: 2, Now: clk.now}) + r := NewRegistry(RegistryOptions{Domain: "example.com", PortBase: 9001, PortCount: 2, Now: clk.now}) if _, err := r.Register(RegisterRequest{Project: "A", Branch: "main"}); err != nil { t.Fatal(err) } diff --git a/cmd/mxcli/tunnelhub/server.go b/cmd/mxcli/tunnelhub/server.go index 0ef836d98..4b4bfa2fa 100644 --- a/cmd/mxcli/tunnelhub/server.go +++ b/cmd/mxcli/tunnelhub/server.go @@ -27,7 +27,7 @@ const ( // ServerOptions configures the multi-tenant hub front. type ServerOptions struct { - // Domain is the wildcard base, e.g. "mxcli.org". App previews are served at + // Domain is the wildcard base, e.g. "example.com". App previews are served at // .. Domain string // HubHost is the control/admin/API host (default "hub."+Domain). Clients dial @@ -148,7 +148,7 @@ func (s *Server) hostPolicy(_ context.Context, host string) error { } // subOf returns the subdomain label of host under Domain (e.g. "app" from -// "app.mxcli.org"), and whether host is under Domain. +// "app.example.com"), and whether host is under Domain. func (s *Server) subOf(host string) (string, bool) { suffix := "." + s.opts.Domain if !strings.HasSuffix(host, suffix) { diff --git a/cmd/mxcli/tunnelhub/server_test.go b/cmd/mxcli/tunnelhub/server_test.go index a30c9a3f0..7561d1c00 100644 --- a/cmd/mxcli/tunnelhub/server_test.go +++ b/cmd/mxcli/tunnelhub/server_test.go @@ -15,7 +15,7 @@ func newTestServer(t *testing.T) (*Server, *Registry) { clk := &fakeClock{t: time.Unix(1_700_000_000, 0)} reg := newTestRegistry(clk) srv, err := NewServer(ServerOptions{ - Domain: "mxcli.org", + Domain: "example.com", Registry: reg, CertCacheDir: t.TempDir(), }) @@ -34,7 +34,7 @@ func req(host, target string) *http.Request { func TestFront_HubHostServesAdmin(t *testing.T) { srv, _ := newTestServer(t) rec := httptest.NewRecorder() - srv.ServeHTTP(rec, req("hub.mxcli.org", "/")) + srv.ServeHTTP(rec, req("hub.example.com", "/")) if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), "mxcli tunnel-hub") { t.Errorf("admin page: code=%d, body has hub title=%v", rec.Code, strings.Contains(rec.Body.String(), "mxcli tunnel-hub")) } @@ -44,7 +44,7 @@ func TestFront_HubHostAPI(t *testing.T) { srv, reg := newTestServer(t) reg.Register(RegisterRequest{Project: "App", Branch: "main"}) rec := httptest.NewRecorder() - srv.ServeHTTP(rec, req("hub.mxcli.org", "/api/backends")) + srv.ServeHTTP(rec, req("hub.example.com", "/api/backends")) if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), `"project":"App"`) { t.Errorf("api/backends: code=%d body=%s", rec.Code, rec.Body) } @@ -53,7 +53,7 @@ func TestFront_HubHostAPI(t *testing.T) { func TestFront_UnknownSubdomain(t *testing.T) { srv, _ := newTestServer(t) rec := httptest.NewRecorder() - srv.ServeHTTP(rec, req("ghost.mxcli.org", "/")) + srv.ServeHTTP(rec, req("ghost.example.com", "/")) if rec.Code != http.StatusNotFound || !strings.Contains(rec.Body.String(), "No preview") { t.Errorf("unknown subdomain: code=%d body=%s", rec.Code, rec.Body) } @@ -64,7 +64,7 @@ func TestFront_RegisteredButOffline(t *testing.T) { b, _ := reg.Register(RegisterRequest{Project: "App", Branch: "main"}) // subdomain "app" rec := httptest.NewRecorder() - srv.ServeHTTP(rec, req("app.mxcli.org", "/")) + srv.ServeHTTP(rec, req("app.example.com", "/")) // Registered but its reverse port has no tunnel -> the proxy error handler // returns the offline page. if rec.Code != http.StatusBadGateway || !strings.Contains(rec.Body.String(), "offline") { @@ -83,11 +83,11 @@ func TestSubOf(t *testing.T) { sub string ok bool }{ - {"app.mxcli.org", "app", true}, - {"my-app-feat.mxcli.org", "my-app-feat", true}, - {"hub.mxcli.org", "hub", true}, // subOf is purely structural; routing handles hub separately - {"a.b.mxcli.org", "", false}, // multi-label not allowed - {"mxcli.org", "", false}, + {"app.example.com", "app", true}, + {"my-app-feat.example.com", "my-app-feat", true}, + {"hub.example.com", "hub", true}, // subOf is purely structural; routing handles hub separately + {"a.b.example.com", "", false}, // multi-label not allowed + {"example.com", "", false}, {"evil.com", "", false}, } for _, c := range cases { @@ -100,9 +100,9 @@ func TestSubOf(t *testing.T) { func TestStripPort(t *testing.T) { for in, want := range map[string]string{ - "app.mxcli.org": "app.mxcli.org", - "app.mxcli.org:443": "app.mxcli.org", - "hub.mxcli.org:8443": "hub.mxcli.org", + "app.example.com": "app.example.com", + "app.example.com:443": "app.example.com", + "hub.example.com:8443": "hub.example.com", } { if got := stripPort(in); got != want { t.Errorf("stripPort(%q) = %q, want %q", in, got, want) diff --git a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md index 7ae4a9d28..edd18c286 100644 --- a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md +++ b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md @@ -425,10 +425,10 @@ removes that unknown** and is cleaner AUP-wise for a pure relay. The load-bearing unknown (Open Q9 — does chisel's WebSocket tunnel survive the Claude Code **web** session's mandatory egress proxy?) is now **closed, positive**, tested live from a Claude Code web container against a **Scaleway VPS** relay on the -registrant's own domain (`hub.mxcli.org`): +registrant's own domain (`hub.example.com`): - **chisel client connected out through the egress proxy:** - `Connecting to wss://hub.mxcli.org:443 via http://127.0.0.1:33451 … Connected`. + `Connecting to wss://hub.example.com:443 via http://127.0.0.1:33451 … Connected`. The web session's proxy re-terminates TLS (MITM) and its README lists "WebSocket upgrades" under *not supported* — but chisel's upgrade **passed** on this path. The egress itself is open (arbitrary external hosts reachable); the proxy is not an @@ -474,12 +474,12 @@ path):** `ApplicationRootUrl=localhost:8080` loads the index + client bundle, but the SPA's session/XAS calls and the `originURI` cookie misbehave across the origin mismatch. `run --local --hub` must boot the runtime with `ApplicationRootUrl` = the assigned - `https://.mxcli.org`, and the hub must serve each preview over **443 on a + `https://.example.com`, and the hub must serve each preview over **443 on a subdomain**, not a raw port. **Reference relay stood up for the test:** stock -`chisel server --reverse --tls-domain hub.mxcli.org -p 443` on a Scaleway instance, -`hub.mxcli.org` A → the instance IP, inbound 80+443 open. `mxcli tunnel-hub` is this +`chisel server --reverse --tls-domain hub.example.com -p 443` on a Scaleway instance, +`hub.example.com` A → the instance IP, inbound 80+443 open. `mxcli tunnel-hub` is this plus the registration API, wildcard-subdomain routing, and the admin overview. ### `mxcli tunnel-hub` (static) + `mxcli dev --hub` (dynamic)