Skip to content

feat: an app UI instance per tenant - #62

Merged
TheMeinerLP merged 10 commits into
mainfrom
feat/per-tenant-app-instance
Aug 17, 2026
Merged

feat: an app UI instance per tenant#62
TheMeinerLP merged 10 commits into
mainfrom
feat/per-tenant-app-instance

Conversation

@TheMeinerLP

Copy link
Copy Markdown
Contributor

Every tenant gets its own deployed instance of the tenant application, served at
https://<host>/t/<tenant>/. One image, N deployments, one environment variable.

This is the second of four subsystems the larger goal decomposes into. The first —
per-tenant options with lock and override — shipped in 0.8.0. Teams and users through Entra,
and impersonation, are still out of scope here and get their own specs.

Design: docs/superpowers/specs/2026-08-16-per-tenant-app-instance-design.md
Plan: docs/superpowers/plans/2026-08-16-per-tenant-app-instance.md

What the operator creates

When tenantUi.host is set — and it is empty by default, which switches the whole feature off —
TenantReconciler adds three objects to each tenant's own namespace:

Resource Notes
Deployment one replica, the apus/ui image, NUXT_APP_BASE_URL=/t/<name>/
Service ClusterIP, http → 8080
Ingress <host> + path /t/<name>, pathType: Prefix

The Ingress has to be per-tenant and in the tenant's namespace: an Ingress may only
reference a Service in its own namespace. A single operator-owned Ingress listing every tenant
was never an option.

Nothing in apps/app changed. buildOidcRedirectUris(origin, baseURL) already derives the
callback from the runtime base path, and its tests already covered a nested prefix.

Four findings that shaped this, two of which contradicted the first draft

One image serves any prefix. NUXT_APP_BASE_URL=/t/acme/ → the built Nitro server serves its
shell and deep links under that prefix, emits assets at /t/acme/_nuxt/…, and 404s on the bare
root. Verified against the actual image before any of this was designed.

Entra permits a wildcard redirect URI here — and it would break the login. Microsoft's docs:
"when a configured wildcard URI matches a redirect URI, query strings and fragments in the
redirect URI are stripped"
. The authorization code lives in that query string. So two explicit
URIs per tenant
, and the 256-URI registration limit is the real ceiling: ~128 tenants.

An earlier attempt to settle this empirically appeared to succeed — and so did the control case
against an unrelated host, because Entra renders its sign-in page before validating the redirect
URI at all. That test proved nothing; the documentation is what settles it.

Path ordering is safe, but not for the reason the platform chart's comment gives. The ingress
class here is cloudflare-tunnel, not nginx. That controller flattens every Ingress in the
cluster into one rule list and sorts it globally by path length descending
(sortIngressRules in pkg/cloudflare-controller/tunnel-client.go), so /t/acme lands ahead of
/ no matter which object declared it. Read from the controller's source, since no host in this
cluster is currently served by two Ingress objects.

A pod in a tenant namespace must declare resource requests or it is never created. The
namespace's ResourceQuota constrains requests.cpu/requests.memory and the LimitRange
beside it is empty (spec={"limits":null} on the live cluster). A Deployment without requests is
accepted and then produces no pod — a healthy-looking Deployment stuck at zero replicas.

Proven with kubectl apply --dry-run=server against a real tenant namespace, which runs the
admission plugins and creates nothing:

pods "..." is forbidden: failed quota: apus-tenant:
  must specify requests.cpu for: ui; requests.memory for: ui

A defect that verification caught

The same dry-run warned that the probe pod violated PodSecurity "restricted". The platform chart
hardens its own ui Deployment — runAsNonRoot, uid 65532, RuntimeDefault seccomp, no
privilege escalation, read-only root, all capabilities dropped — and the first version of this
builder applied none of it to the same image.

Only a warning on this cluster, so nothing would have broken. It would simply have been the same
software running less restricted because a controller created it rather than Helm, and it would
have become a hard rejection the moment tenant namespaces were set to enforce. Fixed, and the
hardened shape now passes the same dry-run with no warning at all.

The manual step, said out loud in three places

Each instance needs two redirect URIs registered before anyone can sign in:

https://<host>/t/<tenant>/auth/callback
https://<host>/t/<tenant>/auth/silent-renew

The operator cannot add them — that needs Microsoft Graph application permissions on the app
registration. And a missing registration does not fail at deploy time: it fails at someone's first
sign-in, at the identity provider (AADSTS50011), with nothing whatsoever in this cluster's logs.

So it is stated where someone will actually meet it: on Tenant.status.redirectUris, in the
chart's NOTES.txt (only when a host is configured), and on the console's tenant page with a copy
action — in front of the person who just created the tenant.

Verification

:operator:test passes — 12 builder cases, 5 new reconciler cases, 5 config cases
:api:test passes, incl. the two redirectUris cases
spotlessCheck clean
pnpm lint / typecheck clean
pnpm test 172 (143 core + 18 app + 11 console)
Chart rendered both ways with helm template; values.schema.json rejects an apiBaseUrl ending in /api — the exact mistake that cost a debugging session once
Quota + PodSecurity --dry-run=server against a real tenant namespace, in all three shapes

One gap, stated rather than hidden: the new k3s integration test — which waits for a Pod
rather than a Deployment, since only a cluster with quota admission can fail that — compiles but
has not been executed. There is no Docker on this machine, and *IntegrationTest classes run in
neither check nor CI (pre-existing for the two that were already there). The claim it would
prove was instead verified directly against the live cluster, as above.

🤖 Generated with Claude Code

https://claude.ai/code/session_019Bff5mpWkUnZA77jys8DiR

TheMeinerLP and others added 10 commits August 16, 2026 23:40
Every tenant gets its own deployed instance of the tenant application under
/t/<tenant>/ on the existing host, provisioned by TenantReconciler.

Four findings settled the design, each read from the source or the cluster
rather than assumed -- two of them contradicted the first draft:

- one image serves any prefix via NUXT_APP_BASE_URL, verified against the
  built image, so this is one env var and not a per-tenant build
- Entra permits wildcard redirect URIs here but strips the query string when
  one matches, which is where the auth code lives; two explicit URIs per
  tenant it is, ~128 tenants against the 256-URI limit
- the ingress class is cloudflare-tunnel, not nginx: it flattens every
  Ingress in the cluster into one rule list sorted by path length descending,
  which is what actually makes a separate per-tenant Ingress work here
- a tenant namespace's ResourceQuota makes resource requests mandatory and
  its LimitRange is empty (spec.limits: null on the live cluster), so a
  Deployment without requests would be created and never produce a pod

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Bff5mpWkUnZA77jys8DiR
Seven tasks: TenantUiConfig, a pure TenantUiResourceBuilder, the redirect
URIs on Tenant.status, the reconciler, the chart, the console display, and a
verification pass that insists on a real API server for the one thing the
mock cannot check -- that the namespace quota actually admits the pod.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Bff5mpWkUnZA77jys8DiR
TenantUiConfig carries the seven settings a per-tenant instance of the tenant
application needs, read from APUS_TENANT_UI_* the same way the rest of
OperatorConfig reads its environment.

A blank host means the feature is off, and off is the default -- asserted, not
assumed: an operator that started provisioning a Deployment per tenant on a
plain upgrade would be a surprise nobody asked for.

The four NUXT_PUBLIC_* values are modelled one by one rather than as a map. A
map would have to be serialised through a single environment variable and would
lose its schema, its documentation, and the ability to test each value.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Bff5mpWkUnZA77jys8DiR
TenantUiResourceBuilder turns a Tenant into the Deployment, Service and Ingress
that serve it its own instance at https://<host>/t/<tenant>/. Pure function,
following HostingResourceBuilder; labels and the owner reference are passed in
so they cannot drift from what TenantReconciler stamps on everything else.

Three things here are load bearing and each has a test that says why:

- resource requests, because the tenant namespace's quota makes them mandatory
  and its limit range supplies no default -- without them the Deployment is
  created and never produces a pod, which no mock API server would catch
- probes on /t/<tenant>/ rather than /, because the bare root 404s once
  NUXT_APP_BASE_URL is set and a root probe would restart a healthy pod forever
- the Ingress being per-tenant and in the tenant's namespace, because an
  Ingress may only reference a Service in its own namespace

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Bff5mpWkUnZA77jys8DiR
Tenant.status.redirectUris carries the two URIs the identity provider must have
registered before anyone can sign in to a tenant's own instance. The operator
cannot register them itself -- that needs Microsoft Graph application
permissions on the app registration -- and a missing registration fails at
sign-in with AADSTS50011 from the broker, leaving nothing in this cluster's
logs. So kubectl answers the question instead.

The list absorbs null and starts empty, both asserted: every reader would
otherwise have to guard, and the console must render nothing at all rather than
an empty box for a tenant that has no instance.

The regenerated CRD adds four lines and nothing else. The generator does not
emit the chart's hand-added helm.sh/resource-policy: keep annotation, so it is
restored -- without it a helm uninstall would delete every Tenant in the
cluster.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Bff5mpWkUnZA77jys8DiR
TenantReconciler now creates a Deployment, Service and Ingress in each tenant's
namespace, serving that tenant its own instance of the tenant application at
https://<host>/t/<name>/, and reports the two redirect URIs that still have to
be registered by hand.

Off unless a host is configured, which is the default and the first thing the
tests assert -- an operator that started provisioning a pod per tenant on a
plain upgrade would be a surprise nobody asked for. Switching the feature back
off clears status.redirectUris rather than leaving it advertising an instance
that no longer exists.

The instance's labels are deliberately not the tenant label set every other
resource here carries: they double as the Deployment's pod selector, and two
workloads in one namespace sharing a selector would each take the other's pods.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Bff5mpWkUnZA77jys8DiR
A tenantUi value block renders the APUS_TENANT_UI_* variables the operator
reads. Off by default -- an empty host, which the operator treats as the
feature being switched off entirely.

The RBAC needed nothing: the operator already has deployments, services and
ingresses from the hosting path. Verified rather than assumed, since a missing
verb produces a clean-looking reconcile and no resources.

values.schema.json rejects an apiBaseUrl ending in /api. That exact mistake
already cost a debugging session once: the suffix produces /api/api/tenants,
which the ingress routes to the API, which has no such route, and the security
filter answers a bare 403 that reads like a missing role.

NOTES.txt prints the manual Entra step, but only when a host is configured.
Both directions were rendered and checked, not just the interesting one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Bff5mpWkUnZA77jys8DiR
TenantResponse gains redirectUris, straight from the status the operator
writes, and the console's tenant page shows both with a copy action.

This is a to-do item rather than a status readout, and the page is the only
place it can be raised in time. The operator cannot register these URIs -- that
needs application permissions on the app registration nobody has granted -- and
a missing registration does not fail at deploy time. It fails at someone's
first sign-in, at the identity provider, with nothing in this cluster to find.
The person who can still act on it is the one who just created the tenant.

Nothing renders for a tenant with no instance, asserted: an empty 'Redirect
URIs' section reads like something failed to load.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Bff5mpWkUnZA77jys8DiR
…he same image

Verifying the quota claim against the real cluster with a server-side dry-run
proved it exactly -- 'must specify requests.cpu for: ui; requests.memory for:
ui' without them, admitted with them -- and the same output carried a
PodSecurity 'restricted' warning that caught a defect I had introduced.

The platform chart hardens its own ui Deployment: runAsNonRoot, uid 65532,
RuntimeDefault seccomp, no privilege escalation, read-only root, all
capabilities dropped. The per-tenant instance runs that identical image and had
none of it. Only a warning on this cluster, so nothing would have broken -- it
would just have been the same software running less restricted because a
controller created it rather than Helm, until someone set tenant namespaces to
enforce and every tenant pod started being rejected.

The hardened shape passes the same dry-run with no warning at all.

Also adds the k3s integration test that waits for a Pod rather than a
Deployment, since only a cluster with quota admission can fail that. It sits
with the module's other *IntegrationTest classes, which need Docker; it
compiles but has not been executed here, and neither check nor CI runs them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Bff5mpWkUnZA77jys8DiR
markdownlint MD032 -- twelve times, all the same shape: a **Files:**/
**Interfaces:** label followed straight by a bullet list. Verified locally with
markdownlint-cli2 this time rather than by pushing and waiting.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Bff5mpWkUnZA77jys8DiR
@github-actions

Copy link
Copy Markdown
Contributor

Test results

603 tests   603 ✅  50s ⏱️
 86 suites    0 💤
 86 files      0 ❌

Results for commit 05bb0fb.

@TheMeinerLP
TheMeinerLP merged commit e0d87d1 into main Aug 17, 2026
10 checks passed
@TheMeinerLP
TheMeinerLP deleted the feat/per-tenant-app-instance branch August 17, 2026 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant