Enable TLS for TCP services (termination + backend re-encryption)#365
Draft
runleveldev wants to merge 10 commits into
Draft
Enable TLS for TCP services (termination + backend re-encryption)#365runleveldev wants to merge 10 commits into
runleveldev wants to merge 10 commits into
Conversation
Contributor
There was a problem hiding this comment.
⚠️ Not ready to approve
There are backend/UI integration bugs around blank externalHostname handling for TLS-enabled TCP services and a missing transaction parameter in port allocation that can cause race-condition failures.
Pull request overview
This PR wires up end-to-end TLS termination for TCP transport services (using an existing external-domain certificate) and surfaces the already-auto-assigned external port for TCP/UDP services in the container edit UI. It also fixes the transport port allocator’s invalid upper bound (65565 → 65535) and updates related documentation and OpenAPI schema.
Changes:
- Add TLS termination support for TCP transport services (API serialization + persistence, NGINX
stream {}generation, UI toggle + domain selection). - Display auto-assigned external ports for TCP/UDP services in the container edit form.
- Correct transport port allocation upper bound to 65535 and update docs/OpenAPI accordingly.
File summaries
| File | Description |
|---|---|
| mie-opensource-landing/docs/users/creating-containers/web-gui.md | Documents external-port visibility and TCP TLS termination behavior in the GUI. |
| mie-opensource-landing/docs/users/creating-containers/command-line.md | Updates CLI/API examples (JSON body, service map) and documents TLS-terminated TCP + external port fields. |
| mie-opensource-landing/docs/developers/system-architecture.md | Updates architecture description to reflect TCP TLS termination support. |
| mie-opensource-landing/docs/admins/core-concepts/external-domains.md | Documents reuse of domain certificates for TCP TLS termination. |
| create-a-container/views/nginx-conf.ejs | Emits listen ... ssl and cert/key directives for TLS-enabled TCP stream servers. |
| create-a-container/routers/templates.js | Eager-loads externalDomain for transport services so TLS stream config can render. |
| create-a-container/routers/api/v1/containers.js | Serializes TLS/domain fields, validates TLS constraints, persists new TLS-related transport fields, fixes port-range upper bound. |
| create-a-container/openapi.v1.yaml | Expands schema/docs for containers/services including transport TLS fields and port constraints. |
| create-a-container/models/transport-service.js | Adds externalHostname/externalDomainId fields, association, and TLS validation rules. |
| create-a-container/migrations/20260617000000-add-transport-service-domain.js | Adds DB columns for TLS-related transport-domain linkage. |
| create-a-container/client/src/pages/containers/ContainerFormPage.tsx | Adds external-port read-only display and TLS toggle + domain inputs for TCP services. |
| create-a-container/client/src/lib/types.ts | Extends transport service typing to include TLS/domain fields. |
Copilot's findings
- Files reviewed: 12/12 changed files
- Comments generated: 4
Note
Your feedback helps us improve the quality of this feature.
Please use 👍 or 👎 to tell us whether this assessment is correct.
Contributor
There was a problem hiding this comment.
⚠️ Not ready to approve
There are validation/safety gaps in newly added TLS paths (notably transport TLS domain ID parsing and domain-name sanitization/validation for NGINX/cert paths) that should be addressed before approval.
Copilot's findings
- Files reviewed: 13/13 changed files
- Comments generated: 2
Note
Your feedback helps us improve the quality of this feature.
Please use 👍 or 👎 to tell us whether this assessment is correct.
Comment on lines
+162
to
+172
| function parseTransportTls(tls, protocol, externalDomainId) { | ||
| const tlsEnabled = tls === true || tls === 'true'; | ||
| if (!tlsEnabled) return false; | ||
| if (protocol !== 'tcp') { | ||
| throw new ApiError(400, 'invalid_service', 'TLS can only be enabled for TCP services'); | ||
| } | ||
| if (!externalDomainId) { | ||
| throw new ApiError(400, 'invalid_service', 'TLS-enabled TCP services must have an externalDomainId'); | ||
| } | ||
| return true; | ||
| } |
Comment on lines
+469
to
+472
| # TLS termination at the load balancer using the external domain's | ||
| # certificate (shared with HTTP services for the same domain). | ||
| ssl_certificate /etc/ssl/certs/<%= transport.externalDomain.name %>.crt; | ||
| ssl_certificate_key /etc/ssl/private/<%= transport.externalDomain.name %>.key; |
The external port for TCP/UDP transport services is auto-assigned by the backend and already serialized by the API, but it was never shown in the UI. Surface it (read-only) in each TCP/UDP service row of the container form; new, unsaved services show "Auto-assigned on save".
Two related service features: 1. Display the auto-assigned external port for TCP/UDP services (read-only) in the container edit form. The port was already serialized by the API; this surfaces it in the UI. 2. Allow TCP services to enable TLS termination at the load balancer. A TLS-enabled TCP service references an external domain (and optional hostname, like HTTP services), and NGINX terminates TLS on the service's auto-assigned port using that domain's existing certificate (/etc/ssl/certs/<domain>.crt), then proxies the decrypted stream to the container. TLS is restricted to TCP; UDP is rejected. Changes: - migration: add nullable externalHostname + externalDomainId to TransportServices (tls column already existed) - model: add fields, externalDomain association, and validation (TLS requires tcp + a domain) - api: serialize tls/hostname/domain, eager-load externalDomain, persist TLS fields on create/update, validate via parseTransportTls() - nginx template: emit ssl + ssl_certificate for TLS-enabled TCP stream servers; plain TCP and UDP unchanged - client: extend ServiceTransport type; add TLS toggle + domain inputs and read-only external port to the service form
The auto-assigned external port range used 65565 as its upper bound, which exceeds the maximum valid TCP/UDP port (65535) and the TransportService model's own validator (max: 65535). Correct both allocation call sites to 65535.
- web-gui: note that TCP/UDP auto-assigned ports are shown in the edit form; add a "TLS for TCP services" section - command-line: document tls/externalHostname/externalDomainId fields for TCP services with an example; fix port range typo (65565 -> 65535) - external-domains: explain that a domain's certificate is reused for TLS-terminated TCP services - system-architecture: note optional TLS termination for TCP at L4
The command-line guide predated the JSON API rewrite and described
form-encoded requests, a non-existent /api/templates endpoint, the old
nodeName,templateVmid template format, and 302-redirect responses.
Rewrite it against the actual /api/v1 JSON API:
- Bearer auth and the { data } / { error } response envelope
- JSON create with services as an object map (not a form array)
- template is now a Docker image reference; node is auto-selected
- accurate endpoints: external-domains, container metadata, list/get,
async job tracking (status + SSE stream), update/delete
- TLS-terminated TCP example in JSON form
- use the {{ manager_url }} macro instead of a hardcoded host
Flesh out the v1 OpenAPI spec, which previously modeled services and
container responses as untyped objects.
- add Service / HttpService / TransportService / DnsService component
schemas matching the serialized container payload, including the
transport tls / externalHostname / externalDomainId / domain fields
- add a ServiceInput schema for create/update request bodies documenting
per-type required fields (tls is tcp-only and needs a domain)
- expand the Container schema (sshHost, httpEntries, nodeApiUrl, createdAt)
- document the container list/get/create/update/delete request and
response shapes with the { data } envelope and error cases
Validated with the Redocly OpenAPI linter (0 errors, 0 warnings).
Add a new "TLS" transport service type: like TCP, but the load balancer
connects to the backend over TLS (nginx stream `proxy_ssl on`) — the
stream analog of the HTTPS service's backendProtocol=https. The backend
certificate is not verified, so self-signed backends work.
It is stored as protocol=tcp plus a new `backendTls` boolean on
TransportService (mirroring HTTPService.backendProtocol), so it composes
with the existing client-side TLS termination flag: a TLS service may
terminate client TLS and/or re-encrypt to the backend.
- migration: add backendTls boolean to TransportServices
- model: add backendTls field + validator (proxy_ssl is TCP-only)
- api: map the `tls` request type to { protocol: tcp, backendTls: true }
via parseTransportType(), persist and serialize backendTls
- nginx template: emit `proxy_ssl on` for backendTls stream servers,
independently of TLS termination
- client: add the TLS service type, derive it from backendTls on load,
reuse the TCP UI (port + termination toggle); relabel the toggle to
"Terminate TLS at load balancer" to distinguish the two TLS concepts
- openapi + docs: document the tls type and backendTls field
Validated: client tsc + build, Redocly lint (0/0), docs build, nginx
template render for all flag combinations, and the model/type-mapping
logic.
Address PR review findings: - Normalize the optional externalHostname for TLS-enabled transport services: trim and treat blank as null. The client defaulted it to '' which failed the model's hostname regex. Add a normalizeHostname helper used by both the create and update paths, and stop sending an empty externalHostname from the client (omit when blank). - Pass the transaction to TransportService.nextAvailablePortInRange in the PUT handler. Without it the SELECT ... FOR UPDATE row lock was skipped, allowing concurrent creates to race and fail the unique (protocol, externalPort) constraint.
The hint shown on existing services referenced "Require authentication", an HTTP-only setting, even for TCP/UDP/TLS/SRV services that have no editable fields. Show the auth-specific message only for HTTP/HTTPS services and a generic "fields cannot be changed" message otherwise.
Previously existing services were immutable (only an HTTP service's authRequired could change); any other change required delete + re-add. Services only drive nginx config and DNS records — never the LXC container — so in-place edits are safe. The PUT handler now updates existing services in place: - HTTP: externalHostname, externalDomainId, backendProtocol (http/https), authRequired; a hostname/domain change rewrites the cross-site DNS record (delete old + create new) - transport: internalPort, TLS termination (tls + hostname/domain), backend TLS (tcp<->tls) - DNS: dnsName - internalPort is editable for all types The service `type` (and a transport's protocol / auto-assigned external port) remain fixed — changing those still requires delete + re-add, and is rejected with a clear error. The UI unlocks the editable fields (keeping Type and the external port read-only) and updates the hint. OpenAPI PUT description updated accordingly.
runleveldev
force-pushed
the
360-enable-tls-for-tcp-services
branch
from
0847350 to
dcb27b1
Compare
runleveldev
changed the title
runleveldev
force-pushed
the
feature/display-transport-port
branch
from
1e26a00 to
206b187
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #360. Draft.
Everything from the original TLS work except the auto-assigned port display (which is split out into #397).
1. TLS termination for TCP services
Per #360, a TCP service can enable TLS termination at the load balancer. The
tlscolumn already existed onTransportServicebut was a no-op; this wires it up end-to-end and adds a domain reference so NGINX knows which certificate to use./etc/ssl/certs/<domain>.crt, the same cert used by HTTP services for that domain) terminates TLS on the service's auto-assigned port; decrypted traffic is proxied to the container.ssl_prereadneeded.2. TLS service type (backend re-encryption)
A new TLS service type: like TCP, but the load balancer connects to the backend over TLS (NGINX stream
proxy_ssl), the L4 analog of the HTTPS service'sbackendProtocol=https. Stored asprotocol=tcp+ abackendTlsboolean, so it composes with client-side termination (a service can terminate and/or re-encrypt). Backend cert is not verified (self-signed backends work).3. Edit existing services in place
Existing services are no longer immutable. The PUT handler updates HTTP (hostname/domain/backendProtocol/auth, with cross-site DNS rewrite), transport (internalPort, TLS termination, backend TLS), and DNS (dnsName) fields in place. The service
typeand a transport's protocol/auto-assigned external port remain fixed (delete + re-add to change).4. Incidental fix
Transport port allocator upper bound
65565→65535(matches the model'smax: 65535), and the PUT port allocation now runs inside the transaction (restores theSELECT ... FOR UPDATErow lock).5. Docs + OpenAPI
web-gui, command-line, external-domains, system-architecture updated; OpenAPI fleshed out with detailed Service/HttpService/TransportService/DnsService/ServiceInput schemas including the
tls/backendTls/domain fields.Commits (9)
Rebased onto the port-display branch, preserving individual commits (TLS termination, port-range fix, docs, OpenAPI, TLS service type, review fixes, hint wording, in-place editing).
Testing notes
tsc -b --noEmit+vite buildpass; backendnode --checkpasses; Redocly OpenAPI lint 0/0; model validators + type-mapping unit-checked.sqlite3needsGLIBC_2.38). Runnpx sequelize-cli db:migratebefore testing.