fix(build): raise file limit for native gateway builds - #3122
Open
purp wants to merge 16 commits into
Open
Conversation
Signed-off-by: Jim Meyer <jimeyer@nvidia.com>
Collaborator
Author
|
LGTM. Moving to ready for review. |
purp
marked this pull request as ready for review
September 2, 2026 17:04
purp
requested review from
a team,
derekwaynecarr,
mrunalp and
sjenning
as code owners
September 2, 2026 17:04
johntmyers
reviewed
Sep 2, 2026
| @@ -54,7 +50,7 @@ ensure_build_nofile_limit() { | |||
| esac | |||
|
|
|||
| if [ "${target}" -gt "${current}" ] && ulimit -n "${target}" 2>/dev/null; then | |||
Collaborator
There was a problem hiding this comment.
nit: Since this runs for building a long-running(ish) gateway, should we only adjust the soft limit? previously this helper only impacted short-lived compilation tasks but how that it would impact a longer running process like the gateway we wouldn't want to lower what might be an already set higher hard limit
* feat(helm): split gateway and workspace charts Signed-off-by: Dhiraj Bokde <dbokde@nvidia.com> * fix(helm): preserve split chart upgrade compatibility Keep workspace manifests valid after value validation and default legacy reused values to the combined resource topology. * fix(ci): preserve VM runtime for E2E The Rust cache restores target/ after VM runtime artifacts are staged, overwriting target/vm-runtime-compressed before openshell-driver-vm is built. Stage the compressed runtime outside target and pass that location through OPENSHELL_VM_RUNTIME_COMPRESSED_DIR so build.rs can embed the supervisor. Also locate the Helm split-ownership test repository root from the script path rather than git rev-parse. The test runs in a container where the GitHub checkout can be owned by a different UID and rejected as dubious ownership. Signed-off-by: Dhiraj Bokde <dbokde@nvidia.com> * fix(ci): install yq for Helm ownership test The split-chart ownership regression uses yq to inspect rendered YAML, but the Helm CI container installs only tools declared in mise. Declare and lock yq so mise install --locked provides the test dependency. Signed-off-by: Dhiraj Bokde <dbokde@nvidia.com> --------- Signed-off-by: Dhiraj Bokde <dbokde@nvidia.com>
Closes NVIDIA#3112 Reuse the dashboard workspace cycle action from the providers pane, advertise the shortcut, and cover the state transition and rendered hint. Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
Signed-off-by: John Myers <johntmyers@users.noreply.github.com> Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(server): add sandbox workload templates Signed-off-by: Gordon Sim <gsim@redhat.com> * feat(go-sdk): add sandbox workload template support Signed-off-by: Gordon Sim <gsim@redhat.com> * feat(rust-sdk): add sandbox workload template support Signed-off-by: Gordon Sim <gsim@redhat.com> * feat(python-sdk): add sandbox workload template support Signed-off-by: Gordon Sim <gsim@redhat.com> * feat(typescript-sdk): add sandbox workload template support Signed-off-by: Gordon Sim <gsim@redhat.com> * docs(agents): document sandbox workload templates Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(cli): support default GPU requests in sandbox templates Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(server): cap sandbox templates per workspace Signed-off-by: Gordon Sim <gsim@redhat.com> * docs(architecture): document sandbox workload template boundaries Signed-off-by: Gordon Sim <gsim@redhat.com> * feat(cli+sdk): expose sandbox workload template provenance Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(cli): include sandbox template annotations in output Signed-off-by: Gordon Sim <gsim@redhat.com> * feat(sandbox): add label selectors to template listing Signed-off-by: Gordon Sim <gsim@redhat.com> * test(e2e): cover sandbox template failure paths Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(server): preserve command and ttl when creating sandbox from template Signed-off-by: Gordon Sim <gsim@redhat.com> * test(server): add field coverage test for template merge Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(go-sdk): add pagination support to fake client Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(cli): warn on env vars that looks like secrets Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(sdk-ts): propagate sandbox workspace through lifecycle calls Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(docs): update workspace management docs Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(ts-sdk): support command and tty when creating from template Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(go-sdk): support command and tty when creating from template Signed-off-by: Gordon Sim <gsim@redhat.com> * test(python-sdk): verify command and tty handling when creating from template Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(go-sdk): update docs and ClientInterface Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(server): validate sandbox create specs before I/O Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(go-sdk): guard empty DNS-1123 label validation Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(python-sdk): allow empty template builder mappings Signed-off-by: Gordon Sim <gsim@redhat.com> * fix(cli): align template GPU JSON default output Signed-off-by: Gordon Sim <gsim@redhat.com> --------- Signed-off-by: Gordon Sim <gsim@redhat.com>
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
* fix(test-guest): support Nix stat on macOS Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(guest): version Ubuntu test guest profiles Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(guest): consolidate rootless Podman provisioning Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Evan Lezar <elezar@nvidia.com>
* ci: add Fedora conformance workflow Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci: enable KVM for Fedora conformance Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci: run Fedora conformance with KVM Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci: copy Fedora conformance script into guest Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci: limit conformance VM setup to KVM Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci: streamline Fedora conformance builds Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci: use dev supervisor for Fedora conformance Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci: pin Fedora RPM build image Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci: cache RPM vendoring dependencies Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci: clarify Fedora conformance job name Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci: make conformance workflow manual only Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci(conformance): use new podman rootless install * fix(ci): build gateway from renamed package Signed-off-by: Simon Scatton <sscatton@nvidia.com> --------- Signed-off-by: Simon Scatton <sscatton@nvidia.com>
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
…dboxes (NVIDIA#3090) * feat(vm): support corporate HTTP forward proxy egress for microVM sandboxes The corporate forward proxy machinery from NVIDIA#1792 is driver-agnostic and already merged: openshell-supervisor-network implements CONNECT chaining, NO_PROXY matching, credentials, https:// proxies and corporate CA trust, and openshell-sandbox exposes it as six argv-only flags. Podman gained the driver half in NVIDIA#2245/NVIDIA#2512 and Kubernetes in NVIDIA#2633; the VM driver had none of it, so VM sandboxes on proxy-only networks could not reach any destination requiring the proxy even when policy allowed it. The blocking piece was not proxy logic but delivery: the VM guest init script runs as PID 1 and execs a fixed supervisor command line, and libkrun's krun_set_exec receives an empty argv, so there was no channel for driver-owned supervisor arguments. The supervisor's proxy flags deliberately have no environment fallback, and build_guest_environment merges user-supplied environment, so the guest env is not a safe transport either. Add a driver-authored argument file, mirroring the existing init.d manifest: the driver writes /opt/openshell/supervisor-args into the overlay upperdir on every launch and the guest reads it verbatim, one argument per line, appending it to every supervisor exec. It is written even when empty, which is what makes the channel unforgeable -- the upperdir always shadows the read-only image layer, so an image can neither supply its own arguments nor disable the operator's by omitting the file. Because both launch backends exec the same init script, this covers libkrun and QEMU without touching either. A microVM has no bind mounts or container secrets, so the credential and CA bundle are staged into the per-sandbox overlay the way the gateway JWT already is: credential root-only at 0600, CA at 0644, both rewritten every launch so a removed setting clears prior material, and both deleted with the sandbox state directory. This places the credential at rest in the overlay image on the gateway host, which differs from the Podman secret model and is documented as an explicit security consideration. Validation is fail-closed and shared: a new openshell_core::driver_utils::validate_upstream_proxy_settings holds the pairing rules the Podman driver established, and both the gateway and the driver call it so an invalid table names the offending key instead of surfacing as an opaque driver-readiness timeout. Guest egress leaves through gvproxy, so a proxy on the gateway host's loopback is reachable only through host.openshell.internal; the guest to gateway callback is unaffected. Closes NVIDIA#3088 Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(vm): bound the proxy CA read and scope the host-loopback recipe Two review findings on the corporate forward proxy support for microVM sandboxes. The driver read the operator's proxy_ca_bundle with an unbounded fs::read and accepted it on a substring match for the PEM BEGIN CERTIFICATE marker. A special file such as /dev/zero therefore grew driver memory without bound on every authorized sandbox create, and a PEM block holding invalid DER passed the host check but contributes no trust anchor in the guest, so every supervisor would fail after boot with an error attributed to the sandbox rather than to the setting. Move the read into openshell-core as read_upstream_proxy_ca_bundle_file: it reuses the credential reader's bounded-read path (non-regular files rejected on fstat, size capped, read bounded even if the file grows), then requires at least one anchor that RootCertStore::add_parsable_certificates accepts. The supervisor's own reader now delegates to it, so host acceptance and guest acceptance are the same function and cannot drift. The published host-loopback recipe was written for libkrun only. gvproxy NATs host.openshell.internal to the gateway host's 127.0.0.1, but GPU sandboxes run on the QEMU/TAP backend where that name resolves to the TAP host address and the driver's own nftables input chain accepts only the gateway port from the guest — no proxy on the gateway host is reachable there at any bind address, so an operator following the generic recipe lost all proxy-required egress while configuration validation succeeded. Scope the recipe to libkrun in every reference and reject a gateway-host proxy URL when a launch plan resolves to QEMU, naming the reason, instead of booting a sandbox whose policy-approved CONNECTs all time out. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(vm): match the QEMU proxy preflight to the selected TAP host The gateway-host proxy guard added for the QEMU/TAP backend classified the wrong set of addresses in both directions. It ran at the top of configure_qemu_launch_plan, before the subnet allocation that settles plan.host_ip, so it could not compare against the address the guest actually reaches the host on. An operator pointing https_proxy at the sandbox's own TAP host address, such as 10.0.128.1, passed the check, and the driver's nftables input chain — which accepts only the gateway port from the guest — then dropped every policy-approved CONNECT, which is exactly the silent timeout the guard exists to prevent. In the other direction it rejected 192.168.127.254 unconditionally. That address is special only to libkrun/gvproxy; on QEMU/TAP it is an ordinary address that may be routable through the guest's masqueraded egress, so the guard refused a working configuration. Run the check after the launch plan's network allocation, on both the freshly-allocated and already-complete paths, and compare IP literals with that sandbox's selected TAP host. Loopback literals, localhost, and the documented host aliases that write_host_gateway_aliases seeds to the TAP host still classify as the gateway host, and the failure names the address. The gvproxy host-loopback constant returns to being a documentation anchor. Signed-off-by: Philippe Martin <phmartin@redhat.com> --------- Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(cli): require ANSI-capable terminal before colorizing Follow-up to NVIDIA#3026, raised in review. `auto` treated any terminal as styleable, so `TERM=dumb openshell ...` still emitted escapes into a terminal that renders them literally. An unset TERM had the same problem. This is partly a regression that NVIDIA#3026 introduced. `console`, which drives indicatif and dialoguer, already refused to colorize when TERM is `dumb` or unset, and miette applies the same check through supports-color. NVIDIA#3026 overrides both with its own switch, so it replaced two working checks rather than only failing to add one. tracing and the owo-colors wrapper never had detection, so those two are a gap rather than a regression. Add the capability check to the `auto` branch only, matching console's unix rule: `dumb` is not capable, and an unset TERM is not capable because nothing identifies a capable terminal. Empty is treated as unset, which diverges from console — it reads `TERM=""` as capable since the value is not `dumb` — because an empty value names no terminal type and every other variable here already treats empty as unset. Because the check sits after the explicit branches, `--color always` and FORCE_COLOR still force styling on a dumb terminal, and `--color never` and NO_COLOR still suppress it on a capable one. TERM is a unix signal; Windows consoles enable virtual terminal processing and do not set it, so the check does not apply there. The existing pty test now pins TERM. It previously inherited the ambient value, which would make its outcome depend on the environment now that capability is consulted — CI runners frequently leave TERM unset. Signed-off-by: Mrunal Patel <mrunalp@gmail.com> * refactor(cli): combine stream and terminal capability checks Signed-off-by: Evan Lezar <elezar@nvidia.com> * docs(cli): clarify table color behavior Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(cli): cover redirected status table colors Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Mrunal Patel <mrunalp@gmail.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> Co-authored-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
* feat(skills): separate public and contributor workflows Closes NVIDIA#2736 Publish the four user-facing OpenShell skills from the top-level skills directory, mark contributor workflows internal, and update portability guidance, validation, and documentation. Signed-off-by: Johnny Greco <jogreco@nvidia.com> * docs(skills): clarify public skill audit scope Signed-off-by: Johnny Greco <jogreco@nvidia.com> * docs(skills): use markdown documentation links Signed-off-by: Johnny Greco <jogreco@nvidia.com> * docs(skills): align public and contributor guidance Signed-off-by: Johnny Greco <jogreco@nvidia.com> --------- Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
* fix(cli): preserve provider type on credential updates provider_update built every UpdateProviderRequest with an empty type and profile workspace. Policy interceptors evaluate the request before the gateway merges it with stored state, so they cannot tell the target is a profile-managed provider and deny credential-only updates. Creating a provider from an interceptor-vended profile works; rotating its credentials does not. Look up the stored provider and forward its type and profile workspace verbatim. The gateway treats profile_workspace as immutable, so the stored value is passed through rather than recomputed from the request workspace. The read is best-effort: a caller holding provider:write without provider:read must still be able to rotate credentials, so a denied read falls back to the previous empty metadata rather than failing the update. --from-existing and --from-oidc-token need the stored type, so they surface the error instead. This also removes a duplicate GetProvider call, since the oidc-token and from-existing paths each fetched the same provider separately. Signed-off-by: Nathan DeMoss <ndemoss28@gmail.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Signed-off-by: Nathan DeMoss <ndemoss28@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Raise the macOS per-process file-descriptor limit before native local gateway builds so sccache does not fail with Too many open files.
Related Issue
No issue required: obvious localized macOS build reliability fix; generalizes the existing #2307 file-descriptor-limit safeguard to native gateway builds.
Changes
Testing
Checklist