Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 68 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions architecture/build.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ collect telemetry: `openshell-server` (gateway), `openshell-sandbox`
(supervisor), and `openshell-driver-vm`. Every crate depends on
`openshell-core` with `default-features = false`, so the binary crate's feature
is the single switch that enables `openshell-core/telemetry` for its build
graph. In-process drivers (`docker`, `kubernetes`, `podman`) inherit the
gateway's setting through feature unification and carry no passthrough.
graph. In-process drivers (`docker`, `kubernetes`, `podman`, `oci`) inherit
the gateway's setting through feature unification and carry no passthrough.

Building a binary with `--no-default-features` compiles out telemetry entirely:
no endpoint, no telemetry HTTP client, and no emission code. With telemetry
Expand Down
8 changes: 6 additions & 2 deletions architecture/compute-runtimes.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,14 @@ of re-querying drivers on each request.
| Podman | Rootless or single-machine deployments. | Container plus nested sandbox namespace. | Uses the Podman REST API, OCI image volumes, and CDI GPU devices when available. |
| Kubernetes | Cluster deployment through Helm. | Pod plus nested sandbox namespace. | Uses Kubernetes API objects, service accounts, secrets, PVC-backed workspace storage, and GPU resources. |
| VM | Experimental microVM isolation. | Per-sandbox libkrun VM. | Managed endpoint-backed driver. The gateway spawns `openshell-driver-vm`, waits for its Unix socket, and then consumes it through the same remote `compute_driver.proto` path used by unmanaged endpoint drivers. The VM driver boots a cached bootstrap `rootfs.ext4`, prepares requested OCI images inside a bootstrap VM with `umoci`, attaches the prepared image disk read-only, and gives each sandbox a writable `overlay.ext4` for merged-root changes and runtime material. The driver persists each accepted launch request beside the overlay and restarts those VMs on driver startup without recreating the overlay. |
| Extension | Out-of-tree drivers operated alongside the gateway. | Whatever boundary the driver implements. | Selected by a non-reserved custom `compute_drivers = ["<name>"]` entry with `[openshell.drivers.<name>].socket_path`, or at launch time by pairing `--drivers <name>` with `--compute-driver-socket=<path>`. Reserved built-in names such as `vm`, `docker`, `podman`, and `kubernetes` cannot be used as unmanaged socket endpoints. The gateway connects to a UDS the operator already provisioned, runs `GetCapabilities`, logs the advertised `driver_name`, and dispatches all sandbox lifecycle calls through `compute_driver.proto`. The driver process and socket lifecycle are operator-owned; the gateway does not spawn, supervise, or remove unmanaged extension drivers. The trust boundary is the socket's filesystem permissions: the operator must ensure only the gateway uid can read/write it. |
| OCI | Early/opt-in kernel-primitive isolation against an existing system `containerd`. | Container built from Linux namespaces + cgroups v2. | In-process driver, like Docker/Podman/Kubernetes. The driver itself — not containerd — spawns the configured `runtime_binary` (`runc` by default, `crun`, or another OCI-runtime-spec-compatible binary) directly through its `create`/`start`/`state`/`delete` CLI contract; containerd is used only for image pull/unpack and snapshot management (protected from containerd's GC via a lease, since no containerd `Container`/`Task` is ever created) and never bundled. Sandboxes are tracked by scanning the driver's own state directory rather than querying containerd or the runtime for a global list. Per-sandbox network namespace + veth pair + nftables ruleset (shared `openshell-nft-ruleset` crate, also used by the VM driver). GPU support is kernel-device-node passthrough only. Rootless mode is implemented but not yet functional (defaults off) — see `crates/openshell-driver-oci/README.md`. |
| Extension | Out-of-tree drivers operated alongside the gateway. | Whatever boundary the driver implements. | Selected by a non-reserved custom `compute_drivers = ["<name>"]` entry with `[openshell.drivers.<name>].socket_path`, or at launch time by pairing `--drivers <name>` with `--compute-driver-socket=<path>`. Reserved built-in names such as `vm`, `docker`, `podman`, `kubernetes`, and `oci` cannot be used as unmanaged socket endpoints. The gateway connects to a UDS the operator already provisioned, runs `GetCapabilities`, logs the advertised `driver_name`, and dispatches all sandbox lifecycle calls through `compute_driver.proto`. The driver process and socket lifecycle are operator-owned; the gateway does not spawn, supervise, or remove unmanaged extension drivers. The trust boundary is the socket's filesystem permissions: the operator must ensure only the gateway uid can read/write it. |

Per-sandbox CPU and memory values currently enter the driver layer through
template resource limits. Docker and Podman apply them as runtime limits.
Kubernetes mirrors each limit into the matching request. VM accepts the fields
but currently ignores them.
but currently ignores them. The OCI driver applies them as cgroup v2 CPU quota/period
and memory limits in the generated OCI spec.

Docker and Podman also accept per-sandbox driver-config mounts for existing
runtime-managed named volumes and tmpfs mounts. Podman additionally accepts
Expand Down Expand Up @@ -76,6 +78,7 @@ Runtime-specific implementation notes belong in the driver crate README:
- `crates/openshell-driver-podman/README.md`
- `crates/openshell-driver-kubernetes/README.md`
- `crates/openshell-driver-vm/README.md`
- `crates/openshell-driver-oci/README.md`

The combined VM topology runs `openshell-sandbox` as guest PID 1. libkrun
executes the driver-owned guest bootstrap as PID 1, and the bootstrap preserves
Expand All @@ -91,6 +94,7 @@ The supervisor must be available inside each sandbox workload:
| Podman | Read-only OCI image volume containing the supervisor binary. |
| Kubernetes | Supervisor image side-loaded into the sandbox pod by image volume or init container. |
| VM | Embedded in the guest rootfs bundle. |
| OCI | Bind-mounted from a host path (`supervisor_binary_path`), not sourced from an image like the other in-tree drivers — see the driver README's Known Gaps. |
| Extension | Defined by the out-of-tree driver. |

Driver-controlled environment variables must override sandbox image or template
Expand Down
9 changes: 8 additions & 1 deletion crates/openshell-core/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ pub enum ComputeDriverKind {
Vm,
Docker,
Podman,
Oci,
}

impl ComputeDriverKind {
Expand All @@ -97,6 +98,7 @@ impl ComputeDriverKind {
Self::Vm => "vm",
Self::Docker => "docker",
Self::Podman => "podman",
Self::Oci => "oci",
}
}
}
Expand Down Expand Up @@ -137,8 +139,9 @@ impl FromStr for ComputeDriverKind {
"vm" => Ok(Self::Vm),
"docker" => Ok(Self::Docker),
"podman" => Ok(Self::Podman),
"oci" => Ok(Self::Oci),
other => Err(format!(
"unsupported compute driver '{other}'. expected one of: kubernetes, vm, docker, podman"
"unsupported compute driver '{other}'. expected one of: kubernetes, vm, docker, podman, oci"
)),
}
}
Expand Down Expand Up @@ -1001,6 +1004,10 @@ mod tests {
"docker".parse::<ComputeDriverKind>().unwrap(),
ComputeDriverKind::Docker
);
assert_eq!(
"oci".parse::<ComputeDriverKind>().unwrap(),
ComputeDriverKind::Oci
);
}

#[test]
Expand Down
4 changes: 4 additions & 0 deletions crates/openshell-core/src/telemetry.rs
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,7 @@ pub enum TelemetryComputeDriver {
Kubernetes,
Podman,
Vm,
Oci,
Unknown,
}

Expand All @@ -172,6 +173,7 @@ impl TelemetryComputeDriver {
Self::Kubernetes => "kubernetes",
Self::Podman => "podman",
Self::Vm => "vm",
Self::Oci => "oci",
Self::Unknown => "unknown",
}
}
Expand All @@ -183,6 +185,7 @@ impl TelemetryComputeDriver {
"k8s" | "kubernetes" => Self::Kubernetes,
"podman" => Self::Podman,
"vm" => Self::Vm,
"oci" => Self::Oci,
_ => Self::Unknown,
}
}
Expand All @@ -194,6 +197,7 @@ impl TelemetryComputeDriver {
Some(crate::ComputeDriverKind::Kubernetes) => Self::Kubernetes,
Some(crate::ComputeDriverKind::Podman) => Self::Podman,
Some(crate::ComputeDriverKind::Vm) => Self::Vm,
Some(crate::ComputeDriverKind::Oci) => Self::Oci,
None => Self::Unknown,
}
}
Expand Down
62 changes: 62 additions & 0 deletions crates/openshell-driver-oci/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

[package]
name = "openshell-driver-oci"
description = "OCI/runc-based compute driver for OpenShell, backed by a system-provided containerd"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true

[lib]
name = "openshell_driver_oci"
path = "src/lib.rs"

[[bin]]
name = "openshell-driver-oci"
path = "src/main.rs"

[dependencies]
openshell-core = { path = "../openshell-core", default-features = false }
openshell-nft-ruleset = { path = "../openshell-nft-ruleset" }
openshell-policy = { path = "../openshell-policy" }

# containerd's own generated gRPC client. This is the "system-provided
# containerd" integration point, used only for image pull/unpack and
# snapshot management (see src/image.rs) -- never bundled, installed, or
# managed by this driver. The driver itself, not containerd, spawns the
# configured low-level OCI runtime (runc/crun/etc.) directly (see
# src/runtime.rs); containerd's container/task services and shim are
# never used.
containerd-client = "0.9"
# OCI runtime-spec and image-spec types (namespaces, cgroups v2 resources,
# seccomp, image manifest/config parsing for chain-ID computation).
oci-spec = { version = "0.10", default-features = false, features = [
"runtime",
"image",
] }

tokio = { workspace = true }
tokio-stream = { workspace = true }
tonic = { workspace = true, features = ["transport"] }
prost = { workspace = true }
prost-types = { workspace = true }
futures = { workspace = true }
clap = { workspace = true }
tracing = { workspace = true }
tracing-subscriber = { workspace = true }
miette = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
nix = { workspace = true }
rand = { workspace = true }
thiserror = { workspace = true }
sha2 = "0.10"

[dev-dependencies]
tempfile = "3"

[lints]
workspace = true
Loading
Loading