Add redis workload with OCI image - #270
Draft
henrybear327 wants to merge 26 commits into
Draft
Conversation
elfuse_launch owns guest bring-up: guest_bootstrap_prepare, the FUSE-temp unlink, the sysroot casefold probe, vCPU creation, GDB init/sync/wait, the run loop, gdb_stub_shutdown, the shim counter and syscall histogram dumps, and guest_destroy. main() retains the original CLI argv (proctitle rewriting), option parsing, sysroot provisioning, the shebang loop, the --gdb x86_64 guard, host cwd, and the heap resource cleanup, and now hands off through launch_args_t so other launchers (the OCI run helper) can share one bring-up path. The launch_args_t envp field generalizes the old hard-coded environ: NULL keeps the host environment, so main()'s behavior is unchanged. Bring-up failures unwind through a single fail label instead of repeating the guest_destroy-plus-unlink tail at every error site. Ownership of the FUSE-materialized temp ELF moves with the bring-up: elfuse_launch owns the unlink from the prepare call onward (teardown and the post-prepare error paths), and main() drops its claim before handing off, so main's shared goto unwind cannot double-unlink a path whose ownership has been transferred. The embedded shim blob include moves along with its only consumer, so shim_bin has a single object definition site. With the guest's whole lifetime inside elfuse_launch, main() no longer tracks one: the guest_t and its initialized flag are gone, and cleanup_main_resources drops the guest_destroy branch that could never fire from that call site. It now releases only what main() owns.
henrybear327
changed the title
henrybear327
marked this pull request as draft
August 5, 2026 18:09
henrybear327
force-pushed
the
oci/redis-workload
branch
from
August 5, 2026 18:21
54916ef to
287881f
Compare
henrybear327
marked this pull request as ready for review
August 5, 2026 18:21
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
An OCI image front end needs to set the guest identity, working directory, and environment without patching the runtime; the new flags map onto launch_args_t fields and `elfuse-oci run` drives exactly this interface. The --user identity is staged before bring-up (proc_set_initial_ids) so the auxv AT_UID/AT_GID snapshot taken by build_linux_stack matches what getuid()/getgid() later report. --workdir rejects non-absolute paths up front instead of silently resolving them against the host cwd, and is applied by elfuse_launch after the casefold probe so the translation sees the sysroot's real case behavior. --env/--clear-env build the guest environment with env(1) semantics: KEY=VAL sets, bare KEY inherits from the host environ, --clear-env starts empty; with neither flag given envp stays NULL and the host environ is used unchanged. The new heap resources join main()'s shared goto unwind: envp, workdir, and the raw --env override array are released at the single cleanup label on every exit path. --fakeroot and a non-root --user are refused together. Fakeroot means the guest starts as uid/gid 0, and the setuid permission check grants every id switch on that basis; a non-root --user would leave that grant in place while the guest reported an unprivileged uid, so the guest could raise itself back to root at will. tests/test-launch-flags.sh covers the refusal along with the --workdir and --user parse rules. The parse-error unwind frees the ELF and sysroot path copies too, so the sysroot-too-long branch reaches it instead of repeating the frees.
henrybear327
force-pushed
the
oci/redis-workload
branch
from
August 5, 2026 19:21
287881f to
57b35c0
Compare
elfuse-oci is a standalone Go binary that owns the OCI image pipeline; elfuse itself stays a pure Linux syscall-to-Darwin runtime with no OCI commands. This first slice is the acquisition half: an OCI image-layout store plus the pull and inspect commands, built on go-containerregistry ($ELFUSE_OCI_STORE or ~/.local/share/elfuse/oci by default). The store is a spec-shape image layout other tools can read, with a refs.json pin table mapping references to manifest digests. An exclusive flock serializes refs.json/index.json updates so concurrent pulls cannot lose pins (the cold-store bootstrap of oci-layout and index.json runs under the same lock, double-checked so a warm store skips it), pin persistence syncs the temp file and directory around the rename, and a nil-object refs.json is rejected as corrupt instead of treated as empty. addImage distinguishes genuinely-absent from unreadable descriptors by scanning index membership, so store corruption surfaces rather than duplicating entries. digestFor returns a distinct errNotPulled for a missing ref so later callers can tell "not pulled" from "store broken". Two helpers keep the plumbing in one place: every subcommand opens the store through commonFlags.openResolvedStore (resolve the store path, then open the layout), and store.withLock scopes lock-held sections; pin and addImage wrap their load-modify-save cycles in it so a critical section cannot leak its lock on an error path. pull resolves the requested platform (default linux/arm64) and validates --platform shape up front; inspect prints the manifest and config summary (or --json) and propagates digest/size and writer errors instead of reporting partial output as success. Credentials come from the ambient default keychain, but its resolution is time-bounded: it shells out to whatever helper the Docker config names, and go-containerregistry drops the context around that exec, so a wedged helper would otherwise hang the pull with no output. A wrapper keychain caps the wait and fails with an explanation and the DOCKER_CONFIG escape hatch instead; a progress line is printed before the pull so it is never silent. --platform is registered per command rather than in the shared flag set, so a subcommand that cannot honor it rejects the flag instead of silently discarding target selection. The test output-capture helper closes its pipe ends in a defer, so a callee that panics or Fatals cannot park the reader goroutines on open write ends. Tests cover the pin table, store locking, error kinds, flag parsing, and the command dispatch, with cranePull as a swappable seam so no test touches the network. `make all` builds elfuse-oci when a Go toolchain is on PATH and skips it with a notice otherwise, so a Go host gets both binaries by default while a C-only host still builds. The Go rule depends on the same version metadata as the C build, so an incremental rebuild restamps --version after a checkout or commit.
Apply a stored image's layers into a rootfs directory, whiteouts and all, so `unpack` (and later `run`) can materialize a filesystem tree from the store without any external tool. Layer application is hardened against hostile archives: extraction is bounded by os.Root so no entry, symlink, or hardlink escapes the destination; member names and hard-link targets archived absolute (GNU tar -P builders) are applied root-relative, as other OCI consumers do; parent components are Lstat'd and a symlinked or non-dir intermediate is replaced with a real directory (containerd/Docker behavior); opaque whiteouts clear through a real directory only, are order-independent within a layer, and an invalid bare .wh. entry fails extraction instead of deleting its parent; a plain whiteout whose parent chain is not all real directories is a no-op, so it cannot remove through a lower layer's symlink; a directory entry replaces a lower-layer non-directory. File bodies are bounded by the tar header size and short bodies are an error, and permissions are finalized with an explicit chmod so the host umask cannot skew modes. The decompressor is drained past tar's end-of-archive marker, so a corrupted gzip trailer fails the unpack instead of vanishing with the discarded Close error. setuid/setgid/sticky bits are re-applied where the host allows it, and degrade gracefully where it does not. An unprivileged chmod that sets setuid/setgid is rejected with EPERM on macOS when the unpacked file's inherited group is one the invoking user is not in (a new file takes its parent directory's group under BSD semantics, e.g. wheel under /tmp, not the tar's root/shadow owner). Since the rootfs is owned by the invoking user those bits could not be honored at runtime there anyway, so they are dropped with a warning naming the lost bit rather than aborting the whole unpack, which is what lets Debian-family images and their shadow suite (chage, passwd, ...) unpack at all. Reported at sysprog21#191 (comment) Unpack stages into a temp sibling directory and renames into the final cache path (keyed by manifest digest under <store>/rootfs/), so a concurrent reader never observes a partial tree and the loser of a rename race adopts the winner's complete one. A failed unpack removes only what it created. Tests drive whiteouts, opaque ordering, parent-symlink replacement, the whiteout-through-symlink no-op, trailer corruption, hardlink identity, exact-size reads, mode preservation, the special-bit degrade decision, and the staged-rename semantics over synthetic layer tarballs. Two malformed-input rules the layer format needs. A whiteout suffix that collapses to a dot name (".wh..", ".wh...") is rejected: Join folds it into the containing directory or its parent, turning the removal of one named entry into a subtree wipe. And a store-managed rootfs cache path that is not a real directory is refused, because os.OpenRoot follows a symlink in the directory name it opens, so a symlink planted at the digest path would redirect the whole extraction out of the store; an explicit --rootfs still follows links, as its merge-in-place contract requires.
run is the last pipeline stage: resolve the image config into a concrete
runspec, materialize the rootfs, and exec the existing `elfuse --sysroot
<rootfs> ...` positional launch path, reusing elfuse's HVF bring-up,
shebang, and dynamic-linker plumbing rather than reinventing guest
launch. elfuse is located as a sibling binary ($ELFUSE_BIN overrides for
tests) and replaced via exec so the shell reaps the same pid and
terminal signals reach the guest directly.
The runspec resolves Entrypoint/Cmd/Env/User/WorkingDir with the usual
precedence (--entrypoint drops image Cmd; --env and --clear-env follow
env(1) semantics; --workdir must be guest-absolute). A PATH is
guaranteed: when neither the image config nor --env supplies one,
Docker's conventional default is appended, so a guest whose image omits
PATH still has a search path after the --clear-env launch. A relative
path command resolves against the working directory and a bare name
against the merged PATH inside the image rootfs, following Docker's
exec-form rules (elfuse resolves the initial ELF before applying
--workdir and does no PATH lookup, so the launcher must); a config-only
image WorkingDir no layer ships is created at run time, as runc does.
Non-absolute PATH elements follow the POSIX rule runc inherits: an
empty element names the working directory and a relative one joins it,
both resolved inside the rootfs like every other candidate. The
workdir is cleaned before use: the guest path resolver folds
doubled slashes and clamps /.. at /, so a config WorkingDir the
runtime accepts cannot fail the pre-launch mkdir. A user --env with an
empty variable name is rejected up front, where elfuse itself would
reject it only after the rootfs work. A
symbolic --user or image User is resolved against the image's own
/etc/passwd and /etc/group through os.Root-bounded, no-follow opens, so
a crafted rootfs cannot redirect resolution to host account files.
run auto-pulls only when the ref is genuinely absent (errNotPulled) and
surfaces store corruption instead of masking it behind a network pull;
an explicit --platform must match the pinned image so a ref pulled for
another architecture is not silently launched. Before exec, host-truth
/etc/{resolv.conf,hosts,hostname} are injected into the rootfs through
the same os.Root bounds.
Tests cover runspec resolution and precedence, workdir normalization,
the empty-env rejection, symbolic user lookup, runtime-file injection
including staging cleanup when the final rename fails, and the exec
argv shape via the execElfuseForRun seam and a subprocess exec probe.
The run path applies the same store-managed rootfs rule as unpack: the
digest-keyed cache must be a real directory, so a planted symlink cannot
redirect the "already unpacked" probe and hand the guest a tree outside
the store.
henrybear327
force-pushed
the
oci/redis-workload
branch
from
August 5, 2026 20:23
57b35c0 to
bbe613a
Compare
A plain directory rootfs on the default case-insensitive APFS volume folds Linux filenames that differ only by case. Default runs now use a case-sensitive APFS sparsebundle per pinned manifest digest, with a per-run clonefile COW rootfs so guest writes never mutate the warm base tree and repeated runs skip the unpack. Liveness and lifecycle transitions are decided by per-digest advisory flocks in the bundle directory (bundlelock.go), not by pids or directory scans: every live run holds run.lock shared from before the volume is attached until guest exit, so a killed run cannot leak liveness, and attach.lock serializes provisioning (always acquired before run.lock, making the exclusive-to-shared downgrade in provision race-free). Provisioning reuses a mount a live run still holds and only force-detaches one proven stale by an exclusive run.lock probe, so a second run of the same digest can never rip the rootfs out from under a live guest. The spawned elfuse child inherits the run.lock descriptor (cmd.ExtraFiles re-opens it without close-on-exec), so a wrapper killed with an uncatchable signal leaves the flock with the still-running guest: the sweeps keep seeing the bundle busy for exactly as long as elfuse executes out of it, and the next provision cannot mistake the mount for stale. A fake-elfuse spawn test pins the inheritance across the wrapper's fd close. The plain-rootfs path remains available with --plain-rootfs, and the non-Darwin stub keeps elfuse-oci buildable for pull/inspect/unpack tests on Linux. Darwin tests drive runCaseSensitive through seams for provisioning, clonefile, spawn, cleanup, and exit, and cover sparsebundle provisioning, mount/detach, attach-failure teardown, and the lock protocol with a fake hdiutil; the mount-probe and force-detach hooks are function variables so tests need no real disk images. The spawn path intercepts SIGHUP alongside INT/TERM/QUIT and installs the handler before the child starts, so a hangup or an early signal still flows through the forward/reap/teardown path. elfuse is invoked with a "--" separator before the guest command, so an image Entrypoint beginning with "-" cannot steer the host launcher. hdiutil attach failures keep stderr in the error message (stdout stays clean for the plist parse), and the parsed mount path is XML-entity-decoded so a store path containing "&" or quotes still round-trips. The bundle directory's layout is named once beside its lock paths (mount point, sparsebundle image) rather than rebuilt from string literals at each use, and the two plain hdiutil invocations share one wrapper that folds the tool's output into the error.
Add list/images, rmi, and prune on top of the OCI image-layout store. rmi and prune use reachability GC: shared manifests, configs, and layers stay on disk while any remaining ref reaches them. Cache cleanup handles plain rootfs caches and macOS sparsebundle caches through platform-specific seams: a prune without --all never touches a still-pinned digest's cache, and one a live run uses is never reclaimed. Liveness is the same flock discipline on both cache kinds, the bundle's run.lock and a sibling <hex>.lock for the plain rootfs, held shared from before the existence probe until guest exit. The plain path execs elfuse in place, so that descriptor is made exec-survivable and the kernel releases it exactly when elfuse exits, SIGKILL included. prune skips busy caches (dry runs never advertise them); rmi refuses one even with --force, before any pin or descriptor is touched. The lock is a sibling of the cache dir, not inside it, because the dir is the guest's / and its existence is unpackImage's publication signal. An explicit --rootfs naming the store's own managed trees is rejected up front: the explicit path runs without the per-digest lock, so the sweeps would see the digest idle and could reclaim the tree under the live guest. The bundle sweep reaps staging directories only; a plain file wearing a clone or unpack-temp name is left alone. Liveness also roots blob reachability, not just cache sweeps. resolveImageForUse takes a digest's run lock inside the store lock and then releases the store lock, so a run reads that manifest's config and layers unlocked; if a concurrent pull repins the tag in that window, rooting only at pins would let the next prune or rmi reclaim the blobs that run is still reading. A busy digest therefore keeps its descriptor and blobs. The dry-run bundle sweep holds run.lock across its clone listing rather than probing and releasing first, because a clone without a keep marker is only proven abandoned while no run can create one. gc resolves every liveness root before reconciling index.json, so a stale or malformed pin fails the pass with the descriptors intact rather than after the reconcile dropped entries it could no longer justify (and then wedged every later prune and rmi on the same error). rmi's cache-existence probe uses Lstat and propagates errors, so a dangling symlink or unreadable cache aborts the removal instead of silently leaving the cache behind. The rootfs sweep also reclaims orphaned sibling .lock files whose cache dir never appeared (every run creates the lock without creating the plain dir), and the bundle busy probe opens run.lock without O_CREATE, so a dry run mutates nothing. The reference lock rides into the spawned guest beside run.lock, so rmi keeps refusing while an orphaned guest still reads the image. prune's GC sweep and list's snapshot run under the store lock, closing the windows where a concurrent pull's fresh blobs could be reclaimed before their descriptor lands or a listing could observe a half-removed ref, and gc reads each liveness root once per pass. list reports the full os/arch/variant platform and the creation time in both output modes, and propagates write errors the way inspect does, so truncated output cannot exit 0. Store writes become crash-durable, because reclamation may now act on what a crash left behind: the pin table, index.json, and a pull's appended blobs are synced before the pin that makes them reachable is committed, so a surviving pin can never name a manifest or layer still sitting in the page cache. Two corrections to the unpack path come with the locking, because both are what the lock discipline needs to be true. unpackImage now takes the caller's already-resolved image rather than re-resolving a ref: the caller keys the cache by the digest it resolved, so a re-resolution could observe a different pin from a concurrent repull and fill digest A's cache with image B's content. Layer application also tracks the ancestors of each entry, so an opaque whiteout arriving after an implicit parent directory no longer clears content the same layer added. Tests cover reachability GC (shared blobs, stale temp blobs, digest prefixes, and blobs held live by a busy unpinned digest), the prune and rmi busy semantics for both cache kinds, the store-path --rootfs rejection, the fail-closed gc ordering and cache probe, orphan-lock sweeping, dry-run probe purity, lock survival across the exec boundary, and end-to-end command wrappers. The darwin sweep runs through the isMountPointFn and detachForce seams, so it needs no disk images.
The on-disk store is the contract: pulls must produce a valid OCI image-layout that other tools can read and that agrees with registry truth on the manifest digest. Conformance tests pin the layout shape, and scripts/oci-interop.sh cross-checks the store with crane, skopeo, and umoci. CI splits by runner capability: a Linux job runs the pure-Go store paths (pull, unpack, inspect, lifecycle) without Hypervisor.framework, a hosted macOS job builds and tests the darwin sparsebundle code and drives a run-less pull/inspect/list/rmi/prune lifecycle smoke, and the self-hosted release leg boots guests end to end under HVF: the alpine:3 default-entrypoint smoke plus a full pull -> inspect -> list -> run -> rmi -> prune lifecycle that runs python:3.12-slim with an --entrypoint override and asserts the teardown half of the lifecycle. The lifecycle teardown covers all three reclamation guardrails: a plain rmi reclaims the cold cache with the image, run --keep output refuses rmi without --force, and a live --plain-rootfs guest parked in sleep pins its cache; prune --cache --all must skip it and rmi must refuse until the guest exits, the only end-to-end exercise of the run-lock descriptor riding through the exec into elfuse. The guest dies by SIGKILL, so the reclamation that follows proves the kernel dropped the flock, not that a graceful teardown ran. The run smoke ends with prune --cache, keeping the persistent store's stranded caches, and not only its blobs, bounded across tag moves. The store durability test drives the writer's rename-failure branch (a non-empty directory at the destination), pinning the staging cleanup the read-only-dir injection cannot reach.
Cover the two-binary model in README and docs: usage.md documents the elfuse-oci commands and flags, testing.md the offline/CI validation split, internals.md the host-literal path fallback semantics, and oci-design.md records the design rationale (the C/Go boundary, the image-layout store with its refs.json pin table, layer application, run paths, and lifecycle GC), plus an explicit scope-and-limitations accounting of which OCI features are and are not implemented. oci-design.md also records the concurrency model: store metadata is lock-serialized, per-digest sparsebundle state is coordinated by the attach.lock/run.lock pair, and the plain digest-keyed rootfs cache is guarded by a sibling per-digest lock a run holds across the exec into elfuse, so prune skips and rmi refuses a cache a live guest still uses. usage.md notes the resolv.conf fallback nameserver and its split-DNS implication. README states the positioning up front: OCI images are consumed as a distribution vehicle for Linux root filesystems, replacing hand-built --sysroot trees, and the non-isolation limitations (host-path fallback, shared network identity, PID space, and clock) are called out explicitly rather than implied.
Issue sysprog21#224 profiled five real images (python:3.12-slim, node:22-alpine, golang:1.23-alpine, eclipse-temurin:21, gcc:14). Add one CI job per image that boots the image under HVF via `elfuse-oci run` and drives that image's operations, so a change that breaks any of them is caught on the PR rather than by hand. A shared driver, scripts/ci/oci-workload.sh <key>, maps each key to its image and guest workload under scripts/ci/workloads/ and asserts a per-image sentinel token: - python: a single-threaded SQLite insert plus an aggregate query, a small file write/read/checksum fan-out, and a JSON round-trip. - node: in-guest compute (fs/crypto/zlib/JSON) plus an HTTP server the job reaches over the host loopback; elfuse maps guest sockets to host sockets and does no network-namespace isolation, so a guest bound to 127.0.0.1 is reachable host-side. - go: gofmt over a tree of misformatted fixtures the workload writes itself, asserting the file set it names, the bytes it produces, and that a second pass names nothing. The toolchain binaries are Go programs, so this drives the runtime's own scheduler and raw syscalls; it compiles nothing, because the guest compiler dies on SIGHUP before finishing a package. - jvm: javac + java exercising collections, file I/O, SHA-256, an 8-thread pool, and a subprocess. - c: a small multi-file make project plus a heavier single translation unit compiled with gcc -O1. Each workload self-check asserts exact known outputs (pinned digests, exact sums, byte-compared read-backs), not just output shape, and the node server request count is validated so a zero cannot pass the request loop vacuously. The go and python lanes stay inside the runtime's current limits; heavier variants that stress those limits, including an in-guest Go build, are submitted separately. The jobs run only on self-hosted Apple Silicon because `run` needs Hypervisor.framework. They share a composite action that fetches the elfuse binary from build-macos and builds elfuse-oci, and each keeps a warm per-key store on the runner's persistent disk so only the first run pulls over the network. gcc:14 and eclipse-temurin:21 ship the shadow suite, so those jobs also exercise the unpack setuid/setgid degrade end to end. Each leg ends with prune --cache, so a moved tag's stranded caches, and not only its blobs, stay bounded on the runner's persistent store. The lanes are legs of one matrixed job, differing only in the workload key and a timeout, so the shared runner, guards, and setup action are stated once. fail-fast is off: each image is an independent signal. The python workload moves under scripts/ci/workloads/ and is rewritten for this lane, dropping the multi-threaded and WAL SQLite stress in favor of a single-threaded insert plus a JSON round-trip; the heavier variant lives on the workload-stress branch. oci-lifecycle.sh follows the new path.
The run smoke proves an image boots and computes; these checks cross the guest-execution seams it does not. A pathname AF_UNIX socket is bound inside a guest-created directory with a getsockname round-trip: the runtime translates sun_path into the sysroot on the way in and must reverse-map it on the way out, and the sparsebundle clone's deep host path additionally forces the over-length shortening indirection. A cold-provision boot (blobs cloned into an ephemeral store with the cs/ bundles dropped, so no network) must report the unpack, and the following warm re-attach of the same digest must boot without unpacking again. A dynamically linked from-image binary runs under an explicit entrypoint so PT_INTERP and its shared-object closure must resolve inside the rootfs. The socket check and the workload's interpreter children carry explicit deadlines (socket timeouts, a bounded thread join, a per-child subprocess timeout), so an exec regression fails the lane promptly instead of holding the self-hosted runner to the job timeout. The python workload's verdicts compare exact values: the SQLite aggregate pins per-thread MIN/MAX/SUM, the JSON round-trip compares the whole parsed document, and the file fan-out compares contents in path order, so value corruption cannot pass as a matching count or an equal multiset. Wired as a runtime-macos Release-leg step beside the run smoke, sharing its warm store (python:3.12-slim joins alpine and debian there).
henrybear327
force-pushed
the
oci/redis-workload
branch
from
August 5, 2026 20:36
bbe613a to
d38fbfc
Compare
Generate Linux-shaped /proc/self/smaps and /proc/<pid>/smaps snapshots from tracked guest VMAs so Redis can complete its ARM64 COW safety check under elfuse. Add growable VMA/string infrastructure and fork-aware accounting with parser regression coverage.
Keep static-analysis checks and the smaps test matrix green after adding the new procfs emulation coverage.
Apply review feedback to the smaps emulation and its regression tests.
Grow formatted string storage safely and silence the Infer warning exposed by the smaps changes.
Avoid collisions when concurrent shared-memory tests create temporary names.
Keep concurrent shared-memory fixture setup consistent with repository formatting conventions.
Append live and shadow VMAs, then sort and merge once. This keeps fragmented maps and smaps snapshots from quadratic insertion work.
Apply repository formatting conventions to proc VMA snapshot calls.
Track whether each guest VMA existed at the fork snapshot so post-fork mappings are excluded from the synthetic Shared_Dirty compatibility signal. Propagate the metadata through fork IPC and memory-region transformations, and add a regression test for post-fork mappings.
Preserve VMA lineage across fork-split mappings and keep synthetic smaps accounting consistent with per-VMA fork state. Add file-backed mremap regression coverage and wire the tests into the build and documentation.
Apply consistent line wrapping in the memory syscall implementation and its mremap regression test without changing behavior.
Preserve fork lineage and smaps consistency when restored regions receive new allocations. Flush shared-file contents before mremap removes source mappings. Make region-boundary preparation transactional so allocation or descriptor failures leave metadata unchanged. Add regressions for repeated post-fork allocations and misaligned moves. Cover PROT_NONE smaps output as well. Related: sysprog21#259
Keep synthesized Pss_Dirty consistent with fork-inherited Shared_Dirty, and avoid flushing writable aliases when moving a read-only MAP_SHARED snapshot. Add regressions for both cases.
Keep failed reservation addresses visible to callers when munmap fails. This lets MAP_FAILED cleanup release still-live mappings instead of leaking them.
Boot redis:7-alpine under HVF with redis-server foreground as the guest process and drive it from a second guest running redis-cli over the shared host loopback: PING, a SET/GET round-trip, then BGSAVE polled through INFO persistence to rdb_bgsave_in_progress:0 with rdb_last_bgsave_status:ok asserted. BGSAVE forks the server and snapshots the dataset copy-on-write, so the lane pins the exact path /proc/self/smaps exists to keep safe. redis cannot announce an ephemeral port (--port 0 disables TCP), so the host probes a free loopback port before boot instead of reading one back as the node lane does. Readiness keeps the node lane's two-stage shape (process alive, then socket accepts), shutdown is an in-band SHUTDOWN NOSAVE, and the server's own exit status is asserted host-side. This leg fails for now: redis-server's ARM64 COW safety check needs /proc/self/smaps (issue sysprog21#258), which elfuse does not yet synthesize; upstream PR sysprog21#259 adds it. The lane is staged in advance so that once that lands and this branch is rebased onto it, the leg turns green with no further changes and keeps the fork/COW path covered from then on. Verified both ways: red on this tree, green with the smaps branch merged locally.
henrybear327
force-pushed
the
oci/redis-workload
branch
from
August 5, 2026 20:38
d38fbfc to
bc9fdee
Compare
henrybear327
marked this pull request as draft
August 5, 2026 21:07
Contributor
|
Avoid modifying unnecessary files. For the Redis workload, deliver the minimal runnable changeset required to validate OCI image support. |
Collaborator
Author
I am stacking git branches locally, thus, the PR would show the commits from the branches I am stacking on top of. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a simple redis workload to the CI pipeline, such that we can ensure future changes are not breaking features used by real software. Please see #259
Do not merge this PR before #259 is merged. Currently, the PR stack is
oci/setup-> elfuse-redis-server (#259) -> this PR #270Summary by cubic
Adds end-to-end OCI image support with the
elfuse-ociCLI and a macOS case‑sensitive APFS sparsebundle rootfs with per‑run COW clones. Includes Linux‑compatible/proc/self/{maps,smaps}to pass Redis BGSAVE fork/COW checks, and CI that boots real images under HVF plus layout conformance and interop.New Features
elfuse-ociCLI:pull,inspect,list,unpack,run,rmi,prune, with a spec‑compliant OCI image‑layout store, ref pinning, and reachability GC; lifecycle commands protect live runs and--keepclones via per‑digest advisory locks.--plain-rootfskeeps a directory mode for portability.runresolves Entrypoint/Cmd/User/Env/WorkingDir, guarantees a sane PATH, injects host‑truth/etc/{resolv.conf,hosts,hostname}, and execselfuse; non‑Darwin reports a clear error without--plain-rootfs.elfuseadds--user,--workdir, and--env, with bring‑up factored intoelfuse_launchand reused byelfuse-oci.crane/skopeo/umoci), darwin sparsebundle tests, run/exec checks (AF_UNIX pathname sockets and dynamic‑interpreter resolution), and per‑image workloads (python/node/go/jvm/c/redis) that boot under HVF.Migration
elfuse-oci;makebuilds it whengois on PATH (otherwise it’s skipped).$ELFUSE_OCI_STOREor~/.local/share/elfuse/oci; self‑hosted macOS runners keep a warm store per workload.Written for commit bc9fdee. Summary will update on new commits.