Add UFFD snapshot pager

[sjmiller609]

Summary

• adds a config-gated Firecracker UFFD memory backend for snapshot restore
• starts a per-restore UFFD pager session backed by the snapshot memory file and an optional shared page cache
• applies resume-network mailbox updates through UFFD overlay pages so restore does not mutate the backing memory file
• shards the pager cache and tracks pager timing counters for page-fault, lookup, backing-read, and copy latency

Tests

• go test ./lib/hypervisor/firecracker ./lib/uffdpager ./lib/mailbox ./lib/guest ./lib/system/guest_agent ./lib/oapi -count=1

---

Note

High Risk
Changes Firecracker snapshot restore and host VM memory handling, depends on a correctly installed/versioned systemd pager, and misconfiguration or pager failure can leave restored VMs unhealthy until recycled.

Overview
Adds an opt-in Firecracker UFFD snapshot memory backend (hypervisor.firecracker_snapshot_memory_backend=uffd, default file) with a bounded shared page cache (hypervisor.firecracker_uffd_cache_max_bytes).

On Linux, enabling UFFD starts a versioned hypeman-uffd-pager via the hypeman-uffd@<version>.service systemd template; Hypeman wires the Firecracker starter to create per-restore pager sessions and passes a mem_backend of type Uffd (per-session socket) instead of mmap’ing the snapshot memory file. Instance metadata tracks session/cache keys, restores go through new RestoreOptions, and stop/delete/standby/restore failure paths close UFFD sessions; state queries can mark instances unknown if the pager is unhealthy.

Also ships the pager in build/release/install (cmd/uffd-pager, Makefile, GoReleaser, install scripts) and adds CI scripts/check-uffd-version.sh so runtime pager changes require bumping lib/uffdpager/VERSION.

^{Reviewed by Cursor Bugbot for commit 6fb4e46. Bugbot is set up for automated code reviews on this repo. Configure here.}

[firetiger-agent]

Created a monitoring plan for this PR.

What this PR does: Adds opt-in lazy-memory paging for Firecracker snapshot restores using Linux UFFD. The default backend remains file — no behavior changes until an operator explicitly sets FIRECRACKER_SNAPSHOT_MEMORY_BACKEND=uffd on a hypeman node.

Intended effect:

• Snapshot restore success rate: baseline 0 WARN-level restore failures/hr; confirmed if "failed to restore from snapshot" and "configure snapshot memory backend" logs remain at 0 after deploy
• Instance spawn rate: baseline 10K–31K/hr; confirmed if rate stays within range (no regression from the RestoreVM interface or mem_backend payload change)
• API 5xx error rate: baseline 0.006–0.026%; confirmed if no sustained increase post-deploy

Risks:

• mem_backend Firecracker API incompatibility — snapshotLoadParams now sends mem_backend struct instead of mem_file_path; alert if any "load firecracker snapshot" ERROR appears post-deploy (baseline: 0/hr)
• Hypeman node crash at startup — NewManagerWithConfig now panics if the UFFD pager fails to start; alert if any hypeman process restart occurs within 1h of deploy (only applies when uffd backend is set, but worth confirming)
• Stale UFFD sessions on failure — on restore failure, UFFD session cleanup must succeed within 2s; alert if "failed to close firecracker uffd session" WARN appears (baseline: 0/hr; relevant when UFFD mode is first activated)
• StateUnknown proliferation — UFFD session health check during instance query can return StateUnknown if pager dies; alert if any "firecracker uffd session is unhealthy" WARN log appears (baseline: 0/hr; relevant when UFFD mode is active)

Status updates will be posted automatically on this PR as monitoring progresses.

View monitor

[hiroTamada]

reviewed the uffd pager slice — architecture looks solid (separate pager process, opt-in file backend, versioned systemd + drain). left a few nits on docs, restore cleanup, cache key wording, and pager vs session health. nice work.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 6fb4e46. Configure here.}