Skip to content

feat(configs): trim kernel configs for faster Firecracker boot#5

Merged
markovejnovic merged 1 commit into
mainfrom
feat/faster-boot
Jul 8, 2026
Merged

feat(configs): trim kernel configs for faster Firecracker boot#5
markovejnovic merged 1 commit into
mainfrom
feat/faster-boot

Conversation

@markovejnovic

@markovejnovic markovejnovic commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

What

All five configs in configs/ were verbatim copies of Firecracker's CI guest configs — built to exercise FC's test matrix, not to boot fast. This PR trims each of them, dropping subsystems a hyper microVM can never reach (USB, input probing, SCSI, thermal, watchdog, NUMA, cpufreq, hibernation, kexec, audit, NFS/XFS, balloon/pmem/virtio-mem, bridge, MPTCP, SEG6, and the BPF_PRELOAD/BPFILTER boot-time userspace helpers). The trims were applied with the kernel's own merge_config.sh + olddefconfig, so every config stays normalized (the CI build's olddefconfig is a no-op on them).

Boot-time verification was done with a local Firecracker benchmark harness (not kept in-tree): FC v1.16.0 (the version hyper pins), hyper's exact default cmdline, a minimal ext4 rootfs whose static /init reboots immediately, wall time measured from VMM spawn to VMM exit.

Measurements (KVM, Firecracker v1.16.0, hyper default cmdline, 20 runs, same host)

kernel boot median before after Δ size before after Δ
x86_64-6.1 257 ms 226 ms −12% 44.5 MB 37.5 MB −15.6%
x86_64-5.10 290 ms 270 ms −7% 39.5 MB 31.9 MB −19.4%
x86_64-5.10-no-acpi 297 ms 257 ms −13% 38.8 MB 31.2 MB −19.5%
aarch64-6.1 n/a (no arm KVM host) n/a 17.7 MB 13.0 MB −26.4%
aarch64-5.10 n/a n/a 17.4 MB 11.8 MB −32.6%

Baselines are the released assets from release-54d6c3f… (sha256-verified). x86 numbers were independently re-measured by a second run; deltas hold under identical host conditions.

Two subsystems the trim was expected to cover stayed — with receipts

  • CONFIG_PCI stays. On x86, Firecracker enumerates virtio-mmio devices through the ACPI DSDT; with PCI=n, ACPICA cannot install its mandatory PCI_CONFIG region handler, acpi_load_tables aborts, and the guest panics with no root device. pci=off on hyper's cmdline already skips bus probing, so keeping it costs size, not time. On aarch64 the rationale doesn't apply (DTB enumeration) — trimming PCI there is noted follow-up.
  • CONFIG_VT (+INPUT) stays. hyper boots with no console=; a VT-less kernel then auto-falls-back to the 8250 serial console and drains the whole boot log through port I/O (~28k PIO vmexits, ≈ +175 ms median). With VT kept (dummycon absorbs printk, VGA_CONSOLE=n, and i8042/serio/atkbd still trimmed) UART writes drop to 0. console=ttyS0 debug boots still work.

Base configs remain Firecracker's CI configs with these trims applied on top.

Deliberate security-posture note (please sign off)

Dropping AUDIT removes SECURITY_SELINUX (it depends on AUDIT), so all five configs now default to DEFAULT_SECURITY_DAC. hyper guests never load an LSM policy, so this is functionally nil there — but anyone using these published kernels with their own rootfs loses the option of SELinux. Revisit if that matters.

Verification

  • x86_64 variants: booted to /init under Firecracker v1.16.0 with hyper's exact cmdline (reboot=k panic=1 pci=off init=/init), verbose-console boot asserting init ran, then 20-run benchmarks (above).
  • aarch64 variants: symbol asserts + clean cross-build (olddefconfig no-op on committed configs) + qemu TCG boot-to-/init smoke test. No arm KVM timing possible on this x86_64 host.
  • Kept in every config: virtio mmio/blk/net/vsock, ext4 rw root, 8250 serial console (+ SERIAL_OF_PLATFORM/PL031 RTC on aarch64), fuse, overlayfs, squashfs, netfilter/iptables core, IPv6 core; ACPI kept on x86_64 non-no-acpi variants.

Rollout

hyper pins kernel sha256s in priv/vmlinux/manifest.json; nothing changes downstream until a release is cut here and that manifest is bumped. Suggested follow-ups: bump the pin after the next release and confirm the win end-to-end via hyper's boot OTel spans; add a CI boot-smoke gate so an unbootable config can't publish; trim PCI on aarch64 (DTB enumeration makes it dead weight there).

@markovejnovic markovejnovic merged commit bab2b3a into main Jul 8, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant