feat(configs): trim kernel configs for faster Firecracker boot#5
Merged
Conversation
24c63fb to
a5c8a31
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
All five configs in
configs/were verbatim copies of Firecracker's CI guest configs — built to exercise FC's test matrix, not to boot fast. This PR trims each of them, dropping subsystems a hyper microVM can never reach (USB, input probing, SCSI, thermal, watchdog, NUMA, cpufreq, hibernation, kexec, audit, NFS/XFS, balloon/pmem/virtio-mem, bridge, MPTCP, SEG6, and the BPF_PRELOAD/BPFILTER boot-time userspace helpers). The trims were applied with the kernel's ownmerge_config.sh+olddefconfig, so every config stays normalized (the CI build'solddefconfigis a no-op on them).Boot-time verification was done with a local Firecracker benchmark harness (not kept in-tree): FC v1.16.0 (the version hyper pins), hyper's exact default cmdline, a minimal ext4 rootfs whose static
/initreboots immediately, wall time measured from VMM spawn to VMM exit.Measurements (KVM, Firecracker v1.16.0, hyper default cmdline, 20 runs, same host)
Baselines are the released assets from
release-54d6c3f…(sha256-verified). x86 numbers were independently re-measured by a second run; deltas hold under identical host conditions.Two subsystems the trim was expected to cover stayed — with receipts
CONFIG_PCIstays. On x86, Firecracker enumerates virtio-mmio devices through the ACPI DSDT; withPCI=n, ACPICA cannot install its mandatory PCI_CONFIG region handler,acpi_load_tablesaborts, and the guest panics with no root device.pci=offon hyper's cmdline already skips bus probing, so keeping it costs size, not time. On aarch64 the rationale doesn't apply (DTB enumeration) — trimming PCI there is noted follow-up.CONFIG_VT(+INPUT) stays. hyper boots with noconsole=; a VT-less kernel then auto-falls-back to the 8250 serial console and drains the whole boot log through port I/O (~28k PIO vmexits, ≈ +175 ms median). With VT kept (dummycon absorbs printk,VGA_CONSOLE=n, and i8042/serio/atkbd still trimmed) UART writes drop to 0.console=ttyS0debug boots still work.Base configs remain Firecracker's CI configs with these trims applied on top.
Deliberate security-posture note (please sign off)
Dropping
AUDITremovesSECURITY_SELINUX(it depends on AUDIT), so all five configs now default toDEFAULT_SECURITY_DAC. hyper guests never load an LSM policy, so this is functionally nil there — but anyone using these published kernels with their own rootfs loses the option of SELinux. Revisit if that matters.Verification
/initunder Firecracker v1.16.0 with hyper's exact cmdline (reboot=k panic=1 pci=off init=/init), verbose-console boot asserting init ran, then 20-run benchmarks (above).olddefconfigno-op on committed configs) + qemu TCG boot-to-/initsmoke test. No arm KVM timing possible on this x86_64 host.SERIAL_OF_PLATFORM/PL031 RTC on aarch64), fuse, overlayfs, squashfs, netfilter/iptables core, IPv6 core; ACPI kept on x86_64 non-no-acpi variants.Rollout
hyper pins kernel sha256s in
priv/vmlinux/manifest.json; nothing changes downstream until a release is cut here and that manifest is bumped. Suggested follow-ups: bump the pin after the next release and confirm the win end-to-end via hyper's boot OTel spans; add a CI boot-smoke gate so an unbootable config can't publish; trim PCI on aarch64 (DTB enumeration makes it dead weight there).