Skip to content

[P3][kernel] Explore an opt-in group-commit (fsync-batching) durability mode #62

Description

@cevheri

Summary

Roadmap item — the one remaining direction from the exploratory durability-hardening issue #9 (its other two directions, the versioned WAL header and the expanded DST fault profiles, shipped in v0.2.0 / PR #43). The kernel fsyncs on every commit: correct, simple, and the right DEFAULT — but per-commit fsync caps write throughput. The standard answer is group commit: several commits share one fsync, trading a bounded, explicitly documented durability window for throughput (cf. Postgres synchronous_commit, SQLite WAL).

Ground rules (carried from #9, still binding)

  • Opt-in only; the default stays fsync-per-commit. Nothing may quietly weaken the default durability contract in docs/RELIABILITY.md.
  • The mode's exact guarantee must be spelled out: what a crash can lose (commits acknowledged inside the open window) and when the window closes.
  • Must stay honest with the synchronous single-process model and interact correctly with the v0.2.0 failure latch (a failed batched fsync must latch exactly like a failed per-commit fsync, and the affected commits must be reported as NOT durable).

Open questions

  • API surface: an open() option (durability: "batched" + flush() fence), or an explicit db.batch(run) scope? The second keeps the window visible in the code that accepts it.
  • Crash semantics with the v1 record format: batching appends multiple records between fsyncs — recovery already tolerates a torn multi-record tail, but the DST needs the reordering fault profile (Explore kernel durability hardening (WAL record headers, fsync-batching mode, expanded DST fault profiles) #9 direction 3's deferred piece) to PROVE it: writes/fsyncs reaching durable storage out of order.
  • Whether acknowledged-but-unfsynced commits should be observable (a pending() count) so callers can build their own fences.

Deliverables

  1. A DESIGN.md addendum ratifying (or rejecting) the mode and its exact contract — documentation is as much the point as implementation (Explore kernel durability hardening (WAL record headers, fsync-batching mode, expanded DST fault profiles) #9's words).
  2. If ratified: implementation behind the opt-in, SimFS reordering profile, DST seeds proving the bounded-loss contract, RELIABILITY.md section for the mode.
  3. Changeset (minor) if shipped.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions