Skip to content

Document sysctl tuning for QUIC UDP buffer sizes#296

Merged
MegaRedHand merged 1 commit intomainfrom
docs-quic-udp-buffer-sysctl
Apr 16, 2026
Merged

Document sysctl tuning for QUIC UDP buffer sizes#296
MegaRedHand merged 1 commit intomainfrom
docs-quic-udp-buffer-sysctl

Conversation

@pablodeymo
Copy link
Copy Markdown
Collaborator

Motivation

Fixes #295. QUIC clients (both Rust/quinn and Go/quic-go) request ~7 MiB UDP receive buffers from the OS, but Linux defaults cap SO_RCVBUF at ~208 KiB. This causes warnings and can degrade networking performance under load.

The underlying SO_RCVBUF/SO_SNDBUF socket options aren't configurable through the libp2p/quinn API chain — they require OS-level sysctl tuning.

Description

Adds a note in the README devnet section documenting:

  • net.core.rmem_max and net.core.wmem_max sysctl settings (7 MiB)
  • How to persist across reboots (/etc/sysctl.conf)
  • Docker --sysctl flags for containerized deployments

How to Test

Documentation-only change — verify the README renders correctly.

Closes

Closes #295

  Add a note in the devnet section about increasing net.core.rmem_max
  and net.core.wmem_max to 7 MiB for optimal QUIC performance, with
  instructions for both bare-metal and Docker environments.

  Closes #295
@github-actions
Copy link
Copy Markdown

🤖 Claude Code Review

Here is the review:


PR #296 — Document sysctl tuning for QUIC UDP buffer sizes

Overview: Documentation-only change adding a callout block to README.md advising operators to raise net.core.rmem_max / net.core.wmem_max to 7 MiB for QUIC performance. No code is modified.


Correctness

  • The value 7340032 is exactly 7 × 1024 × 1024 bytes, matching what quinn and quic-go request. Correct.
  • Both rmem_max and wmem_max are tuned — good; omitting wmem_max would leave send-side constrained.
  • The /etc/sysctl.conf persistence advice is correct for Debian/Ubuntu/RHEL-family systems. Alpine (musl) and NixOS users need different paths, but that is an acceptable omission for a devnet guide.

Potential Issues

  • Docker --sysctl scope: --sysctl sets a sysctl inside the container's network namespace. net.core.rmem_max is a global kernel parameter and is not namespaced in Linux; it is silently ignored (or errors on older kernels) when passed per-container. The correct approach for Docker is to set it on the host or pass it to dockerd via /etc/docker/daemon.json ("default-sysctls"). The current text may give operators a false sense that the Docker flag is sufficient.

    Suggested correction:

    For Docker, set the sysctl on the host (the container inherits the host kernel value).
    Alternatively, configure `"default-sysctls"` in `/etc/docker/daemon.json`.
    
  • sysctl.conf path: On systemd-based distros the canonical location is /etc/sysctl.d/99-quic.conf (or similar drop-in). /etc/sysctl.conf works but is slightly less conventional. This is a minor nit.

Style & Formatting

  • The note renders correctly in GitHub Markdown — the > **Note:** callout with a fenced code block inside a blockquote is valid CommonMark and displays cleanly.
  • Placement between the make run-devnet description and the --is-aggregator warning is logical.

Summary

One substantive concern: the Docker --sysctl guidance is technically inaccurate for non-namespaced sysctls like net.core.rmem_max. Everything else is correct and well-placed. The fix is a single sentence change. The rest of the PR is good to merge once that is addressed.


Automated review by Claude (Anthropic) · sonnet · custom prompt

@github-actions
Copy link
Copy Markdown

🤖 Codex Code Review

No findings.

The diff only updates operational documentation in README.md at lines 43-48, adding Linux/Docker UDP buffer guidance for QUIC. There are no code-path changes affecting fork choice, attestation handling, justification/finalization, state transition, XMSS, SSZ, memory safety, or runtime performance beyond the user-facing guidance itself.

Residual note: I only reviewed the submitted diff in /tmp/pr_diff.txt; based on that diff, this PR looks good.


Automated review by OpenAI Codex · gpt-5.4 · custom prompt

@github-actions
Copy link
Copy Markdown

🤖 Kimi Code Review

Overview: This PR contains only documentation changes to README.md regarding UDP buffer sizing for QUIC performance. No consensus-critical code is modified.

Issues Found:

  1. Docker --sysctl inaccuracy (Line ~46): The suggestion to pass --sysctl net.core.rmem_max=7340032 to Docker is incorrect. Parameters under net.core.* are global kernel parameters, not namespaced, and cannot be modified from within containers. Docker's --sysctl flag only supports namespaced sysctls (e.g., net.ipv4.*). To use these settings in Docker, the host kernel must be configured before container startup.

  2. Privilege/scope clarification (Line ~43): Consider explicitly noting that these sysctl commands require root privileges and affect system-wide networking parameters. While sudo is shown, stating that these are global kernel limits (not per-process) would help users understand the security implications.

Suggestions:

  • Remove the Docker --sysctl line or replace it with instructions to configure the host kernel before running the container.
  • Consider referencing why 7MB (7340032 bytes) specifically is recommended (e.g., matching the QUIC implementation's window size or BDP calculations).
  • For persistence, consider suggesting /etc/sysctl.d/99-quic-buffers.conf instead of /etc/sysctl.conf, as this is the modern convention on systemd-based distributions and avoids modifying the main configuration file.

Consensus Impact: None (documentation only).


Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Apr 16, 2026

Greptile Summary

This PR adds a documentation-only callout in the README devnet section explaining how to tune Linux kernel UDP buffer sizes (net.core.rmem_max/net.core.wmem_max) to satisfy the ~7 MiB requests made by quinn and quic-go. The sysctl values (7,340,032 bytes = 7 MiB) are correct and the note is placed in a logical location.

Confidence Score: 5/5

Safe to merge — documentation-only change with accurate values and no code impact.

All findings are P2 style suggestions (missing sysctl -p step, Docker Compose vs docker run distinction). No code is changed and the documented sysctl values are numerically correct.

No files require special attention.

Important Files Changed

Filename Overview
README.md Adds a callout block documenting sysctl tuning for QUIC UDP buffer sizes; values are correct (7 MiB = 7,340,032 bytes) and placement is appropriate.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[QUIC socket created] --> B{SO_RCVBUF request ~7 MiB}
    B -->|Linux default rmem_max ~208 KiB| C[Kernel caps buffer → warning logged]
    B -->|rmem_max=7340032 tuned| D[Full 7 MiB buffer granted]
    C --> E[Degraded UDP throughput under load]
    D --> F[Optimal QUIC performance]

    G[Tuning options] --> H[Transient: sudo sysctl -w]
    G --> I[Persistent: /etc/sysctl.conf]
    G --> J[Docker: --sysctl flag]
    H & I & J --> D
Loading
Prompt To Fix All With AI
This is a comment left during a code review.
Path: README.md
Line: 48

Comment:
**`/etc/sysctl.conf` note omits apply-without-reboot step**

The note says "add to `/etc/sysctl.conf`" but doesn't show what to write or how to activate it without rebooting. Users will likely need to run `sudo sysctl -p` to apply immediately. A self-contained example would be clearer:

```suggestion
> To persist across reboots, add the following lines to `/etc/sysctl.conf` and run `sudo sysctl -p` to apply without rebooting. For Docker, pass `--sysctl net.core.rmem_max=7340032 --sysctl net.core.wmem_max=7340032`.
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: README.md
Line: 43-48

Comment:
**Docker `--sysctl` flag not applicable to Compose-based devnet**

`make run-devnet` uses `lean-quickstart`, which is Docker Compose based. The `--sysctl` flag shown applies to `docker run`; for Compose deployments the equivalent is a `sysctls:` key in the service definition. Users trying to apply this hint to the devnet may be confused. Consider clarifying that the Docker note applies to standalone `docker run` invocations, or point to modifying the Compose file for the devnet case.

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "Document sysctl tuning for QUIC UDP rece..." | Re-trigger Greptile

Comment thread README.md
> sudo sysctl -w net.core.rmem_max=7340032
> sudo sysctl -w net.core.wmem_max=7340032
> ```
> To persist across reboots, add to `/etc/sysctl.conf`. For Docker, pass `--sysctl net.core.rmem_max=7340032 --sysctl net.core.wmem_max=7340032`.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 /etc/sysctl.conf note omits apply-without-reboot step

The note says "add to /etc/sysctl.conf" but doesn't show what to write or how to activate it without rebooting. Users will likely need to run sudo sysctl -p to apply immediately. A self-contained example would be clearer:

Suggested change
> To persist across reboots, add to `/etc/sysctl.conf`. For Docker, pass `--sysctl net.core.rmem_max=7340032 --sysctl net.core.wmem_max=7340032`.
> To persist across reboots, add the following lines to `/etc/sysctl.conf` and run `sudo sysctl -p` to apply without rebooting. For Docker, pass `--sysctl net.core.rmem_max=7340032 --sysctl net.core.wmem_max=7340032`.
Prompt To Fix With AI
This is a comment left during a code review.
Path: README.md
Line: 48

Comment:
**`/etc/sysctl.conf` note omits apply-without-reboot step**

The note says "add to `/etc/sysctl.conf`" but doesn't show what to write or how to activate it without rebooting. Users will likely need to run `sudo sysctl -p` to apply immediately. A self-contained example would be clearer:

```suggestion
> To persist across reboots, add the following lines to `/etc/sysctl.conf` and run `sudo sysctl -p` to apply without rebooting. For Docker, pass `--sysctl net.core.rmem_max=7340032 --sysctl net.core.wmem_max=7340032`.
```

How can I resolve this? If you propose a fix, please make it concise.

Comment thread README.md
Comment on lines +43 to +48
> **Note:** On Linux, QUIC performance benefits from larger UDP receive buffers. If you see warnings about buffer sizes, increase the kernel limit:
> ```sh
> sudo sysctl -w net.core.rmem_max=7340032
> sudo sysctl -w net.core.wmem_max=7340032
> ```
> To persist across reboots, add to `/etc/sysctl.conf`. For Docker, pass `--sysctl net.core.rmem_max=7340032 --sysctl net.core.wmem_max=7340032`.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Docker --sysctl flag not applicable to Compose-based devnet

make run-devnet uses lean-quickstart, which is Docker Compose based. The --sysctl flag shown applies to docker run; for Compose deployments the equivalent is a sysctls: key in the service definition. Users trying to apply this hint to the devnet may be confused. Consider clarifying that the Docker note applies to standalone docker run invocations, or point to modifying the Compose file for the devnet case.

Prompt To Fix With AI
This is a comment left during a code review.
Path: README.md
Line: 43-48

Comment:
**Docker `--sysctl` flag not applicable to Compose-based devnet**

`make run-devnet` uses `lean-quickstart`, which is Docker Compose based. The `--sysctl` flag shown applies to `docker run`; for Compose deployments the equivalent is a `sysctls:` key in the service definition. Users trying to apply this hint to the devnet may be confused. Consider clarifying that the Docker note applies to standalone `docker run` invocations, or point to modifying the Compose file for the devnet case.

How can I resolve this? If you propose a fix, please make it concise.

@MegaRedHand MegaRedHand merged commit 9a1b1a5 into main Apr 16, 2026
6 of 7 checks passed
@MegaRedHand MegaRedHand deleted the docs-quic-udp-buffer-sysctl branch April 16, 2026 21:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

QUIC UDP buffer size limitation

2 participants