Subsequent requests on reused connection delayed by 40 milliseconds

[tdryer]

When running oxhttp with a client that supports connection reuse on Linux, responses after the first response on the same connection experience a 40 millisecond delay:

$ time curl -s http://localhost:8080 http://localhost:8080 > /dev/null
real 0m0.045s

This delay occurs due to an interaction between oxhttp, Nagle's algorithm, and TCP delayed ACK:

• oxhttp writes the response headers and body to the socket separately, resulting in two separate TCP segments.
• TCP delayed ACK causes the client to wait before acknowledging the first segment, because it is smaller than the maximum segment size.
• Nagle's algorithm causes the server to wait before sending the second segment, because the first segment is unacknowledged.

I think the first response on a connection is unaffected because Linux always acknowledges the first data segment immediately.

This could be fixed in oxhttp by either:

• Writing the response headers and body to the socket together.
• Disabling Nagle's algorithm using TcpStream::set_nodelay.

[Tpt]

Thank you for raising that and the amazingly written issue.

oxhttp writes the response headers and body to the socket separately, resulting in two separate TCP segments.

I am not sure if it's true. If I am not mistaken, oxhttp relies on a BufWriter that writes by default every 8192 bytes and that we manually flush each time a response is finished to be written. So, a request with not too many header and a small body should be written only once.

In any case, enabling nodelay is a good idea, we buffer writes so we should never send small packets except in the case of small requests/responses where it is fine. Thank you!

[Tpt]

By setting NODELAY I indeed get latency down from 52mns to 6mns on a simple example. Thank you!

[tdryer]

oxhttp writes the response headers and body to the socket separately, resulting in two separate TCP segments.

I am not sure if it's true. If I am not mistaken, oxhttp relies on a BufWriter that writes by default every 8192 bytes and that we manually flush each time a response is finished to be written. So, a request with not too many header and a small body should be written only once.

The problem is that encode_body is using std::io::copy to write the response body. The implementation of copy has a specialization for BufWriter that allows it to reuse the writer's internal buffer space. This specialization appears to flush the writer before and after performing the copy if:

• buffer capacity is greater than or equal to DEFAULT_BUF_SIZE; and
• buffer capacity is less than the sum of DEFAULT_BUF_SIZE and the length of the data being copied.

Here's a standalone example. I didn't see this behavior documented anywhere, so to ensure there is only one TCP segment, it might be best to avoid using copy.

[Tpt]

Thank you! I was not aware of that!

I am a bit reluctant to avoid using copy because its implementation seems better than what I can come with easily. However, to avoid the double flush issue on small bodies, I have opened #42 that increases the write (and read) buffers to 16kB from the DEFAULT_BUF_SIZE that is 8kB on most platforms. This way on small bodies we should have only a single flush for bodies that are a bit less than 8kB (the second condition you wrote will be false) and having multiple flushes is likely fine on larger bodies (we have NODELAY enabled now). However the automated benchmark I have setup does not display a reduced number of syscalls (before vs after) so I am a bit confused.

[tdryer]

I can see the reduced number of sendto calls in your pull request branch versus main when I run the benchmark locally:

$ CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUNNER="strace -e trace=sendto" cargo test --benches client_server_fixed_body
[...]
Testing client_server_fixed_body
sendto(7, "GET / HTTP/1.1\r\nhost: localhost:"..., 1086, MSG_NOSIGNAL, NULL, 0) = 1086
Success

+++ exited with 0 +++

Perhaps there's some noise in the overall number of syscalls.

[Tpt]

Amazing! Thank you for checking!