HTTP/2 async streaming ({hackney_response, Ref, ...} messages) never delivered

[cbecker]

I was in the process of upgrading hackney from 1.x to 4.5.2, using the async API, and noticed that it would only work if I force HTTP 1 requests. With HTTP2, I never get the async response I expect.

Maybe I misunderstood how to use the API. Below the details.

Environment

• hackney 4.5.2 (Hex)
• Erlang/OTP 27 (erts-15.2.1)
• Elixir 1.18.2
• Linux

Summary

When using the documented async streaming API (hackney:get(URL, [], <<>>, [async]) or [{async, once}], receiving {hackney_response, Ref, ...} messages) against an HTTPS URL that negotiates HTTP/2 via ALPN, no message is ever delivered to the caller, not even an error. The request just hangs until recv_timeout/connect_timeout elapses. The identical code against a forced to HTTP/1.1 via {protocols, [http1]}) works correctly and near-instantly.

Reproduction

# hackney_h2_async_bug.exs. Run with `elixir hackney_h2_async_bug.exs`
Mix.install([{:hackney, "~> 4.0"}])

Application.ensure_all_started(:hackney)

url = "https://www.google.com/favicon.ico"

IO.puts("--- Plain HTTP/1.1 (works) ---")
t0 = System.monotonic_time(:millisecond)
{:ok, ref1} = :hackney.get("http://www.google.com/favicon.ico", [], "", [:async])

receive do
  {:hackney_response, ^ref1, msg} ->
    IO.puts("Got message after #{System.monotonic_time(:millisecond) - t0}ms: #{inspect(msg)}")
after
  5000 -> IO.puts("TIMEOUT (unexpected for HTTP/1.1)")
end

IO.puts("\n--- HTTPS, negotiates HTTP/2 via ALPN (broken) ---")
t1 = System.monotonic_time(:millisecond)
{:ok, ref2} = :hackney.get(url, [], "", [:async])

receive do
  {:hackney_response, ^ref2, msg} ->
    IO.puts("Got message after #{System.monotonic_time(:millisecond) - t1}ms: #{inspect(msg)}")
after
  5000 -> IO.puts("TIMEOUT after 5000ms - no {hackney_response, Ref, _} message ever arrived")
end

IO.puts("\n--- Same HTTPS URL, forced to HTTP/1.1 (workaround) ---")
t2 = System.monotonic_time(:millisecond)
{:ok, ref3} = :hackney.get(url, [], "", [:async, {:protocols, [:http1]}])

receive do
  {:hackney_response, ^ref3, msg} ->
    IO.puts("Got message after #{System.monotonic_time(:millisecond) - t2}ms: #{inspect(msg)}")
after
  5000 -> IO.puts("TIMEOUT (unexpected)")
end

Actual output (run against deps/hackney 4.5.2 directly in a project with this dependency, equivalent to the above):

--- Plain HTTP/1.1 (works) ---
Got message after 67ms: {:status, 200, "OK"}

--- HTTPS, negotiates HTTP/2 via ALPN (broken) ---
TIMEOUT after 5000ms - no {hackney_response, Ref, _} message ever arrived

--- Same HTTPS URL, forced to HTTP/1.1 (workaround) ---
Got message after 112ms: {:status, 200, "OK"}