Skip to content

Expose an IO-like, Enumerable response body #117

Description

@ZilvinasKucinskas

Priority: P1

Related API contract: Provide a stable error hierarchy and safe, class-specific metadata

Describe the issue

Wreq::Response currently mixes response metadata and body ownership. bytes, text, and json fully buffer the body, while chunks is a one-shot operation that requires a block. There is no body object with the ordinary Ruby protocols for incremental reading, enumeration, and cleanup.

After chunks or close, another body read raises Wreq::MemoryError with a long explanation of Rust ownership. That exposes an implementation detail instead of a clear Ruby stream state.

Proposed Ruby API

response = client.get("https://example.com/archive")
body = response.body

body.read(16_384)       # => binary String or nil at EOF
body.readpartial(16_384)
body.each { |chunk| consume(chunk) }
body.each               # => Enumerator
body.close
body.closed?            # => true

Response#each_chunk can be a convenience delegate:

response.each_chunk { |chunk| consume(chunk) }
response.each_chunk # => Enumerator

Keep Response#chunks as a compatibility wrapper. bytes, text, and json can remain response conveniences implemented on top of the body.

Body semantics

  • The body is IO-like; it does not need to subclass IO.
  • read(length = nil, out_buffer = nil) and readpartial(max_length, out_buffer = nil) follow Ruby IO return/EOF conventions: a positive-length read returns nil when already at EOF, while readpartial raises EOFError.
  • each/each_chunk yield binary strings and return an Enumerator without a block.
  • Only one consumer owns an unbuffered network stream. Concurrent reads raise a concise state error.
  • close is idempotent, closed? is thread-safe, and post-close reads raise IOError or a typed Wreq::BodyStateError < Wreq::BodyError.
  • Breaking enumeration early releases or discards the connection according to documented pool rules.
  • If convenience methods memoize a fully buffered body, document whether later reads are repeatable and whether rewind becomes available. A live network stream need not be rewindable.
  • Caller block exceptions propagate unchanged.
  • The body must make it possible for a future Rust wreq decoded-byte limit to stop reads at the same boundary; that native prerequisite should be filed separately in Rust wreq.

Acceptance criteria

  • Response#body returns a stable body object.
  • The body supports read, readpartial, each, close, and closed? with standard Ruby behavior.
  • Blockless each and Response#each_chunk return Enumerators.
  • EOF, explicit close, double close, early break, block failure, and competing-reader states are tested.
  • No normal body state raises Wreq::MemoryError or exposes Rust ownership terminology.
  • Streaming does not buffer the full response and continues to release the GVL while waiting for network data.

Ruby ecosystem precedent

  • HTTPX response handling treats a response body as both an IO-like reader and an enumerable, with explicit close semantics.
  • HTTP::Response::Body supports incremental reads, enumeration, and conversion to a complete string.
  • Net::HTTPResponse#read_body provides incremental block-based reads.
  • wreq-python exposes the same native response stream through iterable/context-managed Streamer and explicit response close APIs.

Relevant wreq-ruby source

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions