Transient read failures after Aws::S3::TransferManager#download_file with String destination path

[victorwong-panorama]

Describe the bug

Summary

We observed transient read failures immediately after Aws::S3::TransferManager#download_file with a String destination path. First attempt fails, retry succeeds.

Environment

• aws-sdk-s3: 1.213.0
• aws-sdk-core: 3.242.0
• Ruby: 4.0.3p0
• Runtime: Kubernetes / Linux
• Service: nds

Observed error

DuckDB::Error: IO Error: Could not read enough bytes from file "/projection-cache/bundled/.duckdb_hardlink_036c9176-bfd7-4228-a936-a14491ba0be4.duckdb": attempted to read 262144 bytes from location 1323008

Representative log:
2026-05-26T10:00:37.641503943Z stdout F pid=1 tid=4037d WARN: DuckDB::Error: IO Error: Could not read enough bytes from file "/projection-cache/bundled/.duckdb_hardlink_036c9176-bfd7-4228-a936-a14491ba0be4.duckdb": attempted to read 262144 bytes from location 1323008

Usage pattern

We use Aws::S3::TransferManager and call:

transfer_manager.download_file(destination_path, bucket: bucket, key: s3_key)
# then file is read immediately by another component

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

When using Aws::S3::TransferManager#download_file with a String destination path, the destination should never be observed in a partially written state by concurrent readers. Immediately reading the destination path after download_file returns should be safe and should not intermittently fail with short-read / incomplete-file errors.

Current Behavior

We intermittently see read failures immediately after download_file, then success on retry.

Representative error:
DuckDB::Error: IO Error: Could not read enough bytes from file "/projection-cache/bundled/.duckdb_hardlink_036c9176-bfd7-4228-a936-a14491ba0be4.duckdb": attempted to read 262144 bytes from location 1323008

Representative log:
2026-05-26T10:00:37.641503943Z stdout F pid=1 tid=4037d WARN: DuckDB::Error: IO Error: Could not read enough bytes from file "/projection-cache/bundled/.duckdb_hardlink_036c9176-bfd7-4228-a936-a14491ba0be4.duckdb": attempted to read 262144 bytes from location 1323008

Operational pattern:

• first attempt fails
• retry succeeds
  This suggests a transient file-availability/consistency window.

Reproduction Steps

1. Use Aws::S3::TransferManager and call:
   transfer_manager.download_file(destination_path, bucket: bucket, key: key)

2. Immediately open/read destination_path from another component/thread/process.

3. Repeat under concurrent load with repeated writes to the same destination path.

4. Observe occasional short-read/incomplete-read failures on first attempt; retry usually succeeds.

Notes:

• We cannot provide a minimal deterministic standalone repro yet (issue is intermittent).
• We observed this in Kubernetes production workloads.

Possible Solution

No response

Additional Information/Context

We suspect small-object behavior differences in Aws::S3::FileDownloader:

aws-sdk-ruby/gems/aws-sdk-s3/lib/aws-sdk-s3/file_downloader.rb

Line 140 in cef4dd5

def multipart_download(opts)

As a mitigation on our side, we now force temp destination + File.rename to final path at the application layer before exposing the final cache path.

Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version

aws-sdk-s3

Environment details (Version of Ruby, OS environment)

• aws-sdk-s3: 1.213.0 - aws-sdk-core: 3.242.0 - Ruby: 4.0.3p0 - Runtime: Kubernetes / Linux - Service: nds

[jterapin]

Thanks for reporting this, @victorwong-panorama. I've updated the issue title and fixed up the formatting (the rendered output wasn't displaying correctly).

I'll take a look and follow up here once I've had a chance to investigate.

[jterapin]

Hi! A couple of clarifying questions as I investigate:

1. Did you recently upgrade anything before noticing this behavior? (e.g., gem update, Ruby version bump, changes to your download/cache logic)
2. What's the approximate size of the file being downloaded? This helps me determine which internal code path is being taken. Based on the behavior mentioned above, I'm thinking it might be a single-object path.

For context: when a String path is passed as the destination, our multipart download path (files > 5MB) already writes to a temporary file and atomically renames to the destination on completion so this should not be an issue with larger files. It looks like the single-object download path (files ≤ 5MB) doesn't have the same protection, which aligns with what you're seeing. I'm leaning toward applying the same fix there.

[malomalo]

I am having the same issue I believe. When I download a 5.9Mb file to a string path then open the file it is 0 bytes in length, but no errors and when I download the file via browser it downloads. This is reproduce able every time for me. This bug started 5 months ago... so I'm not sure on the changes that could have affected it, could be a Ruby version.

[jterapin]

@malomalo Thanks for reporting this.

Few things:

1. What's your Ruby, aws-sdk-s3, and aws-sdk-core version?
2. How large is the object? and were there any configurations done when calling the download_file? Such as setting the :mode to "get_range"? That would force the ranged/multipart path, so I'm trying to chase down which path your download is actually taking.
3. When you read the file afterward, is that String path also tied to a File/Tempfile object you already had open? We've seen cases where the path and an existing open handle point at the same file and the handle's state is stale on re-read (e.g. position at EOF).

Ultimately, I think we should extend the same durability fix to the single-object path. I think this fix should amend the issue.

[malomalo]

I'm sorry the client was using a Linode S3 backend, and from my research I think that does not support Multipart, so no issue with the client.