Fix hang when closing connections under load by nvartolomei · Pull Request #2575 · Azure/Azurite

Nicolae Vartolomei (nvartolomei) · 2025-07-28T14:55:08Z

fix: Prevent hang when client disconnects under load

In Azurite, all operations are managed through concurrent operation
queues. A bug was identified where operations could hang indefinitely if
the client disconnected before the operation was processed.

This occurred because Azurite would attempt to attach event handlers to
the request's readable stream (body) after it had already been closed by
the client's disconnection. Since a closed stream emits no further
events (like 'data', 'close', or 'error'), the operation would never
complete, causing a permanent hang.

This fix addresses the issue by checking if the request stream is still
readable before attaching any event handlers. This ensures that we only
process requests that are still active, preventing the hang and allowing
the queues to continue processing other operations.

Thanks for contribution! Please go through following checklist before sending PR.

PR Branch Destination

For Azurite V3, please send PR to main branch.
For legacy Azurite V2, please send PR to legacy-dev branch.

Always Add Test Cases

Make sure test cases are added to cover the code change.

Add Change Log

Add change log for the code change in Upcoming Release section in ChangeLog.md.

Development Guideline

Please go to CONTRIBUTION.md for steps about setting up development environment and recommended Visual Studio Code extensions.

Wei Wei (blueww) · 2025-08-05T08:27:46Z

Nicolae Vartolomei (@nvartolomei)
Thanks for the contribution!
It would be better if you could also add changelog.

xiaonlimsft (@XiaoningLiu) , EmmaZhu-MSFT (@EmmaZhu),
Would you please help to review the PR?

…erationQueue processes the request In Azurite, all operations are managed through concurrent operation queues. A bug was identified where operations could hang indefinitely if the client disconnected before the operation was processed. This occurred because Azurite would attempt to attach event handlers to the request's readable stream (body) after it had already been closed by the client's disconnection. Since a closed stream emits no further events (like 'data', 'close', or 'error'), the operation would never complete, causing a permanent hang. This fix addresses the issue by checking if the request stream is still readable before attaching any event handlers. This ensures that we only process requests that are still active, preventing the hang and allowing the queues to continue processing other operations.

Tyler Rockwood (rockwotj) · 2025-09-24T03:40:38Z

EmmaZhu-MSFT (@EmmaZhu) would be great to merge this!

Nicolae Vartolomei (nvartolomei) · 2026-06-01T14:14:07Z

Wei Wei (@blueww) mind taking another look at merging this? Anything blocking it?

Wei Wei (blueww) · 2026-06-02T07:27:08Z

Akanksha Jain (@jainakanksha-msft)

Would you please help to review and approve if no issue found?

Akanksha Jain (jainakanksha-msft) · 2026-08-13T10:14:47Z

Nicolae Vartolomei (@nvartolomei), could you please refresh the PR from main and address copilot comments if any

Copilot

Pull request overview

Fixes an indefinite hang in Azurite’s blob persistence write path by failing fast when an incoming request/body stream is already non-readable (e.g., client disconnected before the operation queue begins processing), and adds regression coverage plus a changelog entry.

Changes:

Add a readability guard in FSExtentStore.streamPipe() to reject immediately when the readable stream is already closed/destroyed.
Add a test that simulates an aborted/destroyed input stream to ensure appendExtent() does not hang.
Document the fix in the upcoming release changelog.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
tests/blob/fsStore.test.ts	Adds a regression test for destroyed/aborted input streams during `appendExtent()`.
src/common/persistence/FSExtentStore.ts	Rejects early in `streamPipe()` when the input stream is not readable to prevent hanging operations.
ChangeLog.md	Adds an “Upcoming Release” entry describing the hang fix.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+    try {
+      await store.appendExtent(stream1);
+      assert.fail("Expected an error to be thrown due to destroyed stream");
+    } catch (err) {
+      assert.deepStrictEqual(err.message, "FSExtentStore:streamPipe() Readable stream is not readable.");
+    }


+        reject(
+          new Error(
+            `FSExtentStore:streamPipe() Readable stream is not readable.`
+          ));


    assert.strictEqual(await readIntoString(readable3), "Test");
  });
+
+  it("should handle garbage collected input stream during appendExtent @loki", async () => {


Nicolae Vartolomei (nvartolomei) marked this pull request as draft July 28, 2025 16:07

This comment was marked as outdated.

Sign in to view

Nicolae Vartolomei (nvartolomei) mentioned this pull request Jul 28, 2025

tests/docker/azurite: run with a fork which does not leak conns redpanda-data/redpanda#27018

Merged

8 tasks

Nicolae Vartolomei (nvartolomei) force-pushed the nv/do-not-hang branch from c3b8e4f to 0551c02 Compare August 3, 2025 19:47

Nicolae Vartolomei (nvartolomei) marked this pull request as ready for review August 3, 2025 19:47

This comment was marked as resolved.

Sign in to view

Nicolae Vartolomei (nvartolomei) force-pushed the nv/do-not-hang branch from 0551c02 to 70d0d06 Compare August 3, 2025 19:58

EmmaZhu-MSFT (EmmaZhu) approved these changes Aug 6, 2025

View reviewed changes

Nicolae Vartolomei (nvartolomei) force-pushed the nv/do-not-hang branch from 70d0d06 to 7382c5c Compare August 6, 2025 12:27

Nicolae Vartolomei (nvartolomei) force-pushed the nv/do-not-hang branch from 7382c5c to fe3b28e Compare August 6, 2025 12:27

Wei Wei (blueww) approved these changes Jun 2, 2026

View reviewed changes

Wei Wei (blueww) assigned Akanksha Jain (jainakanksha-msft) Jun 2, 2026

Akanksha Jain (jainakanksha-msft) requested a lite review from Copilot August 13, 2026 10:14

Copilot started reviewing on behalf of Akanksha Jain (jainakanksha-msft) August 13, 2026 10:15 View session

Copilot AI reviewed Aug 13, 2026

View reviewed changes

Navigation Menu

Uh oh!

Fix hang when closing connections under load - #2575

Fix hang when closing connections under load#2575
Nicolae Vartolomei (nvartolomei) wants to merge 1 commit into
Azure:mainfrom
nvartolomei:nv/do-not-hang

Nicolae Vartolomei (nvartolomei) commented Jul 28, 2025 •

edited

Loading

Uh oh!

This comment was marked as outdated.

This comment was marked as resolved.

Wei Wei (blueww) commented Aug 5, 2025

Uh oh!

Tyler Rockwood (rockwotj) commented Sep 24, 2025

Uh oh!

Nicolae Vartolomei (nvartolomei) commented Jun 1, 2026

Uh oh!

Wei Wei (blueww) commented Jun 2, 2026

Uh oh!

Akanksha Jain (jainakanksha-msft) commented Aug 13, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

Nicolae Vartolomei (nvartolomei) commented Jul 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Branch Destination

Always Add Test Cases

Add Change Log

Development Guideline

Uh oh!

This comment was marked as outdated.

This comment was marked as resolved.

Wei Wei (blueww) commented Aug 5, 2025

Uh oh!

Tyler Rockwood (rockwotj) commented Sep 24, 2025

Uh oh!

Nicolae Vartolomei (nvartolomei) commented Jun 1, 2026

Uh oh!

Wei Wei (blueww) commented Jun 2, 2026

Uh oh!

Akanksha Jain (jainakanksha-msft) commented Aug 13, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Nicolae Vartolomei (nvartolomei) commented Jul 28, 2025 •

edited

Loading