[Rust][Arrow] Move explicit close into supervisor - #680
Open
teodordelibasic-db wants to merge 5 commits into
Open
[Rust][Arrow] Move explicit close into supervisor#680teodordelibasic-db wants to merge 5 commits into
teodordelibasic-db wants to merge 5 commits into
Conversation
teodordelibasic-db
force-pushed
the
effort/zerobus-sdk-supervisor-owned-close
branch
2 times, most recently
from
August 6, 2026 14:33
a4094a4 to
f54f167
Compare
teodordelibasic-db
force-pushed
the
effort/zerobus-sdk-supervisor-owned-close
branch
13 times, most recently
from
August 12, 2026 15:24
fcbf50c to
7381d13
Compare
Signed-off-by: teodordelibasic-db <teodor.delibasic@databricks.com>
teodordelibasic-db
force-pushed
the
effort/zerobus-sdk-supervisor-owned-close
branch
from
August 12, 2026 19:25
7381d13 to
6ca543b
Compare
Supervisor panic or abort left close() waiting on CloseState forever. A detached reaper now finalizes the coordinator, and admission closes before the unacked snapshot is published. Signed-off-by: teodordelibasic-db <teodor.delibasic@databricks.com>
Continuously ready no-progress responses could starve recovery after a request send failure. One response-first tie is allowed; a buffered terminal status or EOF still wins over the local Unavailable error. Signed-off-by: teodordelibasic-db <teodor.delibasic@databricks.com>
A second now_or_never poll consumed a ready PutResult without applying it. Force send-failure on the next loop without polling another response. Signed-off-by: teodordelibasic-db <teodor.delibasic@databricks.com>
Signed-off-by: teodordelibasic-db <teodor.delibasic@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
Arrow Flight close previously split responsibility between foreground flush, connection rotation, recovery, and teardown. That made the result depend on which path observed a close or transport failure first.
This change gives the background supervisor sole ownership of close and terminal finalization:
Open -> Requested -> Finalizedclose coordinator. The firstclose()snapshots its SDK-offset target and absolute deadline under the ingestion mutex, so repeated or cancelled calls await the same request and result.close()waiting forever. A detached reaper finalizes the coordinator, preserves a returned error, and maps panic, cancel, or unexpected success to an invariant error while retaining the unacked suffix.is_closed, so a new ingest cannot return success in the snapshot window.close()re-reads coordinator state under the ingestion mutex so an unrepresentable flush deadline cannot mask a concurrently finalized peer error.PutResultis not discarded from the stream.A close error during recovery or server-requested rotation means that attempt was interrupted. It does not mean the close target is undurable. Callers must inspect
get_unacked_batches(); that set can be empty even whenclose()returns an error.The change is internal to the Rust Arrow Flight implementation. It does not change public signatures, FFI signatures, ABI, or semver compatibility.
Fixes #657.
How is this tested?
ack_before_request_is_latched_timely,request_before_ack_is_latched_timely,ack_at_deadline_is_not_latched_timely, andpreacked_close_preserves_latched_timeoutcover both ACK/publication orderings and the exact flush-deadline boundary.published_close_is_not_starved_by_ready_malformed_responsesandready_terminal_eof_precedes_published_empty_closecover response/close ordering without allowing a ready response stream to starve close.close_during_rotation_ack_wait_preserves_rotation_stateandtest_late_drain_ack_does_not_replace_close_timeoutcover rotation transitions, timeout selection, late ACK application, and rotation-trigger retention.close_publication_precedes_queued_replay_handoffandpublished_close_prevents_sender_commitcover the ingestion-mutex ordering boundary for replay and sender publication.test_close_during_reconnect_transport_handshake_preserves_trigger_and_suffixcovers cancellation while a replacement proxy CONNECT handshake is pending.test_close_during_reconnect_setup_preserves_recovery_triggercovers cancellation while the replacement waits for READY.test_close_after_partial_replay_preserves_trigger_and_suffixandtest_close_during_recovery_backoff_preserves_trigger_and_suffixcover recovery cancellation and exact suffix retention.test_close_during_second_recovery_attempt_preserves_latest_triggercovers close during attempt 2 returning attempt 1's reconnect error and the unacked suffix.test_supervisor_recovery_after_retriable_errorverifies that a committed replacement receives exactly one half-close and no reset.test_unrepresentable_runtime_close_deadline_does_not_publish_close,test_unrepresentable_runtime_recovery_deadline_is_rejected, andtest_close_during_recovery_backoff_preserves_trigger_and_suffixestablish real Flight streams before pausing Tokio time.test_replay_ack_deadline_starts_after_replay_completesprevents implicit clock advancement while its real transports are active.test_cancelled_close_rejects_ingest_and_resumes_teardowncovers cancellation-safe repeated close.test_supervisor_abort_after_close_request_finalizes_exact_suffixcovers abort after a published close request with bounded completion and the exact unacked suffix.non_finalized_supervisor_exit_is_an_invariant_errorandreaper_preserves_worker_error_and_rejects_unfinalized_successcover defensive terminal-state mapping.test_terminal_finalization_rejects_new_ingest_before_snapshot_publishandtest_empty_flush_waits_for_terminal_outcome_during_finalizationcover the admission-closed window beforeis_closed.ready_ack_is_applied_before_reported_request_send_failureandready_server_error_wins_reported_request_send_failurecover the one response-first tie.continuously_ready_nonprogress_responses_do_not_starve_send_failurecovers infinite no-progress and malformed responses.