fix: include PID in ZMQ routing identity for minion/syndic REQ sockets - #70140
Open
waterWang wants to merge 1 commit into
Open
fix: include PID in ZMQ routing identity for minion/syndic REQ sockets#70140waterWang wants to merge 1 commit into
waterWang wants to merge 1 commit into
Conversation
salt-call (and other one-off processes that set __role=minion) collide with the long-running salt-minion daemon on the stable ZMQ identity `salt-req/minion/<minion_id>/0`, because _REQ_IDENTITY_SLOT restarts at 0 for every process. When the master's ROUTER has ROUTER_HANDOVER=1, the newcomer silently takes over the peer slot and drops any reply still in flight to the original owner, stalling the original process for the full REQUEST_TIMEOUT (~60s). Include os.getpid() in the identity so one-off processes never collide with the daemon's slots. Fixes saltstack#70127.
twangboy
requested changes
Aug 26, 2026
twangboy
left a comment
Contributor
There was a problem hiding this comment.
please create this against the 3008.x branch. Also, this will need a changelog and tests
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fix a 3006.27 regression where
salt-call(and other one-off processes that set__role=minion) collide with the long-runningsalt-miniondaemon on the stable ZMQ routing identity.Why was this needed?
Commit
36e914a72("Use stable ZMQ identity for minion + syndic daemon ret-port REQ sockets") introduced a stable identity perAsyncReqMessageClient:_REQ_IDENTITY_SLOTis a module-levelitertools.count()that starts at 0 per process. A runningsalt-miniondaemon and a one-offsalt-callboth setopts["__role"] = "minion"and share the sameminion_id, so both compute the identitysalt-req/minion/<minion_id>/0. Because the master's ROUTER socket hasROUTER_HANDOVER=1, libzmq hands the peer slot over to the newcomer and silently drops any reply still in flight to the original owner — causing requests to stall for the fullREQUEST_TIMEOUT(~60s) before retrying.What changed?
Include
os.getpid()in the identity string for theminion/syndicbranch, so one-off processes never collide with the daemon's identity slots. This mirrors the CLI branch which already usesos.getpid() % 256as the slot component.Tests written?
Commands signed with GPG?
Fixes #70127