Skip to content

fix: include PID in ZMQ routing identity for minion/syndic REQ sockets - #70140

Open
waterWang wants to merge 1 commit into
saltstack:masterfrom
waterWang:fix-70127-zmq-identity-pid
Open

fix: include PID in ZMQ routing identity for minion/syndic REQ sockets#70140
waterWang wants to merge 1 commit into
saltstack:masterfrom
waterWang:fix-70127-zmq-identity-pid

Conversation

@waterWang

Copy link
Copy Markdown

What does this PR do?

Fix a 3006.27 regression where salt-call (and other one-off processes that set __role=minion) collide with the long-running salt-minion daemon on the stable ZMQ routing identity.

Why was this needed?

Commit 36e914a72 ("Use stable ZMQ identity for minion + syndic daemon ret-port REQ sockets") introduced a stable identity per AsyncReqMessageClient:

identity = "salt-req/{role}/{minion_id}/{slot}".format(
    role=_role,
    minion_id=_minion_id,
    slot=next(_REQ_IDENTITY_SLOT),
)

_REQ_IDENTITY_SLOT is a module-level itertools.count() that starts at 0 per process. A running salt-minion daemon and a one-off salt-call both set opts["__role"] = "minion" and share the same minion_id, so both compute the identity salt-req/minion/<minion_id>/0. Because the master's ROUTER socket has ROUTER_HANDOVER=1, libzmq hands the peer slot over to the newcomer and silently drops any reply still in flight to the original owner — causing requests to stall for the full REQUEST_TIMEOUT (~60s) before retrying.

What changed?

Include os.getpid() in the identity string for the minion/syndic branch, so one-off processes never collide with the daemon's identity slots. This mirrors the CLI branch which already uses os.getpid() % 256 as the slot component.

Tests written?

  • Yes
  • No (identity generation is exercised by existing functional transport tests; the change is additive to the identity string format)

Commands signed with GPG?

  • Yes
  • No

Fixes #70127

salt-call (and other one-off processes that set __role=minion) collide
with the long-running salt-minion daemon on the stable ZMQ identity
`salt-req/minion/<minion_id>/0`, because _REQ_IDENTITY_SLOT restarts at 0
for every process. When the master's ROUTER has ROUTER_HANDOVER=1, the
newcomer silently takes over the peer slot and drops any reply still in
flight to the original owner, stalling the original process for the full
REQUEST_TIMEOUT (~60s).

Include os.getpid() in the identity so one-off processes never collide
with the daemon's slots. Fixes saltstack#70127.
@waterWang
waterWang requested a review from a team as a code owner August 26, 2026 05:11
@twangboy twangboy added the test:full Run the full test suite label Aug 26, 2026
@twangboy twangboy added this to the Argon v3008.3 milestone Aug 26, 2026

@twangboy twangboy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please create this against the 3008.x branch. Also, this will need a changelog and tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:full Run the full test suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: salt-call causes ~60s hangs on minions due to ZMQ identity collision with running salt-minion daemon (3006.27 regression)

2 participants