Skip to content

fix(snode): gate SPDK start on the MCP's target render, not its last … - #1338

Open
wmousa wants to merge 1 commit into
mainfrom
fix/mcp-converge-target
Open

wmousa wants to merge 1 commit into
mainfrom
fix/mcp-converge-target

Conversation

@wmousa

@wmousa wmousa commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

…settled one

The converge gate read status.configuration.name as the pool's target. MCO only advances that field once the WHOLE pool has finished rolling, so for the duration of a rollout it still names the PREVIOUS render. A node that had already converged on the new render therefore compared unequal and waited for every other node in the pool; on any pool that takes longer than the 240x10s budget the wait expires with "did not converge on MCP config in time" and SPDK never starts. The check also passed for a node still sitting on the old config, which is the opposite of what the gate exists to assert.

Read spec.configuration.name instead -- the render controller sets it to the target as soon as it renders, so it is correct both mid-rollout and once settled.

Lift _cfg_hash out of its closure and add _mcp_target_config / _node_carries_config at module scope so the predicate is reachable from tests. It was previously inline in a 200-line function that also renders templates, creates Jobs and drives three k8s clients, which is why the suite had no reference to machineconfigpool at all.

…settled one

The converge gate read `status.configuration.name` as the pool's target.
MCO only advances that field once the WHOLE pool has finished rolling, so for
the duration of a rollout it still names the PREVIOUS render. A node that had
already converged on the new render therefore compared unequal and waited for
every other node in the pool; on any pool that takes longer than the 240x10s
budget the wait expires with "did not converge on MCP config in time" and SPDK
never starts. The check also passed for a node still sitting on the old
config, which is the opposite of what the gate exists to assert.

Read `spec.configuration.name` instead -- the render controller sets it to the
target as soon as it renders, so it is correct both mid-rollout and once
settled.

Lift `_cfg_hash` out of its closure and add `_mcp_target_config` /
`_node_carries_config` at module scope so the predicate is reachable from
tests. It was previously inline in a 200-line function that also renders
templates, creates Jobs and drives three k8s clients, which is why the suite
had no reference to machineconfigpool at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant