Give each deltaproxy sub-proxy its own schedule and beacon storage (#65088)#69788
Open
ggiesen wants to merge 1 commit into
Open
Give each deltaproxy sub-proxy its own schedule and beacon storage (#65088)#69788ggiesen wants to merge 1 commit into
ggiesen wants to merge 1 commit into
Conversation
…altstack#65088) subproxy_post_master_init builds each sub-proxy's opts with a shallow opts.copy(), so proxyopts["schedule"] and proxyopts["beacons"] were the same dict objects as the control minion's. The schedule and beacon helpers mutate those dicts in place -- Schedule.add_job does opts["schedule"].update(...) -- so every sub-proxy's add_job("__proxy_keepalive", ...) overwrote the same key in the one shared dict. Only the last sub-proxy kept a keepalive job, so only one of N sub-proxies got a __proxy_keepalive (the reported symptom); per sub-proxy beacons collided the same way. Give each sub-proxy its own schedule and beacon storage. Their jobs and beacons come from their own pillar plus the per-sub-proxy keepalive added below, so they were never meant to share the control minion's dicts (the sharing was an accident of the shallow copy).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Gives each deltaproxy sub-proxy its own
scheduleandbeaconsstorage, soper-sub-proxy
__proxy_keepalivejobs no longer collide.subproxy_post_master_initbuilds each sub-proxy's opts with a shallowopts.copy(), soproxyopts["schedule"]andproxyopts["beacons"]were thesame dict objects as the control minion's. The schedule and beacon helpers
mutate those dicts in place --
Schedule.add_jobdoesopts["schedule"].update(...)-- so every sub-proxy'sadd_job("__proxy_keepalive", ...)overwrote the same key in the one shareddict. Only the last sub-proxy kept a keepalive job, so only one of N sub-proxies
got a
__proxy_keepalive. Per-sub-proxy beacons collided the same way.Each sub-proxy's schedule and beacons come from its own pillar plus the
per-sub-proxy keepalive that
subproxy_post_master_initadds, so they werenever meant to share the control minion's dicts; the sharing was an accident of
the shallow copy. The fix gives each sub-proxy its own storage.
What issues does this PR fix or reference?
Fixes #65088
Previous Behavior
With N napalm sub-proxies, only one
__proxy_keepalivescheduled job existed(the reporter saw 1 of 8), because all sub-proxies shared the control minion's
schedule dict and each keepalive
add_joboverwrote the previous. Schedulesdefined in a sub-proxy's own pillar worked, because those come from
opts["pillar"]["schedule"](distinct per sub-proxy), which is why the reportersaw the pillar route work while
__proxy_keepalivedid not.New Behavior
Each sub-proxy gets its own schedule and beacon dict, so every sub-proxy keeps
its own
__proxy_keepalive(and any beacons), and the control minion's scheduleis untouched.
Merge requirements satisfied?
tests/pytests/unit/metaproxy/test_deltaproxy.py::test_subproxy_post_master_init_isolates_scheduleasserts each sub-proxy's
scheduleandbeaconsare distinct dict objects(not the control minion's and not each other's) and that the control minion's
dicts are left untouched. It fails against the pre-fix code (the sub-proxies
share the one dict).
Also reproduced and verified on real hardware: a napalm deltaproxy over live
Junos devices with four sub-proxies. Before the fix, sub-proxies shared a
schedule dict (colliding keepalives) and all shared one beacons dict; with the
fix each sub-proxy's schedule and beacon dicts are distinct.
Commits signed with GPG?
No