Add salt.cache.etcd3_cache: native etcd v3 minion data cache backend (fixes #61037)#69242
Open
ggiesen wants to merge 1 commit into
Open
Add salt.cache.etcd3_cache: native etcd v3 minion data cache backend (fixes #61037)#69242ggiesen wants to merge 1 commit into
ggiesen wants to merge 1 commit into
Conversation
A from-scratch cache backend for the etcd v3 API. Each cache entry is a
single etcd key holding salt.payload.dumps({"d": data, "t": <epoch>}).
expires= is honoured natively via etcd v3 leases (so salt.auth tokens are
reaped by etcd rather than lingering until a manual flush). list/contains
follow salt.cache.localfs semantics, and listing uses keys-only range
scans so listing a bank does not transfer the stored values.
Connection setup (profile, auth, TLS) reuses
salt.utils.etcd_util.EtcdClientV3 and then operates on the etcd3-py client
directly. Enable with cache: etcd3 (requires the etcd3-py library).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds
salt.cache.etcd3_cache, a minion data cache backend for the etcd v3 API, enabled withcache: etcd3(requiresetcd3-py).It is a from-scratch v3-native design rather than a port of the v2
etcd_cache:salt.payload.dumps({"d": data, "t": <epoch>})-- an atomic single PUT, with no sibling timestamp key to coordinate.store(..., expires=N)is honoured natively via etcd v3 leases, sosalt.authtokens are reaped by etcd rather than persisting until a manual flush.list/containsfollowsalt.cache.localfssemantics (e.g.cache.list('minions')returns the minion IDs), and listing uses keys-only range scans so listing a bank does not transfer the stored values.etcd.path_prefix(a flush safety guard), and the docstring documents the in-etcd key layout and a prefix-scoped etcd RBAC role for shared-cluster isolation.Connection setup (profile, auth, TLS) reuses
salt.utils.etcd_util.EtcdClientV3; all data-plane operations then go through the etcd3-py client directly.What issues does this PR fix or reference?
Fixes #61037 -- provides the etcd v3 cache anticipated by that tech-debt item (and its closed parent #60325). It delivers v3 support as a separate
etcd3backend rather than retrofitting the v2etcd_cachemodule (v2 and v3 differ enough that one code path is more noise than signal); the v2 driver and its existing functional tests are unchanged. If the intent of #61037 was specifically to migrateetcd_cache.pyitself in place, please re-scope.Related / follow-ups
EtcdClientV3.__init__has a latent bug:self.encode_values = encode_values or self.conf.get("etcd.encode_values", True)-- theorshort-circuit means a caller passingencode_values=Falseis silently overridden by the default. This cache avoids the wrapper's codec entirely so it is not affected, but the bug is worth a separate fix (a bug fix, so 3006.x). Flagging rather than folding it in here.client.auth()call; I have suggested there that the call be guarded so it does not break no-auth clusters.Previous Behavior
No native etcd v3 cache backend; only the v2
etcd_cache(python-etcd / v2 API).New Behavior
cache: etcd3provides a v3-native minion data cache.Tests
run_common_cache_tests) plus etcd3-specific integration tests (single-key layout, trailing-slash bank boundary, native lease expiry, the in-tree bank/key patterns), both using the sharedtests.support.pytest.etcdcontainer fixture so they run in CI where docker is available.Validated locally against etcd v3.5: the unit suite, the full cache contract, and an end-to-end run through
salt.cache.factory(loader-resolvedcache: etcd3) including native lease expiry.Merge requirements satisfied?
doc/ref/cache/all)changelog/61037.added.md)Commits signed with GPG?
No