fix: do not open listen events socket in webhook - #70158
Closed
CrazyHackGUT wants to merge 3751 commits into
Closed
Conversation
…-3007x Refresh install/onedir/FAQ/non-root docs (3007.x)
…-3008x Refresh install/onedir/FAQ/non-root docs (3008.x)
Forward-merge 3006.x fixes originating in PRs saltstack#66282, saltstack#68493, saltstack#68930, saltstack#69442, saltstack#69478, saltstack#69614. Conflicts resolved: - doc/topics/development/salt_extensions.rst: took downstream form - salt/crypt.py: forward-ported saltstack#69443 auth_retries outer-loop cap - salt/transport/tcp.py: forward-ported saltstack#66282 concurrent publish_payload writes; adapted to 3007.x async def / await - salt/transport/zeromq.py: forward-ported saltstack#69478 _set_zmq_heartbeat helper; preserved 3007.x raw-socket publisher (no ZMQStream) - salt/utils/asynchronous.py: took HEAD (tornado>=6 asyncio interop) - tests/pytests/unit/test_crypt.py: forward-ported four 3006.x regression tests (saltstack#68493 whitespace-drift, saltstack#68930 gen_signature clean_key, saltstack#69442 auth_retries cap); tornado adaptation - tests/pytests/unit/utils/test_asynchronous.py: union of HelperA/ HelperB tests + new 65702 SyncWrapper asyncio-loop regression test - Lock files + workflow yamls: regenerated via pre-commit DU resolved by removal: - salt/returners/mysql.py - tests/pytests/unit/returners/test_mysql.py
3006.x introduced tests/pytests/functional/transport/tcp/test_pub_server_stability.py via saltstack#66282 using salt.ext.tornado.gen / .ioloop. 3007.x+ removed salt.ext.tornado — adapt imports to plain tornado.gen / tornado.ioloop. tornado.gen.coroutine-returned Futures remain awaitable from async def, so no logic changes needed.
TCPPuller.handle_stream (merged forward from 3006.x #d4e2e075aa3)
called self.io_loop.spawn_callback() to invoke payload_handler. That
API only exists on tornado.ioloop.IOLoop; on 3007.x+ this class runs
under a raw asyncio loop (_UnixSelectorEventLoop), so package tests
saw 'EventPublisher: Exception occurred while handling stream:
_UnixSelectorEventLoop object has no attribute spawn_callback' on
every event and the salt-master service failed to reach a running
state (test_salt_downgrade_minion timed out at factory start).
Match 3008.x/master (salt/transport/tcp.py:1510):
self.io_loop.create_task(self.payload_handler(body))
Fixes CI Rocky Linux 9 downgrade 3007.14 on PR saltstack#69622.
…-26-06-30 Merge forward 3006.x into 3007.x
When a prereq chain is set up such that one state prereq's another state that itself prereq's a third state (e.g. state1 --prereq--> state2 --prereq--> state3), the intermediate state's own prereq check node was created without its own prereq requirements. This allowed the intermediate state to be evaluated in test mode independently of the tail of the chain, so any state that always proposes changes in test mode (e.g. test.succeed_with_changes, module.run) caused the head of the chain to run even when the tail produced no changes. When adding a new prereq requisite (_add_prereq) from a chunk that already has its own prereq check node (because another state prereq's it), also register a PREREQ edge from the requisite's prereq check node into the chunk's own prereq check node. This ensures the intermediate prereq check waits for the tail's prereq check outcome before deciding whether to run in test mode. Also confirms the pre-existing regression test saltstack#68438 now passes on Windows and Linux. Fixes saltstack#68438
When a prereq chain is set up such that one state prereq's another state that itself prereq's a third state (e.g. state1 --prereq--> state2 --prereq--> state3), the intermediate state's own prereq check node was created without its own prereq requirements. This allowed the intermediate state to be evaluated in test mode independently of the tail of the chain, so any state that always proposes changes in test mode (e.g. test.succeed_with_changes, module.run) caused the head of the chain to run even when the tail produced no changes. When adding a new prereq requisite (_add_prereq) from a chunk that already has its own prereq check node (because another state prereq's it), also register a PREREQ edge from the requisite's prereq check node into the chunk's own prereq check node. This ensures the intermediate prereq check waits for the tail's prereq check outcome before deciding whether to run in test mode. Also confirms the pre-existing regression test saltstack#68438 now passes on Windows and Linux. Fixes saltstack#68438
Restore the ``Test Salt / Rocky Linux 9 unit zeromq 4`` CI job to green after the 3006.x→3007.x→3008.x merge-forward chain pulled 3006.x-only regression tests into 3007.x whose expectations don't match the 3007.x runtime API surface. - ``test_verify_master_accepts_cached_key_with_whitespace_drift``, ``test_verify_master_caches_clean_key_on_first_contact``, ``test_authenticate_caps_retry_loop_with_auth_retries_69442``, ``test_authenticate_default_does_not_cap_retry_loop_69442``: switch from the removed 3006.x ``crypt.gen_keys(pki_dir, name, keysize)`` signature to the 3007.x ``crypt.write_keys(...)`` equivalent, and add ``keys.cache_driver`` to opts so ``AsyncAuth.__singleton_init__`` can construct the keystore cache. - ``test_gen_signature_signs_clean_key`` and ``test_gen_signature_signs_clean_key_trailing_newline``: skip on 3007.x. The module-level ``salt.crypt.gen_signature`` was removed by the master-pki cache refactor; the replacement ``MasterKeys.gen_signature`` signs ``pub.public_bytes()`` from a key object rather than the raw file content, so the saltstack#68930 whitespace- drift bug the tests were written against is not reachable on 3007.x. - ``test_maintenance_duration``: add ``eauth_tokens.cache_driver``, ``eauth_tokens.cluster_id``, and ``cluster_id`` to the test's opts dict. ``Maintenance._post_fork_init`` now constructs a long-lived ``LoadAuth`` (as part of the memory-leak fix that caches it across loop iterations); ``LoadAuth.__init__`` reads those keys. - ``test_minion_manager_stop_unblocks_resolve_dns_69466``: assert on either ``io_loop.create_task`` or ``io_loop.add_callback`` being called once. 3007.x refactored ``MinionManager.stop()`` to use ``create_task`` instead of ``add_callback``; the 3006.x-origin test hard-coded the older form. - ``test_event_unpack_with_SaltDeserializationError``: assert on the new debug-level "skipping malformed event (deserialization error)" message that the memory-leak hardening emits from ``SaltEvent._get_event`` instead of the pre-hardening ``log.error("Unable to deserialize received event")`` call the test originally targeted. The hardening intentionally demotes the log level so a single bad IPC frame cannot spam the operator log.
The test used a fixed time.sleep(2) after spawning the salt CLI before sending SIGINT. On slow CI hosts (observed on Photon OS 5 Arm64, both tcp(fips) and zeromq(fips) integration lanes) the CLI had not yet published its job when the signal arrived. Its scripts._handle_signals path then took the AttributeError/KeyError branch (no pub_data), emitted only "Exiting gracefully on Ctrl-c", and skipped the "This job's jid is" message the test asserts on. Wait on the master's salt/job/*/new event via event_listener instead. That guarantees pub_data is populated in the CLI process before we interrupt it, so the jid-bearing signal-handler branch always runs. Fixes flakiness in tests/pytests/integration/cli/test_salt.py::test_interrupt_on_long_running_job seen at: https://github.com/saltstack/salt/actions/runs/28505613358/job/84502049475 https://github.com/saltstack/salt/actions/runs/28505613358/job/84502049337
Restore the ``Test Salt / Rocky Linux 9 unit zeromq 4`` CI job to green after the 3006.x→3007.x→3008.x merge-forward chain pulled 3006.x-only regression tests into 3007.x whose expectations don't match the 3007.x runtime API surface. - ``test_verify_master_accepts_cached_key_with_whitespace_drift``, ``test_verify_master_caches_clean_key_on_first_contact``, ``test_authenticate_caps_retry_loop_with_auth_retries_69442``, ``test_authenticate_default_does_not_cap_retry_loop_69442``: switch from the removed 3006.x ``crypt.gen_keys(pki_dir, name, keysize)`` signature to the 3007.x ``crypt.write_keys(...)`` equivalent, and add ``keys.cache_driver`` to opts so ``AsyncAuth.__singleton_init__`` can construct the keystore cache. - ``test_gen_signature_signs_clean_key`` and ``test_gen_signature_signs_clean_key_trailing_newline``: skip on 3007.x. The module-level ``salt.crypt.gen_signature`` was removed by the master-pki cache refactor; the replacement ``MasterKeys.gen_signature`` signs ``pub.public_bytes()`` from a key object rather than the raw file content, so the saltstack#68930 whitespace- drift bug the tests were written against is not reachable on 3007.x. - ``test_maintenance_duration``: add ``eauth_tokens.cache_driver``, ``eauth_tokens.cluster_id``, and ``cluster_id`` to the test's opts dict. ``Maintenance._post_fork_init`` now constructs a long-lived ``LoadAuth`` (as part of the memory-leak fix that caches it across loop iterations); ``LoadAuth.__init__`` reads those keys. - ``test_minion_manager_stop_unblocks_resolve_dns_69466``: assert on either ``io_loop.create_task`` or ``io_loop.add_callback`` being called once. 3007.x refactored ``MinionManager.stop()`` to use ``create_task`` instead of ``add_callback``; the 3006.x-origin test hard-coded the older form. - ``test_event_unpack_with_SaltDeserializationError``: assert on the new debug-level "skipping malformed event (deserialization error)" message that the memory-leak hardening emits from ``SaltEvent._get_event`` instead of the pre-hardening ``log.error("Unable to deserialize received event")`` call the test originally targeted. The hardening intentionally demotes the log level so a single bad IPC frame cannot spam the operator log.
The test used a fixed time.sleep(2) after spawning the salt CLI before sending SIGINT. On slow CI hosts (observed on Photon OS 5 Arm64, both tcp(fips) and zeromq(fips) integration lanes) the CLI had not yet published its job when the signal arrived. Its scripts._handle_signals path then took the AttributeError/KeyError branch (no pub_data), emitted only "Exiting gracefully on Ctrl-c", and skipped the "This job's jid is" message the test asserts on. Wait on the master's salt/job/*/new event via event_listener instead. That guarantees pub_data is populated in the CLI process before we interrupt it, so the jid-bearing signal-handler branch always runs. Fixes flakiness in tests/pytests/integration/cli/test_salt.py::test_interrupt_on_long_running_job seen at: https://github.com/saltstack/salt/actions/runs/28505613358/job/84502049475 https://github.com/saltstack/salt/actions/runs/28505613358/job/84502049337
…7-01 Merge forward 3007.x into 3008.x
…prereq-chain-dag Fix prereq chain regression in nested prereq DAG setup (saltstack#68438)
Two bugs kept the Sync .lock files job from running for 3008.x PRs, leaving lock files stale: - on.pull_request.branches omitted 3008.x (it only listed master and 3006.x), so the workflow never triggered for PRs targeting 3008.x. Add all four release branches. - The actor guard only matched 'dependabot', so it skipped whenever the salt-pr-bot rebase bot re-pushed a branch. Also fire for salt-pr-bot.
Two bugs kept the Sync .lock files job from running for 3007.x PRs, leaving lock files stale: - on.pull_request.branches omitted 3007.x (it only listed master and 3006.x), so the workflow never triggered for PRs targeting 3007.x. Add all four release branches. - The actor guard only matched 'dependabot', so it skipped whenever the salt-pr-bot rebase bot re-pushed a branch. Also fire for salt-pr-bot.
…c-fix-3007.x Fix dependabot lock-sync workflow triggering on 3007.x
…c-fix-3008.x Fix dependabot lock-sync workflow triggering on 3008.x
…ltstack#69455) The unmaintained `linode-python` 1.1.1 package targets the retired Linode API v3 and uses `is not 0` / `is 1` against literals, which Python 3.12+ emits as `SyntaxWarning` from `linode/api.py` lines 293, 348, and 356. On RHEL/Rocky/Oracle Linux 9.x, the salt-common onedir's post-install scriptlet imports the onedir's Python which in turn imports `linode-python` and the warnings surface during every package install/upgrade. Drop `linode-python` from `requirements/static/pkg/linux.txt` and its dependent CI/pkg linux lockfiles so it is no longer installed into the salt-common onedir. `salt.cloud.clouds.linode` already uses the Linode APIv4 over HTTP/JSON (no `linode-python` import), so the package is purely vestigial. This backports saltstack#69339 (3006.x) and mirrors saltstack#68871 (master/3008.x) to 3007.x. Fixes saltstack#69455
The Fedora 40 skips on test_peer_communication and test_grains_remove_add were added in 5a85699 (2024-05-17) as a temporary workaround referencing saltstack#66539 and saltstack#66540 with no root cause captured. Fedora 40 reached end-of-life on 2025-05-13 and the tests now pass without the workaround on 3007.x and master, so the skips can be dropped. Fixes saltstack#66540 Fixes saltstack#66539
…69636) Since 3008, pillar.get masks scalar string values by default, so signing policies fetched from pillar came back as '**********' and certificate generation failed with "Bad decrypt - is the password correct?". Pass unmask=True so the policy retains its real values. Fixes: saltstack#69253 (cherry picked from commit 27307c5) Signed-off-by: Teddy Andrieux <teddy.andrieux@scality.com>
* fix timeout for salt-api * Add changelog and regression test for salt-api timeout fix Addresses @twangboy's CHANGES_REQUESTED on PR saltstack#62188. - changelog/62187.fixed.md describes the salt-api hang fix. - test_mk_token_missing_password_returns_empty pins the missing-password /username path: mk_token must return {} instead of raising SaltInvocationError, which previously escaped through the master clear-payload handler and hung salt-api workers for ~3 minutes per bad request. Co-authored-by: carrysauce <carrysauce@users.noreply.github.com> * Apply reviewer suggestions: separate try/except for format_call, rename changelog to PR number - Give format_call its own try/except block catching SaltInvocationError with a descriptive debug message, as requested by @twangboy - Rename changelog/62187.fixed.md to changelog/62188.fixed.md (PR number, not issue number) --------- Co-authored-by: Alex Donec <alex.donec@pinely.com> Co-authored-by: Daniel A. Wozniak <dwozniak@broadcom.com> Co-authored-by: carrysauce <carrysauce@users.noreply.github.com> Co-authored-by: Daniel A. Wozniak <daniel.wozniak@broadcom.com>
saltstack#68784) * fix(pip): normalize names in list_freeze_parse and detect modern pip satisfied message * ref: change string formatting * enh: add PR suggestions
…#69712) Since 3008, pillar.get masks scalar string values by default, so modules that read pillar and use the value operationally were receiving the redaction placeholder instead of the real data. Pass unmask=True at each genuine consumer call site, matching the file.managed contents_pillar path and the x509_v2 _get_signing_policy fix (saltstack#69636): - gpg: signing/encryption passphrase (create_key, delete_key, export_key, sign, encrypt, decrypt) - debuild_pkgbuild / rpmbuild_pkgbuild: signing passphrase and public and private key names - x509 / ssh_pki: signing policies - tls: certificate extension definitions - oracle: database connection data (show_dbs consumed by run_query) - pyobjects: the Map renderer merge pillar Display-only reads (oracle show_dbs no-arg, show_pillar) are left masked by design. The pyobjects renderer pillar()/SaltObject shortcuts have the same masking exposure and are left for a separate render-context fix. Refs saltstack#69711
saltstack#69714) Dependabot bumps the shared floor for these packages to a version that no longer supports Python 3.9 (jaraco.functools 4.5.0, jaraco.context 6.1.2, msgpack 1.2.1, opentelemetry 1.43.0 / exporter-prometheus 0.64b0, pyopenssl 26.3.0, virtualenv 21.5.1, xmldiff 3.0, xxhash 3.8.0, zipp 4.1.0). With a single unmarked floor the py3.9 lock targets become unresolvable and every grouped pip-updates PR fails pre-commit/lint/build. Split each into a python_version < '3.10' branch capped at the last 3.9-compatible release plus an open py>=3.10 branch, mirroring the existing cryptography/aiohttp/urllib3 splits. Dependabot then only bumps the uncapped py>=3.10 line and leaves 3.9 resolvable. No lock changes are needed now (uv already backtracks py3.9 to these versions); this only future-proofs the input floors.
saltstack#69715) Dependabot bumps the shared floor for these packages to a version that no longer supports Python 3.9 (jaraco.functools 4.5.0, jaraco.context 6.1.2, msgpack 1.2.1, opentelemetry 1.43.0 / exporter-prometheus 0.64b0, pyopenssl 26.3.0, virtualenv 21.5.1, xmldiff 3.0, xxhash 3.8.0, zipp 4.1.0). With a single unmarked floor the py3.9 lock targets become unresolvable and every grouped pip-updates PR fails pre-commit/lint/build. Split each into a python_version < '3.10' branch capped at the last 3.9-compatible release plus an open py>=3.10 branch, mirroring the existing cryptography/aiohttp/urllib3 splits. Dependabot then only bumps the uncapped py>=3.10 line and leaves 3.9 resolvable. No lock changes are needed now (uv already backtracks py3.9 to these versions); this only future-proofs the input floors.
…#70090) Round-2 verification of PR saltstack#70145 found that the original scenario test in tests/pytests/scenarios/cluster/test_haproxy_isolated_fs.py did not distinguish the fix from the pre-fix code: with the default ``localfs_key`` driver the master-keys cache and the on-disk PEM are the same file, so the cache-refresh in the fix is a tautology and the assertion passes on both sides. Attempting to force ``mmap_key`` in the fixture surfaced a separate 2-master isolated-cluster bring-up bug (peer keys never propagate under mmap_key discover, "Peer key missing" loops), which blocks scenario coverage of the reporter's exact driver until that separate issue is resolved. To actually anchor the fix add a focused unit test in tests/pytests/unit/channel/test_server.py that drives ``MasterPubServerChannel.handle_pool_publish`` with a synthesized ``cluster/peer/join-reply`` payload and asserts: - ``master_key.cache.store("master_keys", "cluster.pem", <wire>)`` is called (regression: pre-fix ``store()`` is never invoked, cache serves the joiner's placeholder to auth replies under mmap_key). - ``master_key.cache.store("master_keys", "cluster.pub", <wire>)`` is called with the wire pub bytes. - ``master_key.cluster_key`` and ``master_key.key`` are rebound from the wire-delivered PEM so the running process signs cluster events with the shared identity immediately (no restart needed). The unit test passes with the fix and fails with a clear ``Fix regression: join-reply handler must write cluster.pem through master_key.cache.store...`` message when the fix's +40-line block in salt/channel/server.py is reverted -- verified locally by git-checkout of the parent commit. The scenario test is retained as a smoke test for the wire-delivery path (``test_isolated_cluster_pem_propagates`` covers the same invariant more thoroughly) with a docstring calling out its tautology under ``localfs_key`` and the mmap_key follow-up work.
…-08-26-26 # Conflicts: # .github/workflows/ci.yml # .github/workflows/depcheck.yml # .github/workflows/nightly-stress-test.yml # .github/workflows/nightly.yml # .github/workflows/scheduled.yml # .github/workflows/staging.yml # .github/workflows/templates/layout.yml.jinja # .pre-commit-config.yaml # doc/topics/development/conventions/style.rst # noxfile.py # requirements/base.txt # requirements/constraints.txt # requirements/static/ci/common.txt # requirements/static/ci/py3.10/cloud.lock # requirements/static/ci/py3.10/darwin.lock # requirements/static/ci/py3.10/docs.lock # requirements/static/ci/py3.10/freebsd.lock # requirements/static/ci/py3.10/lint.lock # requirements/static/ci/py3.10/linux.lock # requirements/static/ci/py3.10/windows.lock # requirements/static/ci/py3.11/cloud.lock # requirements/static/ci/py3.11/darwin.lock # requirements/static/ci/py3.11/docs.lock # requirements/static/ci/py3.11/freebsd.lock # requirements/static/ci/py3.11/lint.lock # requirements/static/ci/py3.11/linux.lock # requirements/static/ci/py3.11/windows.lock # requirements/static/ci/py3.12/cloud.lock # requirements/static/ci/py3.12/darwin.lock # requirements/static/ci/py3.12/docs.lock # requirements/static/ci/py3.12/freebsd.lock # requirements/static/ci/py3.12/lint.lock # requirements/static/ci/py3.12/linux.lock # requirements/static/ci/py3.12/windows.lock # requirements/static/ci/py3.13/cloud.lock # requirements/static/ci/py3.13/darwin.lock # requirements/static/ci/py3.13/docs.lock # requirements/static/ci/py3.13/freebsd.lock # requirements/static/ci/py3.13/lint.lock # requirements/static/ci/py3.13/linux.lock # requirements/static/ci/py3.13/windows.lock # requirements/static/ci/py3.14/cloud.lock # requirements/static/ci/py3.14/darwin.lock # requirements/static/ci/py3.14/docs.lock # requirements/static/ci/py3.14/freebsd.lock # requirements/static/ci/py3.14/lint.lock # requirements/static/ci/py3.14/linux.lock # requirements/static/ci/py3.14/windows.lock # requirements/static/ci/py3.9/freebsd.lock # requirements/static/ci/py3.9/lint.lock # requirements/static/pkg/freebsd.txt # requirements/static/pkg/linux.txt # requirements/static/pkg/py3.10/darwin.lock # requirements/static/pkg/py3.10/freebsd.lock # requirements/static/pkg/py3.10/linux.lock # requirements/static/pkg/py3.10/windows.lock # requirements/static/pkg/py3.11/darwin.lock # requirements/static/pkg/py3.11/freebsd.lock # requirements/static/pkg/py3.11/linux.lock # requirements/static/pkg/py3.11/windows.lock # requirements/static/pkg/py3.12/darwin.lock # requirements/static/pkg/py3.12/freebsd.lock # requirements/static/pkg/py3.12/linux.lock # requirements/static/pkg/py3.12/windows.lock # requirements/static/pkg/py3.13/darwin.lock # requirements/static/pkg/py3.13/freebsd.lock # requirements/static/pkg/py3.13/linux.lock # requirements/static/pkg/py3.13/windows.lock # requirements/static/pkg/py3.14/darwin.lock # requirements/static/pkg/py3.14/freebsd.lock # requirements/static/pkg/py3.14/linux.lock # requirements/static/pkg/py3.14/windows.lock # requirements/static/pkg/py3.9/freebsd.lock # salt/modules/tls.py # tools/pkg/build.py
…-08-25-26 # Conflicts: # changelog/69986.fixed.md
The merge-forward from 3006.x kept the reverted `keybase.io/codecovsecurity` signing-key URL (HTTP 404), which fails gpg --import under bash -e. Point the Codecov Uploader Verification Key fetch at `https://uploader.codecov.io/verification.gpg` (same fingerprint, `27034E7FDB850E0BBC2C62FF806BB28AED779869`), matching 3006.x/master.
…6-26 Merge forward 3006.x into 3007.x
…solated-haproxy [3008.x] Fix cluster isolated-filesystem auth failure behind HAProxy (saltstack#70090)
…-08-25-26 # Conflicts: # .github/workflows/ci.yml # .github/workflows/depcheck.yml # .github/workflows/nightly.yml # .github/workflows/scheduled.yml # .github/workflows/staging.yml # .github/workflows/templates/ci.yml.jinja # .pre-commit-config.yaml # cicd/shared-gh-workflows-context.yml # requirements/constraints.txt # requirements/static/ci/py3.10/freebsd.lock # requirements/static/ci/py3.10/lint.lock # requirements/static/ci/py3.10/linux.lock # requirements/static/ci/py3.10/windows.lock # requirements/static/ci/py3.11/cloud.lock # requirements/static/ci/py3.11/darwin.lock # requirements/static/ci/py3.11/freebsd.lock # requirements/static/ci/py3.11/lint.lock # requirements/static/ci/py3.11/linux.lock # requirements/static/ci/py3.11/windows.lock # requirements/static/ci/py3.12/freebsd.lock # requirements/static/ci/py3.12/lint.lock # requirements/static/ci/py3.12/windows.lock # requirements/static/ci/py3.13/freebsd.lock # requirements/static/ci/py3.13/lint.lock # requirements/static/ci/py3.13/windows.lock # requirements/static/ci/py3.14/cloud.lock # requirements/static/ci/py3.14/darwin.lock # requirements/static/ci/py3.14/docs.lock # requirements/static/ci/py3.14/freebsd.lock # requirements/static/ci/py3.14/lint.lock # requirements/static/ci/py3.14/linux.lock # requirements/static/ci/py3.14/windows.lock # requirements/static/ci/py3.9/freebsd.lock # requirements/static/ci/py3.9/lint.lock # requirements/static/pkg/py3.10/freebsd.lock # requirements/static/pkg/py3.10/windows.lock # requirements/static/pkg/py3.11/freebsd.lock # requirements/static/pkg/py3.11/windows.lock # requirements/static/pkg/py3.12/darwin.lock # requirements/static/pkg/py3.12/freebsd.lock # requirements/static/pkg/py3.12/linux.lock # requirements/static/pkg/py3.12/windows.lock # requirements/static/pkg/py3.13/darwin.lock # requirements/static/pkg/py3.13/freebsd.lock # requirements/static/pkg/py3.13/linux.lock # requirements/static/pkg/py3.13/windows.lock # requirements/static/pkg/py3.14/darwin.lock # requirements/static/pkg/py3.14/freebsd.lock # requirements/static/pkg/py3.14/linux.lock # requirements/static/pkg/py3.14/windows.lock # requirements/static/pkg/py3.9/freebsd.lock # salt/channel/server.py # salt/modules/tls.py # tests/integration/modules/test_tls.py # tests/pytests/functional/utils/test_process.py # tests/pytests/unit/channel/test_server.py # tools/pkg/build.py
- Bump python_version 3.14.6 -> 3.14.7 to match relenv 0.22.25 assets (relenv 0.22.25 ships 3.14.7, not 3.14.6, causing 'HTTP Error 404: Not Found' in Build Salt Onedir for every platform). - Regenerate 8 freebsd lock files via pre-commit to consolidate clr-loader/pythonnet conditionals that pip-compile wanted to simplify (python_full_version < '3.13' -> just sys_platform == 'win32' since the Windows onedir line uses 3.13+ everywhere except py3.9's 3.10 boundary).
relenv 0.22.25 ships Python 3.14.7 whose sysconfig no longer sets -undefined dynamic_lookup in LDSHARED. Source-built extensions on macOS (e.g. timelib) then fail to link because they reference private CPython symbols like _PyBaseObject_Type.
…ite_buffer defaults Four xfail(strict=True) tests for four backpressure findings against salt.transport.tcp on 3008.x head, all cross-corroborated by two independent fuzz runs on 2026-08-26: * test_pub_server_stream_read_unpacker_has_max_buffer_size_cap (P1) -- PubServer._stream_read constructs salt.utils.msgpack.Unpacker() with no max_buffer_size, pinning ~1 MB C-heap per subscriber on msgpack 1.2.1. * test_master_default_ipc_write_buffer_is_bounded (P3 / R2-2026-08) -- the ipc_write_buffer opt exists on 3008.x but defaults to 0 (unbounded), so slow subscribers grow the per-stream write buffer to ~47 MB before publish_drain_timeout fires. * test_pub_server_publish_payload_caps_in_flight_drain_tasks (R1-2026-08 / N2) -- publish_payload schedules one asyncio.ensure_future() per subscriber per event with no cap; a 20k-event burst against 8 subscribers produced 160k pending drain tasks and drove RSS to 820 MB (100k burst hit 2.9 GB and starved the io_loop). Asserts <= 64 outstanding tasks after 1000 publishes to one subscriber. * test_discard_slow_client_cancels_pending_drain_tasks (N1) -- _discard_slow_client does not cancel in-flight drain tasks for the discarded subscriber; their closures pin the payload bytes and client reference for up to publish_drain_timeout seconds. The R1-2026-08 / N1 fixes will remove the xfail markers on the last two tests in the next commit. P1 and P3 fixes are separate work items and their xfail markers stay for the corresponding future PRs.
Under a bursty producer the pre-fix path scheduled one ``asyncio.ensure_future(_make_drain_task(client)(fut))`` per (subscriber x event) with no cap. A 20k-event burst against 8 subscribers produced 160k pending drain tasks (RSS 820 MB); 100k events reached 2.9 GB RSS and starved the io_loop hard enough that consumer subscribers stopped receiving mid-burst. Each drain task pinned an asyncio Task + Future + coroutine + Context + Handle + TimerHandle transitively. Give each Subscriber a per-subscriber drain coroutine reading write futures from a bounded ``asyncio.Queue`` (default 500, configurable via the new ``pub_server_write_queue_size`` master opt). ``publish_payload`` still calls ``stream.write(payload)`` inline (so byte ordering and side effects match the pre-fix semantics), then enqueues the returned future for the drainer to ``asyncio.wait_for``. In-flight drain tasks are now capped at one per subscriber regardless of burst size. When the queue fills, the subscriber is treated as slow and disconnected -- the same fast-fail path ``StreamBufferFullError`` already takes. Also fix N1: ``_discard_slow_client`` now cancels the drainer task and drops its queue entry, so the payload futures captured by the drainer are released immediately rather than pinned until each queued ``wait_for`` hits its ``publish_drain_timeout`` (up to 5s per pending future -- 46 MB retained-bytes measured in the fuzz report). Removes the ``xfail`` markers on ``test_pub_server_publish_payload_caps_in_flight_drain_tasks`` and ``test_discard_slow_client_cancels_pending_drain_tasks`` in ``tests/pytests/unit/transport/test_tcp_pubserver_backpressure.py``; the P1 and P3 xfails there stay (separate fixes). Fixes saltstack#70147
The 500-slot per-subscriber drain queue introduced in the prior commit was too tight for realistic event bursts. Under the CI stress rig, a producer that fires 5000 events in one broadcast pushes publish_payload through 5000 sequential enqueues before the drainer coroutine gets a scheduler slot; the writer trips QueueFull well before the drainer has a chance to make forward progress, and the subscriber gets a false-positive "slow" discard. The failure showed up as ConnectionResetError: EP pub socket closed on test_throughput_floor_single_subscriber and test_multi_subscriber_fanout across every Linux distro's integration split-7 job (zeromq and tcp), reproducible locally in a single-second run of the same stress test. Two changes: 1. Raise ``_DEFAULT_WRITE_QUEUE_MAXSIZE`` from 500 to 10000 (and mirror in ``DEFAULT_MASTER_OPTS[pub_server_write_queue_size]``). Each queue slot holds one tornado Future (a few hundred bytes), so 10k slots x 200 subscribers is ~2 MB retained -- vs. the pre-fix path's 820 MB / 2.9 GB in fire-and-forget asyncio Tasks (issue saltstack#70147). Bursts of a few thousand events fit without backpressure; genuinely runaway bursts still fail-fast. 2. Preserve the QueueFull -> ``_discard_slow_client`` fast-fail path for genuinely wedged subscribers, but the docstring on ``_submit_write`` now spells out why we deliberately do NOT block the writer on ``queue.put`` when the queue fills: awaiting here propagates backpressure through the entire publish_payload loop and stalls broadcast to every other subscriber, and the drainer's own head-of-line ``publish_drain_timeout`` watchdog already detects a truly stuck peer. Local reproduction: with the 500-cap the stress test dies at fut 279 (280 messages received); with the 10k cap all 5000 messages come through in <200 ms.
…-08-25-26 Resolve salt/config/__init__.py conflict: keep both the 3007.x fix (publish_drain_timeout=60.0, bd8ff52) and the 3008.x addition (pub_server_write_queue_size=10000, b8f2622). Also fix a Class-3 silent regression in salt/transport/tcp.py: 3008.x b8f2622 rewrote publish_payload's inline drain logic into a new _drain_loop method whose fallback was still 5.0, which would silently revert 3007.x commit bd8ff52 that bumped the fallback to 60.0. Restore 60.0 in _drain_loop.
The merge silently kept 3007.x's copy of every .lock under
requirements/static/{ci,pkg}/py3.*/ , stranding the environment
on stale pins. Most visibly, redis was left at 3.5.3 while
3008.x had already moved to 7.4.0 -- redis-py 3.5 does not
accept the bylex kwarg on Redis.zrange(), so every subtest in
tests/pytests/functional/cache/test_redis.py::test_caching
tripped 'Redis.zrange() got an unexpected keyword argument
bylex' from salt/cache/redis_cache.py:344. Reproduced locally
with pip install redis==3.5.3 (missing bylex) vs 7.4.0 (present).
Fix: restore every drifted .lock to origin/3008.x, then rerun
the pre-commit pip-compile hooks so the merged 3007.x floors
(attrs>=26.1.0 / charset-normalizer>=3.4.7 for CVE) are honoured
in the regenerated locks.
The previous lock resync started from origin/3008.x's lock files,
which resolved SRP-controlled packages to older versions than
3007.x had shipped. pip-compile then kept them because the
existing floors were satisfied -- so cryptography, pyopenssl,
msgpack, requests, gitpython, and pyasn1 all silently regressed
below the 3007.x SRP baseline across every python x platform
combination.
Bump the base.txt floors to the 3007.x-resolved versions and
re-run the pip-compile hooks:
cryptography >=48.0.0 -> >=50.0.0 (py>=3.10)
gitpython >=3.1.50 -> >=3.1.59
msgpack >=1.1.2 -> >=1.2.1 (py>=3.10; drop the >=3.13 sub-floor,
1.2.1 works on 3.13 -- 3007.x runs it)
pyasn1 >=0.6.3 -> >=0.6.4
pyopenssl >=26.2.0 -> >=26.4.0 (py>=3.10)
requests >=2.32.5 -> >=2.34.2 (py>=3.11)
Locks now resolve to cryptography 50.0.1 / gitpython 3.1.60 /
msgpack 1.2.1 / pyasn1 0.6.4 / pyopenssl 26.4.0 / requests 2.34.2
across every py3.10+ target -- meets or exceeds the 3007.x
baseline everywhere.
The 3006.x SRP PR saltstack#70130 raised setuptools to >=82.0.1 in constraints.txt; our merge inherited only the 3007.x >=78.1.1 floor and pip-compile resolved to setuptools==82.0.0 across every pkg lock (regression from 3007.x's own 84.0.0 baseline). Bump the constraint to 82.0.1 to match the SRP intent. Locks now resolve to setuptools==84.0.0 (>= 3007.x) across every py x platform.
…5-26 Merge forward 3007.x into 3008.x
Two targeted changes so nightly builds sign RPMs when signing secrets
are present on the running repo, without hardcoding a specific key.
templates/build-packages.yml.jinja:
Flip the nightly-environment override from
sign-rpm-packages: false
to
sign-rpm-packages: true
and regenerate nightly.yml. Nightly builds now opt in to signing.
build-packages.yml (not generated):
In the Setup GnuPG step, after `gpg --import`, discover the
fingerprint of the just-imported key via
gpg --list-secret-keys --with-colons | awk -F: '$1=="fpr" ...'
and export SIGN_KEY_ID to $GITHUB_ENV. In the Build RPM step,
replace the hardcoded --key-id=64CBBC8173D76B3F with
--key-id=${env.SIGN_KEY_ID}. rpmsign now uses whatever key
material SIGNING_GPG_KEY contained, regardless of fingerprint.
Combined effect:
- saltstack/salt-nightlies with a dedicated nightly signing secret
set: signs with that key. Blast-radius isolated from the stable
SaltProjectKey held on saltstack/salt.
- Any repo without SIGNING_GPG_KEY set: Setup GnuPG is gated by
`inputs.sign-rpm-packages` so it doesn't run, SIGN_KEY_ID stays
unset, --key-id is not appended, no signing attempted, no failure.
- saltstack/salt stable release path (staging.yml passes sign-rpm-
packages: true and has SIGNING_GPG_KEY set to the SaltProjectKey):
imports as before, discovers 64CBBC8173D76B3F from the keyring,
signs identically. Behavior unchanged.
No caller-side plumbing; no vars; no per-repo workflow config.
|
Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here's some information that may help as you continue your Salt journey. There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar. |
CrazyHackGUT
force-pushed
the
fix/mem-leak-webhook-engine
branch
from
August 27, 2026 16:26
1ff629f to
e25b24e
Compare
…008.x [3008.x] nightly: sign RPM packages with whichever key is in the signing secret
saltstack#70141 enabled RPM signing for nightly builds and pointed the workflow at the nightly environment's SIGNING_GPG_KEY/SIGNING_PASSPHRASE secrets, but the shared template that generates the build-pkgs-onedir / build-pkgs-src callers only sets `secrets: inherit` for the staging environment: <% if gh_environment == 'staging' -%> secrets: inherit <% endif -%> Without `secrets: inherit` (or an explicit `secrets:` block), reusable workflows cannot see caller secrets, so build-packages.yml's Setup GnuPG step receives empty SIGNING_GPG_KEY / SIGNING_PASSPHRASE. `echo "" | gpg --import -` fails with: gpg: no valid OpenPGP data found. gpg: Total number processed: 0 Process completed with exit code 2. Observed in saltstack/salt-nightlies run 33098010828 (3008.x nightly first run with signing enabled). The Setup GnuPG step's env log showed: SIGNING_GPG_KEY: SIGNING_PASSPHRASE: (literal blanks, not `***` redaction) — proving the secrets weren't resolved, not that they were empty. Fix: widen the conditional to `gh_environment != 'ci'`, matching the pattern of `environment:` / `sign-macos-packages:` / `sign-rpm-packages:` lines just below in the same template. Every non-CI caller now inherits secrets. Regenerates nightly.yml with `secrets: inherit` on both build-pkgs-onedir and build-pkgs-src callers. Local pre-commit: Generate GitHub Workflow Templates Passed, Lint GitHub Actions Workflows Passed.
…008.x [3008.x] templates/build-packages: pass secrets: inherit for nightly too
The `SaltVersionsInfo` table in `salt/version.py` had CHLORINE (3007) marked `released=True` but ARGON (3008) not, even though 3008.0, 3008.1, and 3008.2 have all been tagged and shipped. `SaltVersionsInfo.current_release()` iterates the version list and returns the last codename with `released=True`. With ARGON missing the flag it returns CHLORINE, which then gets used as the baseline in `__discover_version()`. On the 3008.x branch that produces version strings like `3007.14+2643.gca4940b8f0` for nightly builds -- packages end up named `salt-3007.14+N-0.x86_64.rpm` instead of the expected `salt-3008.2+N-0.x86_64.rpm`. Regression window on 3008.x: introduced by the 2026-08-25 merge from 3007.x (commit 17c1f44 "Merge remote-tracking branch 'origin/3007.x' into merge/3007.x/3008.x-08-25-26"). Prior nightlies were correctly labeled -- e.g. build 210 from 2026-08-19 shipped as `salt-3008.2+210.g9a41326f33`. Master has the same bug and was similarly affected (though master nightlies have been failing for other reasons and nobody noticed). Fix is a one-line flip: ARGON = SaltVersion("Argon", info=3008, released=True) Needs to be backported to 3008.x. Any active supported release branch whose `current_release()` should return ARGON needs the same. Discovered while verifying nightly RPM signing on a live 3008.x build (salt-nightlies run 33103981309): "Prepare Release" step printed "3007.14+2643.gca4940b8f0" from a 3008.x checkout.
…008.x [3008.x] version: mark ARGON (3008) as released
CrazyHackGUT
force-pushed
the
fix/mem-leak-webhook-engine
branch
from
August 27, 2026 20:47
e25b24e to
5ee30c5
Compare
Author
|
I recreate for merging to 3008. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
16 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes a memory leak caused by using
webhookengine by incorrectget_master_eventparameters.What issues does this PR fix or reference?
Memory leak in EventPublisher
Previous Behavior
get_master_eventreturns an object with openedpull-socket, which was not required for the publication of events, what causes memory leak inEventPublisher:EventPublishersends a message for another process, butwebhookdoesn't even plan to read them.New Behavior
get_master_eventdon't opens apull-socket.Merge requirements satisfied?
[NOTICE] Bug fixes or features added to Salt require tests.
Commits signed with GPG?
No