Detect orphaned Zabbix plugin configs by binary, not by package name - #6
Open
MarkLFT wants to merge 3 commits into
Open
Detect orphaned Zabbix plugin configs by binary, not by package name#6MarkLFT wants to merge 3 commits into
MarkLFT wants to merge 3 commits into
Conversation
added 3 commits
August 2, 2026 13:09
Full interactive run: pinned script verified, both patches applied, compile succeeded, registration completed and the tacticalagent service is running on 2.11.0 with no mesh agent. Remaining checks are UI-side only. Also notes the two benign messages seen during the run — the getcwd warnings (operator's shell was in a deleted directory before the script started; reproduced in a container) and rmmagent's board_serial inventory warning.
A live Debian 13 host got stuck in a dpkg loop that this script was written to fix but would have missed. zabbix-agent2 could never configure because ExecStartPre runs `zabbix_agent2 -T`, which died with: plugin "EmberPlus": fork/exec /usr/libexec/zabbix/zabbix-agent2-plugin-ember-plus: no such file or directory The plugin package had been removed but its plugins.d/ember.conf survived, so the agent kept trying to launch a binary that no longer existed. The failure is self-perpetuating: `dpkg --configure -a` re-runs the same config test. Detection derived the package name from the config filename, but the config, package and binary names differ per plugin: ember.conf -> zabbix-agent2-plugin-ember-plus nvidia.conf -> zabbix-agent2-plugin-nvidia-gpu so it missed precisely the two plugins that break hosts in practice. It now checks whether the binary named in System.Path= is executable, which is name-independent and also catches a config dpkg no longer tracks — the case on the live host, which had no rc-state packages at all, only a leftover file. Repair additionally purges rc-state plugin packages, guarded against an empty package list so it cannot produce a bare `dpkg --purge` invocation. Verified 6/6 in a Debian 13 container staged in the host's state: report names both plugins by missing binary, repair clears them, and the config test passes afterwards. The orphan failure itself was reproduced byte-for-byte first.
clean-zabbix is documented as removing "all Zabbix Agent 2 components", but it ran `rm -rf /etc/zabbix` — a directory shared with zabbix-proxy and zabbix-server. On a host running both, it deleted the proxy's configuration for a service it was never asked to touch, leaving a running proxy that would not survive its next restart. The backup taken beforehand only covered the agent's own paths, so the deleted proxy config was not recoverable from it either. Both halves of that are fixed: - Remove only zabbix_agent2.conf and zabbix_agent2.d. The directory itself is removed only when nothing else lives in it, and anything preserved is listed in the output so it is obvious what was left behind. - Back up the whole of /etc/zabbix when it exists, not just the agent's files, so a mistake in that directory can be undone. Verified 5/5 in a Debian 13 container with the proxy and agent co-installed: the proxy config survives clean-zabbix byte-identical, the agent config and conf.d are removed as intended, and the proxy config is present in the backup archive.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-targets at
mainthe two commits that were pushed to the #3 branch after #3 had already been merged, so they never reachedmain. Same stranding as #2 — my error, not a rebase issue.maintoday still contains the broken detection: 2 occurrences of thezabbix-agent2-plugin-${base}name-guess, 0 occurrences of the binary check.Why the old detection was wrong
A live Debian 13 host (KomuneProxy) got stuck in a dpkg loop that
repair-rmm-zabbix-linux.shexists to fix but would have missed.zabbix-agent2could never configure, becauseExecStartPrerunszabbix_agent2 -T, which died with:The plugin package was removed but its
plugins.d/ember.confsurvived, so the agent kept launching a binary that was gone. Self-perpetuating:dpkg --configure -are-runs the same test, so the host cannot recover on its own.Detection derived the package name from the config filename — but all three names differ per plugin:
ember.confzabbix-agent2-plugin-ember-plus/usr/libexec/zabbix/zabbix-agent2-plugin-ember-plusnvidia.confzabbix-agent2-plugin-nvidia-gpu/usr/libexec/zabbix/zabbix-agent2-plugin-nvidia-gpuSo it missed precisely the two plugins that break hosts in practice.
Change
Check whether the binary named in
System.Path=is executable — what the agent itself does. Name-independent, and it catches a config dpkg no longer tracks, which is the case on the live host: it had norc-state packages at all, only a leftover file.Repair also purges
rc-state plugin packages, guarded against an empty list so it cannot emit a baredpkg --purge(a hand-written version of that command without the guard did exactly that on the live host).Also carries the live-host verification record for the no-mesh TRMM installer into
Progress.md.Verified
Reproduced the orphan failure byte-for-byte in a Debian 13 container (
dpkg --removeleaves the conffile; purging restoresValidation successful), then staged a container in the host's exact state — 6/6:rc-state packages remainbash -nand shellcheck clean.