ci: add a PR test pipeline across distros and init systems

[rvalitov]

Summary

The repository has 13 test scripts in tests/ and no runner, no aggregator, and no workflow that triggers on a pull request. The only workflow, build-engine.yml, is workflow_dispatch-only and builds the engine image. Meanwhile the README lists Ubuntu, Debian, CentOS, RHEL, Fedora, Rocky, AlmaLinux and Alpine as supported, and the script ships two init-system paths — none of which is verified on any commit.

This adds the project's first automated test pipeline. No product code is modified.

What runs

Job	Runs on	Covers
lint	ubuntu-latest	bash -n on every script, plus shellcheck -S error
unit-tests	5 container images	the existing suite, per distro
systemd-integration	ubuntu-24.04	setup_autostart / main_service_remove against real systemd
openrc-integration	alpine:3.20 container	the same against real OpenRC

Triggers on pull_request, push to main, and workflow_dispatch. The workflow requests contents: read only and uses no secrets — it executes fork code, so it uses pull_request and never pull_request_target.

The distro matrix

debian:12, ubuntu:22.04, ubuntu:24.04, alpine:3.20, fedora:41.

Alpine is the row that earns its keep: its busybox userland differs from GNU in sed, grep, mktemp, date, flock and netstat, and it is a documented supported platform that has never been exercised. It has already surfaced real defects — see the quarantine note below.

fail-fast: false, so one red distro still reports all the others.

Design decisions a reviewer may question

docker run, not the job-level container: key. JavaScript actions — including actions/checkout — execute with a node binary inside the job container, and the runner does not inject one. debian:12 and alpine:3.20 ship neither node nor git, so a container: job fails before any step that could install them.

Alpine installs only bash. Adding coreutils or diffutils would shadow busybox and hide exactly the differences that row exists to detect. The suite is expected to meet busybox on its own terms.

Fedora needs diffutils. It ships bash but no diff, and one test in the suite uses diff — which, with 2>/dev/null, made a missing binary look like a content mismatch.

The systemd job runs natively on ubuntu-24.04. That image is a full VM with systemd as PID 1 and Docker already installed, so the generated unit's Requires=docker.service resolves against a real unit. Running it in a privileged container instead would need --privileged --cgroupns=host and would be a less faithful substitute.

Containers need shims that a real host gets for free. OpenRC refuses to run any service without /run/openrc/softlevel and will not start a service whose hard dependencies are unresolved, so the OpenRC test creates the marker and a docker stub. Those are in the test harness, not the workflow.

The quarantine mechanism, and why it is not an excuse

Three tests fail on this repository today. Shipping a pipeline that is red on arrival trains everyone to ignore it, so tests/run-all.sh supports a per-image quarantine list:

• a quarantined test still runs and its output is still printed
• it is reported as QUARANTINE, counted separately, and never counted as passing
• and it does not fail the build

Current entries, each with the fix that retires it:

Test	Where	Reason	Fixed by
test_client_mss.sh	all images	asserts on the stdout of a function that writes to a file — broken everywhere, not a platform difference	#145
test_traffic_reset.sh	alpine	busybox flock has no -w, so the lock call fails while command -v flock succeeds, and the guard never fires	#146
test_guest.sh	alpine	date -d "+24 hours" is invalid on busybox, and the date -r <epoch> fallback fails too	#148

Leaving an entry in place after its fix lands is harmless — a quarantined test that passes is reported as PASS — so this list is correct in any merge order. run-all.sh flags such entries as ! still listed as quarantined, but passing, so a stale entry is visible rather than silently ignored.

Evidence

The full matrix has been run locally against this branch, reproducing the workflow's exact per-image install command:

debian:12      total=13  passed=12  failed=0  quarantined=1  stale-quarantine=0
ubuntu:22.04   total=13  passed=12  failed=0  quarantined=1  stale-quarantine=0
ubuntu:24.04   total=13  passed=12  failed=0  quarantined=1  stale-quarantine=0
alpine:3.20    total=13  passed=10  failed=0  quarantined=3  stale-quarantine=0
fedora:41      total=13  passed=12  failed=0  quarantined=1  stale-quarantine=0

The integration jobs were verified against real init systems: 14/14 on a container running systemd as PID 1, and 16/16 under OpenRC on Alpine.

Both integration scripts are mutation-tested — deliberately breaking ExecStart or the OpenRC start() makes them fail with a readable diagnostic, so they are known to have signal rather than merely being green.

Known interactions

New test files are picked up automatically. run-all.sh globs tests/test_*.sh, so any PR that adds a test file starts running it across the matrix with no workflow change.

#143 adds ten test files, seven of which have not been exercised on Alpine or Fedora. If one fails there, this pipeline will report it — which is the point of the pipeline, but it means a red run immediately after #143 merges may be #143's tests rather than a defect in this PR.

First-time contributor gating. GitHub withholds workflow runs from new contributors until a maintainer approves them, so the first run after this merges may show no checks at all. That is expected, not a failure.

What is not covered

There is no end-to-end test that installs the proxy and pushes traffic through it. The installer is interactive with no non-interactive flag — the script's only [ ! -t 0 ] fallback is for secret removal, not installation — so driving it in CI would need a product change. That is deliberately out of scope here.

The integration tests therefore target setup_autostart() and main_service_remove() directly, which is where the init-system divergence actually lives.

[rvalitov]

I marked this as ready for review, because this PR actually works but skips some tests (quarantine) that fail because other PRs are not merged yet. So you can either:

• merge it now, so it will start working on all new PRs, but I will need to remove the skipped tests with new PR.
• merge other PRs first, then I will commit here to remove the skipped tests, then merge this PR