fix(quality): a required additional app that will not enable must fail the job - #355
fix(quality): a required additional app that will not enable must fail the job#355rubenvdlinde wants to merge 1 commit into
Conversation
…l the job `php occ app:enable "$name" || echo "::warning::Failed to enable $name, continuing..."` let a run proceed WITHOUT the dependency the repository under test is built on. The absence then resurfaced as APPLICATION failures, which is indistinguishable from real debt in the log. Measured on ConductionNL/hermiq, full-scope run 31490144919, six PHPUnit cells, perfect 6/6 correlation between `grep -c 'Failed to enable openregister'` and the error count: 93774658402 8.4/stable33 enable ok -> Tests: 1500, Errors: 5, Failures: 1 93774658407 8.3/stable33 enable ok -> Tests: 1500, Errors: 5, Failures: 1 93774658415 8.4/stable32 enable ok -> Tests: 1500, Errors: 5, Failures: 1 93774658413 8.3/stable31 enable FAILED -> Tests: 1500, Errors: 12 93774658418 8.3/stable32 enable FAILED -> Tests: 1500, Errors: 12 93774658420 8.4/stable31 enable FAILED -> Tests: 1500, Errors: 12 The seven extra "errors" were `Interface "OCA\OpenRegister\Service\Flow\ IFlowNodeLogActions" not found` and `Call to undefined method MockObject_ToolRegistryFacade::describeTools()` — an absent app wearing the costume of app debt. Two causes, both deserving a red job: one deterministic (OpenRegister declares min-version="32"; the cell tested stable31) and one transient (`curl error 60 ... self-signed certificate` during composer install left Twig missing, so enabling crashed). Applied to all four install sites (phpunit, newman, and both playwright legs).⚠️ The failure flag goes through a FILE, not a variable: the loop body runs in a subshell because it is fed by a pipeline, so a variable set inside it is invisible after `done` — the same trap run-hydra-gates.sh documents at its gate-30 loop. Verified with a three-app simulation: exit 0 when all enable, exit 1 when the FIRST fails and exit 1 when the LAST fails.⚠️ Grep for the failure with the APP NAME (`Failed to enable openregister`). The bare `Failed to enable` also matches the workflow's own echoed script line, which contains `$name` unexpanded — it returns 1 on a clean run and 2 on a failing one, so an off-by-one reads as a correlation that is not there.
This PR was right, and it has been sitting here for five daysRediscovered the same defect family tonight from the other end, and independently re-verified this PR's measurement before writing anything. It reproduces exactly. hermiq run
Perfect 6/6. Same commit, same application code — only the environment differs. That is a controlled experiment, not an anecdote, and it is better evidence than most gate findings ever get. The sibling defect, four lines awayThe Measured on decidesk#517 (a diff of exactly one Clear division of ownership so the two PRs do not collide:
They are two halves of one hole — an app that will not enable, and an app whose dependencies never installed — and this PR's own body already names the second as a cause of the first ("…or its composer install did not complete"). They should land together or not at all. A third, #483, fixes the phpcs gate discarding its own verdict. Staleness — measured, and better than it looks
But a dry-run merge of The finding that outlives all three fixesThis defect was found, measured and fixed on 2026-08-11. The fix never landed, and the fleet has since spent real effort misattributing environment failures to application debt — including tonight, on decidesk#517. An unmerged fix is indistinguishable from an undiscovered bug, except that the second time you also pay for the rediscovery. Whatever caused this to stall is now itself the finding. Not touching this branch — no force-push, no rebase from me. Flagging it for a refresh so all three can be sequenced deliberately. |
What is wrong
php occ app:enable "$name" || echo "::warning::Failed to enable $name, continuing..."lets a run proceed without the dependency the repository under test is built on.
The absence then resurfaces as application failures, which is indistinguishable from
real debt in the log — and a warning in a red job is invisible.
Measurement
ConductionNL/hermiq, full-scope run
31490144919, six PHPUnit cells. Perfect 6/6correlation between
grep -c 'Failed to enable openregister'and the error count:Tests: 1500, Errors: 5, Failures: 1Tests: 1500, Errors: 5, Failures: 1Tests: 1500, Errors: 5, Failures: 1Tests: 1500, Errors: 12Tests: 1500, Errors: 12Tests: 1500, Errors: 12The seven extra "errors" were
Interface "OCA\OpenRegister\Service\Flow\IFlowNodeLogActions" not foundandCall to undefined method MockObject_ToolRegistryFacade::describeTools().Two causes, both of which deserve a red job:
min-version="32", the cell testedstable31;curl error 60 … self-signed certificateduring composer install leftTwig missing, so
app:enablecrashed onClass "Twig\Extension\AbstractExtension" not found.Why this is a DRAFT — please sequence it
Merging this today turns 6 repos red on a cell that is currently lying rather than
failing. Measured
nextcloud-test-refsagainstadditional-appsacross the fleet — theserun a
stable31cell while requiringopenregister(min-version 32), so every one oftheir stable31 cells is today measuring an instance without OpenRegister:
app-versions·hermiq·hrmq·petstore·planix·nextcloud-app-templatehermiq is already fixed (ConductionNL/hermiq#180 drops its
stable31cell anddeclares the truthful
min-version="32"). The other five need the same one-line changebefore this is merged. I have left it as a draft rather than merge it and hand five agents
an unexplained red cell mid-session — the sequencing is a fleet call, not mine.
Implementation notes
Applied to all four install sites (phpunit, newman, both playwright legs).
subshell because it is fed by a pipeline, so a variable set inside it is invisible after
done— the same traprun-hydra-gates.shdocuments at its gate-30 loop. Verified with athree-app simulation: exit 0 when all enable, exit 1 when the first fails, exit 1 when
the last fails.
Failed to enable openregister). The bareFailed to enablealso matches the workflow's own echoed script line, which contains$nameunexpanded — it returns 1 on a clean run and 2 on a failing one, so an off-by-onereads as a correlation that is not there. That mis-measurement already happened once today.