feat(e2e): cluster-free plugin sanity check for the catalog index (RHIDP-13508) by gustavolira · Pull Request #4967 · redhat-developer/rhdh

gustavolira · 2026-06-17T20:08:41Z

Summary

Plugin sanity checking for the RHDH catalog index, implementing RHIDP-13508. Two complementary flavors, both running in the nightly OCP job and both driven by the same CATALOG_INDEX_IMAGE (branch-matched nightly index by default; RC index via Gangway --catalog-index-image):

Cluster-free check (new) — validates that every plugin the index declares loads in the real RHDH backend, with no cluster deployment or product image. Builds on the cluster-free harness (feat(e2e): cluster-free local E2E harness #5005): Playwright boots packages/backend from source (the product's own Backstage line and dynamicPluginsFeatureLoader) and the spec compares the installed set against /api/dynamic-plugins-info/loaded-plugins, failing with the exact list of installed-but-not-loaded plugins.
Cluster deployment (existing, now dynamic) — the sanity-plugins Helm deployment no longer hard-codes its plugin list: the enabled set is derived from the catalog index at deploy time, so it tracks the index (including ./dynamic-plugins/dist/* plugins that live inside the product image — enabling them exercises the image content). The shipped RHDH image remains the artifact under test here.

How the dynamic set works (shared between flavors)

local-harness/catalog-index-refs.sh — shared parser: extracts dynamic-plugins.default.yaml from the index image and lists every declared package (uncommented entries only), applying plugin-sanity-excludes.txt (documented known failures, shared by both flavors)
Cluster-free: populate-catalog-index.sh wraps the refs into an install config (the index ships packages disabled: true, so a plain includes would install nothing) and installs via the shared populate.sh
Cluster: generate-catalog-enable-values.sh wraps the refs into a Helm values fragment merged UNDER the curated values (helm::merge_values merge — curated entries win, keeping their env-specific pluginConfig). Packages whose name already has a curated entry are omitted from the fragment: curated entries pin old tags, so exact-ref dedup would enable the same plugin twice (duplicate pluginId → backend crash). Falls back to the curated list if generation fails.

Failure reporting — which plugin broke?

A broken plugin takes the whole backend down by design. To make the culprit obvious:

testing::report_plugin_startup_failures scans the RHDH pod logs (current + previous containers, so CrashLoopBackOff is covered) for threw an error during startup / Backend startup failed and prints a loud PLUGIN STARTUP FAILURES block, saved as a build artifact
The cluster-free wrapper greps its own run log for the same signatures on failure
The cluster-free spec itself fails with the exact list of installed-but-not-loaded plugins when the backend boots but a plugin is missing

CI integration

Both flavors run in handle_ocp_nightly → run_sanity_plugins_check, which exports CATALOG_INDEX_IMAGE (+ derived chart components) before the Helm install so the chart, the in-container install CLI and the cluster-free check all consume the same index. The cluster-free wrapper (testing::run_plugin_sanity_check) follows the testing::run_tests conventions (test_run_tracker, save_overall_result, gzipped JUnit to SHARED_DIR, report artifacts) so Slack reporting includes it like any other run.

RC verification: --tag <rc-image-tag> --catalog-index-image quay.io/rhdh/plugin-catalog-index:<rc-index-tag> via Gangway tests the RC image on the cluster AND the RC index composition cluster-free, in one job.

Local runs

# cluster-free
CATALOG_INDEX_IMAGE=quay.io/rhdh/plugin-catalog-index:next \
  ./e2e-tests/local-harness/populate-catalog-index.sh
yarn --cwd e2e-tests plugin-sanity

# preview the dynamic cluster enable-set
./e2e-tests/local-harness/generate-catalog-enable-values.sh quay.io/rhdh/plugin-catalog-index:next

Verification

Cluster-free against :next: 13 plugins installed (8 backend, 5 frontend), backend booted, 13/13 loaded — 1 passed. Surfaced three real composition issues along the way (orchestrator backend init failure, loki module, argocd startup config) — the class of problem the check exists to catch.
Dynamic cluster values against :next + the current curated file: 43 dynamically enabled packages, argocd/gitlab curated entries correctly deduped by name, helm::merge_values output = 78 deduplicated plugins with curated pluginConfig intact.
oxlint, oxfmt, shellcheck, .ci prettier clean. First nightly run will calibrate the excludes list for the in-cluster environment (accepted trade-off of leaving the frozen curated list behind).

Also declares the yaml dependency already imported by playwright/utils/authentication-providers (phantom-dependency fix).

Jira

Implements: RHIDP-13508
Part of epic: RHIDP-13496 (Test Layer Infrastructure)

rhdh-qodo-merge · 2026-06-17T20:09:56Z

PR Summary by Qodo

feat(e2e): add Playwright plugin sanity-check test
🧪 Tests ✨ Enhancement ⚙️ Configuration changes 🕐 20-40 Minutes

Description

• Add Playwright sanity test that parses default.packages.yaml and validates plugin entries.
• Wire the new spec into the SHOWCASE_SANITY_PLUGINS Playwright project for nightly runs.
• Provide a mock default.packages.yaml in repo root to enable local execution.

Diagram

graph TD
  Config["playwright.config.ts"] --> Project["SHOWCASE_SANITY_PLUGINS"] --> Runner["Playwright runner"] --> Test["plugin-sanity-check.spec.ts"] --> Yaml[("default.packages.yaml")] --> Validate["Validate package entries"]
  CI["Nightly CI deployment"] --> Yaml

High-Level Assessment

The following are alternative approaches to this PR:

1. Resolve/install validation (require.resolve / dynamic import)

➕ Catches missing deps and broken package exports, not just naming issues
➕ Aligns better with the header comment about “can be loaded”
➖ May be flaky if the test environment doesn’t include all plugin packages
➖ Could increase runtime and require dependency management in CI images

2. Use dynamic plugin installer CLI in the test

➕ Validates the real dynamic-plugin acquisition path end-to-end
➕ Closer to how RHDH consumes plugins in deployment
➖ Adds infra complexity (network access, caching, time)
➖ Harder to keep deterministic in CI without pinning and mirrors

3. Schema/structure-only validation + stronger rules

➕ Keeps the test lightweight and deterministic
➕ Can still catch common config errors (missing fields, duplicates, invalid npm scope/name regex)
➖ Won’t detect runtime load failures or missing packages

Recommendation: The PR’s lightweight approach is reasonable for a nightly “sanity” gate, but the spec currently only checks package-name scoping (starts with @) and disabled-list structure. Consider either (a) tightening the structure validation (required fields, non-empty lists, duplicate detection, npm package-name regex) and adjusting the test header comment to match behavior, or (b) adding an optional require.resolve/import-based check when the environment guarantees plugin deps are present. Full CLI-based installation is higher fidelity but likely too heavy unless the team is ready to invest in caching and determinism.

Files changed (3) +149 / -0

Tests (1) +137 / -0

plugin-sanity-check.spec.tsAdd Playwright spec to validate enabled/disabled plugin entries +137/-0

Add Playwright spec to validate enabled/disabled plugin entries

• Introduces a new Playwright spec that reads 'default.packages.yaml', validates enabled plugin package name format (scoped), and asserts the disabled package list is parseable and well-formed. Adds a 'component=plugins' annotation and emits a simple console summary.

e2e-tests/playwright/e2e/plugin-sanity-check.spec.ts

Other (2) +12 / -0

default.packages.yamlAdd mock default plugin package list for local runs +11/-0

Add mock default plugin package list for local runs

• Adds a repo-root 'default.packages.yaml' with enabled/disabled plugin entries to support local execution of the new Playwright sanity test. Notes that the real file is injected during CI deployment.

default.packages.yaml

playwright.config.tsInclude plugin sanity spec in SHOWCASE_SANITY_PLUGINS project +1/-0

Include plugin sanity spec in SHOWCASE_SANITY_PLUGINS project
• Extends the 'SHOWCASE_SANITY_PLUGINS' project 'testMatch' list to include 'plugin-sanity-check.spec.ts', ensuring it runs as part of the sanity-plugins CI job.
e2e-tests/playwright.config.ts

github-actions · 2026-06-17T20:10:18Z

The container image build workflow finished with status: cancelled.

codecov · 2026-06-17T20:13:23Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.77%. Comparing base (849c4e7) to head (4f00cf7).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4967      +/-   ##
==========================================
- Coverage   55.39%   54.77%   -0.62%     
==========================================
  Files         122      110      -12     
  Lines        2365     2147     -218     
  Branches      563      538      -25     
==========================================
- Hits         1310     1176     -134     
+ Misses       1048      969      -79     
+ Partials        7        2       -5

Flag	Coverage Δ
rhdh	`54.77% <ø> (-0.62%)`	⬇️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 849c4e7...4f00cf7. Read the comment docs.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

rhdh-qodo-merge · 2026-06-17T20:14:48Z

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 📜 Skill insights (0)

Context used

✅ Tickets: [RHIDP-13508](https://redhat.atlassian.net/browse/RHIDP-13508)

1. ~~No package resolution performed~~ ✓ Resolved 🐞 Bug ≡ Correctness

Description

The test "All enabled packages can be resolved" never resolves/imports any packages; it only checks
that the string starts with "@" and then records success. This can let CI pass even when a listed
plugin package is missing/unresolvable, making the new sanity check misleading.

Code

e2e-tests/playwright/e2e/plugin-sanity-check.spec.ts[R44-83]

+  test("All enabled packages can be resolved", async () => {
+    // Read default.packages.yaml from rhdh repo root
+    const defaultPackagesPath = join(__dirname, "../../../default.packages.yaml");
+    const yamlContent = readFileSync(defaultPackagesPath, "utf8");
+    const config = yaml.parse(yamlContent) as DefaultPackagesConfig;
+
+    const enabledPackages = config.packages.enabled;
+    console.log(`\n📦 Testing ${enabledPackages.length} enabled packages...\n`);
+
+    const results: {
+      package: string;
+      status: "success" | "failed";
+      error?: string;
+    }[] = [];
+
+    for (const pkg of enabledPackages) {
+      const packageName = pkg.package;
+
+      try {
+        // Attempt to resolve the package
+        // Note: We can't actually import dynamic plugins here as they require
+        // a Backstage runtime, but we can at least verify the package name format
+        // and that it's listed in package.json dependencies
+
+        // Validate package name format
+        if (!packageName.startsWith("@")) {
+          throw new Error("Package name must be scoped (start with @)");
+        }
+
+        // For now, just verify the package is properly formatted
+        // Future enhancement: Use @red-hat-developer-hub/cli-module-install-dynamic-plugins
+        // to actually download and verify the plugins load
+
+        results.push({
+          package: packageName,
+          status: "success",
+        });
+
+        console.log(`✅ ${packageName}`);
+      } catch (error) {

Relevance

⭐⭐ Medium
Mixed history: team accepts improving E2E assertions/coverage (#3147,#4526) but rejected similar
“add runtime validation” checks (#4519).
PR-#3147
PR-#4526
PR-#4519

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The test’s core loop never calls any module-resolution API; it only validates a prefix and then
pushes a success result, so it cannot detect missing/unresolvable packages. Additionally, the E2E
test package itself does not declare the example plugin packages as dependencies, reinforcing that
the current implementation cannot be performing real resolution.

e2e-tests/playwright/e2e/plugin-sanity-check.spec.ts[44-83]
e2e-tests/package.json[57-71]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The test name/comments say it verifies packages can be resolved/loaded, but the implementation only checks a naming convention (`startsWith('@')`) and always marks entries as `success` if that passes. This can silently miss real breakages (missing package, typo, unpublished artifact), because no `resolve`/`import` is ever attempted.

### Issue Context
- This is a Playwright E2E "sanity" test intended for CI signal quality. If the test claims resolvability but doesn’t actually do it, it provides misleading green runs.

### Fix Focus Areas
- e2e-tests/playwright/e2e/plugin-sanity-check.spec.ts[44-83]

### Suggested fix
Choose one (A is lowest-risk for now):

**A) Rename + adjust messaging to match reality**
- Rename the test to something like `All enabled packages have valid package identifiers`.
- Update the header comment and inline comments to remove "resolve/import" language.
- Update summary logs to reflect "validated format" rather than "loaded/resolved".

**B) Actually resolve packages (only if intended + feasible)**
- Implement a real resolution check (e.g., `createRequire(import.meta.url).resolve(packageName)` or Node’s `import.meta.resolve` where available) and fail on thrown errors.
- If doing this, ensure the packages being checked are expected to be present in the environment (dependencies or fetched artifacts), otherwise the test will become permanently red.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

github-actions · 2026-06-17T20:30:12Z

The container image build workflow finished with status: cancelled.

github-actions · 2026-06-17T20:37:16Z

The container image build workflow finished with status: cancelled.

github-actions · 2026-06-17T21:09:11Z

Image was built and published successfully. It is available at:

Implements review findings from PR redhat-developer#4967 code review: **Correctness fixes:** - Fix test.beforeAll signature to use test.info() (follows smoke-test.spec.ts pattern) - Remove unused manifestPath variable in plugin-dynamic-loading.spec.ts **Code organization:** - Extract DEFAULT_PACKAGES_PATH constant to avoid duplication - Move CONFIG_OVERRIDES to module scope for better reusability - Remove unused catalog-index-parser.ts (can be recreated when needed) **Type safety:** - Import and use BackendFeature type for require() calls instead of any - Improves type safety when loading plugin modules **Developer experience:** - Improve console.warn message to explain impact of missing _nodeModulePaths All changes are low-risk refactorings that improve code quality without changing behavior. Type checking passes with no errors. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

github-actions · 2026-06-18T13:01:56Z

The container image build workflow finished with status: cancelled.

github-actions · 2026-06-18T13:14:49Z

The container image build workflow finished with status: cancelled.

github-actions · 2026-06-18T13:17:06Z

The container image build workflow finished with status: cancelled.

github-actions · 2026-06-18T13:36:16Z

The container image build workflow finished with status: cancelled.

github-actions · 2026-06-18T13:39:51Z

The container image build workflow finished with status: cancelled.

github-actions · 2026-06-18T13:40:16Z

The container image build and publish workflows were skipped (either due to [skip-build] tag or no relevant changes with existing image).

gustavolira · 2026-06-18T15:03:40Z

/test ?

gustavolira · 2026-06-18T15:04:09Z

/test e2e-ocp-helm-nightly

gustavolira · 2026-06-18T22:18:05Z

/test e2e-ocp-helm-nightly

github-actions · 2026-06-18T22:49:26Z

Image was built and published successfully. It is available at:

gustavolira · 2026-06-19T12:53:43Z

/test e2e-ocp-helm-nightly

github-actions · 2026-06-19T13:19:14Z

Image was built and published successfully. It is available at:

Implements review findings from PR redhat-developer#4967 code review: **Correctness fixes:** - Fix test.beforeAll signature to use test.info() (follows smoke-test.spec.ts pattern) - Remove unused manifestPath variable in plugin-dynamic-loading.spec.ts **Code organization:** - Extract DEFAULT_PACKAGES_PATH constant to avoid duplication - Move CONFIG_OVERRIDES to module scope for better reusability - Remove unused catalog-index-parser.ts (can be recreated when needed) **Type safety:** - Import and use BackendFeature type for require() calls instead of any - Improves type safety when loading plugin modules **Developer experience:** - Improve console.warn message to explain impact of missing _nodeModulePaths All changes are low-risk refactorings that improve code quality without changing behavior. Type checking passes with no errors. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

github-actions · 2026-06-19T19:42:46Z

The container image build and publish workflows were skipped (either due to [skip-build] tag or no relevant changes with existing image).

github-actions · 2026-06-19T20:12:41Z

The container image build and publish workflows were skipped (either due to [skip-build] tag or no relevant changes with existing image).

gustavolira · 2026-06-20T14:29:13Z

/test e2e-ocp-helm-nightly

…tion, path prefixes - Mark harnesses that are still in review and instruct the assistant to verify paths exist on main before recommending them (the skill referenced redhat-developer#5005/redhat-developer#5044/redhat-developer#2714/redhat-developer#4967 deliverables in present tense). - Describe the overlays native smoke's real interface: yarn smoke --dynamic-plugins <file> [--out] (no --workspace flag exists; the harness does not read workspaces/*/metadata — a workspace run is a dp.yaml listing its oci:// refs). - Replace the fork-only RHIDP-13235-layer3-component-tests branch pointer with the closed PR rhdh#4864. - Prefix every path with its repo (rhdh:/overlays:/plugins:) and use full URLs for cross-repo docs, so the skill resolves from any of its three homes. - Fix skopeo claim: it installs on macOS via brew; CI has it preinstalled. - Name the Docker smoke location (overlays smoke-tests/ + run-workspace-smoke-tests.yaml), replace the vague "any" repo cell, use "n/a (cluster)" for the cluster row's Docker column, move PR numbers to References with status, add trigger phrases to the frontmatter description, and deduplicate the guiding-rule sentence. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…DME link - Fix the Tally L3 count (10, not 9) so the column totals sum to the 30-spec heading; reconcile the heading itself (29 on main + redhat-developer#19 pending in PR redhat-developer#4967). - "runs 4 tests" -> "runs 2 specs (4 test cases)" — the doc's accounting unit is the spec; note that spec numbers refer to the matrix below. - Soften "fully covers the 12 pure-backend workspaces" to load + API surface: scaffolder-backend-module-kubernetes also has a UI e2e that needs the render harness, so "fully" overstated the native-smoke scope. - Replace short commit hashes and the fork-only RHIDP-13235-layer3-component-tests branch name with the durable PR reference (rhdh#4864, closed) — hashes on a mutable branch dangle after a rebase or branch deletion. - Give DRAFT a promotion condition (groomed into RHIDP-13528/13529). - Link the matrix from docs/e2e-tests/README.md ("Adding a Test") so the doc is discoverable outside the Jira comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

gustavolira · 2026-07-03T20:04:57Z

/test e2e-ocp-helm-nightly

github-actions · 2026-07-06T19:44:38Z

Image was built and published successfully. It is available at:

github-actions · 2026-07-06T23:30:48Z

Image was built and published successfully. It is available at:

github-actions · 2026-07-07T14:38:50Z

The container image build and publish workflows were skipped (either due to [skip-build] tag or no relevant changes with existing image).

gustavolira · 2026-07-07T14:42:34Z

/test e2e-ocp-helm-nightly

… wording - 2026-07-07 update: 10 specs (not 9) — the list itself names ten files. - Drop the positional "queue above" reference (the queue list no longer precedes it) and past-tense the 2026-07-02 "now runs" so the two dated updates stop contradicting each other. - Self-date the PR redhat-developer#4967 "open" claim and add PR redhat-developer#5057 to References. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…DME link - Fix the Tally L3 count (10, not 9) so the column totals sum to the 30-spec heading; reconcile the heading itself (29 on main + redhat-developer#19 pending in PR redhat-developer#4967). - "runs 4 tests" -> "runs 2 specs (4 test cases)" — the doc's accounting unit is the spec; note that spec numbers refer to the matrix below. - Soften "fully covers the 12 pure-backend workspaces" to load + API surface: scaffolder-backend-module-kubernetes also has a UI e2e that needs the render harness, so "fully" overstated the native-smoke scope. - Replace short commit hashes and the fork-only RHIDP-13235-layer3-component-tests branch name with the durable PR reference (rhdh#4864, closed) — hashes on a mutable branch dangle after a rebase or branch deletion. - Give DRAFT a promotion condition (groomed into RHIDP-13528/13529). - Link the matrix from docs/e2e-tests/README.md ("Adding a Test") so the doc is discoverable outside the Jira comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

… wording - 2026-07-07 update: 10 specs (not 9) — the list itself names ten files. - Drop the positional "queue above" reference (the queue list no longer precedes it) and past-tense the 2026-07-02 "now runs" so the two dated updates stop contradicting each other. - Self-date the PR redhat-developer#4967 "open" claim and add PR redhat-developer#5057 to References. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

gustavolira · 2026-07-07T16:50:33Z

/test e2e-ocp-helm-nightly

openshift-ci · 2026-07-07T18:01:35Z

@gustavolira: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-ocp-helm-nightly	`6564a0f`	link	false	`/test e2e-ocp-helm-nightly`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

…IDP-13508) Validates that every plugin enabled by the catalog index loads in the real RHDH backend, without a cluster deployment or product image. Builds on the cluster-free local E2E harness (redhat-developer#5005): Playwright boots packages/backend from source - the product's own Backstage line and dynamicPluginsFeatureLoader - and the spec compares the installed plugin set against /api/dynamic-plugins-info/loaded-plugins. Runs in the nightly OCP job alongside the existing cluster-based sanity-plugins deployment: the cluster deployment validates the curated plugin set on the shipped image; this validates the FULL index composition against the current backend line. - populate-catalog-index.sh: extracts dynamic-plugins.default.yaml from CATALOG_INDEX_IMAGE (skopeo dir: layout, layers via manifest.json) and generates an install config re-enabling every oci:// package the index declares (the index ships them disabled by default), then installs via the shared populate.sh (now accepting a config argument) - playwright.plugin-sanity.config.ts: backend-only webServer boot; shared harness pieces extracted to playwright/support/local-harness-servers.ts (also used by playwright.legacy-local.config.ts) plus app-config.plugin-sanity.yaml (dummy values for plugins that abort the backend on missing config, e.g. the roadie argocd instance URL) - plugin-dynamic-loading.spec.ts: request-only spec (guest token -> loaded-plugins) failing with the list of installed-but-not-loaded plugins; static frontend bundle validation on top - plugin-sanity-excludes.txt: documented known failures skipped at install time (orchestrator backend + loki module cannot initialize in a standalone boot and would abort the backend); extended-regex patterns anchored with (@|:) so entries survive the index flipping between digest- and tag-style references - testing::run_plugin_sanity_check: CI wrapper following the testing::run_tests conventions (test_run_tracker, save_overall_result, junit gzip to SHARED_DIR, artifacts), called from handle_ocp_nightly after the cluster-based sanity check; index overridable via Gangway --catalog-index-image for RC verification - declares the yaml dependency already imported by playwright/utils/authentication-providers (phantom dependency fix) Verified locally against quay.io/rhdh/plugin-catalog-index:next: 13 plugins installed (8 backend, 5 frontend), backend booted, 13/13 reported loaded - 1 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The cluster-based sanity-plugins deployment hard-coded its plugin list in diff-values_showcase-sanity-plugins.yaml with pinned tags - a frozen snapshot of the index that rots on every index rebuild. Derive the enabled set dynamically from the same CATALOG_INDEX_IMAGE the cluster-free check uses, so both flavors track the index (and an RC index passed via Gangway --catalog-index-image drives both): - catalog-index-refs.sh: shared parser (extracted from populate-catalog-index.sh) listing every package the index declares - oci:// refs AND ./dynamic-plugins/dist paths (the latter live inside the product image, so enabling them exercises the image content) - with plugin-sanity-excludes.txt applied - generate-catalog-enable-values.sh: emits a Helm values fragment enabling every declared package; packages whose NAME already has a curated entry are omitted - exact-ref dedup is not enough, since curated entries pin old tags and the same plugin under a newer index ref would be enabled twice (duplicate pluginId -> backend crash) - initiate_sanity_plugin_checks_deployment: merges the generated fragment UNDER the curated values (helm::merge_values merge - diff precedence keeps curated pluginConfig), falling back to the curated list when generation fails; fragment and final values saved as artifacts - run_sanity_plugins_check: exports CATALOG_INDEX_IMAGE (+ derived CATALOG_INDEX_* chart components) BEFORE the Helm install so helm::get_image_params passes the index to the chart - testing::report_plugin_startup_failures: scans pod logs (current and previous containers - CrashLoopBackOff keeps the culprit in -p) for 'threw an error during startup' / 'Backend startup failed' and prints a loud PLUGIN STARTUP FAILURES summary, saved as an artifact; the cluster-free wrapper greps its own run log for the same signatures on failure - a broken plugin takes the whole pod down by design, so the report names the culprit without log digging Verified locally: generated fragment from :next (43 dynamic packages, argocd/gitlab curated entries deduped by name), helm::merge_values produces 78 deduplicated plugins with curated pluginConfig intact; cluster-free flavor revalidated end-to-end after the shared-parser refactor (13/13 loaded, 1 passed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Addresses the SonarCloud bash best-practices findings on the local-harness scripts: single-bracket [ ] tests replaced with the safer [[ ]] construct (and the two chained [ ] && [ ] tests in populate.sh collapsed into one [[ ... && ... ]]). Behavior-neutral - shellcheck, bash -n and a generator run against the live index confirm identical output. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

github-actions · 2026-07-07T20:46:30Z

The container image build workflow finished with status: cancelled.

sonarqubecloud · 2026-07-07T20:46:51Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

github-actions · 2026-07-07T21:18:46Z

Image was built and published successfully. It is available at:

openshift-ci · 2026-07-09T14:00:12Z

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci Bot requested review from rm3l and subhashkhileri June 17, 2026 20:08

rhdh-qodo-merge Bot added Tests Enhancement labels Jun 17, 2026

gustavolira force-pushed the main branch from 6628116 to 1ef2bc2 Compare June 19, 2026 19:42

gustavolira force-pushed the main branch from 165931c to a8d76ac Compare June 23, 2026 13:41

gustavolira mentioned this pull request Jul 2, 2026

docs(ai): add test-placement skill for the plugin testing strategy (RHIDP-13501) #5045

Closed

gustavolira force-pushed the main branch from 4496383 to d9c4915 Compare July 6, 2026 18:37

openshift-ci Bot added the needs-rebase label Jul 6, 2026

gustavolira changed the title ~~feat(e2e): add plugin sanity check test (RHIDP-13508)~~ feat(e2e): cluster-free plugin sanity check for the catalog index (RHIDP-13508) Jul 6, 2026

gustavolira mentioned this pull request Jul 6, 2026

feat: notify RHDH plugin sanity check after publishing plugins redhat-developer/rhdh-plugin-export-overlays#2732

Closed

gustavolira force-pushed the main branch from d9c4915 to 130dc47 Compare July 6, 2026 19:12

openshift-ci Bot removed the needs-rebase label Jul 6, 2026

gustavolira force-pushed the main branch from 130dc47 to 53f8b54 Compare July 6, 2026 23:00

gustavolira mentioned this pull request Jul 7, 2026

feat: add test-placement skill for the plugin testing strategy (v0.7.0) redhat-developer/rhdh-skill#58

Open

gustavolira and others added 2 commits July 7, 2026 17:14

gustavolira force-pushed the main branch from 6564a0f to 8768c47 Compare July 7, 2026 20:31

openshift-ci Bot added the needs-rebase label Jul 9, 2026

Uh oh!

Conversation

gustavolira commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How the dynamic set works (shared between flavors)

Failure reporting — which plugin broke?

CI integration

Local runs

Verification

Jira

Uh oh!

rhdh-qodo-merge Bot commented Jun 17, 2026

PR Summary by Qodo

Uh oh!

github-actions Bot commented Jun 17, 2026

Uh oh!

codecov Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

rhdh-qodo-merge Bot commented Jun 17, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review by Qodo

Uh oh!

github-actions Bot commented Jun 17, 2026

Uh oh!

github-actions Bot commented Jun 17, 2026

Uh oh!

github-actions Bot commented Jun 17, 2026

Uh oh!

github-actions Bot commented Jun 18, 2026

Uh oh!

github-actions Bot commented Jun 18, 2026

Uh oh!

github-actions Bot commented Jun 18, 2026

Uh oh!

github-actions Bot commented Jun 18, 2026

Uh oh!

github-actions Bot commented Jun 18, 2026

Uh oh!

github-actions Bot commented Jun 18, 2026

Uh oh!

gustavolira commented Jun 18, 2026

Uh oh!

gustavolira commented Jun 18, 2026

Uh oh!

gustavolira commented Jun 18, 2026

Uh oh!

github-actions Bot commented Jun 18, 2026

Uh oh!

gustavolira commented Jun 19, 2026

Uh oh!

github-actions Bot commented Jun 19, 2026

Uh oh!

github-actions Bot commented Jun 19, 2026

Uh oh!

github-actions Bot commented Jun 19, 2026

Uh oh!

gustavolira commented Jun 20, 2026

Uh oh!

gustavolira commented Jul 3, 2026

Uh oh!

github-actions Bot commented Jul 6, 2026

Uh oh!

github-actions Bot commented Jul 6, 2026

Uh oh!

github-actions Bot commented Jul 7, 2026

Uh oh!

gustavolira commented Jul 7, 2026

Uh oh!

gustavolira commented Jul 7, 2026

Uh oh!

openshift-ci Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 7, 2026

Uh oh!

sonarqubecloud Bot commented Jul 7, 2026

Quality Gate passed

gustavolira commented Jun 17, 2026 •

edited

Loading

codecov Bot commented Jun 17, 2026 •

edited

Loading

rhdh-qodo-merge Bot commented Jun 17, 2026 •

edited by openshift-ci Bot

Loading

openshift-ci Bot commented Jul 7, 2026 •

edited

Loading