Skip to content

feat: enable pluginDivisionMode schema tests for OCP Operator nightly jobs [release-1.10]#4884

Merged
openshift-merge-bot[bot] merged 7 commits into
redhat-developer:release-1.10from
Fortune-Ndlovu:RHIDP-13221-operator-add-test-for-plugin-division-mode-schema-1.10
May 26, 2026
Merged

feat: enable pluginDivisionMode schema tests for OCP Operator nightly jobs [release-1.10]#4884
openshift-merge-bot[bot] merged 7 commits into
redhat-developer:release-1.10from
Fortune-Ndlovu:RHIDP-13221-operator-add-test-for-plugin-division-mode-schema-1.10

Conversation

@Fortune-Ndlovu
Copy link
Copy Markdown
Member

@Fortune-Ndlovu Fortune-Ndlovu commented May 26, 2026

Summary

Backport of (RHIDP-13221) to release-1.10.

  • Enable pluginDivisionMode: schema E2E tests for OCP Operator deployments
  • Wire up Crunchy PostgreSQL and schema-mode env configuration in the operator CI pipeline
  • Add runtime config change tests with INSTALL_METHOD=operator and cross-platform base64 encoding

Commits cherry-picked

  • docs(ci): clarify operator runtime test deployment method
  • feat(e2e): enable pluginDivisionMode schema tests for OCP Operator nightly
  • fix(ci): create dynamic-plugins ConfigMap for showcase-runtime operator deployment
  • fix(ci): use minimal dynamic-plugins config for showcase-runtime operator
  • fix(e2e): retry deployment restart for schema-mode on slow PVC creation
  • fix(e2e): treat ephemeral volume PVC scheduling as transient in pod failure check
  • feat(ci): set INSTALL_METHOD to operator in ocp-operator.sh

(Skipped commits already present on release-1.10: stale ConfigMap ref removal, Crunchy PostgreSQL restore.)

Test plan

  • /test e2e-ocp-operator-nightly on release-1.10
  • Verify plugin-division-mode-schema tests run in showcase-runtime namespace

Made with Cursor

Fortune-Ndlovu and others added 7 commits May 26, 2026 07:06
Signed-off-by: Fortune-Ndlovu <fortune.ndlovu2@gmail.com>
…ghtly

Enable pluginDivisionMode: schema E2E tests for OCP Operator deployments.
The tests previously skipped because schema-mode-setup.ts patched the
operator-managed Deployment spec to inject POSTGRES_* env vars, which the
operator reconciliation loop reverted — crashing the init container.

Fix: for operator deployments, update the existing postgres-cred secret
(already mounted via extraEnvs.secrets in the Backstage CR) instead of
creating a new secret and patching the Deployment. Skip
ensureDeploymentEnvVars() entirely for operator since env vars are
injected by the operator from the secret automatically.

CI pipeline changes:
- Wire up real Crunchy PostgreSQL via configure_external_postgres_db
- Configure SCHEMA_MODE_* env vars via configure_schema_mode_runtime_env
- Export INSTALL_METHOD=operator for correct deployment naming
- Fall back to placeholder secrets if Crunchy setup fails

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…or deployment

The Backstage CR references dynamicPluginsConfigMapName: dynamic-plugins
but the ConfigMap was never created in the showcase-runtime namespace,
causing the operator reconciler to fail.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ator

Use values-showcase-postgres.yaml (plugins: []) instead of the full
values_showcase.yaml to avoid the install-dynamic-plugins init container
hanging while downloading dozens of plugins. The runtime namespace only
needs default bundled plugins for schema-mode testing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The ephemeral volume controller can be slow to create PVCs on CI
clusters, causing the deployment restart to fail on the first attempt.
Add retry logic (up to 3 attempts with 30s delay) and increase the
beforeAll timeout to 15 minutes to accommodate retries.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ailure check

checkPodFailureStates() was treating all PodScheduled:False conditions as
immediate hard failures, bypassing the 10-minute timeout in
waitForDeploymentReady(). When the ephemeral volume controller is still
creating the PVC for dynamic-plugins-root, this is a transient state that
resolves on its own. Now logs a warning and continues polling instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This change establishes the INSTALL_METHOD environment variable as 'operator' for the deployment process. Additionally, it updates the base64 encoding method for the runtime URL in the operator runtime config change tests, ensuring compatibility with the common encoding function.
@rhdh-qodo-merge
Copy link
Copy Markdown

rhdh-qodo-merge Bot commented May 26, 2026

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0)

Context used
✅ Tickets: RHIDP-13221

Grey Divider


Advisory comments

1. PVC failures treated transient 🐞 Bug ☼ Reliability
Description
checkPodFailureStates now returns null (no failure) for any PodScheduled=False condition whose
message contains the substring persistentvolumeclaim, which can suppress genuine unschedulable
errors and prevent waitForDeploymentReady from taking its fail-fast diagnostics/throw path. It
also returns immediately on the first such pod/condition, potentially masking failures in other pods
selected by the same label selector.
Code

e2e-tests/playwright/utils/kube-client.ts[R606-615]

Relevance

⭐ Low

PR #4685 intentionally adds transient PVC handling in checkPodFailureStates to reduce restart
flakiness; likely desired behavior.

PR-#4685

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The updated logic returns null for a wide class of PodScheduled=False messages and
short-circuits the scan, while waitForDeploymentReady only throws (and gathers diagnostics) when a
non-null failure reason is returned; this can convert real scheduling failures into long timeouts
without early diagnostics.

e2e-tests/playwright/utils/kube-client.ts[599-617]
e2e-tests/playwright/utils/kube-client.ts[731-753]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`checkPodFailureStates()` treats any scheduler message containing `persistentvolumeclaim` as transient and returns `null`, which (a) can classify permanent scheduling errors as non-failures and (b) exits the pod scan early.

### Issue Context
`waitForDeploymentReady()` relies on `checkPodFailureStates()` returning a non-null reason to trigger early failure diagnostics and throw. Returning `null` here causes the loop to keep waiting until timeout, delaying feedback and potentially hiding other pod failures.

### Fix Focus Areas
- e2e-tests/playwright/utils/kube-client.ts[599-617]
- e2e-tests/playwright/utils/kube-client.ts[731-753]

### What to change
- Replace the broad `msg.includes("persistentvolumeclaim")` heuristic with a narrower check for known transient provisioning phrases/reasons (and/or gate on `condition.reason === "Unschedulable"`).
- Do not `return null` immediately; prefer `continue` to keep scanning other pods/conditions, and only return `null` after checking all pods.
- Consider adding a bounded “grace period”/retry counter for PVC-related unschedulable conditions so truly-stuck PVCs still fail fast with diagnostics.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

@sonarqubecloud
Copy link
Copy Markdown

@Fortune-Ndlovu Fortune-Ndlovu changed the title [release-1.10] RHIDP-13221: enable pluginDivisionMode schema tests for OCP Operator nightly feat: enable pluginDivisionMode schema tests for OCP Operator nightly jobs [release-1.10] May 26, 2026
@Fortune-Ndlovu
Copy link
Copy Markdown
Member Author

/test e2e-ocp-helm

@Fortune-Ndlovu
Copy link
Copy Markdown
Member Author

/test e2e-ocp-operator-nightly

@Fortune-Ndlovu
Copy link
Copy Markdown
Member Author

/test e2e-ocp-helm-nightly

@codecov
Copy link
Copy Markdown

codecov Bot commented May 26, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.60%. Comparing base (6141cd4) to head (ca472a5).
⚠️ Report is 1 commits behind head on release-1.10.

Additional details and impacted files
@@                Coverage Diff                @@
##           release-1.10    #4884       +/-   ##
=================================================
+ Coverage         41.03%   69.60%   +28.57%     
=================================================
  Files               121      111       -10     
  Lines              2220     4702     +2482     
  Branches            539      512       -27     
=================================================
+ Hits                911     3273     +2362     
- Misses             1304     1429      +125     
+ Partials              5        0        -5     
Flag Coverage Δ
install-dynamic-plugins 92.44% <ø> (?)
rhdh 38.81% <ø> (-2.23%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6141cd4...ca472a5. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@rhdh-qodo-merge
Copy link
Copy Markdown

Review Summary by Qodo

Enable pluginDivisionMode schema tests for OCP Operator nightly

✨ Enhancement 🧪 Tests

Grey Divider

Walkthroughs

Description
• Enable pluginDivisionMode schema E2E tests for OCP Operator deployments
• Update secret handling to use operator-managed postgres-cred instead of patching Deployment
• Add retry logic for deployment restart on slow ephemeral volume PVC creation
• Wire up Crunchy PostgreSQL and schema-mode environment configuration in operator CI pipeline
• Treat ephemeral volume PVC scheduling as transient in pod failure checks
Diagram
flowchart LR
  A["OCP Operator Deployment"] -->|"Update postgres-cred secret"| B["Schema Mode Setup"]
  B -->|"Skip Deployment patching"| C["Operator Reconciliation"]
  B -->|"Retry with backoff"| D["Deployment Restart"]
  E["Crunchy PostgreSQL"] -->|"Configure external DB"| F["Runtime Namespace"]
  F -->|"Set schema-mode env vars"| G["E2E Tests Run"]
  D -->|"Apply changes"| G

Loading

Grey Divider

File Changes

1. e2e-tests/playwright/e2e/plugin-division-mode-schema/schema-mode-setup.ts ✨ Enhancement +36/-6

Operator-aware secret handling and deployment restart retry logic

• Return postgres-cred secret name for operator install method instead of
 ${releaseName}-postgresql
• Skip ensureDeploymentEnvVars() for operator deployments since env vars are injected via
 extraEnvs.secrets in Backstage CR
• Add retry logic (up to 3 attempts with 30s backoff) for deployment restart to handle slow
 ephemeral volume PVC creation

e2e-tests/playwright/e2e/plugin-division-mode-schema/schema-mode-setup.ts


2. e2e-tests/playwright/e2e/plugin-division-mode-schema/verify-schema-mode.spec.ts ✨ Enhancement +1/-1

Increase test timeout for operator deployments

• Increase test timeout from 300s to 900s to accommodate slower operator deployments and PVC
 creation

e2e-tests/playwright/e2e/plugin-division-mode-schema/verify-schema-mode.spec.ts


3. e2e-tests/playwright/utils/kube-client.ts 🐞 Bug fix +11/-1

Treat ephemeral volume PVC scheduling as transient

• Treat ephemeral volume and persistentvolumeclaim PVC scheduling failures as transient (non-fatal)
• Return null instead of error message for transient PVC creation delays
• Log transient PVC wait status for debugging

e2e-tests/playwright/utils/kube-client.ts


View more (4)
4. .ci/pipelines/jobs/ocp-operator.sh ✨ Enhancement +43/-7

Wire up Crunchy PostgreSQL and schema-mode configuration

• Source schema-mode-env.sh library for schema-mode environment configuration
• Export INSTALL_METHOD=operator for correct deployment naming
• Update run_operator_runtime_config_change_tests() to configure external Crunchy PostgreSQL via
 configure_external_postgres_db()
• Add schema-mode environment configuration via configure_schema_mode_runtime_env() with operator
 install method parameter
• Use minimal dynamic-plugins config (values-showcase-postgres.yaml) instead of full showcase
 config
• Add NAME_SPACE_POSTGRES_DB variable for external PostgreSQL namespace
• Implement fallback to placeholder secrets if external PostgreSQL setup fails

.ci/pipelines/jobs/ocp-operator.sh


5. .ci/pipelines/lib/schema-mode-env.sh ✨ Enhancement +7/-6

Add install method parameter to schema-mode configuration

• Add install_method parameter (defaults to helm) to configure_schema_mode_runtime_env()
 function
• Update all log messages to include install method in output for better debugging

.ci/pipelines/lib/schema-mode-env.sh


6. docs/e2e-tests/CI-medic-guide.md 📝 Documentation +1/-1

Update documentation for enabled operator runtime tests

• Update OCP Operator nightly documentation to reflect that runtime config tests are now enabled
• Note that pluginDivisionMode schema tests now run with external Crunchy PostgreSQL

docs/e2e-tests/CI-medic-guide.md


7. e2e-tests/playwright/e2e/plugin-division-mode-schema/README.md 📝 Documentation +1/-1

Update README for enabled operator schema-mode tests

• Update CI behavior section to indicate schema-mode tests now run for OCP Operator nightly jobs
• Note that tests are auto-configured by schema-mode-env.sh with operator install method

e2e-tests/playwright/e2e/plugin-division-mode-schema/README.md


Grey Divider

Qodo Logo

@github-actions
Copy link
Copy Markdown
Contributor

Image was built and published successfully. It is available at:

@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented May 26, 2026

@Fortune-Ndlovu: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-ocp-helm-nightly ca472a5 link false /test e2e-ocp-helm-nightly

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@Fortune-Ndlovu
Copy link
Copy Markdown
Member Author

/test e2e-ocp-helm-nightly

@zdrapela
Copy link
Copy Markdown
Member

zdrapela commented May 26, 2026

/lgtm

@openshift-ci openshift-ci Bot added the lgtm label May 26, 2026
@openshift-merge-bot openshift-merge-bot Bot merged commit 63fb98c into redhat-developer:release-1.10 May 26, 2026
39 of 42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants