Skip to content

NO-JIRA: Re-enable tests for the recommend cmd if alertsByCVO - #31440

Open
hongkailiu wants to merge 1 commit into
openshift:mainfrom
hongkailiu:OTA-1814-re-enable
Open

NO-JIRA: Re-enable tests for the recommend cmd if alertsByCVO#31440
hongkailiu wants to merge 1 commit into
openshift:mainfrom
hongkailiu:OTA-1814-re-enable

Conversation

@hongkailiu

@hongkailiu hongkailiu commented Jul 28, 2026

Copy link
Copy Markdown
Member

It was disabled in #31417.

Requiring openshift/oc#2322

/hold

Summary by CodeRabbit

  • Tests
    • Updated upgrade command recommendation tests to respect feature-gate settings for risk/alert evaluation.
    • Adjusted expected results for HyperShift environments, using branched assertions based on whether alerts are evaluated.
    • Improved coverage for conditional recommendations and for the --version target, including updated accept-guidance expectations when alerts-by-CVO is enabled.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jul 28, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@hongkailiu: This pull request explicitly references no jira issue.

Details

In response to this:

It was disabled in #31417.

Requiring openshift/oc#2322

/hold

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Jul 28, 2026
@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: c1b0b5b5-be55-4a27-aa41-1fb7335b31d8

📥 Commits

Reviewing files that changed from the base of the PR and between f1b04d8 and 7ba69fd.

📒 Files selected for processing (1)
  • test/extended/cli/adm_upgrade/recommend.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/extended/cli/adm_upgrade/recommend.go

Walkthrough

The upgrade recommendation test detects HyperShift and the FeatureGateClusterUpdateAcceptRisks setting for the target ClusterVersion, then adjusts conditional recommendation and --version expectations based on whether CVO evaluates update-accept risks.

Changes

Upgrade risk test behavior

Layer / File(s) Summary
Alert evaluation state and setup
test/extended/cli/adm_upgrade/recommend.go
Initializes shared HyperShift and alert-evaluation state, adds feature-gate lookup through alertsEvaluatedByCVO, and reuses the computed HyperShift value during setup.
Conditional recommendation expectations
test/extended/cli/adm_upgrade/recommend.go
Branches recommendation and --version assertions on alertsByCVO, retaining regex-based checks when CVO does not evaluate alerts and expecting accept guidance otherwise.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: petr-muller, davidhurta


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (1 error)

Check name Status Explanation Resolution
No-Sensitive-Data-In-Logs ❌ Error The PR adds a failure message that prints raw oc adm upgrade recommend output, which can include internal service URLs/IPs. Remove the raw output dump or redact URLs/IPs before including command output in assertion messages.
✅ Passed checks (14 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: re-enabling recommend command tests when alertsByCVO is enabled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed All Ginkgo titles in the changed file are static strings; the PR only changed test bodies and left names deterministic.
Test Structure And Quality ✅ Passed PASS: The suite keeps each It focused, uses Ordered BeforeAll/AfterAll with explicit clusterrolebinding cleanup, and only uses finite sleeps plus a timed deployment-ready helper.
Microshift Test Compatibility ✅ Passed The test is MicroShift-guarded in BeforeAll via exutil.IsMicroShiftCluster() and g.Skip() before any ClusterVersion/FeatureGate use.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The changed recommend tests only exercise ClusterVersion/update-service behavior and create one deployment/service; no SNO-unsafe node-count, drain, anti-affinity, or topology assumptions were found.
Topology-Aware Scheduling Compatibility ✅ Passed Only a test file changed; the diff adds no manifests, controllers, or pod scheduling logic/topology assumptions.
Ote Binary Stdout Contract ✅ Passed No stdout writes or process-level hooks (main/init/TestMain/BeforeSuite/etc.) were added; the file only uses test setup and helper functions.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No IPv4-only assumptions or public-internet dependencies found; the helper uses net.JoinHostPort and binds the local server on ::.
No-Weak-Crypto ✅ Passed Changed file is test-only and the diff adds no weak-crypto APIs, custom crypto, or secret comparisons.
Container-Privileges ✅ Passed The PR only changes a test file and adds no privileged or securityContext settings in any manifest.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
test/extended/cli/adm_upgrade/recommend.go (1)

280-281: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Keep the helper comment rationale-focused.

Replace the behavior restatement and pinned implementation link with a short explanation of why HyperShift is excluded.

As per coding guidelines, Go comments should be minimal and explain why rather than what.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/cli/adm_upgrade/recommend.go` around lines 280 - 281, Update
the comment for alertsEvaluatedByCVO to briefly explain why HyperShift or hosted
clusters are excluded, removing the behavior restatement and pinned upstream
implementation link.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/extended/cli/adm_upgrade/recommend.go`:
- Around line 243-247: Implement the CVO-specific expectation in the alertsByCVO
branch of the recommendation check: define the expected output pattern for
CVO-evaluated accept risks and assert it there instead of allowing the shared
matchRegexp call to use the legacy pattern. Preserve the existing legacy regex
path when alertsByCVO is false.
- Around line 286-289: Update the FeatureGates().Get error handling to return
false, nil when apierrors.IsNotFound(err) indicates the cluster FeatureGate
resource is missing, while continuing to return false, err for all other errors.

---

Nitpick comments:
In `@test/extended/cli/adm_upgrade/recommend.go`:
- Around line 280-281: Update the comment for alertsEvaluatedByCVO to briefly
explain why HyperShift or hosted clusters are excluded, removing the behavior
restatement and pinned upstream implementation link.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 761a1860-2775-4386-a781-fd1df64ba976

📥 Commits

Reviewing files that changed from the base of the PR and between 18c766f and 23d8c3b.

📒 Files selected for processing (1)
  • test/extended/cli/adm_upgrade/recommend.go

Comment thread test/extended/cli/adm_upgrade/recommend.go
Comment on lines +286 to +289
featureGate, err := client.ConfigV1().FeatureGates().Get(ctx, "cluster", metav1.GetOptions{})
if err != nil {
return false, err
}

@coderabbitai coderabbitai Bot Jul 28, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Inspect existing repository conventions for Kubernetes NotFound handling.
rg -n --type go -C2 'apierrors\.IsNotFound|FeatureGates\(\)\.Get\(' test/extended vendor/github.com/openshift

Repository: openshift/origin

Length of output: 50373


🏁 Script executed:

#!/bin/bash
set -euo pipefail

file=test/extended/cli/adm_upgrade/recommend.go

echo "== relevant slice =="
sed -n '240,320p' "$file" | cat -n

echo
echo "== imports =="
sed -n '1,60p' "$file" | cat -n

echo
echo "== function references =="
rg -n 'alertsEvaluatedByCVO|FeatureGates\(\)\.Get|apierrors\.IsNotFound|FeatureGate' "$file" test/extended/cli/adm_upgrade -C 2

Repository: openshift/origin

Length of output: 11157


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Find the exact function and its call sites to understand intended fallback behavior.
rg -n 'func alertsEvaluatedByCVO|alertsEvaluatedByCVO\(' test/extended/cli/adm_upgrade -C 3

Repository: openshift/origin

Length of output: 1498


🌐 Web query:

OpenShift cluster-version-operator cvo.go FeatureGateClusterUpdateAcceptRisks FeatureGates Get cluster not found disabled

💡 Result:

The FeatureGateClusterUpdateAcceptRisks is a feature gate introduced to the OpenShift Cluster Version Operator (CVO) to support the reconciliation of accepted risks during cluster updates [1][2][3]. This feature allows cluster administrators to specify accepted conditional update risks in the ClusterVersion CRD under spec.desiredUpdate.acceptRisks, provided the feature gate is enabled [1][3]. Regarding your observation of a "cluster not found" error related to feature gates in CVO: 1. Nature of the Error: In the context of the CVO and its feature gate processing, errors like 'Get "https://127.0.0.1:6443/apis/config.openshift.io/v1/featuregates/cluster": dial tcp 127.0.0.1:6443: connect: connection refused' often occur during operator startup [4]. These typically indicate that the CVO attempted to access the FeatureGate resource before the API server was fully responsive or before the informer cache had finished syncing [5][4][6]. 2. Resolution and Handling: Historically, the CVO has been updated to handle these transient failures more gracefully [7]. Recent architectural improvements have shifted FeatureGate retrieval to rely on informers rather than direct API polling [5]. If a "cluster not found" error occurs, it is frequently due to an incorrect initialization order where a 'Get' call on a lister is executed before the informer factory has been started and synchronized [6]. The fix generally involves ensuring the informer factory is fully prepared (started and synchronized) before attempting to retrieve the 'cluster' FeatureGate instance [5][6]. 3. Context for Development: If you are encountering this while modifying or debugging code in cvo.go or related packages, ensure that your logic respects the lifecycle of the controller's informer factory [6]. Using cached listers properly after ensuring cache synchronization is the standard pattern to avoid "not found" or "connection refused" errors [5][6].

Citations:


🌐 Web query:

site:github.com/openshift/cluster-version-operator FeatureGates().Get("cluster") FeatureGateClusterUpdateAcceptRisks NotFound

💡 Result:

The message "FeatureGates.Get("cluster") FeatureGateClusterUpdateAcceptRisks NotFound" refers to an interaction within the OpenShift Cluster Version Operator (CVO) [1][2]. This error (or diagnostic log) indicates that the CVO is attempting to retrieve the cluster-wide FeatureGate configuration, specifically checking for the presence or status of the FeatureGateClusterUpdateAcceptRisks feature [2][3]. Key technical context: 1. FeatureGate Interaction: The CVO uses the standard Kubernetes/OpenShift API to retrieve the "cluster" FeatureGate object [3][4]. If this object is not found (returned as a NotFound error by the API), the operator typically assumes a default state [3]. 2. Specific Feature Gate: FeatureGateClusterUpdateAcceptRisks is a feature introduced to allow the CVO to reconcile user-accepted risks during cluster updates [2]. 3. Expected Behavior: In the OpenShift architecture, if the FeatureGate object for "cluster" is not found, the operator defaults to standard behavior [3]. If you are encountering this as an error message or log entry, it generally signifies that the operator is operating under the assumption that the feature is disabled or not explicitly configured [2][3]. If you are seeing this during cluster operations, it usually indicates that the ClusterVersion Operator has successfully handled the absence of this feature gate configuration and is proceeding with default logic [3]. If the operator is failing to start or function correctly, verify the status of your FeatureGate configuration in the cluster [3].

Citations:


Treat a missing FeatureGate/cluster as disabled. Return false, nil for apierrors.IsNotFound(err) so clusters without that resource don't fail BeforeAll; keep propagating other GET errors.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/cli/adm_upgrade/recommend.go` around lines 286 - 289, Update
the FeatureGates().Get error handling to return false, nil when
apierrors.IsNotFound(err) indicates the cluster FeatureGate resource is missing,
while continuing to return false, err for all other errors.

Source: Learnings

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer in the change of this pull.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hongkailiu Understood—this is no longer in scope for #31440. Would you like me to open a follow-up GitHub issue for handling a missing FeatureGate/cluster as disabled?


🧠 Learnings used
Learnt from: hongkailiu
Repo: openshift/origin PR: 31417
File: test/extended/cli/adm_upgrade/recommend.go:283-285
Timestamp: 2026-07-28T11:06:40.515Z
Learning: In OpenShift Origin e2e tests, when looking up the `config.openshift.io/v1` `FeatureGate` named `cluster`, a “not found”/missing `FeatureGate/cluster` should be treated as “feature gate disabled” (i.e., not an error). Helper code (e.g., upgrade/CLI helpers like `alertsEvaluatedByCVO`) should distinguish NotFound from real retrieval failures and avoid propagating retrieval errors that would incorrectly fail the test when the FeatureGate resource simply doesn’t exist.

@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hongkailiu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 28, 2026
@hongkailiu
hongkailiu force-pushed the OTA-1814-re-enable branch from 23d8c3b to f1b04d8 Compare July 28, 2026 17:07
@hongkailiu
hongkailiu marked this pull request as ready for review July 29, 2026 10:34
@hongkailiu

hongkailiu commented Jul 29, 2026

Copy link
Copy Markdown
Member Author

/payload-job periodic-ci-openshift-release-main-ci-5.0-e2e-aws-ovn-techpreview-serial-2of3

@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 29, 2026
@openshift-ci

openshift-ci Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

@hongkailiu: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-ci-5.0-e2e-aws-ovn-techpreview-serial-2of3

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/091b0b00-8b39-11f1-843e-e9ec4b2febbb-0

@openshift-ci
openshift-ci Bot requested review from DavidHurta and petr-muller July 29, 2026 10:35
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@hongkailiu
hongkailiu force-pushed the OTA-1814-re-enable branch from f1b04d8 to 7ba69fd Compare July 29, 2026 14:13
@hongkailiu

Copy link
Copy Markdown
Member Author

/payload-job periodic-ci-openshift-release-main-ci-5.0-e2e-aws-ovn-techpreview-serial-2of3

@openshift-ci

openshift-ci Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

@hongkailiu: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-ci-5.0-e2e-aws-ovn-techpreview-serial-2of3

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/ced35cd0-8b57-11f1-9d0e-6f83db47a6b0-0

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@openshift-ci

openshift-ci Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

@hongkailiu: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-fips 7ba69fd link true /test e2e-aws-ovn-fips
ci/prow/e2e-gcp-ovn 7ba69fd link true /test e2e-gcp-ovn
ci/prow/e2e-aws-ovn-serial-1of2 7ba69fd link true /test e2e-aws-ovn-serial-1of2
ci/prow/e2e-metal-ipi-ovn-ipv6 7ba69fd link true /test e2e-metal-ipi-ovn-ipv6

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@hongkailiu

Copy link
Copy Markdown
Member Author

install failure on the previous one.

/payload-job periodic-ci-openshift-release-main-ci-5.0-e2e-aws-ovn-techpreview-serial-2of3

@openshift-ci

openshift-ci Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

@hongkailiu: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-ci-5.0-e2e-aws-ovn-techpreview-serial-2of3

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/9749fb90-8b87-11f1-92da-599146efaf38-0

@hongkailiu

Copy link
Copy Markdown
Member Author

/retest-required

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants