Skip to content

Improve test output for nodes should be ready test#31419

Open
dgoodwin wants to merge 1 commit into
openshift:mainfrom
dgoodwin:more-info-for-verify-tests
Open

Improve test output for nodes should be ready test#31419
dgoodwin wants to merge 1 commit into
openshift:mainfrom
dgoodwin:more-info-for-verify-tests

Conversation

@dgoodwin

@dgoodwin dgoodwin commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Now allows us to see the problematic conditions directly in test output instead of fishing through artifacts

Summary by CodeRabbit

  • Bug Fixes

    • Improved machine-node consistency failure messages with a multi-line, per-node breakdown of problematic conditions (type, status, reason, and message), including whether nodes are marked unschedulable.
  • Tests

    • Updated node-condition assertions to validate full returned node details (including condition subsets and unschedulable flag) instead of only node names.
    • Added table-driven coverage for the logic that determines which node conditions are considered “problematic.”

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci
openshift-ci Bot requested review from deads2k and sjenning July 23, 2026 12:40
@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 23, 2026
@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown

Walkthrough

Node consistency analysis now returns unready or unschedulable nodes with problematic conditions and includes those details in failure messages. Tests validate the structured result and condition-classification rules.

Changes

Node consistency reporting

Layer / File(s) Summary
Structured node condition model
pkg/e2eanalysis/e2e_analysis.go, pkg/e2eanalysis/e2e_analysis_test.go
Introduces unreadyNode, replaces the name-only helper, classifies problematic node conditions, and adds table-driven predicate coverage.
Consistency failure report
pkg/e2eanalysis/e2e_analysis.go
Updates Case 2 to format node names, unschedulable state, and each node’s problematic condition details.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested reviewers: sjenning, deads2k


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (1 error)

Check name Status Explanation Resolution
No-Sensitive-Data-In-Logs ❌ Error FAIL: the new node failure report prints raw node names and unredacted condition Reason/Message text, which can leak internal hostnames or other cluster data. Redact or whitelist condition fields before emitting them, and avoid printing raw node/message text unless it is sanitized for sensitive data.
✅ Passed checks (14 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: improving output for the nodes should be ready test.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The changed tests use only static, descriptive titles; no dynamic names, timestamps, node names, or generated IDs appear in test titles.
Test Structure And Quality ✅ Passed These are table-driven unit tests; no cluster setup, waits, or cleanup concerns apply, and the tests each target one behavior in repo style.
Microshift Test Compatibility ✅ Passed The PR only changes unit tests in pkg/e2eanalysis; no new Ginkgo e2e It/Describe/Context tests or MicroShift-sensitive APIs were added.
Single Node Openshift (Sno) Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; the changes are only unit tests and helper logic in pkg/e2eanalysis, with no multi-node/SNO assumptions.
Topology-Aware Scheduling Compatibility ✅ Passed PR only updates e2e-analysis node failure reporting and tests; no manifests, controllers, replicas, affinity, selectors, or topology assumptions were added.
Ote Binary Stdout Contract ✅ Passed Patch only changes node-condition formatting and tests; no stdout writes were added in main/init/TestMain/setup paths.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PR only changes pkg/e2eanalysis unit tests/helper logic; no new Ginkgo e2e tests, IPv4 literals, or external connectivity requirements were added.
No-Weak-Crypto ✅ Passed Touched code only changes node-readiness reporting/tests; no weak crypto, custom crypto, or secret comparisons found.
Container-Privileges ✅ Passed The PR only changes Go logic/tests in pkg/e2eanalysis; no container/K8s manifests or privilege-related settings were added in the touched files.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
pkg/e2eanalysis/e2e_analysis_test.go (1)

204-211: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Test the new structured condition payload.

The updated test verifies only node names, so a regression that drops n.conditions would still pass. Add expected condition data to the table and assert it, including NodePIDPressure, NodeNetworkUnavailable, and the default classification branch.

As per coding guidelines, validate unit-test changes with go vet ./..., go test ./pkg/..., make verify, and make check.

Also applies to: 214-254

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/e2eanalysis/e2e_analysis_test.go` around lines 204 - 211, Add structured
condition expectations to the test cases driving getUnreadyOrUnschedulableNodes,
covering NodePIDPressure, NodeNetworkUnavailable, and the default classification
branch. Update the assertions to compare each returned node’s conditions as well
as names, preserving the existing name matching, then validate with go vet
./..., go test ./pkg/..., make verify, and make check.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/e2eanalysis/e2e_analysis.go`:
- Around line 663-674: Update getUnreadyOrUnschedulableNodes and the unreadyNode
failure rendering to preserve scheduling blockers, including Spec.Unschedulable,
even when no problematic status condition exists. Record the unschedulable
reason alongside collected conditions for every blocker recognized by
IsNodeSchedulable, and ensure the output prints that reason instead of an empty
explanation.

---

Nitpick comments:
In `@pkg/e2eanalysis/e2e_analysis_test.go`:
- Around line 204-211: Add structured condition expectations to the test cases
driving getUnreadyOrUnschedulableNodes, covering NodePIDPressure,
NodeNetworkUnavailable, and the default classification branch. Update the
assertions to compare each returned node’s conditions as well as names,
preserving the existing name matching, then validate with go vet ./..., go test
./pkg/..., make verify, and make check.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 231d5edd-9bbd-4ee5-bc3e-3a9928d0bb07

📥 Commits

Reviewing files that changed from the base of the PR and between c73a0bb and bb9b0ee.

📒 Files selected for processing (2)
  • pkg/e2eanalysis/e2e_analysis.go
  • pkg/e2eanalysis/e2e_analysis_test.go

Comment thread pkg/e2eanalysis/e2e_analysis.go
@dgoodwin

Copy link
Copy Markdown
Contributor Author

/test images

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@mstaeble mstaeble left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor comments.

Comment thread pkg/e2eanalysis/e2e_analysis.go Outdated
names[i] = n.name
}
message := fmt.Sprintf("Found %d out of %d Nodes not Ready or unschedulable: %s\n",
len(notReadyNodes), len(nodeList.Items), strings.Join(names, ", "))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the names listed out here, if they are going to be listed out one separate lines below, too? Or is the point to make the names prominent without having to scan through the rest of the message?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove them.

Comment thread pkg/e2eanalysis/e2e_analysis.go Outdated
for i, n := range notReadyNodes {
names[i] = n.name
}
message := fmt.Sprintf("Found %d out of %d Nodes not Ready or unschedulable: %s\n",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using strings.Builder to build out the message.

Comment thread pkg/e2eanalysis/e2e_analysis.go Outdated
return c.Status != k8sv1.ConditionTrue
case k8sv1.NodeMemoryPressure, k8sv1.NodeDiskPressure, k8sv1.NodePIDPressure, k8sv1.NodeNetworkUnavailable:
return c.Status != k8sv1.ConditionFalse
default:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am confused by this default case. If we want to include all unknown conditions that are not False, why are we explicitly specifying the four conditions above?

I suggest we either drop the case k8sv1.NodeMemoryPressure, k8sv1.NodeDiskPressure, k8sv1.NodePIDPressure, k8sv1.NodeNetworkUnavailable: or ignore unknown conditions.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that doesn't make a lot of sense. I'll just do these four, I don't know what else is in there and if we see not ready but all of these are false, there are other artifacts to check.

Now allows us to see the problematic conditions directly in test output instead of fishing through artifacts
@dgoodwin
dgoodwin force-pushed the more-info-for-verify-tests branch from c7f2a67 to b978b31 Compare July 24, 2026 18:33

@dgoodwin dgoodwin left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ready for another look

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
pkg/e2eanalysis/e2e_analysis.go (1)

575-584: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Handle the formatting write errors.

fmt.Fprintf and strings.Builder.WriteString return errors here, but all four are discarded. Propagate them through checkMachineNodeConsistency.

Proposed fix
-       fmt.Fprintf(&b, "Found %d out of %d Nodes not Ready or unschedulable:\n",
-           len(notReadyNodes), len(nodeList.Items))
+       if _, err := fmt.Fprintf(&b, "Found %d out of %d Nodes not Ready or unschedulable:\n",
+           len(notReadyNodes), len(nodeList.Items)); err != nil {
+           return -1, fmt.Errorf("format node consistency report: %w", err)
+       }
        for _, n := range notReadyNodes {
-           fmt.Fprintf(&b, "Node %s:\n", n.name)
+           if _, err := fmt.Fprintf(&b, "Node %s:\n", n.name); err != nil {
+               return -1, fmt.Errorf("format node consistency report: %w", err)
+           }
            if n.unschedulable {
-               b.WriteString("  Spec.Unschedulable=true (cordoned)\n")
+               if _, err := b.WriteString("  Spec.Unschedulable=true (cordoned)\n"); err != nil {
+                   return -1, fmt.Errorf("format node consistency report: %w", err)
+               }
            }
            for _, c := range n.conditions {
-               fmt.Fprintf(&b, "  %s=%s reason=%s: %s\n", c.Type, c.Status, c.Reason, c.Message)
+               if _, err := fmt.Fprintf(&b, "  %s=%s reason=%s: %s\n", c.Type, c.Status, c.Reason, c.Message); err != nil {
+                   return -1, fmt.Errorf("format node consistency report: %w", err)
+               }
            }
        }

As per path instructions, “Never ignore error returns.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/e2eanalysis/e2e_analysis.go` around lines 575 - 584, Update
checkMachineNodeConsistency to handle and propagate errors from every
fmt.Fprintf and strings.Builder.WriteString call while constructing the node
consistency report, returning immediately on failure instead of discarding the
errors.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@pkg/e2eanalysis/e2e_analysis.go`:
- Around line 575-584: Update checkMachineNodeConsistency to handle and
propagate errors from every fmt.Fprintf and strings.Builder.WriteString call
while constructing the node consistency report, returning immediately on failure
instead of discarding the errors.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: d0da093a-4315-423d-a73e-b849cd7ed4b1

📥 Commits

Reviewing files that changed from the base of the PR and between c7f2a67 and b978b31.

📒 Files selected for processing (2)
  • pkg/e2eanalysis/e2e_analysis.go
  • pkg/e2eanalysis/e2e_analysis_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/e2eanalysis/e2e_analysis_test.go

@mstaeble mstaeble left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 24, 2026
@openshift-ci

openshift-ci Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dgoodwin, mstaeble

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@openshift-ci

openshift-ci Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

@dgoodwin: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-fips b978b31 link true /test e2e-aws-ovn-fips
ci/prow/e2e-vsphere-ovn b978b31 link true /test e2e-vsphere-ovn

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants