Skip to content

CMP-3831: Add test for namespace exemption test for resource limit checks#71

Open
taimurhafeez wants to merge 4 commits intoComplianceAsCode:mainfrom
taimurhafeez:CMP-3831
Open

CMP-3831: Add test for namespace exemption test for resource limit checks#71
taimurhafeez wants to merge 4 commits intoComplianceAsCode:mainfrom
taimurhafeez:CMP-3831

Conversation

@taimurhafeez
Copy link
Copy Markdown
Collaborator

@taimurhafeez taimurhafeez commented Mar 24, 2026

Implements test case to validate that namespace exemption variables work correctly for resource limit compliance rules. The test creates workloads without resource limits in exempted namespaces and verifies that the exemption regex patterns properly exclude them from failing the daemonset, deployment, and statefulset resource limit checks.

Test it using:
go test -v -timeout 60m -test-type platform -run TestNamespaceExemptionVariables -content-directory /your/path/to/content/repo -install-operator=true -log-dir ./logs
On OCP:

2026/03/24 16:47:54 Using default content image: quay.io/redhat-user-workloads/ocp-isc-tenant/compliance-operator-content-dev:master
2026/03/24 16:47:54 ProfileBundle ocp4 is valid
2026/03/24 16:47:55 ProfileBundle rhcos4 is valid
2026/03/24 16:47:55 Setup completed successfully
=== RUN   TestNamespaceExemptionVariables
    e2e_test.go:528: Created test namespace: ns-76797-test-1
    e2e_test.go:528: Created test namespace: ns-76797-test-2
    e2e_test.go:543: Created test workloads without resource limits in ns-76797-test-1
    e2e_test.go:563: Created TailoredProfile: ns-exemption-test-profile with exemption pattern: ^ns-76797-test-.*$
2026/03/24 16:48:02 Created new ScanSettingBinding ns-exemption-scan-binding
    e2e_test.go:576: Created ScanSettingBinding: ns-exemption-scan-binding
2026/03/24 16:48:02 ComplianceSuite ns-exemption-scan-binding doesn't exist yet, waiting for it to be created
2026/03/24 16:48:02 ComplianceSuite ns-exemption-scan-binding is not DONE: no statuses available yet
2026/03/24 16:48:07 ComplianceSuite ns-exemption-scan-binding is not DONE: suite ns-exemption-scan-binding scan ns-exemption-test-profile is RUNNING
2026/03/24 16:48:12 ComplianceSuite ns-exemption-scan-binding is not DONE: suite ns-exemption-scan-binding scan ns-exemption-test-profile is RUNNING
2026/03/24 16:48:18 ComplianceSuite ns-exemption-scan-binding is not DONE: suite ns-exemption-scan-binding scan ns-exemption-test-profile is RUNNING
2026/03/24 16:48:23 ComplianceSuite ns-exemption-scan-binding is not DONE: suite ns-exemption-scan-binding scan ns-exemption-test-profile is RUNNING
2026/03/24 16:48:28 ComplianceSuite ns-exemption-scan-binding is not DONE: suite ns-exemption-scan-binding scan ns-exemption-test-profile is RUNNING
2026/03/24 16:48:33 ComplianceSuite ns-exemption-scan-binding is not DONE: suite ns-exemption-scan-binding scan ns-exemption-test-profile is RUNNING
2026/03/24 16:48:38 ComplianceSuite ns-exemption-scan-binding is not DONE: suite ns-exemption-scan-binding scan ns-exemption-test-profile is RUNNING
2026/03/24 16:48:43 ComplianceSuite ns-exemption-scan-binding is not DONE: suite ns-exemption-scan-binding scan ns-exemption-test-profile is RUNNING
2026/03/24 16:48:48 ComplianceSuite ns-exemption-scan-binding is not DONE: suite ns-exemption-scan-binding scan ns-exemption-test-profile is AGGREGATING
2026/03/24 16:48:53 ComplianceSuite ns-exemption-scan-binding is not DONE: suite ns-exemption-scan-binding scan ns-exemption-test-profile is AGGREGATING
2026/03/24 16:48:59 ComplianceSuite ns-exemption-scan-binding is DONE
2026/03/24 16:48:59 Created result map with 3 rules for suite ns-exemption-scan-binding
    e2e_test.go:591: Found 3 results:
    e2e_test.go:593:   - ns-exemption-test-profile-resource-requests-limits-in-statefulset: PASS
    e2e_test.go:593:   - ns-exemption-test-profile-resource-requests-limits-in-daemonset: PASS
    e2e_test.go:593:   - ns-exemption-test-profile-resource-requests-limits-in-deployment: PASS
    e2e_test.go:616: ✓ Rule resource-requests-limits-in-statefulset: PASS (namespace exemption working correctly)
    e2e_test.go:616: ✓ Rule resource-requests-limits-in-daemonset: PASS (namespace exemption working correctly)
    e2e_test.go:616: ✓ Rule resource-requests-limits-in-deployment: PASS (namespace exemption working correctly)
2026/03/24 16:48:59 Saved YAML data to logs/namespace-exemption-test-results.yaml
    e2e_test.go:630: Namespace exemption test passed successfully - all exempted workloads passed the resource limit checks
--- PASS: TestNamespaceExemptionVariables (64.06s)
PASS
ok  	github.com/ComplianceAsCode/ocp4e2e	66.103s

Assisted by Claude.

@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Mar 24, 2026

Hi @taimurhafeez. Thanks for your PR.

I'm waiting for a ComplianceAsCode member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Tip

We noticed you've done this a few times! Consider joining the org to skip this step and gain /lgtm and other bot rights. We recommend asking approvers on your previous PRs to sponsor you.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@taimurhafeez taimurhafeez changed the title CMP-3881: Upstream Improvement: Enhance ocp4e2e suite for namespace exemption test for resource limit checks CMP-3831: Upstream Improvement: Enhance ocp4e2e suite for namespace exemption test for resource limit checks Mar 24, 2026
@taimurhafeez taimurhafeez changed the title CMP-3831: Upstream Improvement: Enhance ocp4e2e suite for namespace exemption test for resource limit checks CMP-3831: Add test for namespace exemption test for resource limit checks - WIP Mar 24, 2026
@taimurhafeez
Copy link
Copy Markdown
Collaborator Author

/ok-to-test

@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Mar 24, 2026

@taimurhafeez: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ocp4-stig 9a54826 link true /test e2e-aws-ocp4-stig
ci/prow/e2e-aws-ocp4-cis 9a54826 link true /test e2e-aws-ocp4-cis
ci/prow/e2e-aws-rhcos4-moderate 9a54826 link true /test e2e-aws-rhcos4-moderate
ci/prow/e2e-aws-openshift-node-compliance 9a54826 link true /test e2e-aws-openshift-node-compliance

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@taimurhafeez taimurhafeez changed the title CMP-3831: Add test for namespace exemption test for resource limit checks - WIP CMP-3831: Add test for namespace exemption test for resource limit checks Mar 25, 2026
Copy link
Copy Markdown
Collaborator

@rhmdnd rhmdnd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely seems like a good thing to add coverage for, but I'm wondering if this should be a functional test in the compliance operator repository?

Do you have strong opinions about it living here as opposed to the compliance-operator e2e tests?

Comment thread e2e_test.go

// Test namespace names
testNamespaces := []string{
"ns-76797-test-1",
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does 76797 carry significance?

Comment thread helpers.go

// deleteNamespace deletes a namespace.
func deleteNamespace(c dynclient.Client, name string) error {
c.Delete(context.TODO(), &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: name}})
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should return what c.Delete returns, like createNamespace does. Otherwise, we'll miss any errors in cleaning up that could be important.

Comment thread helpers.go
return result
}
for resultName, resultValue := range results {
if strings.Contains(resultName, ruleName) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rule names would always be a suffix of the result name? But what happens if we have the same rule running in two different profiles? This would return the first found, right?

Comment thread e2e_test.go
t.Logf("Created test workloads without resource limits in %s", testNamespaces[0])

// Wait for workloads to be created
time.Sleep(5 * time.Second)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'll be safer to add polling here instead of a blanket 5 second wait.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants