Skip to content

CMP-3830: Added testing TailoredProfile variable customization#76

Open
taimurhafeez wants to merge 1 commit intoComplianceAsCode:mainfrom
taimurhafeez:CMP-3830
Open

CMP-3830: Added testing TailoredProfile variable customization#76
taimurhafeez wants to merge 1 commit intoComplianceAsCode:mainfrom
taimurhafeez:CMP-3830

Conversation

@taimurhafeez
Copy link
Copy Markdown
Collaborator

@taimurhafeez taimurhafeez commented Apr 23, 2026

Added TestVariableCustomization() which:

  1. Creates test resources (namespaces, SCCs) to trigger compliance failures
  2. Builds dynamic variable values from cluster state
  3. Runs baseline scan with default variable values
  4. Creates TailoredProfile extending base profile with variable overrides
  5. Runs customized scan with variable exemptions
  6. Compares results and validates that variables affect compliance outcomes
  7. Cleans up all test resources

Variables tested:

ocp4-var-network-policies-namespaces-exempt-regex - Namespace exemptions
ocp4-var-sccs-with-allowed-capabilities-regex - SCC capability exemptions

Run on OCP 4.22:

go test -v -timeout 120m . -run=^TestVariableCustomization$ \
  -variable-test=true \
  -install-operator=true

Output:

e2e_test.go:702: Customized scan completed with 91 results
    e2e_test.go:705: Comparing baseline vs. customized results...
    e2e_test.go:714: WARNING: No differences found between baseline and customized scans
    e2e_test.go:718: Validating that variables affected expected rules...
    e2e_test.go:748: ✓ Variable ocp4-var-network-policies-namespaces-exempt-regex changed configure-network-policies-namespaces: FAIL → PASS
    e2e_test.go:745: ℹ Variable ocp4-var-sccs-with-allowed-capabilities-regex did not change result for scc-limit-container-allowed-capabilities (both: FAIL)
    e2e_test.go:763: ✓ Variable customization test completed successfully
    e2e_test.go:764: Variables tested: 2
    e2e_test.go:765: Total result differences: 0
2026/04/23 15:42:45 Deleted ScanSettingBinding customized-scan-binding
2026/04/23 15:42:45 Waiting for ComplianceSuite and results cleanup for customized-scan-binding
2026/04/23 15:42:45 Still waiting for cleanup after 5s: 91 ComplianceCheckResults still exist for suite customized-scan-binding
2026/04/23 15:42:50 Scan cleanup completed for customized-scan-binding
2026/04/23 15:42:50 Deleted ScanSettingBinding baseline-scan-binding
2026/04/23 15:42:50 Waiting for ComplianceSuite and results cleanup for baseline-scan-binding
2026/04/23 15:42:51 Still waiting for cleanup after 5s: 96 ComplianceCheckResults still exist for suite baseline-scan-binding
2026/04/23 15:42:56 Scan cleanup completed for baseline-scan-binding
2026/04/23 15:42:56 Deleted test SCC: scc-test-var-74227
2026/04/23 15:42:56 Deleted test namespace: test-ns-var-74227-2
2026/04/23 15:42:56 Deleted test namespace: test-ns-var-74227-1
--- PASS: TestVariableCustomization (100.94s)
PASS

Assisted by Claude.

@taimurhafeez taimurhafeez changed the title CMP-3830: Added testing TailoredProfile variable customization in the ocp4e2e t… CMP-3830: Added testing TailoredProfile variable customization Apr 23, 2026
@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Apr 23, 2026

@taimurhafeez: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-rhcos4-moderate 37fd0b7 link true /test e2e-aws-rhcos4-moderate
ci/prow/e2e-aws-ocp4-stig 37fd0b7 link true /test e2e-aws-ocp4-stig
ci/prow/e2e-aws-ocp4-cis 37fd0b7 link true /test e2e-aws-ocp4-cis

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant