diff --git a/docs/en/solutions/Pipeline_Policy_Constraints_with_Tekton_and_Kyverno.md b/docs/en/solutions/Pipeline_Policy_Constraints_with_Tekton_and_Kyverno.md new file mode 100644 index 000000000..f076d3a3f --- /dev/null +++ b/docs/en/solutions/Pipeline_Policy_Constraints_with_Tekton_and_Kyverno.md @@ -0,0 +1,11534 @@ +--- +products: + - Alauda Container Platform + - Alauda DevOps +kind: + - Solution +ProductsVersion: + - 4.3.x +--- +# Pipeline Policy Constraints with Tekton and Kyverno + +:::info Applicable versions + +**Applies to: Alauda DevOps Pipelines v4.14.x and later** — that version is the criterion, not the ACP version (this document depends on the Tekton APIs and features shipped with Alauda DevOps Pipelines; the ACP version only determines whether the Kyverno plugin can be installed). On earlier versions these features are incomplete: the policy assets and examples in this document cannot be applied as-is (see [§3.2](#s3-2) for the hard prerequisites), though the mechanisms and design trade-offs are still worth reading. All mechanism explanations, policy assets, and quantitative figures in this document were produced against the following combination of versions: + +| Component | Version | Role | +|---|---|---| +| Alauda DevOps Pipelines (the ACP distribution of Tekton Pipelines) | v4.14.x | **Applicability criterion** — below this version, the policy assets do not apply | +| Alauda Artifact Hub Shim (the built-in ACP hub: an Artifact Hub-compatible API consumed by Tekton's hub resolver; the release source of the catalog Task / Pipeline definitions this document references) | v1.0.0 | The template / Task definitions in the [§3.2](#s3-2) contract matrix ship with it | +| Kyverno (ACP Compliance Management plugin) | v1.15.9-v4.3.2 | Policy engine; delivered by ACP's Compliance Management plugin | +| Alauda Container Platform | 4.3 | The platform hosting both of the above (the verification environment for this document) | + +**Re-test whenever you change versions.** The mechanisms are usually backward compatible, but result and parameter contracts change with Task and template versions (see the matrix in [§3.2](#s3-2)), and applying them across versions fails as a **silent mismatch** — the failure shape is not an error: the policy stays `Ready`, the reports stay clean, and the path you care about is simply no longer watched. The concrete numbers in this document likewise depend on the runtime environment (scale, network, load); re-measure in the target environment before putting them into a change request. For any combination outside this table, run the positive/negative probe regression per [§3.4](#s3-4) before switching to Enforce; after go-live, whenever any of Kyverno / Tekton / templates / Tasks / ACP is upgraded, use [§3.6](#s3-6) to locate the affected criteria and run the minimal regression set per [§3.8](#s3-8). + +::: + +## 1. Overview {#s1} + +In platform engineering practice, the CI/CD pipeline is the mandatory path through which every change reaches production — which makes it the key leverage point for enforcing an organization's engineering standards. Common governance requirements include: + +- **Template sprawl**: business teams bypass the platform-approved pipeline templates and assemble their own pipelines that lack quality steps; +- **Gates switched off**: the code scanning and quality gates in a template are disabled with a single parameter (for example, setting the scan switch to false) — the pipeline "appears to run the template" while the critical steps never execute; +- **Unauthorized sources and targets**: artifacts pulled from unapproved repositories, applications deployed to unauthorized namespaces; +- **Substandard results still shipped**: coverage or vulnerability counts miss the bar, yet the pipeline proceeds through the release stages anyway. + +This document describes how to enforce policy constraints on **Tekton**-based pipelines using **Kyverno** on Alauda Container Platform (ACP). Rather than a rule-by-rule how-to, it focuses on **mechanisms**: what Kyverno can see across the pipeline lifecycle, when it sees it, what actions it can take (block, audit, inject, cancel) — and how to build out a policy system tailored to your organization on top of these mechanism points, using custom Tasks and Task results. + +### 1.0 What you will be able to do after reading {#s1-0} + +Once your environment is prepared per [§3](#s3), you should be able to: + +- **Decide which layer owns a given governance requirement** — what Kyverno can block at admission, what only the construction of a trusted template can guarantee, and what must be left to RBAC or after-the-fact audit ([§1.4](#s1-4) boundaries, [§2.3](#s2-3) the seven contracts); +- **Lock template and Task identity**, so business teams cannot change "which template, which version" ([§4.1](#s4-1)); +- **Validate the effective values of gate parameters**, so that changes like "set the scan switch to false" or "drop the threshold to 0" are rejected the moment the gate TaskRun is created ([§4.2](#s4-2)); +- **Constrain sources and release targets** — only pull material from approved repositories/registries, only release to authorized namespaces ([§4.5](#s4-5)); +- **Close the entrances that bypass the pipeline** — bare TaskRuns, unapproved inline definitions and resolver types ([§4.5.4](#s4-5-4)); +- **Consume custom Task results** for audit, reporting, and automatic cancellation, bringing your in-house checks into the same governance system ([§2.4](#s2-4), [§4.4](#s4-4), [§4.6](#s4-6)); +- **Differentiate and exempt safely** — the two-tier model of platform baseline plus per-project tightening, controlled exemptions via PolicyException, and making sure the scoping itself cannot be bypassed ([§5](#s5)); +- **Operate the whole system** — staged rollout order, change and upgrade triggers, scale and failure budgets, and the minimal regression set to run after upgrades ([§3.5](#s3-5)–[§3.8](#s3-8)). + +**Out of scope**: image signing and supply-chain attestation (see the companion document *Software Supply Chain Security of ACP with Tekton and Kyverno*), the installation and operation of Kyverno itself (see the ACP Compliance Management documentation), and how to write pipeline templates — this document only states the contracts a template must satisfy. + +**Shortest evaluation path**: if all you want is to confirm whether this machinery can block the scenarios you care about, read [§1.4](#s1-4) + [§2.3](#s2-3). To get hands-on, follow the role-based paths in [§1.1](#s1-1). + +### 1.1 Audience and reading paths {#s1-1} + +| Role | Focus | Suggested path | +|---|---|---| +| Platform administrator (writes policies, manages scope) | The full mechanism picture, scope safety, policy assets | [§2](#s2) mechanisms overview (first learn what can be seen and done) → [§3](#s3) common configuration (install, verify, build fixtures) → [§5](#s5) scope control → [§4](#s4) Cookbook → [§6](#s6) FAQ | +| Project administrator (maintains per-project constraints) | Namespaced `Policy`, per-project tightening, permission boundaries | [§1.3](#s1-3) per-project differentiation and scope safety (the two-tier model) → [§5.1](#s5-1)–[§5.2](#s5-2) scope and RBAC → [§4](#s4) Cookbook (pick what you need; remember to convert the demo's cross-namespace scoping into a `Policy` in your own namespace) | +| Template / Task author (supplies governed pipelines) | Hard-gate contracts, extension contracts | [§2.3](#s2-3) hard-gate contracts → [§2.4](#s2-4) extension model → [§3.2](#s3-2) versions and dependent features → [§3.3](#s3-3) fixtures → [§4.3](#s4-3) genuine gate failure → the relevant parts of [§4.1](#s4-1)–[§4.2](#s4-2) | +| Pipeline user (runs pipelines, gets blocked by policy) | Quick reference of failure shapes, exemption path | [§1.5](#s1-5) quick reference of outcome shapes → [§6.2](#s6-2) user-side FAQ (only read [§6.2.3](#s6-2-3) if your run was auto-cancelled) | +| Walkthrough operator (runs the whole document as a lab) | Policies and run manifests copy-pasteable; probes assembled yourself from the [§3.4.1](#s3-4-1) skeleton (nine sections give only an expectation table); and no leftovers on a shared cluster | [§3.1](#s3-1) verification → **[§3.2](#s3-2) first confirm object results are enabled** (`enable-api-fields`; acceptable values per [§3.2](#s3-2) — if it is off, the very first fixture creation is rejected, with an error that looks like a Kyverno problem) → **[§4.0.3](#s4-0-3) placeholders + [§4.0.4](#s4-0-4) cleanup discipline (read before creating anything: self-created namespaces plus a pre-check for cluster-scoped name collisions are what make deletion possible afterwards)** → [§3.3](#s3-3) build the fixtures and **keep your walkthrough id at hand** → [§4.0.1](#s4-0-1) install order + **[§4.0.5](#s4-0-5) cross-section interference between demos** (the number-one reason "the probe won't run") → your target sections (**run each section's "cleanup" immediately after finishing it** — do not batch them up for the end) → the "final cleanup" in [§3.3](#s3-3) to delete the two shared namespaces; and if you did [§5.3](#s5-3), go back to [§3.1.1](#s3-1-1) at the very end to revert the platform configuration | + +**A few items in the [§3.1](#s3-1) checklist are forward references** (`--exceptionNamespace` in [§3.1.1](#s3-1-1), the mutate-existing RBAC in the [§4.6](#s4-6) introduction, replica planning in [§6.1.8](#s6-1-8)): that checklist is a **capability inventory**, not an "all green before you may proceed" gate — items 1 and 2 are shared prerequisites; come back for the rest depending on which chapter's capabilities you actually use. + +### 1.2 Kyverno in brief {#s1-2} + +Kyverno is a Kubernetes-native policy engine (a CNCF project), delivered on ACP through Compliance Management (the Kyverno plugin). The core concepts relevant to pipeline governance: + +- **Architecture**: the admission controller (admission webhook — enforces validate / mutate / image verification), the background controller (scans existing resources, executes mutate-existing / generate), the reports controller (produces compliance reports), and the cleanup controller (periodic cleanup). +- **Policy resources**: `ClusterPolicy` is a cluster-scoped resource, maintained by platform administrators, that can match namespaced resources across the whole cluster as well as cluster-scoped resources; `Policy` is a namespaced resource that only applies to resources inside its own `metadata.namespace` — the right vehicle for letting project administrators self-maintain their project's constraints. A rule is not a standalone Kubernetes resource; it is embedded in the policy's `spec.rules`, and each rule = `match/exclude` (which resources and operations to select) + optional `preconditions` (further filtering) + an action. +- **Action types**: + - `validate`: validates a resource. In `Enforce` mode it rejects at admission; in `Audit` mode it allows the request but records the result in a **PolicyReport**; + - `mutate`: modifies a resource at admission (injecting defaults); the **mutate-existing** variant can, upon a triggering event, modify other resources that **already exist** in the cluster; + - `generate`: creates new resources when triggered; + - `verifyImages`: image signature verification (not covered here — see the companion document *Software Supply Chain Security of ACP with Tekton and Kyverno*). +- **PolicyException**: the controlled exemption mechanism — it turns "who may bypass which rule" into a separate resource governed by RBAC ([§5.3](#s5-3)). +- **How it works**: once loaded, policies are registered as admission webhooks; every matching API request (CREATE/UPDATE/…) goes through policy evaluation. Audit results and background scan results both land in PolicyReports. + +For Kyverno's full capabilities, see the ACP Compliance Management documentation and the upstream Kyverno documentation ([§8.2](#s8-2) References); this document only develops the usage relevant to pipeline governance. + +**Terminology used throughout** (these words live at different layers; conflating them makes you misread where a policy acts): + +| Term | What it means | What it is not | +|---|---|---| +| **policy** | One Kyverno `ClusterPolicy` / `Policy` resource | Not a gate step inside a pipeline | +| **rule** | One entry in a policy's `spec.rules` (`match` + optional `preconditions` + an action) | Not a standalone Kubernetes resource | +| **criterion** | The boolean expression inside a rule that decides compliant / non-compliant (usually written as a JMESPath variable in `context`) | Not the name of a YAML structure | +| **`deny.conditions`** | The YAML structure that carries the criteria; under `any:` a single hit denies, under `all:` every condition must hold | — | +| **guard (precondition)** | A condition that decides whether the rule applies to this request at all: identity, terminal state, list uniqueness, and so on. A non-match is a **skip (allow)**, not a denial | Not a criterion; writing a criterion as a guard amounts to allowing everything | +| **gate / gate Task** | The Tekton Task in the pipeline that renders the quality verdict (`exit 1` when below the bar), e.g. `sonarqube-scanner`, `trivy-scanner` | Not a Kyverno action | +| **DAG** (directed acyclic graph) | The dependency graph among a pipeline's tasks. Tekton derives it from `runAfter` plus result references between tasks: dependent tasks run in order, independent ones run in parallel, and cycles are not allowed. "The gate's DAG successors" are the tasks that depend on the gate directly or transitively; when the gate fails they are **skipped** — never created at all | Does not include finally tasks — finally is not part of the DAG; it is scheduled only after the whole DAG has finished (this distinction is the key to the outcome-shape table in [§2.3](#s2-3)) | +| **profile** | A set of criteria written against a **specific version** of a real template / Task | Not a generic template | + +One sentence to tie it together: **the gate Task's job is to stop unqualified builds; Kyverno's job is to make sure the gate Task is present when it should be and its parameters have not been tampered with** ([§1.4](#s1-4)) — and note that "present when it should be" is not the same as "guaranteed to run": a gate skipped wholesale by `when` / matrix never produces a TaskRun, admission never sees it, and only after-the-fact audit can catch it ([§4.1.5](#s4-1-5)). + +### 1.3 Per-project differentiation and scope safety {#s1-3} + +Different projects almost inevitably need different constraints: project A sets the coverage bar at 80, project B at 60; and the platform has a set of lines nobody may cross. **Differentiation is a hard requirement — but the way it is implemented must not open a loophole around the policies.** Every policy in this document follows a two-tier model (detailed and verified in [§5](#s5)): + +- **Platform baseline**: a `ClusterPolicy` covering **all workload namespaces**, using a **negative `exclude`** to carve out the platform's own system namespaces. The baseline must **not** depend on "this namespace carries a certain label" — otherwise a newly created unlabeled namespace, or one whose label gets changed, naturally escapes the baseline. +- **Per-project tightening**: the main path for project administrators is to maintain namespaced `Policy` resources inside their own project namespaces — they do not need, and should not be granted, `ClusterPolicy` permissions. Where the platform team centrally manages policies for multiple projects, a `ClusterPolicy` + `namespaceSelector` (e.g. on the `cpaas.io/project` label) can select the target projects. + +**This section describes the target governance model, not the current state of this document's demo assets**: every policy in [§4](#s4) has its scope hard-coded to the demo namespace `policy-poc` so that installation and cleanup can be uniform ([§4](#s4) introduction, [§4.0.2](#s4-0-2)). **"Covering all workload namespaces" is something you change yourself at production deployment time** — copying the demo YAML verbatim will not cover any real project, and newly created namespaces will of course not be picked up automatically either (that is exactly the first trigger listed in [§3.6](#s3-6)). + +The accompanying semantics (which likewise only hold once you deploy per the target model above): an unclassified namespace necessarily falls under the baseline; when multiple policies match the same resource the relationship is **AND** (all must pass; there is no precedence semantics under which a project `Policy` overrides or weakens the platform baseline); and permission to change the scoping labels themselves must also be controlled ([§5.0](#s5-0)). Note that a `Policy`'s scope is a single Kubernetes namespace; if one ACP project spans several namespaces, deploy a corresponding `Policy` in each of them, or have the platform distribute them through a controlled central mechanism. + +### 1.4 Roles and boundaries: what Kyverno does and does not govern {#s1-4} + +The division of labor in one sentence: **hard gates are implemented by gate Tasks inside the pipeline (below the bar → `exit 1` → the pipeline fails natively); Kyverno's role is to narrow the paths by which a gate gets removed, tampered with, or bypassed from the side — and to provide audit and response actions.** + +**Deliberately, this does not say "impossible to bypass"** — that property only emerges from **policies + RBAC + template design combined**; Kyverno alone cannot deliver it. The **last three items** under "cannot do" below correspond to the two responsibilities **not borne by Kyverno** — "the wiring between the gate and the release" belongs to **template design**, while "the path that avoids Tekton" and "protecting the policy system itself" belong to **RBAC**. The document-wide conditional phrasing is in [§4.0.1](#s4-0-1), "what the minimal usable set guarantees is conditional"; the item-by-item exposures are in [§2.5](#s2-5). + +What Kyverno can do: + +- **Hard validation at admission**: block at PipelineRun / TaskRun / Pod creation — non-compliant template identity, gate parameters switched off, unauthorized image sources; the object simply cannot be created, and the pipeline terminates with a clear failure shape ([§2.1](#s2-1), [§4](#s4)); +- **Audit visibility**: read run results (coverage, vulnerability counts, scan verdicts) on resource status updates, and record misses in PolicyReports ([§4.4](#s4-4)); +- **Inject defaults**: mutate at admission (default timeouts, labels, etc., [§4.2](#s4-2)); +- **Response actions**: perform a controlled cancellation of a running pipeline (mutate-existing patch of `spec.status`, [§4.6](#s4-6)). + +What Kyverno explicitly cannot do (the boundaries): + +- **It cannot turn a running pipeline into Failed**: the terminal state of a PipelineRun/TaskRun is decided by the Tekton controller. If you want "results below the bar → failure", the correct answer is for the gate Task itself to `exit 1`; what Kyverno can do is **cancel** (terminal state Cancelled, [§4.6](#s4-6)). +- **Never block writes to `*/status` subresources with Enforce**: what you would be blocking is the Tekton controller's status write-back. The result is a resource stuck in Running with the controller retrying forever (a wedge) — not a failure ([§2.2](#s2-2), [§6.1.4](#s6-1-4)). +- **Remotely referenced definitions (hub / git resolver) never pass through cluster admission**: Kyverno can only lock the **identity** (which catalog entry, which commit); trust in the content comes from external governance (catalog release process, repository permissions). The three tiers of strength are in [§2.1](#s2-1). +- **It cannot see a skipped gate**: when a `when` expression is false, or a matrix expands to nothing, that gate **never produces a TaskRun**, so admission has no object to reject — "the gate must run" can only be guaranteed by template design (do not give the gate a `when` that business teams can switch off) plus the **after-the-fact Audit** of `status.skippedTasks` in [§4.1.5](#s4-1-5). It is not an admission-time hard block. +- **It cannot see whether the wiring between the gate and the release is right**: whether the gate consumes the result of **the intended task** ([§2.3](#s2-3) contract 4), whether release-type tasks are ordered after the gate (contract 5), whether finally hides a gate-protected side effect (contract 6) — these three are **template design responsibility**. Contracts 4 / 5 / 6 have not even a ready-made after-the-fact Audit on the admission side (in the [§2.3](#s2-3) table their only guarantor is `T`; [§4.1.4](#s4-1-4) audits only the gate's **identity** and reads neither `runAfter` nor `finally` — the resolved-definition snapshot it hangs on is the hook if you want to build such an Audit yourself, trade-offs at the end of the [§4.1](#s4-1) intro). **"The gate is present and its parameters were not switched off" does not equal "the gate actually governed the release"** — this is the **template design** share of the "combined" sentence above. +- **It cannot block the path that avoids Tekton entirely**: an identity with workload API permissions can create Pods / Jobs / Deployments directly, or use the deployment credentials somewhere else, without a single PipelineRun. **Only RBAC can close this layer** ([§4.5.4](#s4-5-4)) — the entry-closure policies in this document seal off bare `TaskRun` / `CustomRun`, not every API capable of running a container. +- **It cannot protect itself**: every conclusion in this document rests on "the policy system and Kyverno's own configuration are controlled". Whoever can modify `ClusterPolicy` / `PolicyException` can modify the gates ([§5.3](#s5-3) / [§5.0](#s5-0)); whoever can modify Kyverno's `resourceFilters` or its webhooks can make an entire chapter of policies **silently stop enforcing** ([§3.1](#s3-1) checklist item 7 / [§5.0](#s5-0)); whoever can modify Tekton's platform configuration can swap out the template resolution source ([§4.1.1](#s4-1-1)) or break the scoping labels the image policy relies on ([§3.6](#s3-6)). **These identities are outside this document's threat model** — they are closed off by RBAC separation of duties, change auditing, and the policy system's self-protection ([§5.0](#s5-0)), not by writing yet another policy. + +### 1.5 Quick reference of outcome shapes (for pipeline users) {#s1-5} + +When a policy acts on your pipeline, you will see one of the following six shapes (mechanisms in [§2](#s2), troubleshooting in [§6](#s6)) — **note that the last one is "you see nothing at all"**: + +| What you see | What it means | Where to look for the reason | +|---|---|---| +| Creating the PipelineRun is rejected outright (kubectl / UI shows an admission error) | Admission blocking: template / parameters / entry non-compliant | The error message itself is the policy message (policy name, rule name, reason) | +| PipelineRun fails with reason `CreateRunFailed`; some mid-pipeline Task was never created | Mid-run admission blocking: the effective parameters of a gate Task are non-compliant | `kubectl describe pipelinerun`; the condition message carries the full policy message | +| PipelineRun fails with reason `Failed`; the gate Task is red | A genuine quality-gate failure (coverage / vulnerabilities below the bar). **Exception**: a `spec.status` holding **a cancellation value** (`Cancelled` / `CancelledRunFinally` / `StoppedRunFinally`) means **someone — or some policy — did request cancellation** and the task's own failure merely outranked it; `spec.status` alone cannot tell you who wrote it | The gate Task's logs; if `spec.status` holds a cancellation value, follow [§6.2.3](#s6-2-3) to look for `cancel-reason` / `statusMessage` — those markers only point at a policy cancellation (confirming the writer takes the audit log, see that section); without them the origin is unknown (a manual cancel looks exactly the same). **Do not equate "non-empty" with "cancelled"**: the field has one more legitimate value unrelated to cancellation, `PipelineRunPending` (see [§6.2.3](#s6-2-3)) | +| TaskRun fails with reason `PodCreationFailed`; the Pod never appeared | Pod-level admission blocking: the container image for this step is not on the approved list ([§4.5.3](#s4-5-3)) | `kubectl describe taskrun`; the message carries the full policy message | +| The PipelineRun turns `Cancelled` (and you didn't cancel it) | **The first suspect is a policy cancellation — but do not jump to that conclusion**: the cancellation field is a public Tekton field, and another user, an ops tool, or other automation writes it identically; the markers of [§6.2.3](#s6-2-3) only point at a policy cancellation (confirming the writer takes the audit log — see that section). On the policy side **there are four possible origins**, listed here in the troubleshooting order of [§6.2.3](#s6-2-3): gate TaskRun cancelled ([§4.2.3](#s4-2-3)), parent run cancelled ([§4.2.2](#s4-2-2)), definition drift ([§4.6.2](#s4-6-2)), results below the bar ([§4.6.1](#s4-6-1)) | The evidence lives in only two places: for the first, on that gate TaskRun; the latter three share the parent run's `cancel-reason` annotation, distinguished by its text. Work through them in the order given in [§6.2.3](#s6-2-3) (the mechanism differences are summarized in the table in the [§4.6](#s4-6) introduction) | +| **The pipeline is perfectly fine, green — yet a violation was recorded anyway** | Audit-mode policies record without blocking ([§4.4](#s4-4)). **"It ran through" does not equal "it is compliant"**: [§4](#s4) contains several pure-Audit policies, and [§4.2.4](#s4-2-4) has one more policy carrying an Audit rule — all of them completely invisible to you (which ones are Audit is in the policy quick reference of [§4.0.2](#s4-0-2)) | Only in the PolicyReport: `kubectl get policyreport -n `, look for entries with `result: fail` ([§6.1.5](#s6-1-5)) | + +## 2. Understanding the Mechanisms {#s2} + +This chapter is the core of the document. Two models run through everything that follows: + +- **Model 1: the lifecycle observation/action matrix ([§2.1](#s2-1)–[§2.2](#s2-2))** — what Kyverno can see along the pipeline lifecycle, when it sees it, and what it can do; +- **Model 2: the trust and hard-gate contracts ([§2.3](#s2-3))** — the seven contracts that make up an "unbypassable quality gate", and who guarantees each one. + +Every section of the Cookbook ([§4](#s4)) is an instantiation of these two models in a concrete scenario. + +### 2.1 The lifecycle observation/action matrix {#s2-1} + +The typical lifecycle of a reference-style pipeline (`pipelineRef` pointing at a template), with Kyverno's intervention points: + +```text +Pipeline/Task definition stored (CREATE/UPDATE) ← observation point 1 (in-cluster definitions only) + │ +PipelineRun CREATE ── admission ─────────────── ← observation point 2 (the primary hard blocking point) + │ resolver resolution (cluster/hub/git) +PipelineRun status UPDATE (resolution written) ── ← observation point 3 (the only place a referenced definition can be introspected) + │ TaskRuns created one by one +TaskRun CREATE ── admission ─────────────────── ← observation point 4 (hard blocking point after parameter expansion) + │ execution Pod created +Pod CREATE ── admission ─────────────────────── ← observation point 5 (hard blocking point for the images that actually run) + │ execute, write back results +TaskRun status UPDATE (results written) ──────── ← observation point 6 (the only source of results) + │ +PipelineRun status UPDATE (terminal state, pipelineResults) +``` + +| # | Observation point | What is visible | What can be done / caveats | +|---|---|---|---| +| 1 | Pipeline / Task definition resource CREATE/UPDATE (**in-cluster definitions only**) | The full definition spec is introspectable: tasks, finally, parameter defaults, labels | In theory two things can be done here: Enforce-validate the stored content (must contain the gate task, etc.) + lock change permissions. **This document uses only the latter**, and hands it to standard RBAC rather than a policy ([§4.1.2](#s4-1-2)) — **not a single policy in this document matches the `Pipeline` / `Task` definition resources**; why not is in the [§4.1](#s4-1) introduction (including "what kind of site is worth building its own"). **Coverage comes in three tiers**: ① inline / in-cluster direct ref — introspectable and lockable at admission; ② hub / git **immutable reference** (pinned version / commit SHA) — in-cluster you can only lock the **identity**; content trust comes from external catalog / repository governance; ③ hub / git **mutable reference** (branch / tag) — content changes take effect automatically as the remote moves; Kyverno can only lock "which branch / tag is referenced". Using this tier requires repository-side permission controls (protected branches / tags); otherwise it should be rejected | +| 2 | `PipelineRun` CREATE admission | `pipelineRef` (resolver type + all resolver parameters), **`spec.params` with values**, workspaces, labels, **`request.userInfo`** (creator identity) | Enforce: template identity allowlist, PipelineRun-level parameter contracts, entry identity constraints; mutate: inject defaults (timeout / label, [§4.2.6](#s4-2-6)). ⚠️ For a reference-style pipeline, `spec.pipelineSpec` is **empty** at this moment — the definition content is invisible, and so are task-level parameters | +| 3 | `PipelineRun/status` UPDATE (subresource) | The resolver-resolved **`status.pipelineSpec`** (the only place in the cluster a referenced definition can be introspected), `status.childReferences`, **`status.skippedTasks`** (each skipped task's `name` + `reason` + `whenExpressions`; `reason` values come from Tekton's `SkippingReason` enum), `status.pipelineResults` (only present after completion — far too late for admission) | Past admission = after-the-fact view. **Never Enforce-deny** (wedge, [§2.2](#s2-2)). Correct uses: **Audit as defense in depth** (resolved definition missing the gate task → record in PolicyReport, [§4.1.4](#s4-1-4); gate skipped by `when` / empty matrix → read `status.skippedTasks` and record, [§4.1.5](#s4-1-5)); **response action**: trigger self-cancellation ([§4.6.2](#s4-6-2)) | +| 4 | `TaskRun` CREATE admission | `spec.taskRef` (resolver + kind/catalog/name/version/namespace), labels (visible but **untrusted**: `tekton.dev/pipeline` / `tekton.dev/pipelineTask` / `tekton.dev/pipelineRun` can be overridden via `taskRunSpecs` — usable as troubleshooting hints, never to locate a trusted profile or the parent run), `request.userInfo`, the controller ownerReference, and **`spec.params` = the expanded, effective parameter values** (`$(params.x)` already resolved to concrete values — **task-level gate parameters can be validated without being surfaced at the PipelineRun level**); step images visible only for inline taskSpec. ⚠️ `tekton.dev/task` is visible on the final TaskRun but may not yet be present at real CREATE admission time, so it likewise cannot serve as an identity precondition at this stage; parent identity must be derived from the controller ownerReference + an `apiCall` for the live parent's UID/`spec.pipelineRef` | Enforce: **validation of the gate task's effective parameters** (deny → the parent run fails cleanly with `CreateRunFailed`, the policy message passed through verbatim into the run condition, [§4.2](#s4-2)), bare-TaskRun closure ([§4.5.4](#s4-5-4)), taskRef allowlist. ⚠️ A parameter the pipeline did not bind **does not appear** in `spec.params` (the task definition's default takes effect) — a policy may interpret absence as that trusted default only when `spec.taskRef` is already locked to an exact Task version whose defaults are trusted; with untrusted identity or unknown defaults it must fail closed | +| 5 | **Pod CREATE / plain UPDATE / `Pod/ephemeralcontainers` UPDATE admission** (Tekton execution Pods, in-flight image updates, and debug containers injected after the fact) | CREATE and plain UPDATE expose the actual step / sidecar / init container images, securityContext, labels (`tekton.dev/taskRun` etc.), volumes; the subresource UPDATE exposes `spec.ephemeralContainers` | **The reliable hard blocking point for the images that actually run** (non-compliant execution image → TaskRun `PodCreationFailed`; non-compliant main/init images on plain UPDATE and non-compliant ephemeral-image patches are rejected the same way, [§4.5.3](#s4-5-3)). **What this layer can do**: registry allowlist, digest requirements, no-privileged, image signature verification (verifyImages); **this document ships only the registry-prefix allowlist** ([§4.5.3](#s4-5-3)) — digest / privileged / signing each need their own policy; for verifyImages see the companion document | +| 6 | `TaskRun/status` UPDATE (subresource) | **Task results** (object-result drill-down / aggregate-string parsing) and the terminal state — **the only source of results** | One run triggers multiple UPDATEs, so a terminal-state guard is required ([§4.4](#s4-4)); usable only for **Audit** or as a **mutate-existing trigger** (cancellation, [§4.6](#s4-6)) — **never Enforce** (wedge); these policies must also declare `failurePolicy: Ignore` — otherwise, during a Kyverno outage, the API server rejects the status write-backs on their behalf ([§3.7](#s3-7) tiering) | +| 7 | Pod status / events | The failure scene at runtime | Troubleshooting observation only ([§6](#s6)); carries no policy action | +| 8 | External data sources | `context.apiCall` (query other in-cluster resources during admission: Pipeline definitions, the parent PipelineRun, …), `context.imageRegistry` (read image config; usage in [§4.5.2](#s4-5-2)) | apiCall's JMESPath syntax is strict ([§6.1.7](#s6-1-7)); imageRegistry can only read images that already exist in the registry, and it puts external network calls on the admission path (latency and timeout risks in [§4.5.2](#s4-5-2)). **Which direction a failed apiCall takes is decided by the rule it sits on, not by the mechanism itself**: on a synchronous `validate` rule ([§4.2.1](#s4-2-1)) a lookup that cannot be satisfied — target unreachable, absent, or forbidden — makes the rule error out and the request is denied (fail-closed); on a mutate-existing rule ([§4.2.2](#s4-2-2) / [§4.6.1](#s4-6-1)) it runs inside background-controller, outside the admission verdict entirely, so a failed lookup merely makes the patch vanish silently while the original request is allowed (fail-open) — see the "asynchronous delivery chain" row of [§3.7](#s3-7) | + +### 2.2 Enforcement and action modes {#s2-2} + +| Mode | Use for | Key boundaries | +|---|---|---| +| `validate` + **Enforce** | Template / parameter / definition / Pod constraints (CREATE on observation points 1/2/4, plus Pod CREATE / plain UPDATE / `Pod/ephemeralcontainers` UPDATE on point 5) — non-compliant requests are rejected outright | For CREATE/UPDATE on main resources, or on **non-status subresources** explicitly brought under governance such as `Pod/ephemeralcontainers`; never on `*/status` UPDATE. Operational boundary: the webhook's `failurePolicy` decides whether Kyverno being unavailable means allow-everything (Ignore) or reject-everything (Fail) — verify it in [§3.1](#s3-1) and have a playbook in [§6.1](#s6-1) | +| `validate` + **Audit** | Result constraints (status UPDATE on observation points 3/6) — allowed through, but recorded in PolicyReport | **Reading status is Audit-only.** ⚠️ Subresource match and `background: true` are mutually exclusive — result-type Audit only has the admission moment; there is no background-scan backstop | +| `mutate` (admission injection) | Injecting default timeout / labels / SA etc. (observation point 2) | The `+(field)` anchor = add-if-absent: it never overwrites a user's explicit value ([§4.2.6](#s4-2-6)) | +| **mutate-existing** | Response action: on a triggering event, patch other resources that **already exist** in the cluster — used in this document to cancel pipelines ([§4.6](#s4-6)) | Requires the background controller to hold update RBAC on the target resource (**Kyverno validates that RBAC at policy-creation time; without it the policy fails to install**, [§3.1](#s3-1)). When triggered by admission events and using `subjects` / `request.userInfo`, it must set `background: false`; only enable `background: true` when you genuinely need policy updates to scan pre-existing trigger resources and the rule uses none of those request variables | +| `generate` | Auto-provisioning namespaced Policies for new project namespaces, etc. | Lifecycle management is complex; not covered here (Advanced) | +| `verifyImages` | Image signatures / attestations | See the companion document; one of the trust prerequisites for the "identity" contract in [§2.3](#s2-3) | + +**The anti-mechanism (burn this in)**: hanging `validate + Enforce` on UPDATE of `tekton.dev/v1/TaskRun/status` or `PipelineRun/status` blocks **the Tekton controller's completion write-back** — the TaskRun sticks at Running, the controller retries `UpdateFailed` forever, and the pipeline neither fails nor ends until a human intervenes (reproduction and recovery steps in [§6.1.4](#s6-1-4)). This is the single easiest trap on the road to "I want the pipeline to fail": **denying the status write ≠ making it fail**. + +### 2.3 Trust and the hard-gate contracts {#s2-3} + +**Positioning**: hard gates (coverage bars, vulnerability thresholds — "below the bar shall not pass") are implemented by a **gate Task inside the pipeline** — the gate reads the results of earlier tasks and exits 1 when the bar is missed; the pipeline fails natively (`Failed`), and the release tasks ordered after the gate (`runAfter`) are skipped by the DAG, **never created at all**. Kyverno's responsibility is to **verify the statically verifiable parts of this contract set**; the rest is guaranteed by the construction of trusted templates (by construction) and by external governance. + +An "unbypassable hard gate" = all seven of the following contracts holding at once. Guarantors come in three kinds: **K** = statically verifiable by Kyverno, **T** = promised by trusted-template construction (true by the way the template is built, not by runtime checks), **E** = external governance. The skeleton first: + +| # | Contract | One-liner | Guarantor | Details | +|---|---|---|---|---| +| 1 | Identity | The gate uses a trusted Task with an immutable reference (pinned / digest) | K + E | [§4.1](#s4-1) | +| 2 | Effective parameter values | Switches / thresholds validated on the expanded, effective values | K | [§4.2.1](#s4-2-1) | +| 3 | Must-run | The gate is not skipped via `when` / matrix / defaults | T + K post-hoc Audit | [§4.1.5](#s4-1-5) | +| 4 | Data binding | The gate consumes the results of the intended task | T | — (template responsibility) | +| 5 | DAG dominance | Release-type side-effect tasks must be ordered after the gate | T | — (template responsibility; hook and trade-offs for building your own Audit: end of the [§4.1](#s4-1) intro) | +| 6 | finally safety | No gate-protected side effects inside finally | T | — (template responsibility; finally execution semantics: [§4.2.2](#s4-2-2)) | +| 7 | Entry closure | No bypassing the pipeline via bare TaskRuns / inline definitions / unapproved resolvers | K + RBAC | [§4.5](#s4-5) | + +In detail: + +1. **Identity** (K + E): the gate uses a trusted Task with an immutable reference (pinned version / digest). K locks the reference identity ([§4.1](#s4-1)); the integrity of step images (digest / signature), registry push permissions, and the credential safety of external scanning services belong to the external trust surface (E; image signing is verifyImages / the companion document). +2. **Effective parameter values** (K): gate switches, thresholds, target branches, and so on are validated on the **expanded, effective values**. Validation site = **gate TaskRun CREATE** — at that moment `$(params.x)` has been resolved to concrete values; identity is derived from the controller `ownerReference` + the live parent run + `spec.taskRef` (child labels can be forged by the caller and are unusable), and template authors owe zero changes. Response: Enforce deny (the gate TaskRun cannot be created → the parent run fails cleanly with `CreateRunFailed`) or cancel the parent run ([§4.6](#s4-6)); when a template already exposes the parameters at the PipelineRun level, **blocking early** at PipelineRun CREATE is an optional optimization. Full derivation and policy in [§4.2.1](#s4-2-1). +3. **Must-run** (T + K post-hoc Audit): the gate is not skipped by `when` expressions / matrix / conditional branches / parameter defaults. The classic trap: the scan-URL parameter defaults to empty + `when: sonarURL != ''` ⇒ by default the scan is skipped entirely and the gate becomes opt-in. ⚠️ **A skipped gate produces no TaskRun** — contract 2's admission validation is blind to absence (admission cannot block what never happens). So must-run is grounded in T (the template offers no skip path); on the K side, post-hoc Audit of **`status.skippedTasks`** (the controller records every skip with its `reason` into the PipelineRun status) determines whether the gate was opted out — still Audit, and it cannot stop the current run. How to classify the reasons and write the policy: [§4.1.5](#s4-1-5). +4. **Data binding** (T): the gate really consumes the results of the designated producer task (the `$(tasks.scan.results.x)` wiring is correct). Admission cannot see expression-level bindings; the template guarantees this. +5. **DAG dominance** (T): **every** release / push / promotion side-effect task must depend on the gate transitively (`runAfter`, directly or indirectly). The gate can only stop its DAG successors — **tasks ordered before or parallel to the gate may already have finished, and failure does not roll back side effects that already happened**. Making side effects dominated by the gate is a template design responsibility; this document ships no ready-made DAG-dominance Audit (judging transitive dependency means computing a closure — trade-offs at the end of the [§4.1](#s4-1) intro), and the resolved-definition snapshot of [§4.1.4](#s4-1-4) (which contains `runAfter`) is the hook for building such a criterion yourself. +6. **finally safety** (T): finally tasks execute when the pipeline fails, or when it is cancelled with **`CancelledRunFinally`** (whether finally runs under deny vs. cancel is contrasted in the table in [§4.2.2](#s4-2-2); the full trade-off across the three response shapes is in [§4.2.3](#s4-2-3)); a plain `spec.status: Cancelled` does not guarantee scheduling of finally tasks that have not started — so finally must not contain any gate-protected side effect (release, push). There is likewise no ready-made Audit of finally contents (the [§4.1.4](#s4-1-4) snapshot contains the `finally` list; you can build one — same trade-offs as above). +7. **Entry closure** (K + RBAC): business identities must not bypass the pipeline by creating bare TaskRuns, must not use unapproved inline definitions, must not use unapproved resolver types; `CustomRun` is denied by default or explicitly declared unsupported ([§4.5.4](#s4-5-4)). + +**Kyverno's three verifiable things** (the taxonomy for every Enforce policy in this document): template identity allowlisting (per the three tiers in [§2.1](#s2-1); **change permissions** on in-cluster definitions are separately closed off by standard RBAC, see [§4.1.2](#s4-1-2) — that one does not count as Kyverno-verifiable); parameter contracts (effective values at the TaskRun level as the main path, early blocking at the PipelineRun level as the auxiliary path); entry closure. **Audit / PolicyReport is the second, after-the-fact line of defense — for spotting drift and backstop alerting; it does not count toward the hard gate's guarantee.** Audit blocks nothing. + +**Failure / termination shape comparison** (pipeline users' quick reference is [§1.5](#s1-5)): + +| Shape | Trigger | Run terminal reason | Downstream release tasks | finally | How the failure is surfaced | +|---|---|---|---|---|---| +| Admission rejects the gate TaskRun's creation (contract 2 response) | Gate's effective parameters non-compliant | `CreateRunFailed` (terminal; "no retry" here means **it will not retry forever**, and does not promise a single attempt; it also **has a precondition** — see the last item in the info block below) | Never created (`skippedTasks` empty) | **Does not run** | The Kyverno policy message is passed through verbatim into the PipelineRun condition | +| Gate task exits 1 (the mainline hard gate) | Results below the bar | `Failed` | Skipped by the DAG; listed in `skippedTasks` (reason `PipelineRun was stopping`) | **Runs** | The gate task's logs + "Tasks Completed: N (Failed: 1)" | +| mutate-existing cancellation ([§4.6.1](#s4-6-1)) | Results below the bar (status-event triggered); a missing / malformed result triggers the same way, fail-closed (**criterion direction fail-closed ≠ delivery guarantee**: the cancellation is delivered asynchronously in the background and silently fails to happen when that chain breaks — see the "asynchronous delivery chain" row in [§3.7](#s3-7)) | Usually `Cancelled`; when the result-producing task itself failed first it is `Failed` (the failure verdict outranks the cancellation; `spec.status` still reads `CancelledRunFinally`) | In-flight ones are stopped with `TaskRunCancelled` | **Runs** | The parent run's `cancel-reason` annotation (written by the same patch; the text names the triggering TaskRun and the out-of-bounds result value) + events; with the companion Audit rule there is also a PolicyReport record | +| mutate-existing self-cancellation ([§4.6.2](#s4-6-2)) | Resolved-definition drift (the `pipelineSpec` written back into `status` does not match the approved identity) | `Cancelled` | Same as above | **Runs** | The parent run's `cancel-reason` annotation (stating the drift) + events | +| **mutate-existing cancellation (RunFinally) replacing deny on non-compliant gate parameters ([§4.2.2](#s4-2-2))** | Non-compliant effective parameters detected on the gate TaskRun | Usually `Cancelled`; `Failed` when the cancellation races a task failure (same adjudication rule as two rows above: the failure verdict outranks the cancellation, `spec.status` still reads `CancelledRunFinally`; [§4.6.1](#s4-6-1)'s initialization window applies to this path as well) | Tasks before the gate already ran; from the gate onward, cancelled | **Runs** | Generic cancellation text + the `cancel-reason` annotation | +| **Admission-mutate cancelling the gate TaskRun itself (the synchronous alternative to deny, [§4.2.3](#s4-2-3))** | Gate's effective parameters non-compliant | `Cancelled` | Skipped by the DAG; listed in `skippedTasks` (reason `PipelineRun was stopping`) | **Runs** | The TaskRun condition carries the policy-written `statusMessage` verbatim (visible in tkn / the UI); no violation record in PolicyReport | + +:::info Why "admission rejecting the gate TaskRun" skips finally (a known community issue) + +The behavior has been reported upstream: https://github.com/tektoncd/pipeline/issues/10514 (*finally tasks are not executed when a child run creation is permanently rejected*; still open as of this writing). The community Pipelines version used by the current ACP release carries this issue; until the upstream fix lands, the selection guidance below applies. The mechanism: + +- **Mechanism**: finally is scheduled only after the whole DAG has finished, and "finished" requires every DAG task to land in one of succeeded / failed / **skipped**. A gate TaskRun rejected at admission **was never created**, so that node can never reach any of the three — the DAG never counts as finished, finally is never scheduled, and the controller promptly moves the run to the `CreateRunFailed` terminal state. +- **Contrast**: when the gate task exits 1, the TaskRun was created, ran, and failed — the node has a terminal state, the DAG can finish, and finally runs as usual. The dividing line is **whether the gate node reached a terminal state**, not whether the run failed. +- **How to recognize this shape**: the run is `CreateRunFailed`, there are no child TaskRuns, finally never got created, and `skippedTasks` is empty. +- **Why a terminal state, rather than endless retries**: when creating a child run fails, the controller first classifies the error (`handleRunCreationError` in upstream `pkg/reconciler/pipelinerun/pipelinerun.go`), and **only errors judged "permanent" are written as `CreateRunFailed`** — everything else is treated as retryable. An admission rejection lands in the permanent bucket because the API server uniformly returns 400 for a "webhook denied but supplied no status code" response. **So one more shape exists**: if your rejection response carries a known failure reason (such as `Forbidden`), the error may be classified as retryable — the symptom becomes a run **stuck in Running while the controller retries creating the same child run over and over**, rather than failing outright. When you see that stuck shape, do not go inspect the DAG; look at the rejection response's status code and reason. +- **"Permanent" does not mean "attempted exactly once"**: once the error lands in the permanent bucket the run terminates outright, but the controller **does not guarantee that it issued exactly one creation request** — the `TaskRunsCreationFailed` event of a single run can carry a `count` greater than 1 (`Failed` / `InternalError` are counted together). So what this section promises is that the run **reaches a terminal state quickly**, not that exactly one request was sent: when troubleshooting, **do not read `count > 1` as an anomaly** — the shape to worry about is the previous item's, a run **stuck in Running while the controller retries creating the same child run over and over**. To count precisely per run, use `kubectl get events -n --field-selector involvedObject.uid=`; querying by name folds in stale events left by an earlier run of the same name. + +::: + +:::warning Selection note: teams that rely on finally for notifications / cleanup, pay attention + +- Under the admission-rejection shape (`CreateRunFailed`), finally **does not run**; finally runs per the comparison above only when the gate task landed and then failed, or when the run is cancelled explicitly with `CancelledRunFinally`. +- If your notification / cleanup must fire even when gate parameters are blocked, do not hang it on finally alone — replace deny with a **cancel (RunFinally)**, via either of two routes: + - **[§4.2.2](#s4-2-2) (cancel the parent run)**: on scan TaskRun CREATE, trigger mutate-existing to patch the parent PipelineRun's `spec.status=CancelledRunFinally` and stamp a reason annotation; the run terminates as `Cancelled` but finally runs as usual (cancellation triggered by below-the-bar results is [§4.6](#s4-6)). + - **[§4.2.3](#s4-2-3) (the other synchronous shape: cancel the gate TaskRun itself)**: leave the parent run alone and instead mutate the gate TaskRun itself to `spec.status=TaskRunCancelled` during admission — it completes within the same admission pass, has no race window, and needs no extra background-controller RBAC. +- The trade-off across the three shapes is in the comparison table in [§4.2.3](#s4-2-3). + +::: + +### 2.4 The extension model: growing policies from custom Tasks and results {#s2-4} + +Beyond the platform's built-in scanning / gating capabilities, every organization has checks of its own (in-house linters, license scans, security baselines, artifact conventions, …). The extension path has three steps: + +1. **The Task produces declarative results**: a custom Task writes its conclusions as **decidable results** — a number (`error-count`), an enum verdict (`verdict: pass|fail`), or a structured object — not "the path to a report file". Tekton results come in three declared types — `string` / `array` / `object` — and the policy side can consume all three: `status.results[].value` is serialized per the declared type (string → a string, array → an array of strings, object → a map of strings), so JMESPath receives the corresponding native structure: + - **`type: object` (use it for multi-field structures)**: the Task declares `type: object` + `properties`, and the policy drills straight down with JMESPath `.value.xxx` — fields have names and a schema, and the policy never parses any text format. Note that the values under `properties` can only be `string` (no nested objects / arrays); when you need hierarchy, flatten the field names; + - **`type: array` (use it for homogeneous lists)**: the value is an array of strings; the policy filters with `[?...]`, `contains(...)`, `length(...)` — e.g. "the list of unfixed critical CVEs must be empty". It solves "many values", not "many fields" — semantically distinct fields still belong in an object; + - **`type: string` (the default)**: most direct for single values — one number or one enum verdict per result; the policy converts with `to_number` or compares directly, with zero parsing risk. + - **Aggregate strings (a convention layered on `type: string`; a compatibility measure, not recommended)**: packing several fields into one string result as `key=value` concatenation, unpacked on the policy side with `split` + regex + `to_number` ([§4.4.2](#s4-4-2)). **It does work** — but only do it when consuming an **existing Task contract you cannot change yet**: a text format is not a stable contract; field order, separators, added fields, and "count unknowable" sentinel values all cause silent mismatches — and a mismatch typically shows up as **falsely scored as passing**. When you own the contract and have several fields to aggregate, use `type: object`. +2. **For a hard gate**: the Task renders its own verdict and exits 1 (or is immediately followed by a gate task that reads the result) — entering the [§2.3](#s2-3) contract system, subject to identity locking and parameter validation; +3. **For visibility / backstop**: an Audit policy reads the result into PolicyReport ([§4.4](#s4-4)); below-the-bar results can additionally trigger automatic cancellation ([§4.6](#s4-6)). + +**Two-layer parameter validation** (same as contract 2): the main path = validating the expanded, effective values at TaskRun CREATE (works for any template as-is, zero design obligations on template authors); the optional optimization = when the template already exposes the switch / threshold as a PipelineRun-level parameter, block early at PipelineRun CREATE. + +**Trust prerequisite**: custom Tasks fall under contract 1 like everything else — immutable reference + trusted image. Otherwise "push a script version that always prints pass" is the cheapest bypass there is. + +[§4](#s4), the Cookbook, threads this extension path through a fictional, self-contained scanner task (`policy-demo-scanner`); real Tasks from the platform catalog such as sonarqube / trivy appear as profile subsections with their real result contracts. + +### 2.5 The residual-risk ledger (once the minimal usable set is installed, which paths remain) {#s2-5} + +[§1.4](#s1-4) said what Kyverno does and does not govern, [§2.3](#s2-3) said who guarantees each of the seven contracts, and every section carries its own "what this does not cover" note. This section merges them into one table: **assuming you installed the minimal usable set per [§4.0.1](#s4-0-1) and fixed the scopes, this is the actual set of guarantees and exposures in your hands.** The table is also this document's scope statement — rows marked ❌ are **explicitly not covered** by this document; they are not omissions. + +Legend: ✅ = hard admission Enforce block (the precondition shared by the allowlist types is a fully populated list, see [§4.0.7](#s4-0-7) — not repeated row by row below); 🟡 = after-the-fact Audit / asynchronous response only, or blocking depends on conditions outside Kyverno such as template design; ❌ = not covered by this document. + +| # | Bypass or failure path | Coverage | What closes it | +|---|---|---|---| +| 1 | Not going through Tekton: creating Pods / Jobs / Deployments directly, or using the deployment credentials somewhere else | ❌ | RBAC narrowing of workload APIs and credentials ([§1.4](#s1-4) / [§4.5.4](#s4-5-4)) — this document cannot seal this layer | +| 2 | Bare `TaskRun` / `CustomRun` bypassing the pipeline | ✅ | [§4.5.4](#s4-5-4); this row's "list" is **the legitimate automation creator identities** — omitting one blocks a legitimate path outright | +| 3 | Referencing an unapproved template, or an inline definition | ✅ | The three-channel allowlist of [§4.1.1](#s4-1-1) — inline is **naturally denied** by it (not in any of the three channels). For a cluster-wide flat ban there is also `disable-inline-spec` in [§4.1.2](#s4-1-2), but that is **Tekton's own webhook, not Kyverno**; [§4.1.3](#s4-1-3) covers the reverse operation (carefully opening an exception), not a blocking means for this row | +| 4 | The reference coordinates unchanged, but **the remote definition's content** swapped out | 🟡 | Identity locking only; content trust comes from catalog / repository governance (the three tiers in [§2.1](#s2-1)), layered with the after-the-fact drift Audit of [§4.1.4](#s4-1-4) | +| 5 | Gate skipped via `when` / an empty matrix (no TaskRun produced at all) | 🟡 | Admission has no object to reject; rely on the template offering no skip path + the after-the-fact Audit reading `skippedTasks` in [§4.1.5](#s4-1-5) | +| 6 | Gate switches turned off, thresholds tuned, override injection (`taskRunSpecs` / `taskRunTemplate`) | ✅ | Official templates take the real profiles of [§4.2.5](#s4-2-5) / [§4.2.4](#s4-2-4), **usable once scope and placeholders are fixed**; self-built templates take [§4.2.1](#s4-2-1), but that one **is a template, not a ready-made implementation** — its identity and parameter contract must be rewritten for your gate ([§4.0.1](#s4-0-1) stage 3) | +| 7 | Release-type tasks not dominated by the gate, or a gate-protected side effect placed in finally | 🟡 | Contracts 5 / 6 are template design responsibility; the K side ships no ready-made criterion ([§4.1.4](#s4-1-4) audits only the gate's identity; its snapshot is the hook for building such an Audit). **The definition-side admission route was not taken in this document**; its shape and cost are at the end of the [§4.1](#s4-1) introduction | +| 8 | The gate consumes a result that is not the intended task's (mis-wired, or rewired) | ❌ | Contract 4: admission cannot see expression-level bindings; only the template can guarantee this | +| 9 | The execution image swapped for another image **inside an approved registry**, or a mutable tag's content replaced | 🟡 | [§4.5.3](#s4-5-3) judges prefixes; for more strength pin digests or add `verifyImages` (the companion document) | +| 10 | The other Pod-level surfaces: privileged / `securityContext` / `automountServiceAccountToken` / mounts | ❌ | The same observation point could do it ([§2.1](#s2-1) row 5), but **this document ships only the registry-prefix allowlist**; governing these needs additional policies | +| 11 | Workspace bindings: Secrets / PVCs other than the [§4.5.5](#s4-5-5) kubeconfig mounted into the pipeline | ❌ | This document governs only the one binding "where the release step's kubeconfig comes from"; the credential surface as a whole is borne by RBAC and Secret governance | +| 12 | A forged result (the scan step writing a `pass` of its own) | 🟡 | Falls under contract 1: immutable reference + trusted image; [§4.6.1](#s4-6-1) additionally has an identity anti-forgery check | +| 13 | A cancellation that should have happened but did not (mutate-existing's asynchronous delivery chain broken) | 🟡 | Fail-open; monitor per the "asynchronous delivery chain" row in [§3.7](#s3-7); for a synchronous hard guarantee switch to [§4.2.1](#s4-2-1) / [§4.2.3](#s4-2-3) | +| 14 | Modifying Kyverno's own configuration, the policy objects, or PolicyException | ❌ | Outside this document's threat model; closed off by RBAC separation of duties and change auditing ([§5.0](#s5-0) / [§5.3](#s5-3)) | +| 15 | A new namespace / a new cluster not brought under governance | 🟡 | Both are **silently allowed**: update the scopes per the first row of [§3.6](#s3-6); there is no cross-cluster distribution mechanism ([§7.3](#s7-3)) | +| 16 | Submitting `PipelineRun` / `TaskRun` as `v1beta1` (while the environment still serves that version) | ✅ | **This row is not an exposure; it is listed because it is routinely mistaken for one**: the webhooks Kyverno generates are `matchPolicy: Equivalent` and register only `v1`, so the API server converts `v1beta1` requests to `v1` before sending them for review — writing only `tekton.dev/v1` in `kinds` already covers it. **What actually opens a hole is "adding `v1beta1` to `kinds` to be safe"** — from then on the conversion no longer happens, **field paths renamed across versions** read empty, and the criteria that depend on them silently skip (paths shared by both versions still resolve, so it is a **partial** failure — harder to notice). Details in [§3.2](#s3-2), "API group-version prerequisite"; `CustomRun` is the exception — it exists only as v1beta1 | +| 17 | `StepAction` (step-level remote references), Tekton Chains / provenance, resource quotas and concurrency abuse | ❌ | Out of this document's scope; no analysis and no criteria given — govern each by its own mechanism when needed | +| 18 | The "effective values" a criterion depends on have sources outside the request (sonar's properties file can come from the scanned repository or a workspace) | 🟡 | Admission sees only the request. For the branch value [§4.2.4](#s4-2-4) is already immune to the file source (when the parameter is non-empty the Task overrides the file value with it; when it is absent the criteria handle it per the protection scope); the remaining path is a non-empty `sonar.pullrequest.key` injected in the file silently switching the analysis into PR mode — borne by repository governance ([§2.1](#s2-1)) and content control of reviewed objects ([§2.3](#s2-3) contract 1) | +| 19 | The known false-rejection surface of [§4.2.4](#s4-2-4)'s contract narrowing: ① out-of-contract forms are denied across the board — governed keys inside `sonarProperties` (even though a parameter would override them), comment lines, leading whitespace, a single element carrying an embedded newline, duplicated PR declarations or values containing whitespace; ② the combination of `sonarBranchName` absent + the repository properties file pointing the analysis at a feature branch + the gate explicitly switched off | 🟡 | Direction fail-closed: ① rewrite to the recommended form per the denial message and the reference table in the first warning of [§4.2.4](#s4-2-4), and it is allowed; ② explicitly pass the feature-branch value for that run. Legacy shapes genuinely outside the contract go through the explicit exemption of [§5.3](#s5-3) | + +**How to use this table**: ① before go-live, walk the rows marked ❌ / 🟡 and confirm "who in my organization owns this" — a row with no owner is a real exposure; ② when reporting "what this policy set guarantees", quote the rows marked ✅, and never present a 🟡 as a ✅; ③ come back and re-read it after every upgrade or scope change ([§3.6](#s3-6)). + +## 3. Common Configuration and Operating Discipline {#s3} + +This chapter completes, in one pass, the environment verification and shared resources that every later chapter depends on ([§3.1](#s3-1)–[§3.4](#s3-4)), and lays down the operating discipline you keep observing once these policies are live ([§3.5](#s3-5)–[§3.8](#s3-8): staged rollout, change triggers, scale and failure budgets, the upgrade regression set). **Only the first half is needed to get started; the second half is what you come back to, again and again, once the policies are in production.** + +:::warning Which cluster do the commands run on + +**The `kubectl` commands in this document run, by default, on the workload cluster that hosts Kyverno and Tekton** (the target cluster from here on) — including this chapter's verification checklist and fixtures, and every policy and probe in [§4](#s4)–[§6](#s6). + +**The only exception is [§3.1.1](#s3-1-1)**: changing the configuration of platform-managed components goes through the `ModuleInfo` on the global management cluster; that section's commands explicitly carry `--kubeconfig ` — write them exactly as shown and do not reuse the current context. + +Before doing anything, confirm your current context points at the target cluster; do not create demo resources on the global cluster: + +```bash +kubectl config current-context +# Expect the context of the cluster that runs Kyverno and Tekton. If it points +# anywhere else, switch first: kubectl config use-context +kubectl get deploy -n kyverno kyverno-admission-controller +# Expect the controller to exist here. NotFound means you are on the wrong +# cluster (or Kyverno is not installed yet -- see the checklist below). +``` + +::: + +### 3.1 Installation and capability verification checklist {#s3-1} + +Both components install through ACP's modular mechanisms, and both support air-gapped environments: + +- **Kyverno**: administrator view → **Marketplace → Cluster Plugins** → search `kyverno` → install **"Alauda Container Platform Compliance for Kyverno"**. Once installed, Kyverno is managed by the platform as Helm / AppRelease, with the four controllers deployed in the `kyverno` namespace. +- **Tekton Pipelines**: administrator view → **Marketplace → OperatorHub** → install **"Alauda DevOps Pipelines"**; from then on `TektonConfig` manages Pipelines / Triggers / Chains and the resolver switches. + +Product documentation: Compliance Management (Kyverno) installation and configuration, DevOps (Tekton) installation — see the official ACP documentation links in [§8.2](#s8-2). + +:::warning Do not change managed configuration on the Deployment directly + +ACP's Kyverno is managed by a platform module (Helm / AppRelease) and **periodically reconciled** — any argument change made by directly `kubectl patch`ing the controller Deployment (for example manually adding `--exceptionNamespace`) **will be reverted by the next reconcile**. All controller-level configuration must be persisted through the platform module's configuration entry point (how-to in [§3.1.1](#s3-1-1)). + +::: + +**Confirm three prerequisites first, or the commands below will give misleading results**: + +```bash +# 1) Tekton's namespace: this document (including the checklist below) writes the +# literal tekton-pipelines for readability, but on ACP the operator decides it +# and it may be something else. TektonConfig is authoritative. Every later code +# block that uses it starts with a fallback line : "${TEKTON_NS:=tekton-pipelines}", +# so the blocks run even when read out of order; but **tekton-pipelines inside +# policy YAML is a literal** (controller ServiceAccount subjects, +# system:serviceaccount:tekton-pipelines:... and the like) -- a shell variable +# cannot be substituted in. When targetNamespace is not that name, every +# occurrence must be edited by hand; a missed one means the rule silently skips. +TEKTON_NS=$(kubectl get tektonconfig config -o jsonpath='{.spec.targetNamespace}') +# Exported so the commands you run from this shell (including subshells and scripts) +# see it. It does NOT survive a new terminal, which is why later blocks re-assert the +# default on their first line instead of trusting the variable to be there. +export TEKTON_NS=${TEKTON_NS:-tekton-pipelines} +echo "Tekton namespace: $TEKTON_NS" + +# 2) Checklist items 3 and 4 use --as to query someone else's permissions, which +# requires impersonate permission; without it the command itself reports +# forbidden (which is NOT a "permission missing" verdict). If you lack +# impersonate permission, inspect the ClusterRoleBindings directly instead: +# kubectl get clusterrolebinding -o json | jq '…kyverno…' +echo "can impersonate serviceaccounts: $(kubectl auth can-i impersonate serviceaccounts)" + +# 3) Client tools: besides kubectl, the commands in this document use jq (parsing +# childReferences / PolicyReport / result JSON) and python3 (generating the +# regex in §4.5.3). Install whichever is missing -- you can read without them, +# but the corresponding steps cannot be followed along. +for tool in kubectl jq python3; do + command -v "$tool" >/dev/null 2>&1 && echo "$tool: ok" || echo "$tool: MISSING" +done +# §4.5.2 reads image labels, and for that EITHER skopeo OR crane is enough -- so this +# one is an either-or, not a per-tool requirement. Missing both only blocks §4.5.2. +if command -v skopeo >/dev/null 2>&1 || command -v crane >/dev/null 2>&1; then + echo "skopeo/crane: ok (at least one)" +else + echo "skopeo/crane: BOTH MISSING -- only §4.5.2 needs them" +fi +# The kyverno CLI is a LOCAL binary, separate from the in-cluster Kyverno install -- +# having Kyverno running does not give you this command. Only §6.1.6's offline +# evaluation uses it, so missing it blocks nothing on the walkthrough path. +# Probed by running it rather than by resolving its path, so a broken install is +# reported as missing instead of as "ok". +kyverno version >/dev/null 2>&1 \ + && echo "kyverno (CLI): ok" \ + || echo "kyverno (CLI): MISSING or not runnable -- optional, only §6.1.6 uses it" +``` + +Once installed, verify each capability this solution depends on, item by item. This checklist is a **capability inventory, not an "all green before you may proceed" gate**: items 1 and 2 are shared prerequisites; items 3, 4, and 5 only need to hold when you use the corresponding chapter's capabilities; item 6's **choice of tier** has no right or wrong — that part is planning input — but its **declaration and the generated grouping must agree** (fix a mismatch per the remediation table); and item 7 is the **single check where "not as expected" means an entire chapter of policies is void**. **Where to fix each item that does not match expectations is in the remediation table after the code block.** + +```bash +# 1. All four controllers must be Ready +# Expect kyverno-admission-controller / background-controller / cleanup-controller / +# reports-controller with all replicas Ready. A single replica is not acceptable +# long term in production; size the replica count per your HA plan (§6.1.8). +# Every item below prints an "== N) ... ==" banner first, so the combined output of +# this block reads back against the checklist numbers without guessing. +echo "== 1) Kyverno controllers ==" +kubectl get deploy -n kyverno + +# 2. Tekton controllers and resolver feature flags +# TEKTON_NS is set by the prerequisite block above; this line only fills it in if you +# copied this block alone. It is not cosmetic: with the variable unset, `-n ""` reads +# the CURRENT namespace and still exits 0, so the three checks would report an empty +# Tekton namespace instead of failing loudly. +: "${TEKTON_NS:=tekton-pipelines}" +echo "== 2) Tekton controllers and resolver flags (ns: $TEKTON_NS) ==" +kubectl get deploy -n "$TEKTON_NS" +echo "resolver feature flags:" +kubectl get cm -n "$TEKTON_NS" resolvers-feature-flags -o jsonpath='{.data}{"\n"}' +# Expect enable-cluster-resolver / enable-hub-resolver / enable-git-resolver to be +# "true" as required by the resolvers you actually use +echo "hub default-type: $(kubectl get cm -n "$TEKTON_NS" hubresolver-config -o jsonpath='{.data.default-type}')" +HUB_API=$(kubectl get cm -n "$TEKTON_NS" hubresolver-config -o jsonpath='{.data.artifact-hub-api}') +echo "artifact-hub-api: $HUB_API" +# Expect the in-cluster Artifact Hub (the Shim service) here. A public https://artifacthub.io/ +# means every hub reference in this document resolves against the public hub and 404s -- +# and the flags above stay green while it happens, which is why the next probe exists. + +# 2b. Hub endpoint smoke test: the flags only say the resolver is ON, never that its endpoint +# can actually serve the coordinates this document pins. Resolve-side failures surface far +# later as CouldntGetPipeline / CouldntGetTask, so probe the five coordinates up front. +# Pass criterion: every exact version detail endpoint returns HTTP 200 AND a non-empty +# data.manifestRaw. A package-list 200 is insufficient: the pinned version or its +# manifest can still be absent. Any failed coordinate makes the whole block exit non-zero. +echo "== 2b) hub endpoint smoke (expect five usable exact-version manifests) ==" +kubectl -n '' run hub-smoke-$$ --rm -i --restart=Never \ + --image='/busybox:latest' --env="HUB_API=$HUB_API" --command -- sh -c ' +failed=0 +for coordinate in \ + tekton-task/catalog/sonarqube-scanner/0.7 \ + tekton-task/catalog/trivy-scanner/0.6 \ + tekton-task/catalog/skopeo-copy/0.1 \ + tekton-pipeline/catalog/java-image-build-scan-deploy/0.3 \ + tekton-pipeline/catalog/python-image-build-scan-deploy/0.3; do + body=/tmp/hub-detail.json + headers=$(wget -S -O "$body" "${HUB_API%/}/api/v1/packages/$coordinate" 2>&1) || true + code=$(printf "%s\n" "$headers" | awk "/^ HTTP\// { code=\$2 } END { print code }") + if [ "$code" != 200 ]; then + echo "$coordinate -> ${code:-UNREACHABLE}" + failed=1 + elif ! grep -Eq "\"manifestRaw\"[[:space:]]*:[[:space:]]*\"[^\"].*\"" "$body"; then + echo "$coordinate -> 200 but data.manifestRaw is empty or absent" + failed=1 + else + echo "$coordinate -> 200 + non-empty data.manifestRaw" + fi +done +exit "$failed"' +# The detail path is +# /api/v1/packages////: package type +# is tekton-task / tekton-pipeline, and is the value pinned by taskRef / +# pipelineRef (this document pins `catalog`) -- NOT the default-*-catalog keys, which only +# apply when the reference omits the catalog param. The Shim accepts normalized exact +# SemVer forms (for example 0.1 and 0.1.0), but the probe should use the exact coordinates +# present in your Run references. Adjust catalog, name and version together. + +# 3. RBAC prerequisite for mutate-existing (required by the three mutate-existing +# cancellation policies: §4.2.2 / §4.6.1 / §4.6.2. §4.2.3 is an ADMISSION mutate +# on the incoming object and needs no extra RBAC) +echo "== 3) mutate-existing RBAC (only needed for §4.2.2 / §4.6) ==" +echo "background-controller can update pipelineruns: $(kubectl auth can-i update pipelineruns.tekton.dev \ + --as=system:serviceaccount:kyverno:kyverno-background-controller -A)" +# "no" means you must grant it as described in the §4.6 preamble; without the grant Kyverno +# rejects those policies at creation time + +# 4. Effective reports-controller permissions on the Tekton /status subresource +# (all three verbs: get / list / watch). "no" is usually fine -- see the notes below +echo "== 4) reports-controller perms on /status (no is usually fine) ==" +for resource in pipelineruns.tekton.dev taskruns.tekton.dev; do + for verb in get list watch; do + echo " $resource status/$verb: $(kubectl auth can-i "$verb" "$resource" \ + --subresource=status \ + --as=system:serviceaccount:kyverno:kyverno-reports-controller -A)" + done +done + +# 5. PolicyException feature flags (required by §5.3) +echo "== 5) PolicyException flags ==" +kubectl get deploy -n kyverno kyverno-admission-controller \ + -o jsonpath='{.spec.template.spec.containers[0].args}' | tr ',' '\n' | grep -i exception +# Expect BOTH --enablePolicyException=true and --exceptionNamespace=. +# Only the first one present is the ACP default -- configure the second per §3.1.1 + +# 6. Webhook failure policy (fail-open or fail-closed while Kyverno is unavailable) +# and the per-request timeout every rule -- including its external calls (§3.7) -- must fit inside. +# Read BOTH layers: the per-policy intent declared in spec.webhookConfiguration, +# then the generated webhook groups (-fail / -ignore) it must land in +echo "== 6) webhook failurePolicy / timeout (declared intent vs generated grouping) ==" +kubectl get clusterpolicy -o \ + custom-columns='NAME:.metadata.name,FAILURE_POLICY:.spec.webhookConfiguration.failurePolicy,TIMEOUT:.spec.webhookConfiguration.timeoutSeconds' +# Namespaced Policy objects (§5 project autonomy) carry the same field and are +# NOT in the clusterpolicy listing -- read them too when §5 is in use +kubectl get policy -A -o \ + custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,FAILURE_POLICY:.spec.webhookConfiguration.failurePolicy,TIMEOUT:.spec.webhookConfiguration.timeoutSeconds' +kubectl get validatingwebhookconfiguration -o \ + custom-columns='NAME:.metadata.name,WEBHOOK:.webhooks[*].name,POLICY:.webhooks[*].failurePolicy,TIMEOUT:.webhooks[*].timeoutSeconds' \ + | grep kyverno + +# 7. Which resources Kyverno ignores outright, BEFORE any policy is consulted +echo "== 7) Kyverno resourceFilters (silent, pre-policy exemptions) ==" +kubectl get cm -n kyverno kyverno -o jsonpath='{.data.resourceFilters}' | tr ' ' '\n' | grep -n ',' +# Expect no entry covering a namespace where pipelines run, and none covering +# PipelineRun / TaskRun / Pod. A match here produces no denial and no report at all +``` + +**The expected value for each item, and where to go when the result does not match** (read this table first, then the three easily misjudged interpretations below it): + +| Check | Expected | If not as expected | +|---|---|---| +| 1 Controllers ready | All four controllers Ready | First check the plugin's installation status (Marketplace → Cluster Plugins) and the Pod events to locate the failure; size the replica count per the HA plan in [§6.1.8](#s6-1-8), and make that change through the [§3.1.1](#s3-1-1) `ModuleInfo.spec.valuesOverride` entry point too — the corresponding chart-values keys are `admissionController.replicas` / `backgroundController.replicas` / `cleanupController.replicas` / `reportsController.replicas` (all four can be checked directly in the values of the deployed `AppRelease`; before writing, confirm the actual key names of that chart in your environment the same way as in [§3.1.1](#s3-1-1)) — **do not edit the Deployment directly** (the platform reconcile reverts it) | +| 2 Resolver switches and hub endpoint | The resolvers you actually use are `true`; Hub's `default-type` is `artifact`; `artifact-hub-api` points at the in-cluster Artifact Hub (Shim) service; **all five coordinates of the 2b smoke test return 200** | These two ConfigMaps are managed by the Tekton operator and direct edits get reverted — change `TektonConfig.spec.pipeline` instead: set `enable-cluster-resolver` / `enable-hub-resolver` / `enable-git-resolver` to `true` as needed; the Hub endpoint and the output type both live in **the same place**, `TektonConfig.spec.pipeline.hub-resolver-config` (a single string map whose keys match the ConfigMap: `artifact-hub-api` / `default-type` / `default-artifact-hub-task-catalog` / `default-artifact-hub-pipeline-catalog`), which the operator reconciles into `tekton-pipelines/hubresolver-config`. **Do not go through `spec.hub`** — that section configures the Tekton Hub component itself, not the hub resolver. If you would rather not touch platform configuration, have every Hub reference carry an explicit `type=artifact` ([§4.5.1](#s4-5-1)). **404s in the 2b smoke test**: first check whether `artifact-hub-api` is the in-cluster Shim address (pointed at the public hub, every hub reference in this document fails as `CouldntGetPipeline` / `CouldntGetTask` while the three switches above stay green), then check whether the catalog and package names in the coordinates match what your environment actually publishes; if the endpoint points at the public Artifact Hub, treat it as an environment configuration problem — have a platform administrator point it back at the in-cluster Shim before continuing. **UNREACHABLE in the smoke test**: the probe Pod has no network / DNS path to that address; fix connectivity before talking about policy | +| 3 mutate-existing RBAC | Returns `yes` if you use the mutate-existing cancellation capability ([§4.2.2](#s4-2-2) and [§4.6](#s4-6), three policies in total) | On `no`, grant the aggregated ClusterRole given in the [§4.6](#s4-6) preamble (the `rbac.kyverno.io/aggregate-to-background-controller: "true"` label in its labels aggregates it into the background controller's permissions). **If you want to use a namespaced Role instead, you must also change `mutate.targets[].namespace` from `{{ request.namespace }}` to a namespace literal** — otherwise Kyverno's creation-time authorization check cannot resolve that variable, recognizes only cluster-level permissions, and the policy still fails to install (see the [§4.6](#s4-6) preamble). **If you do not install the [§4.2.2](#s4-2-2) / [§4.6](#s4-6) mutate-existing cancellation policies, this permission is not needed — the admission mutate of [§4.2.3](#s4-2-3) modifies the incoming request object and does not need it** | +| 4 reports-controller reads status | All six `yes` (optional, not required) | A `no` **usually needs no action** (rationale in the third interpretation below). Only when some other feature genuinely needs the reports-controller to read status directly, add one more least-privilege ClusterRole the same aggregated way as item 3, with the aggregation label swapped to `rbac.kyverno.io/aggregate-to-reports-controller: "true"` | +| 5 PolicyException switches | Both `--enablePolicyException=true` and `--exceptionNamespace=` present | Seeing only the former is ACP's default state — per [§3.1.1](#s3-1-1), write the `enabled` / `namespace` of `features.policyExceptions` into the kyverno `ModuleInfo`'s `spec.valuesOverride["ait/chart-kyverno"]` (**`ModuleInfo` exists only on the global management cluster**, see the warning in [§3.1.1](#s3-1-1)); **do not patch the Deployment args**. [§3.1.1](#s3-1-1) provides copy-pasteable atomic patch and rollback commands. **If you do not plan to use PolicyException exemptions ([§5.3](#s5-3)), you need not configure this** | +| 6 Webhook failure policy and timeout | **Read the intent declared in the policy body first, then check the generated result** (field semantics, the ⚠️ timing trap on the generated side, and the impact of the platform-wide override switch are in [§3.1.2](#s3-1-2) — that is the complete version of this mechanism): `kubectl get clusterpolicy -o custom-columns='NAME:.metadata.name,FAILURE_POLICY:.spec.webhookConfiguration.failurePolicy,TIMEOUT:.spec.webhookConfiguration.timeoutSeconds'` for the declared intent (when the namespaced `Policy` objects of [§5](#s5) are in use, also read `kubectl get policy -A` with the same columns — they never appear in the clusterpolicy listing, and skipping them leaves their declarations unchecked), then the generated webhooks, which **take effect grouped by value** (`validate.kyverno.svc-fail` / `validate.kyverno.svc-ignore`, each carrying its own `failurePolicy` / `timeoutSeconds`). All policy assets in this document declare it explicitly (tiering rationale in [§3.7](#s3-7)) | If declaration and grouping disagree, or a policy needs a different tier: **change that policy body's `spec.webhookConfiguration` and manage it with GitOps** — the only entry point that can express per-policy tiers; the three traps (`ModuleInfo` can only override platform-wide, `timeoutSeconds` is a single-request total budget, never hand-edit the `ValidatingWebhookConfiguration`) are in [§3.1.2](#s3-1-2) | +| 7 Resources Kyverno ignores outright | The filter list has **no** entry covering a namespace where pipelines run, and none covering `PipelineRun` / `TaskRun` / `Pod` | The `resourceFilters` in the `kyverno` ConfigMap take effect **before any policy**: a matched request is neither denied, nor recorded in a PolicyReport, nor logged — a **completely silent** exemption channel. The factory value generally excludes four namespaces (**take the value the command above actually read as authoritative**) — `kyverno` / `kube-system` / `kube-public` / `kube-node-lease`: the same violating Pod is denied in `policy-poc` yet sails straight through in `kube-system`. Therefore ① do not run pipelines in an excluded namespace; ② know that a policy written with `namespaces: ["*"]` carries this hole by construction; ③ write permission on this configuration must be controlled at the same level as `ClusterPolicy` ([§5.0](#s5-0)) | + +Three of the interpretations above are easy to get wrong: + +- **Item 2's `default-type`**: this document allows Hub references to omit the `type` parameter, on the premise that this platform setting outputs `artifact`. If it does not, either govern that platform setting first, or require every Hub reference to write `type=artifact` explicitly ([§4.5.1](#s4-5-1)). +- **Item 4 must carry `--subresource=status`**: passing `taskruns.tekton.dev/status` as a positional argument to `kubectl auth can-i` gets parsed as `TYPE/NAME` — what you queried is not the status-subresource permission but an object named `status`. +- **Item 4 returning `no` does not mean widen permissions right away**: status Audit with `background: false` is aggregated through the admission-report chain and does not require the reports-controller to read TaskRun / PipelineRun status directly; even with all six permissions at `no`, [§4.4.1](#s4-4-1) / [§4.4.2](#s4-4-2) still produce terminal-state PolicyReports. So **do not enlarge the ClusterRole merely because a permission warning appears at policy creation time** — first run one real controlled request and confirm whether the PolicyReport converges from the early skip to a terminal pass/fail; only when some other feature genuinely needs the reports-controller to read status directly, grant it separately with least privilege. +#### 3.1.1 Enabling PolicyException (optional; required by §5.3) {#s3-1-1} + +ACP's "Compliance for Kyverno" plugin **ships by default with only `--enablePolicyException=true`, without `--exceptionNamespace`**. This default state is the most deceptive one: a PolicyException object **can be created successfully**, with nothing but a warning `The exceptionNamespace flag is not set` — yet it **has no effect at all**: the exemption is in place, and the target resource is still denied. The two flags must be configured together, and Kyverno only honors PolicyExceptions in the namespace `--exceptionNamespace` points at (which is exactly where exemption authority is closed off, [§5.3](#s5-3)). The flag **accepts a single namespace name, or `*`** (meaning PolicyExceptions in any namespace take effect) — **multiple namespaces are not supported** (confirmed on the Kyverno 1.15 line; the multi-namespace-list request was raised upstream — [kyverno#6980](https://github.com/kyverno/kyverno/issues/6980) — and closed as not-planned in 2026-01, because the informer only comes in "single namespace / whole cluster" flavors and the implementation is complex). In a multi-project / multi-tenant environment, this single-value constraint lands in one of two ways: + +- **Central approval (used in this document)**: the trusted namespace **belongs to the approving side (the platform)**; project members never enter it — exemptions go through a request-and-approval flow, issued on the requester's behalf by the approver identity (this is exactly the model [§5.3](#s5-3) demonstrates). The natural isolation between projects is unaffected: this namespace is not a space projects share, it is the landing point of the approval flow. Do **not** let multiple projects share one trusted namespace and self-serve their exemptions — RBAC can only govern "who may create a PolicyException", not "whether the exemption's content stays in bounds" (`spec.match` can name any namespace), so project A could create an exception that exempts project B's pipelines. +- **Project autonomy (`*`)**: each project creates PolicyExceptions in its own namespace, and issuing authority follows project RBAC. In this mode you **must** add a meta-policy restricting a PolicyException to **exempting only resources in its own namespace** — without it, the "content out of bounds" problem above holds in every namespace; and write permission on `policyexceptions` must be explicitly tightened in each project — default roles should not carry it. + +:::warning ModuleInfo exists only on the global management cluster; workload clusters do not have this resource + +`ModulePlugin` / `ModuleConfig` / `ModuleInfo` are all platform management-plane objects and **exist only on the global management cluster**. Running `kubectl get moduleinfo` on the workload cluster where Kyverno runs finds nothing — that cluster does not even have the CRD. The **locate and patch commands in this section must therefore be executed with the global cluster's kubeconfig**; whereas, of the three confirmations in point 4, ② the Deployment args and ③ the rollout and the Pods' actual arguments must be executed on **the cluster Kyverno runs on**. + +Note also that on global, one plugin has **one `ModuleInfo` per installation target cluster**, so before asserting "exactly one match" you must first narrow by target cluster — the platform marks the delivery target with `cpaas.io/cluster-name`; an instance installed on the global cluster itself may not carry that label, in which case identify it by the ownerReference pointing at its `Cluster` object. + +The commands below are written for Kyverno and Tekton on the same cluster, so there is no cross-cluster switching; if your environment deploys the two separately, split the commands into the two sides as described above. + +::: + +The correct enablement path has four essentials: + +1. **Never patch the controller Deployment's args directly** — the platform reconcile will revert it (see the warning above). +2. **The override entry point is the plugin's `ModuleInfo` `spec.valuesOverride`**, not `spec.config`. The kyverno `ModuleInfo` has only `version` in its spec by default; `spec.config` is the module instance's user configuration, not an override surface for chart values — change the wrong field and nothing takes effect. `valuesOverride` is layered by **chart name** (isomorphic to `ModuleConfig.spec.valuesTemplates`), and the chart name is `ait/chart-kyverno`. +3. **Locating the ModuleInfo must assert uniqueness**: on the global cluster, query precisely by the module label, narrow by the target-cluster label, then hard-assert exactly 1 match; do not guess by version or a `global-` prefix, and do not silently take `items[0]`. +4. **After the change, confirm in three places — every one of them**: ① the `AppRelease` has merged the values; ② the Deployment template args carry the flag; ③ the rollout has finished and **every Ready admission Pod** actually runs the new arguments. Looking only at the Deployment template, or hitting only one new Pod, is not enough to prove that every serving instance has switched over during an HA rolling update. + +:::warning Single-node / CPU-starved clusters: the configuration can be right and the flag still not in effect + +The admission-controller rollout **starts the surge pod first, then retires the old pod** (`maxUnavailable` is effectively 0); on a node short of CPU the surge pod goes Pending, the rollout wedges, the old pod keeps serving, and the symptom is that PolicyException still reports `exceptionNamespace flag is not set` — this is not a configuration error. **There is exactly one criterion: what arguments the serving pods are actually running** (③ of point 4); the flag being on the Deployment template does not mean it is on the serving pods. When wedged, free up node resources and let the rollout complete on its own — do not count on deleting a single old pod being enough (the new pod's actual resource request need not equal the template value). + +::: + +⚠️ **First look at where it currently points**: `--exceptionNamespace` **accepts exactly one value**. If the cluster already has it enabled, pointing at another namespace that carries real exemptions, changing it to a demo value makes **all of those exemptions stop working immediately** (and they stay broken until you change it back). In that case do not change it — reuse the existing trusted namespace to run [§5.3](#s5-3) (the opening of [§5.3](#s5-3) reads exactly that value; the `policy-exceptions` in the text is merely the value configured by this document's [§3.1.1](#s3-1-1), not a constant you must match). This change is a switch that is **globally unique on the target cluster**; only one person should be touching it at any given time. + +**Every value this section needs from you is gathered in the input block below** — every later block (a)–g), the save-to-disk block, the read-back block) only references the variables set here and carries no further `<...>` placeholders, so this block must run first: + +```bash +# The ONLY user-supplied inputs of this section, gathered in one place so a pasted +# block never hides a in its middle; later blocks validate these +# variables instead of re-declaring them. +GLOBAL_KUBECONFIG='' # kubeconfig of the GLOBAL management cluster +TARGET_CLUSTER='' # the cluster Kyverno runs on; a) narrows its query by it +TRUSTED_EXCEPTION_NS='' # namespace that will hold PolicyExceptions (§5.3) +# ModuleInfo lives only on the global management cluster, so every command in this +# section goes through this one wrapper. A shell FUNCTION, not a KGLOBAL="kubectl ..." +# string: zsh keeps an unquoted expansion as one word, so the string form pasted into +# an interactive zsh looks for a command literally named "kubectl --kubeconfig ...". +# The :? inside makes every call refuse by name in a shell that never ran this block. +KGLOBAL() { + kubectl --kubeconfig "${GLOBAL_KUBECONFIG:?run the inputs block at the top of §3.1.1 in this shell first}" "$@" +} +KGLOBAL config view --minify -o jsonpath='{.clusters[0].cluster.server}{"\n"}' +``` + +The API server address that last command prints must be **the global cluster you intend to change**; if it is not, fix the kubeconfig first, then continue. + +**Execution order overview** — a)–g) all live in the collapsible block below; the order must not change, and **do not paste the whole collapsible block in one go** (e) is the rollback — running it all at once amounts to enabling and immediately reverting): + +1. **Check for an old ledger before starting**: if `ls moduleinfo-target.txt moduleinfo-original.json moduleinfo-expected.json 2>/dev/null` prints anything, a previous enable round was never rolled back — first use the "Recovering rollback state in a new terminal" block to reload that state, run e)–g) to close out that round, then start a new one. This step must happen before a): once c) has run, the globally unique switch has already been changed. +2. **Enable**: a) locate and assert uniqueness → b) save the original value → **write to disk** (persist the rollback state into the three files above; this must come before c) — c) is irreversible, and until the state is on disk the "original value" lives only in the current shell: close the terminal at that moment and it is gone forever, and re-running b) afterwards would only record the modified value as the original) → c) atomic write → d) confirm in three places. +3. **Use**: go run [§5.3](#s5-3); come back to roll back only after all of it is done and cleaned up. +4. **Rollback**: e) atomic restore → f) confirm it took effect the same way d) did → g) delete the rollback files. If you switched terminals along the way, first rebuild state from the files with the "Recovering rollback state in a new terminal" block — **never re-run b)**. This restoration is platform-side configuration; it belongs to no section's "cleanup" subsection and can only be performed manually here. + +:::details Enable and rollback commands (atomic JSON Patch, copy-paste ready) + +```bash +# a) Locate the ModuleInfo on the GLOBAL management cluster and assert the match is unique. +# ModuleInfo exists only there -- the cluster running Kyverno has no such resource. +# KGLOBAL and TARGET_CLUSTER come from the inputs block at the top of §3.1.1; stop +# here if this shell never ran it, rather than query the wrong cluster. +: "${GLOBAL_KUBECONFIG:?run the inputs block at the top of §3.1.1 in this shell first}" +# Presetting GLOBAL_KUBECONFIG by hand is not enough -- the KGLOBAL wrapper +# function must exist too, or every call below dies as "command not found". +command -v KGLOBAL >/dev/null || : "${KGLOBAL:?run the inputs block at the top of §3.1.1 in this shell first}" +: "${TARGET_CLUSTER:?run the inputs block at the top of §3.1.1 in this shell first}" +# One plugin gets one ModuleInfo per target cluster, so narrow the query to the cluster +# Kyverno runs on before asserting uniqueness. An instance installed onto the global +# cluster itself may carry no cpaas.io/cluster-name label -- identify that one by the +# ownerReference pointing at its Cluster object instead of by this selector. +# ModuleInfo is CLUSTER-SCOPED -- it has no namespace, so nothing here passes -n. +MODULES=$(KGLOBAL get moduleinfo -o json \ + -l cpaas.io/module-name=kyverno,cpaas.io/cluster-name="$TARGET_CLUSTER") +# `test ... -eq 1` on its own line does NOT stop an interactive shell: it only sets $?, +# and the next line would take items[0] anyway -- the very thing point 3 above forbids. +# Branch instead, so a non-unique match leaves MODULE unset and c) cannot run. +if [ "$(jq '.items | length' <<<"$MODULES")" -ne 1 ]; then + echo "expected exactly ONE ModuleInfo, got $(jq '.items | length' <<<"$MODULES") --" + echo "narrow the selector by target cluster first; do NOT continue to b)/c)." + unset MODULE +else + MODULE=$(jq -r '.items[0].metadata.name' <<<"$MODULES") + echo "target ModuleInfo: $MODULE" +fi + +# b) Save the complete original spec and compute the target spec to write. +# Keeping the original verbatim is what lets the rollback restore an absent field, +# an explicit null, or an arbitrary non-empty object exactly as it was. +: "${TRUSTED_EXCEPTION_NS:?run the inputs block at the top of §3.1.1 in this shell first}" +# a) prints "do NOT continue to b)/c)" when the match is not unique -- but printing is not +# stopping, and the whole block is pasted in one go, so b) has to refuse for itself. A bare +# `: "${MODULE:?...}"` would not do it either: in an INTERACTIVE shell that fails only that +# one command and the next line still runs. Branch, exactly as a) does. +if [ -z "${MODULE:-}" ]; then + echo "a) did not settle on exactly one ModuleInfo -- fix a) first; b) and c) are skipped." +else + ORIGINAL_MODULEINFO_SPEC=$(KGLOBAL get moduleinfo "$MODULE" -o json | jq -c '.spec') + TEST_MODULEINFO_SPEC=$(jq -c --arg ns "$TRUSTED_EXCEPTION_NS" ' + .valuesOverride = (.valuesOverride // {}) | + .valuesOverride["ait/chart-kyverno"].features.policyExceptions = { + enabled: true, + namespace: $ns + } + ' <<<"$ORIGINAL_MODULEINFO_SPEC") +fi +``` + +**After b) finishes, write to disk before touching c)** — the state e) depends on (`GLOBAL_KUBECONFIG`, `MODULE`, the two specs) at this point lives only in the current shell; write it into the three rollback files first, and continue only after seeing `saved:`: + +```bash +# Everything here comes from earlier blocks IN THIS SHELL: GLOBAL_KUBECONFIG (which +# the KGLOBAL wrapper reads) from the inputs block at the top of §3.1.1, the rest +# from a)-b). Checked first and by name -- a bare "command not found: KGLOBAL" +# further down would not say which piece of state is missing. +if [ -z "$GLOBAL_KUBECONFIG" ] || ! command -v KGLOBAL >/dev/null \ + || [ -z "$MODULE" ] \ + || [ -z "$ORIGINAL_MODULEINFO_SPEC" ] || [ -z "$TEST_MODULEINFO_SPEC" ]; then + echo "missing state in this shell -- run the inputs block (GLOBAL_KUBECONFIG +" + echo "the KGLOBAL wrapper) and a)+b)" + echo "(MODULE / the two specs) here first, then this block." + # Refuse to overwrite: if these files are already here, an earlier enable was never + # rolled back, and b) has just captured the ALREADY-MODIFIED spec as "the original". + # Overwriting would destroy the only record of the true original value. +elif [ -e moduleinfo-target.txt ] || [ -e moduleinfo-original.json ] \ + || [ -e moduleinfo-expected.json ]; then + # Any of the three still here means an earlier enable was never rolled back -- and + # b) has just captured the ALREADY-MODIFIED spec as "the original". Overwriting + # would destroy the only record of the true original value. + echo "rollback files from an earlier run are still here, so what this shell is" + echo "holding as 'the original' is really the PREVIOUS round's modified spec." + echo "Do NOT run c). The true original is in moduleinfo-original.json: load it with" + echo "the read-back block below, run e)+f)+g) to finish THAT round, then start over." + # Not just a printed refusal: e) reads these variables, and running it with + # what this shell currently holds would write the previous round's change back as + # if it were the original. Clearing them makes e) fail until the read-back block + # has reloaded the real values from the files. + unset MODULE ORIGINAL_MODULEINFO_SPEC TEST_MODULEINFO_SPEC + # The API server URL goes in too: a name alone does not identify a CLUSTER, and + # e)'s test would happily pass against a same-named ModuleInfo on another global + # cluster whose current spec matches -- writing this cluster's original onto it. + # The uid is the tie-breaker: one kubeconfig can spell the same API server several + # ways (DNS alias, load balancer, :443 written out, a tunnel), so a URL mismatch on + # the way back is not proof of a different cluster -- the uid settles it. + # Each value is read and checked separately: inside `printf "$(...)"` a failed + # command substitution is invisible, and an empty field would still print "saved". +elif ! saved_api=$(KGLOBAL config view --minify \ + -o jsonpath='{.clusters[0].cluster.server}') || [ -z "$saved_api" ]; then + echo "could not read the API server URL out of this kubeconfig -- fix that first." +elif ! saved_uid=$(KGLOBAL get moduleinfo "$MODULE" \ + -o jsonpath='{.metadata.uid}' 2>&1) || [ -z "$saved_uid" ]; then + echo "could not read the ModuleInfo uid ($saved_uid)." + echo "Do NOT run c) yet: with no uid there is nothing to bind the rollback files to," + echo "and c) is the step that makes this shell's variables irreplaceable." +elif ! printf '%s %s %s\n' "$MODULE" "$saved_api" "$saved_uid" \ + > moduleinfo-target.txt \ + || ! printf '%s' "$ORIGINAL_MODULEINFO_SPEC" > moduleinfo-original.json \ + || ! printf '%s' "$TEST_MODULEINFO_SPEC" > moduleinfo-expected.json; then + # "Run this block again" is not enough on its own: a partial write can leave one or + # two of the three files behind, and the guard at the top would then read them as an + # earlier round's rollback and refuse -- with the true values still only in this + # shell. They came from THIS block, seconds ago, so deleting them is safe here and + # nowhere else; say so explicitly rather than leaving the reader in that deadlock. + echo "writing the rollback files failed -- do NOT run c), and do NOT close this shell:" + echo "its variables are the only copy. Free space / fix permissions, then delete" + echo "whatever this attempt left behind and run this block again:" + echo " rm -f moduleinfo-target.txt moduleinfo-original.json moduleinfo-expected.json" + echo "(safe ONLY right here: at the top of this block none of the three existed.)" +else + echo "saved: rollback for $MODULE (uid $saved_uid)" +fi +``` + +```bash +# Same-shell state from the inputs block, a)-b) and the save block; fail by name here +# instead of feeding jq an empty --argjson or patching a nameless object. +# Collected and branched, not `: "${VAR:?msg}"` -- see block b) for why that shape does +# not guard a block that writes. +# +# `$MODULE` is also checked against the name the save block recorded. An unset variable is +# caught by the emptiness test; a STALE one -- left in a reused shell by an earlier attempt +# -- is not, and it is the dangerous case, because the patch would then rewrite a DIFFERENT +# ModuleInfo that the rollback files do not describe. +missing= +for v in GLOBAL_KUBECONFIG TRUSTED_EXCEPTION_NS MODULE ORIGINAL_MODULEINFO_SPEC TEST_MODULEINFO_SPEC; do + eval "[ -n \"\${$v:-}\" ]" || missing="$missing $v" +done +command -v KGLOBAL >/dev/null || missing="$missing KGLOBAL(the wrapper function)" +# The rollback files are inputs here too: this is a block that CHANGES the cluster, and +# it must not run unless the on-disk record to roll back from exists. The target file +# carries three fields (name, API server URL, uid) -- the recovery block needs all +# three -- so the stale-shell comparison reads only the first field, not the whole line. +for f in moduleinfo-target.txt moduleinfo-original.json moduleinfo-expected.json; do + [ -s "$f" ] || missing="$missing $f(missing or empty -- the save block has not written it)" +done +if [ -z "$missing" ]; then + read -r saved_name _ < moduleinfo-target.txt + if [ "$MODULE" != "$saved_name" ]; then + missing="$missing MODULE(='$MODULE' but the save block recorded '$saved_name' -- stale shell?)" + fi +fi +if [ -n "$missing" ]; then + echo "NOT RUN -- missing or inconsistent state from earlier blocks IN THIS SHELL:$missing" + echo "Run the inputs block at the top of §3.1.1, then a), b) and the save block, then paste this block again." +else + # c) Atomic write (still on the global cluster): the test op guarantees no concurrent + # modification happened -- on conflict the whole patch fails instead of silently overwriting + KGLOBAL patch moduleinfo "$MODULE" --type json -p \ + "$(jq -cn \ + --argjson expected "$ORIGINAL_MODULEINFO_SPEC" \ + --argjson replacement "$TEST_MODULEINFO_SPEC" ' + [ + {op:"test",path:"/spec",value:$expected}, + {op:"replace",path:"/spec",value:$replacement} + ] + ')" + + # d) Confirm in three places -- after waiting out the reconcile. The platform + # propagates asynchronously (ModuleInfo -> AppRelease -> Deployment -> rollout), and + # until the Deployment TEMPLATE has actually changed, (3)'s `rollout status` returns + # success for the PREVIOUS, already-finished rollout and the closing jq prints false: + # pasted in one go straight after c), every check below races the operator and + # proves nothing (live run on the validation environment: apprelease empty, args unchanged, + # "successfully rolled out", `false` -- and 30s later all four converged). So first + # wait, bounded, for the observable precondition: the template carrying the flag. + # Steps (2) and (3) inspect the workloads, so run them against the cluster Kyverno runs + # on -- that is the global cluster only when Kyverno is installed there. + EXPECTED_ARG="--exceptionNamespace=$TRUSTED_EXCEPTION_NS" + elapsed=0 + # `--` before the pattern is required, not tidiness: the pattern itself starts with + # `--`, and without the separator grep parses it as an option and dies with + # "unrecognized option" on every iteration. The loop would then never succeed -- + # it burns the full timeout and reports the reconcile as stuck on an enable that + # actually worked, sending you off to debug an operator that is fine. + until kubectl get deploy -n kyverno kyverno-admission-controller \ + -o jsonpath='{.spec.template.spec.containers[0].args}' | grep -qF -- "$EXPECTED_ARG"; do + if [ "$elapsed" -ge 120 ]; then + echo "no $EXPECTED_ARG on the Deployment template after ${elapsed}s -- the reconcile" + echo "is stuck, not merely slow. Check the kyverno AppRelease/operator, then re-run d)." + break + fi + sleep 5; elapsed=$((elapsed + 5)) + done + + # (1) AppRelease has merged the values; expect {"enabled":true,"namespace":""} + kubectl get apprelease -n cpaas-system kyverno \ + -o jsonpath='{.spec.values.features.policyExceptions}' + + # (2) The Deployment template args now carry the flag (re-run item 5 of the checklist) + kubectl get deploy -n kyverno kyverno-admission-controller \ + -o jsonpath='{.spec.template.spec.containers[0].args}' | tr ',' '\n' | grep -i exception + + # (3) Rollout finished AND every Ready admission Pod actually runs the new arg + kubectl rollout status deployment/kyverno-admission-controller -n kyverno --timeout=5m + # rollout status can return in the brief window before the new admission Pod flaps + # NotReady to reload config with the changed arg; for that instant there are zero + # Ready Pods and the jq below (which requires `($ready|length)>0`) would print false + # on an enable that in fact succeeded. Wait for a Ready Pod first so the check reads + # steady state, not the flap. Best-effort: on timeout the jq still runs and prints + # the real verdict. + kubectl wait --for=condition=Ready pod -n kyverno \ + -l app.kubernetes.io/component=admission-controller --timeout=120s + kubectl get pod -n kyverno -l app.kubernetes.io/component=admission-controller -o json | \ + jq -e --arg expected "$EXPECTED_ARG" ' + [.items[] | select(any(.status.conditions[]?; .type == "Ready" and .status == "True"))] as $ready + | ($ready | length) > 0 + and all($ready[]; + any(.spec.containers[]?; + .name == "kyverno" and any(.args[]?; . == $expected))) + ' +fi +``` + +Once d)'s three confirmations pass, go run [§5.3](#s5-3); come back and execute e)–g) only after **all of [§5.3](#s5-3)'s steps** are done and cleaned up. If you have switched terminals, first rebuild state with the "Recovering rollback state in a new terminal" collapsible block below. + +```bash +# Same-shell state again -- from the shell that ran a)-d), or rebuilt by the recovery +# block below. Refuse by name rather than patch a nameless object as the admin user. +# Collected and branched, not `: "${VAR:?msg}"` -- see block b) for why that shape does +# not guard a block that writes. +# +# `$MODULE` is also checked against the name the save block recorded. An unset variable is +# caught by the emptiness test; a STALE one -- left in a reused shell by an earlier attempt +# -- is not, and it is the dangerous case, because the patch would then rewrite a DIFFERENT +# ModuleInfo that the rollback files do not describe. +missing= +for v in GLOBAL_KUBECONFIG MODULE ORIGINAL_MODULEINFO_SPEC TEST_MODULEINFO_SPEC; do + eval "[ -n \"\${$v:-}\" ]" || missing="$missing $v" +done +command -v KGLOBAL >/dev/null || missing="$missing KGLOBAL(the wrapper function)" +# The rollback files are inputs here too: this is a block that CHANGES the cluster, and +# it must not run unless the on-disk record to roll back from exists. The target file +# carries three fields (name, API server URL, uid) -- the recovery block needs all +# three -- so the stale-shell comparison reads only the first field, not the whole line. +for f in moduleinfo-target.txt moduleinfo-original.json moduleinfo-expected.json; do + [ -s "$f" ] || missing="$missing $f(missing or empty -- the save block has not written it)" +done +if [ -z "$missing" ]; then + read -r saved_name _ < moduleinfo-target.txt + if [ "$MODULE" != "$saved_name" ]; then + missing="$missing MODULE(='$MODULE' but the save block recorded '$saved_name' -- stale shell?)" + fi +fi +if [ -n "$missing" ]; then + echo "NOT RUN -- the rollback would target the wrong object or fail halfway:$missing" + echo "Rebuild state with the 'Recovering rollback state in a new terminal' block below, then paste this block again." +else + # e) Rollback (global cluster again): test that the current spec still equals what we wrote, + # then replace it with the complete original spec. A failing test means someone else + # changed the ModuleInfo meanwhile -- do a manual three-way merge and revert only the + # policyExceptions change. + KGLOBAL patch moduleinfo "$MODULE" --type json -p \ + "$(jq -cn \ + --argjson expected "$TEST_MODULEINFO_SPEC" \ + --argjson original "$ORIGINAL_MODULEINFO_SPEC" ' + [ + {op:"test",path:"/spec",value:$expected}, + {op:"replace",path:"/spec",value:$original} + ] + ')" + + # f) Confirm the rollback the same way d) confirmed the enable -- a patched ModuleInfo is + # not a withdrawn flag. Until the platform has reconciled and the Pods have rolled, + # `--exceptionNamespace` is still live on the admission controllers actually serving + # requests, which means every PolicyException in that namespace is still in force. + # The asymmetry is the trap: enabling has three confirmations, and a rollback that + # just ends looks equally finished while leaving the exemption entrance open. + # Expect: an empty/absent policyExceptions value, no exception flag in the args, and + # the jq below printing true (every Ready admission Pod is free of the flag). + # (1)-(3) inspect workloads, so like d) they run against the cluster Kyverno runs on, + # not the global one -- plain kubectl, not the KGLOBAL wrapper. + # Same operator race as d), mirrored: until the Deployment template has dropped the + # flag, `rollout status` blesses the PREVIOUS rollout and the jq below prints false + # while the exemption entrance is still open. Wait, bounded, for the drop first. + elapsed=0 + until ! kubectl get deploy -n kyverno kyverno-admission-controller \ + -o jsonpath='{.spec.template.spec.containers[0].args}' | grep -q 'exceptionNamespace'; do + if [ "$elapsed" -ge 120 ]; then + echo "the Deployment template still carries --exceptionNamespace after ${elapsed}s --" + echo "the reconcile is stuck and the exemption entrance is STILL OPEN. Check the" + echo "kyverno AppRelease/operator, then re-run f); do not proceed to g)." + break + fi + sleep 5; elapsed=$((elapsed + 5)) + done + # Re-check once, explicitly: the loop above exits BOTH when the flag dropped and when + # the timeout branch broke out of it, and g) below must not have to guess which. A + # failed read answers "no match" too, so capture the read and require it to succeed + # before interpreting emptiness as absence. + if ARGS_NOW=$(kubectl get deploy -n kyverno kyverno-admission-controller \ + -o jsonpath='{.spec.template.spec.containers[0].args}' 2>&1) \ + && ! printf '%s' "$ARGS_NOW" | grep -q 'exceptionNamespace'; then + flag_dropped=yes + else + flag_dropped=no + fi + kubectl get apprelease -n cpaas-system kyverno \ + -o jsonpath='{.spec.values.features.policyExceptions}{"\n"}' + kubectl get deploy -n kyverno kyverno-admission-controller \ + -o jsonpath='{.spec.template.spec.containers[0].args}' | tr ',' '\n' | grep -i exception + kubectl rollout status deployment/kyverno-admission-controller -n kyverno --timeout=5m + # Same readiness flap as d): rollout status can return just before the admission Pod + # flaps NotReady to reload config, and the jq below requires at least one Ready Pod, so + # a single shot would print false on a rollback that in fact completed. Wait for a Ready + # Pod first; best-effort, the jq still runs and prints the real verdict on timeout. + kubectl wait --for=condition=Ready pod -n kyverno \ + -l app.kubernetes.io/component=admission-controller --timeout=120s + kubectl get pod -n kyverno -l app.kubernetes.io/component=admission-controller -o json | \ + jq -e ' + [.items[] | select(any(.status.conditions[]?; .type == "Ready" and .status == "True"))] as $ready + | ($ready | length) > 0 + and all($ready[]; + all(.spec.containers[]?; + .name != "kyverno" or all(.args[]?; (. | test("exceptionNamespace")) | not))) + ' + + # g) Only now retire the rollback files. Leaving them behind is not harmless: the check + # you are told to run before the NEXT enable ("ls moduleinfo-*") reads any of them as + # "the previous round was never rolled back", and the save block then refuses to + # record the new round and clears its variables. Delete them only after f) came back + # clean -- while any of it is unconfirmed, these three files are still the record. + if [ "$flag_dropped" = yes ]; then + rm -f moduleinfo-target.txt moduleinfo-original.json moduleinfo-expected.json + else + echo "KEEPING the rollback files: the Deployment template still carries (or could not" + echo "be confirmed free of) --exceptionNamespace, so the withdrawal is unconfirmed and" + echo "these three files are still the only record. Re-run f); delete only when it is clean." + fi + unset flag_dropped +fi +``` + + +::: + +:::details Recovering rollback state in a new terminal (as needed, before running e)) + +**Read the target from the files; do not pick it by querying again**: + +```bash +# A new terminal has none of the variables, so re-declare the wrapper here (this is the +# one place it is re-declared on purpose -- everywhere else it comes from the block at +# the top of this section). +GLOBAL_KUBECONFIG='' +KGLOBAL() { + kubectl --kubeconfig "${GLOBAL_KUBECONFIG:?fill GLOBAL_KUBECONFIG in this block first}" "$@" +} +# The saved target is the authority. Re-running a) would pick an object by querying +# again -- point it at the wrong cluster and e)'s test could pass against a DIFFERENT +# ModuleInfo whose current spec happens to equal the saved one, writing this cluster's +# original spec onto somebody else's object. +# Guarded on purpose: a missing or empty file must stop you here, not leave MODULE +# empty and let the patch below run against a name the API server fills in for you. +if [ -s moduleinfo-target.txt ] && [ -s moduleinfo-original.json ] \ + && [ -s moduleinfo-expected.json ] \ + && read -r MODULE SAVED_API SAVED_UID < moduleinfo-target.txt \ + && [ -n "$SAVED_UID" ]; then + # The read is kept OUT of the condition above and its exit status kept: an + # unreachable API server, a missing token and a deleted object all answer "empty" + # to a `2>/dev/null` query, and only one of those means "wrong cluster". + if ! live_uid=$(KGLOBAL get moduleinfo "$MODULE" \ + -o jsonpath='{.metadata.uid}' 2>&1); then + echo "could not read $MODULE ($live_uid)." + echo "NotFound means wrong cluster or a deleted object; anything else (Forbidden," + echo "connection refused, timeout) says nothing at all about what is there." + echo "Fix the kubeconfig / RBAC / connectivity and run this block again." + # Cleared AFTER the message, so the message can still name the target. + unset MODULE ORIGINAL_MODULEINFO_SPEC TEST_MODULEINFO_SPEC + elif [ "$live_uid" != "$SAVED_UID" ]; then + echo "same name, DIFFERENT object (live $live_uid vs saved $SAVED_UID): the" + echo "ModuleInfo was recreated, or this is another cluster. The saved spec belongs" + echo "to an object that no longer exists -- do a manual three-way merge instead." + unset MODULE ORIGINAL_MODULEINFO_SPEC TEST_MODULEINFO_SPEC + else + # The uid is what binds this file to an OBJECT; the URL below is only a hint about + # which cluster you were on. Same uid = same object, whatever the URL says. + ORIGINAL_MODULEINFO_SPEC=$(cat moduleinfo-original.json) + TEST_MODULEINFO_SPEC=$(cat moduleinfo-expected.json) + echo "rollback target: $MODULE (uid $SAVED_UID)" + [ "$SAVED_API" = "$(KGLOBAL config view --minify \ + -o jsonpath='{.clusters[0].cluster.server}')" ] \ + || echo "note: the API server is spelled differently than when saved ($SAVED_API) -- same object though" + fi +else + # A printed refusal is only a refusal if something downstream reads it. Nothing + # stops you from pasting e) anyway, and stale values left in this shell from an + # earlier session would let it patch the WRONG ModuleInfo -- successfully. So + # clear them: e) then stops at its state guard, which is the intended outcome. + unset MODULE ORIGINAL_MODULEINFO_SPEC TEST_MODULEINFO_SPEC + echo "the three saved files are not all here (or the target line has no uid) --" + echo "do NOT run e) from memory. Recover them from the shell that ran a)-d), or do" + echo "a manual three-way merge: read the live spec, remove only the policyExceptions" + echo "change, write it back." +fi +``` + +Re-running a) as a cross-check is fine, but **the query result must match `moduleinfo-target.txt` word for word — if it differs, stop and investigate**. **Never re-run b)** — by then the spec on the cluster is already the modified one, b) would record the "original value" as the changed value, and the rollback would be lost for good; apart from the target and these two specs, e) depends on nothing from b). + +::: +#### 3.1.2 Webhook failure policy and timeout: field semantics, read timing, and how to change tiers {#s3-1-2} + +This subsection expands checklist item 6 and is the **single source of truth** for the `failurePolicy` mechanics in this document — the tiering trade-off in [§3.7](#s3-7), the deployment check in [§4.0.7](#s4-0-7) step 1, and the control-plane observation in [§6.1.8](#s6-1-8) all point back here; mechanism revisions land only in this subsection. + +- **Field semantics**: the policy-level entry point is each policy's own `spec.webhookConfiguration.failurePolicy` / `.timeoutSeconds` (shared by all rules within one policy; allowed values `Ignore` / `Fail`, defaulting to `Fail`; timeout defaults to `10` with a 1–30 range — per the 1.15 CRD). The old top-level `spec.failurePolicy` / `spec.webhookTimeoutSeconds` are deprecated, and declaring old and new together is rejected at install time. `timeoutSeconds` is the **total budget for a single request**, not a per-rule allowance — the external calls in [§3.7](#s3-7) must fit inside that number. +- ⚠️ **Reading the generated side is timing-sensitive**: `kyverno-resource-validating-webhook-cfg` (the one that actually governs `PipelineRun` / `TaskRun` / `Pod`) is **generated dynamically by Kyverno from the installed policies** — with none of this document's policies installed its `webhooks` is empty; the `Fail/10` lines you can read at that point all belong to the webhooks of Kyverno's **own CRs** (policy / exception / cleanup / ttl). **Come back and read the generated side after installing any [§4](#s4) policy.** +- **The platform-wide override switch cannot express tiers**: for this setting the [§3.1.1](#s3-1-1) `ModuleInfo` entry point serves **platform-wide overrides only** — e.g. with `features.forceFailurePolicyIgnore.enabled` on, every policy takes effect as `Ignore` and every declared `Fail` is defeated. **Do not use it in place of the declaration in the policy body**; conversely, when checking, **reading declarations alone is not enough either**: only the generated grouping reflects the effective value after the override — a policy declaring `Fail` whose webhook lands in the `-ignore` group has been force-overridden by the platform; resolve the override before discussing tiers. Each cluster's state of this switch must be compared as a cluster-level item in baseline drift checks (the new-cluster row in [§3.6](#s3-6); same scope as [§7.3](#s7-3)). +- **Never hand-edit the `ValidatingWebhookConfiguration`**: it is an object Kyverno itself maintains (it carries `webhook.kyverno.io/managed-by=kyverno`), and a manual edit is overwritten when the per-policy grouping is recomputed. The only correct path to a different tier is the policy body's `spec.webhookConfiguration`, managed with GitOps — which is also the only entry point that can express the per-policy tiering of [§3.7](#s3-7) ("hard gates `Fail`, bookkeeping Audit may `Ignore`"). + +### 3.2 Applicable versions and dependent features {#s3-2} + +The applicable range is stated in the "Applicable versions" box at the top of this document: the criterion is **Alauda DevOps Pipelines v4.14.x and later**, not the ACP version. On earlier versions the dependent features below are incomplete, and policies may silently stop enforcing instead of raising an error — the mechanism chapters read just as well there, but do not apply this document's policy assets and examples as-is. + +The specific features depended on (your degradation checklist on older versions): + +- **Tekton**: the `tekton.dev/v1` API, object results (`enable-api-fields: beta`), the `status.pipelineSpec` write-back, `status.childReferences`, `spec.status: CancelledRunFinally`, cluster / hub / git resolvers; +- **Kyverno**: subresource match (the `kind/subresource` form), mutate-existing (`targets`), `context.apiCall`, `foreach` + `element`, PolicyException v2 (`--enablePolicyException` + `--exceptionNamespace`). + +**API group-version prerequisite**: the `match` blocks of this document's policies write `tekton.dev/v1` throughout for `PipelineRun` / `TaskRun` and their `/status` subresources, on the grounds that in the applicable versions Tekton makes `v1` the storage and served version for all three. **The one exception is `CustomRun`** (the entry-closure policies of [§4.5.4](#s4-5-4) and [§5.3](#s5-3)): Tekton defines and registers this type only in `v1beta1` — it simply does not exist in `v1` — so writing `tekton.dev/v1beta1/CustomRun` in those two places is not an omission, and must not be "tidied up to v1" in passing — change it and the rules **silently mismatch**. + +**Their `v1beta1` is usually still being served as well**: in the CRDs upstream Tekton Pipelines ships per release, `pipelineruns.tekton.dev` and `taskruns.tekton.dev` have **both `v1beta1` and `v1` at `served: true`** (only `v1` is `storage: true`) — "both versions submittable at once" is the default shape, not an unusual configuration. **But this does not constitute a bypass** — the warning below explains why (in one sentence: the request has already been converted to `v1` by the API server before it reaches Kyverno, **so do not** add `v1beta1` to `kinds` on account of this). Evidence from the upstream CRDs is not the same as the copy in your environment; after installing, it is still advisable to confirm the served versions once: + +```bash +# Which tekton.dev versions this cluster actually serves. A v1beta1 row for +# PipelineRun / TaskRun is NORMAL and does not bypass these policies -- see the +# warning below for why (the API server converts such requests to v1 first). +kubectl get crd pipelineruns.tekton.dev taskruns.tekton.dev customruns.tekton.dev \ + -o jsonpath='{range .items[*]}{.metadata.name}{": "}{range .spec.versions[*]}{.name}{"(served="}{.served}{",storage="}{.storage}{") "}{end}{"\n"}{end}' +``` + +:::warning Submitting `v1beta1` does not bypass these policies — writing `v1beta1` into `kinds` does + +**Bottom line: add nothing** — write only `tekton.dev/v1` in `kinds`. The resource webhook Kyverno generates is `matchPolicy: Equivalent` and registers only `v1`, so the API server **first converts a `v1beta1` request to `v1` and then sends it for admission**, field names already normalized (`spec.serviceAccountName` → `spec.taskRunTemplate.serviceAccountName`, `taskPodTemplate` → `podTemplate`, and so on). **Conversely, the moment `v1beta1` appears in `kinds`, that conversion no longer happens** and what goes to admission is the raw `v1beta1` object — **the field paths that moved house between the two versions** read empty from then on, the criteria that depend on them silently skip, and that is the actual allow hole. + +**Note that the failure here is "partial", not "total"** — do not expect the whole rule to collapse where you can see it: fields the two versions share at unchanged paths (`spec.taskRef` with its resolver params, `spec.params`, and so on) still read fine on a `v1beta1` object, and criteria built on them keep denying as usual. What genuinely reads empty are the ones that moved house — `spec.serviceAccountName` → `spec.taskRunTemplate.serviceAccountName`, `taskPodTemplate` → `podTemplate` and their like. So the symptom is **part of the criteria within one rule going dead**, which is harder to notice than a whole-rule skip. + +For a policy declaring only `tekton.dev/v1/PipelineRun`, the actual behavior of the two spellings is as follows (applicable versions per the table at the top of this document): + +| Submitting as `v1beta1`, with the policy's `kinds` being | Object Kyverno sees | Values the criteria read | +|---|---|---| +| `v1` only (this document's spelling) | `apiVersion: tekton.dev/v1` (`requestKind` still `v1beta1`) | Everything reads normally | +| `v1` **plus** `v1beta1` | `apiVersion: tekton.dev/v1beta1` | Shared paths read as usual; **paths renamed across versions** come back `ABSENT`, and the criteria depending on them skip | + +Self-check once after installing (the object **has content only once policies are installed**; empty output only means no policy is installed yet): + +```bash +# matchPolicy must be Equivalent, and apiVersions must NOT list v1beta1. +kubectl get validatingwebhookconfiguration kyverno-resource-validating-webhook-cfg \ + -o jsonpath='{range .webhooks[*]}{.name}{" matchPolicy="}{.matchPolicy}{" apiVersions="}{range .rules[*]}{.apiVersions}{end}{"\n"}{end}' +``` + +**`CustomRun` is unaffected by this passage**: it has only the one version, `v1beta1`, and no counterpart to be converted to; writing `tekton.dev/v1beta1/CustomRun` in [§4.5.4](#s4-5-4) / [§5.3](#s5-3) is mandatory. + +::: + + + +Of these, **only `enable-api-fields` will stop you at the very start**: the fixture Task in [§3.3](#s3-3) declares a result of `type: object`, and when this switch is not `beta` (or `alpha`), Tekton's own admission rejects `kubectl apply -f public-fixtures.yaml` outright — **the blocking point is in the shared fixtures, not in any policy**, and it is easy to misdiagnose as a Kyverno problem. So read it first (`TEKTON_NS` per [§3.1](#s3-1)): + +```bash +# Either read is fine; they must agree. Expect: beta (alpha also enables object +# results). Anything else -- including empty output -- means object results are off. +: "${TEKTON_NS:=tekton-pipelines}" # §3.1 sets it; this only covers a fresh shell +kubectl -n "$TEKTON_NS" get configmap feature-flags \ + -o jsonpath='{.data.enable-api-fields}{"\n"}' +kubectl get tektonconfig config \ + -o jsonpath='{.spec.pipeline.enable-api-fields}{"\n"}' +``` + +When it is not `beta`, **change the `TektonConfig` — do not edit the ConfigMap directly**: the operator's next reconcile reverts a hand-edited ConfigMap (the same discipline as in [§3.1.1](#s3-1-1)). On the verification environment both reads return `beta`. + +**The template → Task → result contract version matrix.** Every real profile in the Cookbook is pinned per version: different versions may carry different result contracts, and applying one across versions fails as a **silent mismatch**. + +**The table below is this document's single contract baseline**: parameter names, types, defaults, and result shapes are authoritative here. **The action items for upgrading these versions are in [§3.6](#s3-6) (which criteria are affected) and [§3.8](#s3-8) (what to run after the upgrade).** Later sections repeat, in place, the one or two rows relevant to their own criteria (so you can write policies as you read), but **when upgrading a template / Task version you only need to come back to this table and re-verify it row by row** — no hunting for the scattered notes in each section. The template and Task definitions in the matrix ship with **Alauda Artifact Hub Shim v1.0.0** (the built-in ACP hub: an Artifact Hub-compatible API consumed by Tekton's hub resolver); **later Shim versions may change these definitions** — upgrading the Shim is handled the same way as upgrading a template / Task version, per [§3.6](#s3-6) / [§3.8](#s3-8). + +| Template / scenario | Key Tasks contained (version) | Consumed result / parameter contract | +|---|---|---| +| Official `java-image-build-scan-deploy` 0.3, `python-image-build-scan-deploy` 0.3 | `sonarqube-scanner` 0.7 | `code-scan-results` (object: result/reportURL/taskID/projectID), `code-scan-metrics` | +| Same as above | `trivy-scanner` **0.6** (both templates pin this version) | `trivy-summary-metadata` (object, 11 keys, **the recommended consumption shape**) + `trivy-summary` (array, whose first element is a string mirror of the same aggregate); the gate parameters are the structured `trivyExitCode` (string, **default `"1"`**) and `trivySeverity` (array); `trivyExtraArgs` (array) carries only the remaining native arguments | +| Same as above | `deploy-or-upgrade` alias → `kubectl` 0.1 | The release switch and target come from the PipelineRun's `workloadName` / `workloadNamespace` / `kubeconfig` workspace; the resolved TaskRun carries only `args` / `script` | +| **Standalone profile** (not contained in the templates above) | `skopeo-copy` 0.1 | Parameters `srcImage` / `srcTransport` / `imageMappings` (validated in [§4.5.1](#s4-5-1)) | + +:::warning Four points that are easy to get wrong + +1. **The vulnerability gate is controlled by structured parameters — do not compare `trivyExtraArgs` literals**: the gate switches are `trivyExitCode` (string, default `"1"`) and `trivySeverity` (array), which the templates pass straight through to `trivy-scanner`'s `exitCode` / `severity`. `trivyExtraArgs` is an **array** (one complete argument per element) carrying only the remaining native arguments — the criterion should require it to be empty, not equal to some approved list (see [§4.2.5](#s4-2-5)). +2. **Parameters are passed to the Task structurally, no longer concatenated into a shell command string**: `scanType` / `scanTargets` / `severity` / `exitCode` / `extraArgs` each travel in their own slot. So the main risk on the scanning side is not command injection but "has the gate been switched off"; what still genuinely needs injection defense are the string-typed `buildExtraArgs` / `pushExtraArgs` in the same templates (this document does not govern the build/push side, see [§4.2.5](#s4-2-5)). +3. **java 0.3 and python 0.3 have different DAG shapes**: in java 0.3, `deploy-or-upgrade` only has `runAfter: [trivy-scanner]`; in python 0.3 it is `runAfter: [sonarqube-scanner, trivy-scanner]` — "the Sonar verdict dominates the release" is expressed only in the python DAG (details in [§4.3](#s4-3)). Carrying a conclusion from one over to the other gets it backwards. Their **parameter surfaces** differ too (python replaces the maven group with a `preBuildScript` / `pythonImage` group; workspaces number **12** versus java's **16**; `trivy-config` exists in both); but **the trivy-gate-related parameters are field-for-field identical on both sides** (the sonar-side parameter names are also the same; only the `sonarProperties` default differs, which does not affect the criteria), so the gate criteria in [§4.2.5](#s4-2-5) cover both templates with a single rule — only the build inputs and workspace allowlists are split per template. +4. **Neither of these pipelines contains `skopeo-copy`**: [§4.5.1](#s4-5-1) is a standalone profile for the artifact-transfer scenario. + +The Task versions in the table above defer to **whatever your environment's templates actually pin**; the field names in your policies must match the real contract of the target version. + +::: + +Degradation on older versions: fall back to the aggregate-string result only when object results are unavailable (the parsing pattern in [§4.4.2](#s4-4-2) is exactly that backstop shape) — **this is the degradation path, not the target shape**. Since 0.6, `trivy-scanner` also publishes an object result, so **consume trivy results directly via the drill-down pattern in [§4.4.1](#s4-4-1)**; [§4.4.2](#s4-4-2) is reserved for third-party / in-house Tasks that "only give you a string and cannot be changed any time soon". The reasoning is in [§2.4](#s2-4). + +### 3.3 Shared fixtures {#s3-3} + +:::info What the walkthrough leaves behind (see where things land before copy-pasting) + +- **The local working directory**: the rollback files of [§3.1.1](#s3-1-1) — `moduleinfo-target.txt` / `moduleinfo-original.json` / `moduleinfo-expected.json` (**deleted only by rollback step g); if they are still there, that round was never wrapped up**); the `cluster-scoped-ownership.tsv` of [§4.0.4](#s4-0-4); the snapshots and verdict files written along the six steps of [§5.3](#s5-3) (`gate-snapshot.txt`, `step*-verdict.txt`, `exemption-id.txt` / `exemption-uid.txt` and the like — whatever each step actually writes); the side files `*.err` used to split off stderr (**empty on success, and left in the directory all the same**); plus the YAML / JSON you copied out in each section. Cluster cleanup never touches these local files — keeping them as evidence is your call. +- **On the cluster**: the two shared namespaces this section creates, `policy-poc` / `tekton-templates`; the namespaces created by the probe block of [§5.2](#s5-2) (`proj-a` / `proj-b` / `rogue-ns` — defer to that section's creation loop); and [§5.3](#s5-3)'s `policy-exempt-runs` / `policy-exceptions` (**all of them get the walkthrough-id label only when this walkthrough created them by hand** — pre-existing ones are never labelled and never touched by cleanup). The namespaced demo objects — `PipelineRun` / `TaskRun`, the fixture `Task` / `Pipeline` objects, allowlist-type `ConfigMap`s, the `Role` / `RoleBinding` of [§4.2.2](#s4-2-2) and [§5.3](#s5-3), `PolicyException` — all live inside these namespaces. Beyond that, individual sections also create **cluster-scoped objects**: `ClusterPolicy` and the aggregated `ClusterRole` of [§4.6](#s4-6) — **deleting the namespaces does not take those along**. +- **Where cleanup lands ([§4.0.4](#s4-0-4)'s two rules)**: cluster-scoped objects are deleted one by one, by the UID in the creation-time ledger, in each section's closing "cleanup"; namespaces are deleted after checking the walkthrough-id label, cascading away everything inside ([§5.2](#s5-2) / [§5.3](#s5-3)'s namespaces are handled by their own cleanup passages; `policy-poc` / `tekton-templates` by the "final cleanup" at the end of this section). Hence **clean up as each section finishes — do not batch it up for the end**. And one thing **no cleanup passage will do for you**: the platform configuration changed per [§3.1.1](#s3-1-1) for [§5.3](#s5-3) (the PolicyException switch in the `ModuleInfo`) — after finishing [§5.3](#s5-3), go back to [§3.1.1](#s3-1-1) yourself and run its rollback step. + +::: + +Resources shared by all later chapters. First create the two namespaces: `policy-poc` hosts the business-side runs and probes, `tekton-templates` hosts the trusted template and Task definitions. + +```bash +# Record which namespaces THIS walkthrough created, so the final cleanup never +# deletes one that was already there (§4.0.4 keeps the same discipline per object). +# The marker is a LABEL on the namespace carrying an id UNIQUE TO THIS RUN. A fixed +# value like "created-here" would not do: on a shared cluster an earlier unfinished +# walkthrough may have left its own marked namespaces behind, and a fixed marker +# cannot tell the two apart -- the cleanup would delete somebody else's work. +# WRITE THE ID DOWN. Without it the cleanup refuses to delete anything, which is the +# safe direction, but you then have to compare the label by hand. +# date+PID alone is not unique across machines (same second, same PID happens); +# $RANDOM makes an accidental collision between two parallel walkthroughs unlikely. +# Any unique string works -- what matters is that it is not a constant. +WALKTHROUGH_ID=$(date +%Y%m%d-%H%M%S)-$$-$RANDOM +export WALKTHROUGH_ID +echo "walkthrough id: $WALKTHROUGH_ID" + +for ns in policy-poc tekton-templates; do + # --ignore-not-found gives three distinguishable outcomes without matching any error + # text: exit 0 + a name = it exists, exit 0 + empty = it does not, non-zero = the + # query itself failed (no RBAC, API server down) and you must not create anything. + if ! out=$(kubectl get namespace "$ns" -o name --ignore-not-found 2>&1); then + echo "$ns: CHECK FAILED ($out)" + elif [ -n "$out" ]; then + # §4.0.4's premise: every demo object lives in a namespace THIS walkthrough + # created, because cleanup is a namespace cascade. A pre-existing namespace has + # no removal path here, so going on inside it would strand everything you make. + echo "$ns: pre-existing -- STOP: this walkthrough must own its namespaces (§4.0.4)." + echo " Pick your own names and substitute them throughout, or finish the earlier" + echo " walkthrough that left this one behind." + elif ! kubectl create namespace "$ns" >/dev/null 2>&1; then + # Somebody created it between the check and the create: it is theirs, not yours. + echo "$ns: create failed -- do NOT label it, and treat it as pre-existing (STOP)" + elif ! kubectl label namespace "$ns" "policy.alauda.io/walkthrough=$WALKTHROUGH_ID" >/dev/null; then + # Created but unlabelled: the cleanup loop keys on that label and would skip it + # forever. The namespace is seconds old, empty, and certainly yours -- delete it + # by hand and re-run this loop rather than going on without the marker. + echo "$ns: created but LABEL FAILED -- the cleanup loop will not touch it." + echo " Run: kubectl delete namespace $ns # then re-run this loop" + else + echo "$ns: created" + fi +done +``` + +The heart of the fixtures is a **SonarQube Scanner 0.7 contract fixture** (`policy-demo-scanner`). It is not a real scanner, but it **fully mirrors the 0.7 external contract surface this document depends on**, so every policy expression the Cookbook writes against that contract holds on the real Task as well: + +- `enableScanQualityGate` / `enableAnalyzeQualityGate` are both `string` with default `"true"`; +- `analyzeQualityGateRules` is an `array` with default `[]`; `sonarBranchName` is a `string`, default empty; +- `code-scan-results` is an object result declaring only `result` / `reportURL` / `taskID` / `projectID`; the real schema of all four properties is the empty map `{}`, with no additional `type: string`; +- `code-scan-metrics` is an object result whose schema declares only the property the real 0.7 always has, `bugs: {}` (the real Task can collect more fields dynamically via its `metrics` parameter, but **a policy must not assume undeclared fields necessarily exist**); +- `code-scan-results.result` uses the real value range `Succeeded` / `Failed` / `Skipped` / `Canceled`. + +The fixture additionally uses `demoCoverage` / `demoBugs` / `demoDelaySeconds` / `demoResult` to drive repeatable pass / fail / cancellation and four-value-range audit tests, and the template layer adds a `demoSkipScan` (default `"false"`; set to `"true"` it skips `scan` entirely via `when`, letting [§4.1.5](#s4-1-5) reproduce "the gate opted out"). These `demo*` parameters are **explicitly not a productized Task contract** — do not keep them when substituting a real Task. There is no separate gate task: the fixture failing by itself is what blocks the `release` behind it. + +:::warning Replace the placeholder + +Replace `` in the fixtures with a registry prefix from which your environment can pull busybox. In production, pin step images to a digest — otherwise anyone with registry push permission can swap out the scanning logic outright (contract 1, [§2.3](#s2-3)). + +**If you do not know what to put there, first look at where the platform itself pulls from** — in an air-gapped environment this is the easiest starting point: + +```bash +# Where the platform itself pulls from. Output shape: [:port]//... +: "${TEKTON_NS:=tekton-pipelines}" # §3.1 sets it; this only covers a fresh shell +kubectl -n "$TEKTON_NS" get deploy tekton-pipelines-controller \ + -o jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' + +# Wider sample: every distinct prefix in use in that namespace. +kubectl -n "$TEKTON_NS" get pods \ + -o jsonpath='{range .items[*]}{range .spec.containers[*]}{.image}{"\n"}{end}{end}' \ + | sed 's#/[^/]*$##' | sort -u +``` + +⚠️ **These are candidates, not the answer**: that the platform namespace can pull does not mean `policy-poc` can too (pull credentials are granted per namespace), and both commands print **platform image** paths, which may not carry a `busybox` at all. **The only verification that counts is the fixture actually running** — after building the fixtures per [§3.3](#s3-3), run `demo-run-pass`; if the Pod will not start, look for the `ImagePullBackOff` / `ErrImagePull` events in `kubectl -n policy-poc describe pod`. That is not a Tekton or Kyverno problem — the prefix is wrong or the credentials are missing. + +::: + +:::details Complete shared-fixture YAML (Task, templates, negative template — copy-paste ready) + +One YAML file contains five objects; later chapters reference them as needed: + +- `Task/policy-demo-scanner` (`tekton-templates`) — the contract fixture itself; +- `Pipeline/gated-build` — the standard governed template: scan → release, finally does notification only; +- `Pipeline/gated-build-with-prep` — used by [§4.2.2](#s4-2-2) to prove "work already completed before scan + RunFinally cancellation + finally still executes"; +- `Task/policy-demo-scanner` (`policy-poc`) — the **same-name, different-source** Task, [§4.6.2](#s4-6-2)'s definition-drift target; +- `Pipeline/gated-build-rogue` — the negative template: the `scan` alias keeps the trusted name but resolves from `policy-poc`. + +```yaml +apiVersion: tekton.dev/v1 +kind: Task +metadata: + name: policy-demo-scanner + namespace: tekton-templates +spec: + # This fixture mirrors the sonarqube-scanner 0.7 contract surface consumed by + # this document. Parameters prefixed with demo are test drivers, not product + # task parameters. + params: + - name: enableScanQualityGate + type: string + default: "true" + - name: enableAnalyzeQualityGate + type: string + default: "true" + - name: analyzeQualityGateRules + type: array + default: [] + - name: sonarBranchName + type: string + default: "" + - name: demoCoverage + type: string + default: "85" + - name: demoBugs + type: string + default: "0" + - name: demoDelaySeconds + type: string + default: "0" + - name: demoResult + type: string + default: Auto + results: + - name: code-scan-results + description: quality-gate verdict object (result/reportURL/taskID/projectID) + type: object + properties: + # Empty property schemas exactly match the catalog 0.7 Task. + result: {} + reportURL: {} + taskID: {} + projectID: {} + - name: code-scan-metrics + description: metrics collected after the scan; real 0.7 always declares bugs + type: object + properties: + bugs: {} + steps: + - name: scan + # pin to a digest in production so a registry pusher cannot swap the scan logic + image: /busybox:latest + # params passed via env (NOT text-substituted into the script body) to avoid + # Tekton parameter injection; the script reads shell variables only + env: + - name: ENABLE_SCAN_QG + value: $(params.enableScanQualityGate) + - name: ENABLE_ANALYZE_QG + value: $(params.enableAnalyzeQualityGate) + - name: DEMO_COVERAGE + value: $(params.demoCoverage) + - name: BUGS + value: $(params.demoBugs) + - name: DEMO_DELAY_SECONDS + value: $(params.demoDelaySeconds) + - name: DEMO_RESULT + value: $(params.demoResult) + script: | + #!/bin/sh + set -eu + case "$ENABLE_SCAN_QG" in true|false) ;; *) exit 1 ;; esac + case "$ENABLE_ANALYZE_QG" in true|false) ;; *) exit 1 ;; esac + case "$DEMO_COVERAGE" in ''|*[!0-9]*) exit 1 ;; esac + case "$BUGS" in ''|*[!0-9]*) exit 1 ;; esac + case "$DEMO_DELAY_SECONDS" in ''|*[!0-9]*) exit 1 ;; esac + # The numeric-looking "1" is an intentional invalid-contract probe. It + # does not extend the scanner 0.7 result enum. + case "$DEMO_RESULT" in Auto|Succeeded|Failed|Skipped|Canceled|1) ;; *) exit 1 ;; esac + [ "$DEMO_DELAY_SECONDS" -le 300 ] || exit 1 + sleep "$DEMO_DELAY_SECONDS" + + RESULT="$DEMO_RESULT" + if [ "$RESULT" = Auto ]; then + RESULT=Succeeded + if [ "$DEMO_COVERAGE" -lt 80 ]; then RESULT=Failed; fi + fi + + # The fixture self-gates whenever either 0.7 quality-gate phase is enabled. + # Setting both switches false is reserved for the explicit negative fixture + # that proves §4.2 rejects a fully disabled gate. + FAIL=0 + if [ "$RESULT" != "Succeeded" ] && { [ "$ENABLE_SCAN_QG" = "true" ] || [ "$ENABLE_ANALYZE_QG" = "true" ]; }; then + FAIL=1 + fi + + printf '{"result":"%s","reportURL":"https://sonar.example/dashboard?id=demo","taskID":"demo-task-001","projectID":"demo-proj"}' "$RESULT" > "$(results.code-scan-results.path)" + printf '{"bugs":"%s"}' "$BUGS" > "$(results.code-scan-metrics.path)" + echo "scan: demoCoverage=$DEMO_COVERAGE result=$RESULT fail=$FAIL" + if [ "$FAIL" = 1 ]; then + echo "task-side quality gate FAILED"; exit 1 + fi + echo "quality gate not enforced or passed" +--- +apiVersion: tekton.dev/v1 +kind: Pipeline +metadata: + name: gated-build + namespace: tekton-templates +spec: + params: + - name: coverage + type: string + default: "85" + - name: enableScanQualityGate + type: string + default: "true" + - name: enableAnalyzeQualityGate + type: string + default: "true" + - name: analyzeQualityGateRules + type: array + default: [] + - name: demoDelaySeconds + type: string + default: "0" + - name: demoResult + type: string + default: Auto + # §4.1.5 needs a run where the gate is skipped BY CONFIGURATION. The default keeps + # `scan` running, so every other section behaves exactly as before; passing "true" + # is the opt-out that section's Audit is supposed to catch. + - name: demoSkipScan + type: string + default: "false" + tasks: + - name: scan + # the scanner self-gates; failing it blocks `release` (no separate gate task) + when: + - input: $(params.demoSkipScan) + operator: notin + values: + - "true" + taskRef: + resolver: cluster + params: + - name: kind + value: task + - name: name + value: policy-demo-scanner + - name: namespace + value: tekton-templates + params: + - name: demoCoverage + value: $(params.coverage) + - name: enableScanQualityGate + value: $(params.enableScanQualityGate) + - name: enableAnalyzeQualityGate + value: $(params.enableAnalyzeQualityGate) + - name: analyzeQualityGateRules + value: + - $(params.analyzeQualityGateRules[*]) + - name: demoDelaySeconds + value: $(params.demoDelaySeconds) + - name: demoResult + value: $(params.demoResult) + - name: release + runAfter: + - scan + taskSpec: + steps: + - name: release + image: /busybox:latest + script: | + #!/bin/sh + echo "releasing..." + finally: + - name: notify + taskSpec: + steps: + - name: notify + image: /busybox:latest + script: | + #!/bin/sh + echo "notify: run finished" +--- +# 4.2.2 uses this profile to prove that work completed before `scan` can be +# followed by a RunFinally cancellation and still execute the final notifier. +apiVersion: tekton.dev/v1 +kind: Pipeline +metadata: + name: gated-build-with-prep + namespace: tekton-templates +spec: + params: + - name: coverage + type: string + default: "85" + - name: enableScanQualityGate + type: string + default: "true" + - name: enableAnalyzeQualityGate + type: string + default: "true" + - name: demoDelaySeconds + type: string + default: "0" + tasks: + - name: prep + taskSpec: + steps: + - name: prep + image: /busybox:latest + script: | + #!/bin/sh + echo "prep completed" + - name: scan + runAfter: + - prep + taskRef: + resolver: cluster + params: + - name: kind + value: task + - name: name + value: policy-demo-scanner + - name: namespace + value: tekton-templates + params: + - name: demoCoverage + value: $(params.coverage) + - name: enableScanQualityGate + value: $(params.enableScanQualityGate) + - name: enableAnalyzeQualityGate + value: $(params.enableAnalyzeQualityGate) + - name: demoDelaySeconds + value: $(params.demoDelaySeconds) + - name: release + runAfter: + - scan + taskSpec: + steps: + - name: release + image: /busybox:latest + script: | + #!/bin/sh + echo "release completed" + finally: + - name: notify + taskSpec: + steps: + - name: notify + image: /busybox:latest + script: | + #!/bin/sh + echo "finally notification completed" +--- +# 4.6.2 uses a same-name Task from another namespace as the resolved-definition +# drift target. The name still looks trusted, but the complete source does not. +apiVersion: tekton.dev/v1 +kind: Task +metadata: + name: policy-demo-scanner + namespace: policy-poc +spec: + steps: + - name: wait + image: /busybox:latest + script: | + #!/bin/sh + sleep 30 +--- +# Negative fixture for 4.6.2: the scan alias keeps the trusted Task name but +# resolves it from policy-poc instead of tekton-templates. +apiVersion: tekton.dev/v1 +kind: Pipeline +metadata: + name: gated-build-rogue + namespace: tekton-templates +spec: + tasks: + - name: prep + taskSpec: + steps: + - name: prep + image: /busybox:latest + script: | + #!/bin/sh + sleep 30 + - name: scan + runAfter: + - prep + taskRef: + resolver: cluster + params: + - name: kind + value: task + - name: name + value: policy-demo-scanner + - name: namespace + value: policy-poc + - name: release + runAfter: + - scan + taskSpec: + steps: + - name: release + image: /busybox:latest + script: | + #!/bin/sh + echo "release must not complete after self-cancel" + finally: + - name: notify + taskSpec: + steps: + - name: notify + image: /busybox:latest + script: | + #!/bin/sh + echo "finally notification completed" +``` + +::: + +Save the YAML above as `public-fixtures.yaml` (with `` replaced) and create it on the target cluster — **every later section's probes assume these five objects exist**: + +```bash +# If either namespace pre-existed, check for same-named objects FIRST: `apply` would +# overwrite somebody else's Task or Pipeline with this document's fixture, and the +# cleanup at the end of §3.3 would then delete what you overwrote (§4.0.4). +# Fail-closed on purpose: a query that ERRORS (no RBAC, API server hiccup, CRD not +# installed) must stop you too -- silencing stderr and reading "empty" as "absent" is +# how a guard turns into decoration. +FIXTURES_SAFE=yes +# Heredoc + read, not `set -- $spec`: zsh keeps an unquoted expansion as ONE word, so +# a splitting-based loop pasted into an interactive zsh queries an empty resource type. +# `read` splits on IFS in bash and zsh alike, and the redirect (no pipe) keeps the +# FIXTURES_SAFE assignment in the current shell. +while read -r ns kind name; do + # Same three-way outcome as the namespace check: exists / absent / query failed -- + # decided by the exit code and whether anything was printed, not by error text. + if ! out=$(kubectl get "$kind" -n "$ns" "$name" -o name --ignore-not-found 2>&1); then + echo "CHECK FAILED for $ns/$kind/$name: $out"; FIXTURES_SAFE=no + elif [ -n "$out" ]; then + echo "COLLISION: $ns/$out already exists -- stop, and use namespaces of your own"; FIXTURES_SAFE=no + fi +done <<'FIXTURE_LIST' +tekton-templates task policy-demo-scanner +policy-poc task policy-demo-scanner +tekton-templates pipeline gated-build +tekton-templates pipeline gated-build-with-prep +tekton-templates pipeline gated-build-rogue +FIXTURE_LIST +echo "FIXTURES_SAFE=$FIXTURES_SAFE" +# Expect FIXTURES_SAFE=yes and nothing else. COLLISION means the name is taken (change +# the two namespace names in the block above and in every later probe). CHECK FAILED +# means you do not know yet -- fix that query before applying anything. +``` + +**This probe is written for a first install: it cannot tell "somebody else's same-named object" from "the same batch of fixtures you built last time"** — both report `COLLISION`. Hence: + +- **First install**: the probe should print nothing; if it does, switch namespaces as instructed above. +- **Re-running the same walkthrough**: those five objects are the ones you built last time. Verify they really are yours (`kubectl get -o yaml` — is the content this fixture, and is the namespace's walkthrough label your previous id), then **set `FIXTURES_SAFE=yes` by hand** and run the next block — `apply` on the same YAML is idempotent. Or delete last time's batch first and start over. +- **For "never overwrite"**: replace the next block's `kubectl apply -f` with `kubectl create -f`; when a same-named object exists it fails with `AlreadyExists` instead of overwriting. There is still a window between the probe and the creation (someone may create a same-named object exactly in between) — the value of `create` is precisely that in that case it fails rather than silently overwrites. + +**The probe and the apply below are split into two blocks on purpose**: in a single block, pasting the whole thing would run `apply` regardless, reducing the probe to an after-the-fact notice. The next block checks `FIXTURES_SAFE` once more — both guards exist because **splitting only stops "pasted along the way"; it cannot stop "skipped the previous block and pasted this one"**: + +```bash +# Refuse to run if the check above did not pass (or was never run at all). +if [ "${FIXTURES_SAFE:-no}" != yes ]; then + echo "run the collision check above first, and fix what it reported" +else + + # `apply` on purpose, so that re-running the whole walkthrough is idempotent. It is + # NOT collision-proof: the check above and this line are separate requests, and a + # same-named object created in between would be overwritten rather than reported. On + # a shared cluster prefer `kubectl create -f public-fixtures.yaml` -- it fails with + # AlreadyExists instead, which is the answer you want there (see the bullet above). + kubectl apply -f public-fixtures.yaml + # Expect five objects created. Verify all five before going on: a missing template + # makes the cluster resolver fail later, and the run will report a resolution error + # instead of the gate behaviour this document describes. + kubectl get task -n tekton-templates policy-demo-scanner + kubectl get task -n policy-poc policy-demo-scanner + kubectl get pipeline -n tekton-templates gated-build gated-build-with-prep gated-build-rogue + +fi +``` + +If any line reports `NotFound`, go back to that YAML and find the corresponding object — the most common causes are an unreplaced `` failing the whole apply midway, or the two namespaces not yet created (the loop at the start of this section). + +⚠️ **The two shared namespaces must have been created by this walkthrough** ([§4.0.4](#s4-0-4)'s prerequisite discipline — cleanup relies on the namespace-deletion cascade, and the cascade presumes nothing of anybody else's is inside). When the creation loop above prints `pre-existing`, someone on this cluster already occupies that namespace name — **do not demo inside it**: substitute your own names for `policy-poc` / `tekton-templates` throughout (and run the final cleanup under your names too); or first confirm it is what your own previous walkthrough left behind (the walkthrough id in the label is the one you wrote down), wrap that round up, and start again. + +This template embodies the template-side responsibilities of the [§2.3](#s2-3) contracts: the gate is carried by the scanner itself (contract 3 "must-run" + contract 4 "consuming the real effective values" cohere inside one task), `release` is ordered after the scanner (contract 5, DAG dominance), and finally does notification only (contract 6). + +The standard business-side usage references the template through the cluster resolver: + +```yaml +apiVersion: tekton.dev/v1 +kind: PipelineRun +metadata: + name: demo-run-pass + namespace: policy-poc +spec: + pipelineRef: + resolver: cluster + params: + - name: kind + value: pipeline + - name: name + value: gated-build + - name: namespace + value: tekton-templates + params: + - name: coverage + value: "85" +``` + +Save it as `demo-run-pass.yaml` and create it (on the target cluster; the observation commands below need it to really exist): + +```bash +kubectl create -n policy-poc -f demo-run-pass.yaml +kubectl wait -n policy-poc pipelinerun/demo-run-pass \ + --for=condition=Succeeded --timeout=5m +``` + +The `code-scan-results.result` in the last column of the table below is **a Tekton task result produced by the scan task** — neither a Pipeline-level field nor a Kyverno concept. Get it straight first; the "result-type" policies of the coming chapters all hinge on it: + +- **Who produces it**: the `scan` task (`policy-demo-scanner` in the fixture) writes a piece of JSON into `$(results.code-scan-results.path)` in its step script; +- **Where it lands**: Tekton records it on `status.results` of **the TaskRun corresponding to that task**. The PipelineRun does not hold this data itself — look at the child TaskRun ([§2.1](#s2-1) observation point 6); +- **What `.result` is**: this result is of type `object` ([§2.4](#s2-4)), and its `result` field is **the scan verdict**, with the real value range `Succeeded` / `Failed` / `Skipped` / `Canceled`; +- **Why this document keeps returning to it**: the result audit of [§4.4](#s4-4) and the automatic cancellation of [§4.6.1](#s4-6-1) both read this field. The table lists it so you can confirm the verdicts your fixture environment produces match expectations. + +To see it with your own eyes (using the `demo-run-pass` above): + +```bash +# The verdict lives on the scan TaskRun, not on the PipelineRun. +# childReferences is the API-level mapping from pipeline task name to TaskRun name -- +# unlike the tekton.dev/pipelineTask label, it cannot be overridden by the submitter. +TR=$(kubectl get pipelinerun -n policy-poc demo-run-pass -o json \ + | jq -r '.status.childReferences[] | select(.pipelineTaskName == "scan") | .name') +kubectl get taskrun -n policy-poc "$TR" -o jsonpath='{.status.results}{"\n"}' +``` + +Three runs cover the gate's three shapes and double as an environment-readiness check: + +| run | Input | scan | release | finally notify | Scan verdict (scan's task result `code-scan-results.result`) | +|---|---|---|---|---|---| +| pass | `coverage=85` | ✅ succeeds | ✅ runs | ✅ runs | `Succeeded` | +| gate-fail | `coverage=30` (both gate switches `true`) | ❌ fails itself | ⏭ skipped (reason `PipelineRun was stopping`) | ✅ runs | `Failed` | +| gates-off | `coverage=30` + both gate switches `false` | ✅ fixture succeeds | ✅ runs (**the deliberately exposed bypass**) | ✅ runs | `Failed` | + +The latter two runs differ from `demo-run-pass` **only in params** (beyond the template identity, only `metadata.name` and the params differ). Save as `demo-runs-negative.yaml`: + +```yaml +# gate-fail: coverage below the bar; both gate switches keep the template default "true" +apiVersion: tekton.dev/v1 +kind: PipelineRun +metadata: + name: demo-run-gate-fail + namespace: policy-poc +spec: + pipelineRef: + resolver: cluster + params: + - name: kind + value: pipeline + - name: name + value: gated-build + - name: namespace + value: tekton-templates + params: + - name: coverage + value: "30" +--- +# gates-off: below the bar as well, but both gate switches explicitly off (the deliberately exposed bypass) +apiVersion: tekton.dev/v1 +kind: PipelineRun +metadata: + name: demo-run-gates-off + namespace: policy-poc +spec: + pipelineRef: + resolver: cluster + params: + - name: kind + value: pipeline + - name: name + value: gated-build + - name: namespace + value: tekton-templates + params: + - name: coverage + value: "30" + - name: enableScanQualityGate + value: "false" + - name: enableAnalyzeQualityGate + value: "false" +``` + +Create both together and wait for their terminal states — **note the two end opposite**, so the wait conditions are opposite too: + +```bash +kubectl create -n policy-poc -f demo-runs-negative.yaml + +# gate-fail must end NOT Succeeded (the scanner fails itself and stops the run) +kubectl wait -n policy-poc pipelinerun/demo-run-gate-fail \ + --for=condition=Succeeded=false --timeout=5m +# gates-off must end Succeeded -- that "green" run is the exposed bypass, not a pass +kubectl wait -n policy-poc pipelinerun/demo-run-gates-off \ + --for=condition=Succeeded --timeout=5m + +# Then read the scan verdict of each: expect Failed for BOTH (the table's last column) +for run in demo-run-gate-fail demo-run-gates-off; do + TR=$(kubectl get pipelinerun -n policy-poc "$run" -o json \ + | jq -r '.status.childReferences[] | select(.pipelineTaskName == "scan") | .name') + printf '%s -> %s\n' "$run" \ + "$(kubectl get taskrun -n policy-poc "$TR" -o jsonpath='{.status.results}')" +done +``` + +A `wait` that times out rather than returning promptly usually means the run is stuck on resolution (the template was never built — go back to the five-object verification above); when either run's terminal state disagrees with the table, first confirm the fixture's `demo*` parameters have not been altered. + +The first two rows are the hard gate's baseline shape (the scanner fails itself → `release` is skipped → finally runs anyway — exactly the second row of the comparison table in [§2.3](#s2-3)). + +The third row is a **fixture-only negative test**, and its `Failed` is not a typo — two things are deliberately pulled apart in this row: the **verdict** still computes to `Failed` (`demoResult` defaults to `Auto`; coverage 30 < 80 judges `Failed`), but the fixture converts a failing verdict into `exit 1` only when **at least one gate switch is `true`**. With both switches off, scan exits successfully and `release` runs — while the scan TaskRun's `code-scan-results.result` says `Failed` in plain sight. **The verdict says non-compliant, yet the pipeline is green all the way** — exactly the harm shape of "the gate switches were turned off", and exactly why [§4.2.1](#s4-2-1) must block non-compliant switch values at TaskRun CREATE: by the time the result is out, the release has already run. This row describes only the fixture's deterministic behavior; it does not claim that a real SonarQube service necessarily produces the same combination with both gates disabled. + +#### Final cleanup (after walking the whole document) + +Each section's closing "cleanup" deletes only that section's own policies and run objects; **these two shared namespaces are deleted separately after the whole document is done** — otherwise the fixtures stay on the cluster forever: + +```bash +# First LOOK: which namespaces carry a walkthrough marker at all, and whose? +kubectl get namespace -l policy.alauda.io/walkthrough \ + -o custom-columns='NAME:.metadata.name,WALKTHROUGH:.metadata.labels.policy\.alauda\.io/walkthrough' +# Expect your own id (printed when you created them) on the namespaces you created. +# A DIFFERENT id belongs to another run of this document -- leave it alone and go ask +# its owner. +# +# Then delete BY NAME, with your own id as the precondition. Deliberately not +# `kubectl delete namespace -l