Skip to content

ci: add sigstore e2e test suite#2062

Open
stealthybox wants to merge 2 commits into
mainfrom
sigstore-testing
Open

ci: add sigstore e2e test suite#2062
stealthybox wants to merge 2 commits into
mainfrom
sigstore-testing

Conversation

@stealthybox
Copy link
Copy Markdown
Member

Add a parallel CI job that deploys a local sigstore stack and runs
verification tests against it.

hack infra:

  • kind cluster with zot (OCI 1.1 referrers API) and registry:2 (tag fallback)
  • sigstore scaffold Helm chart (fulcio, rekor, ctlog, trillian)
  • Scripts for setup, build, and teardown

tests:

  • v2/v3 key-pair signing with secretRef
  • v2/v3 keyless signing with trustedRootSecretRef
  • v3 key-pair with tlog entry
  • Combined secretRef + trustedRootSecretRef
  • Registry auth + verify
  • v3 bundles on registry:2 (referrers tag fallback)
  • Sad paths: wrong key, wrong identity, wrong rekor key in trusted root

Runs concurrently with the existing kind-linux-amd64 job -- hopefully i've set this up right

@stealthybox stealthybox force-pushed the sigstore-transport branch 2 times, most recently from 211fb0a to ffa18cc Compare May 26, 2026 16:04
@stealthybox stealthybox force-pushed the sigstore-transport branch from ffa18cc to b00f7ba Compare June 4, 2026 05:19
@stealthybox stealthybox marked this pull request as draft June 4, 2026 06:41
@stealthybox stealthybox force-pushed the sigstore-testing branch 3 times, most recently from 9599840 to c030efd Compare June 4, 2026 07:19
Copy link
Copy Markdown
Member

@matheuscscp matheuscscp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@stealthybox stealthybox force-pushed the sigstore-transport branch from b00f7ba to 34c7c9c Compare June 4, 2026 08:00
@stealthybox stealthybox marked this pull request as ready for review June 4, 2026 08:26
Base automatically changed from sigstore-transport to main June 4, 2026 08:37
@stefanprodan stefanprodan added area/ci CI related issues and pull requests area/testing Testing related issues and pull requests labels Jun 4, 2026
Scripts and testdata for running cosign verification tests against a
local sigstore stack on kind. Uses zot (referrers API) and registry:2
(tag fallback) with the scaffold Helm chart.

Covers v2/v3 key-pair, v2/v3 keyless with trustedRootSecretRef, tlog,
combined refs, wrong key/identity/rekor material, and registry auth.

Signed-off-by: leigh capili <leigh@null.net>
New sigstore-linux-amd64 job deploys a local sigstore stack and runs
the verification test suite concurrently with kind-linux-amd64.

Signed-off-by: leigh capili <leigh@null.net>
@stealthybox
Copy link
Copy Markdown
Member Author

stealthybox commented Jun 4, 2026

rebased, if this is green, I don't see a problem merging it
edit: it's green ✅

I am extending this significantly to test some auto policy features based off of what's in the the trustbundle.
In other words, we'll need more changes to this harness, but it does already cover the custom trust bundle fairly well.

The idea is to bridge the gap between our existing heuristic/implicit behaviors for Fulcio/Rekor to also support TSA (which GitHub uses for immutable releases)

Once we uniformly implement our existing auto behaviors, Flux users with private/hybrid/third-party sigstore stacks will be able to combine whichever key and trust materials they desire.
This can be done without adding too many fields, however, our end goal should be to add enough configuration that these auto-heuristic behaviors can be made fully deterministic and client-driven, since that is what detailed by the sigstore spec.

Realistically, people will be dynamically loading trustBundles/TUF-roots into these SecretRefs which could change how the verify policy evaluates.

Flux already supports overriding the global TUF root, so that is a considerable part of this as well. Also, whether we choose to support or not-support per-resource TUF roots.

Lastly, there's the topic of TLS/mTLS connections to verification endpoints that are not the OCI registry. We may need to plumb some additional cert fields in. Alternatively, we could load them from a separate, optional data field in the TrustBundle Secret? Not sure if that would violate the spirit of the verification material or not.

@stefanprodan stefanprodan added the hold Issues and pull requests put on hold label Jun 4, 2026
Comment on lines +46 to +55
- name: Setup cosign
env:
COSIGN_VERSION: v3.0.6
COSIGN_SHA256: c956e5dfcac53d52bcf058360d579472f0c1d2d9b69f55209e256fe7783f4c74
run: |
curl -fsSL "https://github.com/sigstore/cosign/releases/download/${COSIGN_VERSION}/cosign-linux-amd64" -o /usr/local/bin/cosign
echo "${COSIGN_SHA256} /usr/local/bin/cosign" | sha256sum -c -
chmod +x /usr/local/bin/cosign
- name: Setup Flux CLI
run: curl -fsSL https://fluxcd.io/install.sh | bash
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should be GH Actions so that dependabot can update them.

Comment on lines +62 to +74
kubectl -n fulcio-system get cm fulcio-server-config -o json | \
python3 -c "
import json, sys
cm = json.load(sys.stdin)
config = json.loads(cm['data']['config.json'])
config['OIDCIssuers']['https://kubernetes.default.svc.cluster.local'] = {
'IssuerURL': 'https://kubernetes.default.svc.cluster.local',
'ClientID': 'sigstore',
'Type': 'kubernetes'
}
cm['data']['config.json'] = json.dumps(config, indent=2)
json.dump(cm, sys.stdout)
" | kubectl apply -f -
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's rewrite this with yq, no need to introduce python3 in our test suite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/ci CI related issues and pull requests area/testing Testing related issues and pull requests hold Issues and pull requests put on hold

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants