Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
8 changes: 4 additions & 4 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
# resources is thousands of lines of emitted Go that nobody should read line by
# line -- the reviewable change is the blueprint, so point reviewers there.
blueprints/** linguist-generated=true
probe-evidence/** linguist-generated=true
openapi-specs/** linguist-generated=true
interop-specs/** linguist-generated=true
recordings/** linguist-generated=true
openapi/** linguist-generated=true
specs/** linguist-generated=true
pilot/*/internal/services/** linguist-generated=true

# Pinned specifications are upstream documents, not ours to review line by line.
openapi-specs/**/api.yaml linguist-generated=true
openapi/**/api.yaml linguist-generated=true
4 changes: 2 additions & 2 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,6 @@ cmd/** @ShocOne
# blueprint or a template -- a diff here on its own usually means something was
# hand-edited.
blueprints/** @ShocOne
probe-evidence/** @ShocOne
openapi-specs/** @ShocOne
recordings/** @ShocOne
openapi/** @ShocOne
pilot/** @ShocOne
2 changes: 1 addition & 1 deletion .github/linters/.checkov.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,4 @@ skip-path:
# See the note in trivy.yaml. The pinned ThousandEyes document's
# client-certificate example is a sample PEM, not a credential, and the document is
# committed verbatim so it cannot be edited to satisfy a scanner.
- openapi-specs
- openapi
2 changes: 1 addition & 1 deletion .github/linters/.gitleaks.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,4 @@ Scoped to this path only. Every other file in the repository is still scanned,
including the probe cassettes that a later phase will commit -- those carry real
recorded HTTP traffic and must keep being checked.
"""
paths = ['''^openapi-specs/''']
paths = ['''^openapi/''']
2 changes: 1 addition & 1 deletion .github/linters/.tflint.hcl
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# are no reusable modules, and that distinction is what the one disabled rule below
# turns on. Trivy and tflint both walk the tree rather than reading super-linter's
# changed-file list, so FILTER_REGEX_EXCLUDE's `^pilot/.*` does not reach them --
# the same reason openapi-specs is handled by trivy's scan.skip-dirs.
# the same reason openapi is handled by trivy's scan.skip-dirs.

config {
# Nothing here is called as a module, so there are no variables to infer.
Expand Down
2 changes: 1 addition & 1 deletion .github/linters/.trivyignore.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
#
# Referenced explicitly from trivy.yaml. Trivy's filesystem scanner walks the tree
# rather than reading super-linter's changed-file list, which is why an exception
# has to live here and not in FILTER_REGEX_EXCLUDE -- the same reason openapi-specs
# has to live here and not in FILTER_REGEX_EXCLUDE -- the same reason openapi
# is handled by scan.skip-dirs.

vulnerabilities:
Expand Down
4 changes: 2 additions & 2 deletions .github/linters/trivy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,9 @@ scan:
# that as a leaked key. It is illustrative text in a vendor's documentation, and
# the only way to satisfy the scanner would be to edit a pinned artefact -- which
# would defeat the point of pinning it, and is why VALIDATE_OPENAPI is off too.
- openapi-specs
- openapi

# Deliberately NOT skipped: probe-evidence. Those cassettes are our own recorded
# Deliberately NOT skipped: recordings. Those cassettes are our own recorded
# HTTP traffic against a live tenant, so a secret appearing there is exactly the
# thing worth being told about. The Authorization header is absent by construction
# -- it is not on the recorder's allow list -- but that is a property to keep
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/acceptance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ jobs:
# Deliberately no sweep step, and the reason is worth recording rather than leaving as an
# apparent omission.
#
# `probe -mode sweep` exists, but it sweeps the *prober's* ledger -- the objects it recorded
# `probe sweep` exists, but it sweeps the *prober's* ledger -- the objects it recorded
# creating before it issued each request. It knows nothing about objects an acceptance test
# created, because those are Terraform's, tracked in its state and destroyed by the test
# framework at the end of the run. Invoking it here would clean up nothing and look like it
Expand Down
52 changes: 27 additions & 25 deletions .github/workflows/codegen-verify.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Fails when the committed generated code no longer matches the blueprints it was
# generated from.
#
# Everything under pilot/ is produced by cmd/tfpluginframeworkgen. Nothing stops a change
# Everything under pilot/ is produced by cmd/tfpfgen. Nothing stops a change
# to the generator, a template or a blueprint from landing without the output being
# regenerated, and stale generated code compiles perfectly well -- so the drift
# would be invisible. This regenerates and fails if anything moved.
Expand All @@ -16,8 +16,8 @@ on:
- "blueprints/**"
- "cmd/**"
- "internal/**"
- "interop-specs/**"
- "probe-evidence/**"
- "specs/**"
- "recordings/**"
- "pilot/**"
- "go.mod"
- "go.sum"
Expand Down Expand Up @@ -66,12 +66,12 @@ jobs:
- name: Regenerate
run: |
echo "::group::🔁 Regenerating the pilot provider"
go run ./cmd/tfpluginframeworkgen emit \
go run ./cmd/tfpfgen provider generate \
-blueprint blueprints/thousandeyes \
-out pilot/thousandeyes
echo "::endgroup::"

# git diff rather than `tfpluginframeworkgen verify`, deliberately. verify compares
# git diff rather than `tfpfgen provider generate -check`, deliberately. verify compares
# the files the blueprints produce against disk, so it cannot see a stale file
# the blueprints no longer produce. Diffing the worktree after a real
# regeneration catches that too.
Expand All @@ -82,14 +82,14 @@ jobs:
exit 0
fi

echo "::error::Generated code is out of date. Run 'go run ./cmd/tfpluginframeworkgen emit -blueprint blueprints/thousandeyes -out pilot/thousandeyes' and commit the result."
echo "::error::Generated code is out of date. Run 'go run ./cmd/tfpfgen provider generate -blueprint blueprints/thousandeyes -out pilot/thousandeyes' and commit the result."
{
echo "### ❌ Generated code is out of date"
echo ""
echo "Run this locally and commit the result:"
echo ""
echo '```bash'
echo 'go run ./cmd/tfpluginframeworkgen emit -blueprint blueprints/thousandeyes -out pilot/thousandeyes'
echo 'go run ./cmd/tfpfgen provider generate -blueprint blueprints/thousandeyes -out pilot/thousandeyes'
echo '```'
echo ""
echo "<details><summary>Files that differ</summary>"
Expand Down Expand Up @@ -196,10 +196,13 @@ jobs:
- name: Verify bindings against the pinned SDK
run: |
echo "::group::🔗 Bindings"
go run ./cmd/tfpluginframeworkgen bindings \
go run ./cmd/tfpfgen bindings check \
-blueprint blueprints/thousandeyes \
-module pilot/thousandeyes
go run ./cmd/tfpfgen bindings facts \
-blueprint blueprints/thousandeyes \
-module pilot/thousandeyes \
-facts-check blueprints/thousandeyes/static.facts.json
-out blueprints/thousandeyes/static.facts.json -check
echo "::endgroup::"

interop:
Expand Down Expand Up @@ -231,34 +234,34 @@ jobs:
- name: Re-export and diff
run: |
echo "::group::🔀 Exporting"
go run ./cmd/tfpluginframeworkgen interop export \
go run ./cmd/tfpfgen spec export \
-blueprint blueprints/thousandeyes \
-out interop-specs/thousandeyes/provider-code-spec.json
-out specs/thousandeyes/provider-code-spec.json
echo "::endgroup::"

if git diff --quiet -- interop-specs/; then
if git diff --quiet -- specs/; then
echo "✅ The exported specification matches the blueprints."
exit 0
fi

echo "::error::interop-specs/ is out of date. Re-export it and commit the result."
echo "::error::specs/ is out of date. Re-export it and commit the result."
{
echo "### ❌ The exported specification is out of date"
echo ""
echo '```bash'
# Double quotes with an escaped backslash, not a single-quoted string ending
# in one: the latter reads as an attempt to escape the closing quote. The text
# has no expansion in it, so the quoting style makes no other difference.
echo "go run ./cmd/tfpluginframeworkgen interop export \\"
echo "go run ./cmd/tfpfgen spec export \\"
echo " -blueprint blueprints/thousandeyes \\"
echo ' -out interop-specs/thousandeyes/provider-code-spec.json'
echo ' -out specs/thousandeyes/provider-code-spec.json'
echo '```'
} >> "$GITHUB_STEP_SUMMARY"

git diff --stat -- interop-specs/
git diff --stat -- specs/
exit 1

# This job is the reason internal/interop exists.
# This job is the reason internal/spec exists.
#
# Our own tests assert the export satisfies HashiCorp's embedded JSON schema,
# which is a structural claim. It is not the same claim as "a tool can render
Expand All @@ -279,7 +282,7 @@ jobs:

echo "::group::🏗️ Generating"
tfplugingen-framework generate resources \
--input interop-specs/thousandeyes/provider-code-spec.json \
--input specs/thousandeyes/provider-code-spec.json \
--output "$RUNNER_TEMP/gen"
echo "::endgroup::"

Expand Down Expand Up @@ -362,22 +365,21 @@ jobs:
- name: Verify facts reproduce with no network
run: |
echo "::group::🔬 Replaying committed cassettes"
go run ./cmd/tfpluginframeworkgen probe \
-blueprint blueprints/thousandeyes \
-mode verify
go run ./cmd/tfpfgen probe verify \
-blueprint blueprints/thousandeyes
echo "::endgroup::"

# Committed evidence that nobody has folded in is evidence nobody is acting on.
- name: The blueprint reflects the recorded facts
run: |
echo "::group::🔀 merge -check"
for facts in probe-evidence/*/*/*/facts.json; do
echo "::group::🔀 blueprint merge -check"
for facts in recordings/*/*/*/facts.json; do
echo "checking $facts"
go run ./cmd/tfpluginframeworkgen merge \
go run ./cmd/tfpfgen blueprint merge \
-blueprint blueprints/thousandeyes \
-facts "$facts" \
-check \
-accept-conflicts
-allow-conflicts
done
echo "::endgroup::"

Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/linter.yml
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ jobs:
# ^pilot/.* generated output. Third-party style opinions about
# generated code are noise: the shape is decided by the
# templates.
# ^openapi-specs/. vendored third-party API documents, pinned verbatim
# ^probe-evidence/ recorded HTTP cassettes and derived facts, replayed
# ^openapi/. vendored third-party API documents, pinned verbatim
# ^recordings/ recorded HTTP cassettes and derived facts, replayed
# byte-for-byte by the probe verification job
FILTER_REGEX_EXCLUDE: '(.*\.md$|.*_test\.go$|.*/test/.*|.*\.json$|^pilot/.*|^openapi-specs/.*|^probe-evidence/.*)'
FILTER_REGEX_EXCLUDE: '(.*\.md$|.*_test\.go$|.*/test/.*|.*\.json$|^pilot/.*|^openapi/.*|^recordings/.*)'
10 changes: 5 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -35,18 +35,18 @@ golangci-lint-report.sarif
go.work
go.work.sum

# tfpluginframeworkgen
/bin/tfpluginframeworkgen
.tfpluginframeworkgen/cache/
# tfpfgen
/bin/tfpfgen
.tfpfgen/cache/

# A probe ledger records live objects in somebody's tenant. It is per-run rather than
# per-snapshot, so committing one would be both meaningless and a disclosure.
.tfpluginframeworkgen/probe/
.tfpfgen/probe/

# A sandbox profile carries assertions.accountGroupId, and a tenant identifier in a committed
# file turns a vulnerability into a targeted one. See docs/examples/probe-profile.example.json
# for a version with nothing real in it.
.tfpluginframeworkgen/sandbox/
.tfpfgen/sandbox/

# Emitted when a template renders unparsable Go, kept for inspection only.
*.broken
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{"kind":"intent","seq":1,"probe":"write.required","path":"/account-groups","name":"tfpfgen-probe-write-required-1"}
{"kind":"created","seq":1,"probe":"write.required","path":"/account-groups","name":"tfpfgen-probe-write-required-1","id":"281474976748142","status":201}
{"kind":"intent","seq":2,"probe":"write.writable-returned","path":"/account-groups","name":"tfpfgen-probe-write-writable-returned-1"}
{"kind":"created","seq":2,"probe":"write.writable-returned","path":"/account-groups","name":"tfpfgen-probe-write-writable-returned-1","id":"281474976748143","status":201}
{"kind":"intent","seq":3,"probe":"write.rehearsal","path":"/account-groups","name":"tfpfgen-probe-write-rehearsal-1"}
{"kind":"created","seq":3,"probe":"write.rehearsal","path":"/account-groups","name":"tfpfgen-probe-write-rehearsal-1","id":"281474976748144","status":201}
{"kind":"deleted","seq":1,"probe":"write.required","path":"/account-groups","name":"tfpfgen-probe-write-required-1","id":"281474976748142","status":204}
{"kind":"deleted","seq":2,"probe":"write.writable-returned","path":"/account-groups","name":"tfpfgen-probe-write-writable-returned-1","id":"281474976748143","status":204}
{"kind":"deleted","seq":3,"probe":"write.rehearsal","path":"/account-groups","name":"tfpfgen-probe-write-rehearsal-1","id":"281474976748144","status":204}
{"kind":"intent","seq":4,"probe":"write.required","path":"/account-groups","name":"tfpfgen-probe-write-required-1"}
{"kind":"created","seq":4,"probe":"write.required","path":"/account-groups","name":"tfpfgen-probe-write-required-1","id":"281474976748146","status":201}
{"kind":"intent","seq":5,"probe":"write.writable-returned","path":"/account-groups","name":"tfpfgen-probe-write-writable-returned-1"}
{"kind":"created","seq":5,"probe":"write.writable-returned","path":"/account-groups","name":"tfpfgen-probe-write-writable-returned-1","id":"281474976748147","status":201}
{"kind":"deleted","seq":5,"probe":"write.writable-returned","path":"/account-groups","name":"tfpfgen-probe-write-writable-returned-1","id":"281474976748147","status":204}
{"kind":"intent","seq":6,"probe":"write.rehearsal","path":"/account-groups","name":"tfpfgen-probe-write-rehearsal-1"}
{"kind":"created","seq":6,"probe":"write.rehearsal","path":"/account-groups","name":"tfpfgen-probe-write-rehearsal-1","id":"281474976748148","status":201}
{"kind":"deleted","seq":6,"probe":"write.rehearsal","path":"/account-groups","name":"tfpfgen-probe-write-rehearsal-1","id":"281474976748148","status":204}
{"kind":"deleted","seq":4,"probe":"write.required","path":"/account-groups","name":"tfpfgen-probe-write-required-1","id":"281474976748146","status":204}
Loading
Loading