Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 88 additions & 8 deletions .agents/references/gcp-backplane.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
description: GCP backplane identity conventions for meshstack-hub modules under modules/gcp/. Covers the mandatory workload identity federation pattern (pool + provider + service account), why there is no service account key path, project API enablement with disable_on_destroy = false, required permissions for the applying identity, the workload identity pool soft-delete constraint, and the GCP backplane checklist.
description: GCP backplane identity conventions for meshstack-hub modules under modules/gcp/. Covers the mandatory workload identity federation pattern (pool + provider + service account), why there is no service account key path, scoping the subject to one building block definition and the two steps that keep its uuid from closing a dependency cycle, project API enablement with disable_on_destroy = false, required permissions for the applying identity, the workload identity pool soft-delete constraint, and the GCP backplane checklist.
---

# GCP Backplane Conventions
Expand Down Expand Up @@ -56,10 +56,10 @@ resource "google_iam_workload_identity_pool_provider" "meshstack" {
"google.subject" = "assertion.sub"
}

# Restrict token acceptance to the configured subjects.
# Exact match, never startsWith() — see "Scope the subject to one building block definition".
attribute_condition = join(" || ", [
for subject in var.workload_identity_federation.subjects :
"google.subject.startsWith('${subject}')"
for subject in var.workload_identity_subjects :
"google.subject == '${subject}'"
])
}

Expand All @@ -81,7 +81,7 @@ resource "google_project_iam_member" "buildingblock" {
}
```

`issuer`, `audience` and `subjects` must always come from `data.meshstack_integrations` in
`issuer`, `audience` and the subjects must always come from `data.meshstack_integrations` in
`meshstack_integration.tf` — never hardcoded. See the AWS and Azure references for the shared
subject-derivation idiom.

Expand All @@ -90,6 +90,73 @@ The backplane's `credentials_json` output is then an
document (`type = "external_account"`) pointing at the runner's token file, passed to the building
block as a `FILE` input with `GOOGLE_APPLICATION_CREDENTIALS` set to its path.

## Scope the subject to one building block definition

The building block runner names its per-run Kubernetes service account
`workspace.<workspace>.buildingblockdefinition.<bbd-uuid>`, so the token's `sub` claim is
`system:serviceaccount:<namespace>:workspace.<workspace>.buildingblockdefinition.<bbd-uuid>`. The
subject must carry that uuid and be matched **exactly**, as every other provider's backplane already
does. A subject that stops at `buildingblockdefinition`, or an exact subject matched with
`startsWith()`, admits **every** building block definition in the workspace — a second definition in
the same workspace can then federate into this backplane's service account and inherit its roles.

```hcl
# meshstack_integration.tf
workload_identity_subjects = [
"${trimsuffix(data.meshstack_integrations.integrations.workload_identity_federation.replicator.subject, ":replicator")}:workspace.${var.meshstack.owning_workspace_identifier}.buildingblockdefinition.${meshstack_building_block_definition.this.metadata.uuid}"
]
```

### Breaking the dependency cycle the uuid creates

The building block definition carries `credentials_json` as a `FILE` input, so it already depends on
the backplane. Feeding its uuid back in as a subject closes that loop — unless **nothing on the
credential path depends on a resource that consumes the subjects**. GCP needs two deliberate steps
for that.

**1. Keep the subjects out of `workload_identity_federation`.** OpenTofu tracks module input
dependencies **per variable**, not per attribute. A `subjects` field inside that object taints every
resource that reads any other field of it — the pool via its identifier, the
`roles/iam.workloadIdentityUser` binding via the pool, and the credentials via the IAM propagation
wait. A separate `variable "workload_identity_subjects"` confines the dependency to the pool
provider, the only resource that needs it.

**2. Assemble the pool provider's resource name instead of reading the attribute back** — the same
trick `modules/aws/` uses for its role ARN.

```hcl
data "google_project" "this" {
project_id = var.project_id
}

locals {
# Not google_iam_workload_identity_pool_provider.meshstack.name: that resource consumes
# var.workload_identity_subjects, and the subjects name the building block definition that
# carries these credentials.
workload_identity_pool_provider_name = "projects/${data.google_project.this.number}/locations/global/workloadIdentityPools/${var.workload_identity_federation.workload_identity_pool_identifier}/providers/${var.workload_identity_federation.workload_identity_pool_identifier}"
}

output "credentials_json" {
# ...
value = jsonencode({
audience = "//iam.googleapis.com/${local.workload_identity_pool_provider_name}"
# ...
})
}
```

The pool itself does **not** consume the subjects, so `google_service_account_iam_binding` and
`time_sleep.wait_for_iam` may keep referencing it. That is why the binding stays pool-wide
(`principalSet://.../workloadIdentityPools/<pool>/*`) rather than naming the subject with
`principal://.../subject/<sub>`: a subject-scoped member would put the subjects back on the credential
path through the IAM propagation wait, and dropping the binding from that wait would trade a real
`iam.serviceAccounts.getAccessToken` 403 for a theoretical second gate. The pool-wide set has exactly
one member anyway — one pool, one provider, one accepted subject.

`tofu validate` on the backplane alone surfaces neither cycle. Check both by initialising and
validating a root that wires `meshstack_integration.tf` against a local `./backplane` — that is where
OpenTofu reports `Error: Cycle: ...`.

<!-- scorecard-checks: gcp_project_service_disable_on_destroy -->
## Project API enablement

Expand Down Expand Up @@ -262,12 +329,18 @@ variable "workload_identity_federation" {
workload_identity_pool_identifier = string
audience = string
issuer = string
subjects = list(string)
subject_token_file_path = string
})
nullable = false # required: there is no service account key fallback
description = "Workload identity federation settings sourced from data.meshstack_integrations."
}

# Deliberately not a field of the object above — see "Breaking the dependency cycle the uuid creates".
variable "workload_identity_subjects" {
type = list(string)
nullable = false
description = "Full `sub` claims of the OIDC tokens the pool provider accepts, matched exactly."
}
```

<!-- scorecard-checks: gcp_credentials_output -->
Expand Down Expand Up @@ -295,7 +368,11 @@ Both existing modules expose these two. Do not add a `documentation_md` output
- ❌ `google_service_account_key` — use workload identity federation
- ❌ Conditional WIF-vs-key logic: a nullable `workload_identity_federation` and a `count` on every
federation resource
- ❌ Hardcoded `issuer`, `audience` or `subjects` — source them from `data.meshstack_integrations`
- ❌ Hardcoded `issuer`, `audience` or subjects — source them from `data.meshstack_integrations`
- ❌ A subject that stops at `buildingblockdefinition`, or `startsWith()` on the subject — both admit
every building block definition in the workspace
- ❌ `subjects` as a field of `workload_identity_federation` — it drags the whole object onto the
credential path and the configuration no longer plans
- ❌ Hardcoded workload identity pool identifier — soft-delete makes it unreusable for ~30 days
- ❌ Granting the building block's service account `roles/serviceusage.serviceUsageAdmin` when the
building block does not enable services
Expand All @@ -310,7 +387,10 @@ Both existing modules expose these two. Do not add a `documentation_md` output
- [ ] Workload identity pool + provider present
- [ ] No `google_service_account_key` anywhere in `backplane/`
- [ ] `workload_identity_federation` is `nullable = false` — no key fallback, no `default = null`
- [ ] `attribute_condition` restricts `google.subject` to the configured subjects
- [ ] The accepted subjects live in their own `workload_identity_subjects` variable, not in that object
- [ ] `attribute_condition` matches `google.subject` **exactly** against the configured subjects — no `startsWith()`
- [ ] The subject in `meshstack_integration.tf` ends in `.${meshstack_building_block_definition.<name>.metadata.uuid}`
- [ ] `credentials_json` reads no attribute of `google_iam_workload_identity_pool_provider` — the audience is assembled from `data.google_project.<x>.number` and the pool identifier
- [ ] `google_service_account_iam_binding` grants `roles/iam.workloadIdentityUser` on the pool
- [ ] Workload identity pool identifier is an input, not a hardcoded literal
- [ ] A `time_sleep` absorbs IAM propagation and the `credentials_json` output `depends_on` it
Expand Down
39 changes: 36 additions & 3 deletions modules/gcp/budget-alert/backplane/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,47 @@ The building block authenticates by **workload identity federation**; `workload_
is required. The backplane creates a workload identity pool and provider alongside the service
account, grants the pool `roles/iam.workloadIdentityUser` on it, and exports `credentials_json` as an
[external account](https://cloud.google.com/iam/docs/workload-identity-federation) document.
`issuer`, `audience` and `subjects` must come from `data.meshstack_integrations` — see
`meshstack_integration.tf`.
`issuer`, `audience` and `workload_identity_subjects` must come from `data.meshstack_integrations` —
see `meshstack_integration.tf`.

GCP **soft-deletes** workload identity pools and providers for ~30 days and will not reissue their
identifiers in that window. `workload_identity_pool_identifier` is an input for exactly that reason:
a backplane that has to be recreated needs a fresh one, and a caller that creates and destroys
backplanes repeatedly (an e2e test) must derive a unique identifier per run. The soft-deleted pools
count against the project's pool limit while they linger.

#### Subject matching is exact

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

d: this is now a general pattern for all GCP backplanes. don't repeat this blurb in every README.md when its already in the shared instructions. Same applies to the other backplane/README.md touched in this PR


The provider's `attribute_condition` compares `google.subject` for **equality** against each entry of
`workload_identity_subjects`, so each entry must be the complete `sub` claim of the token to accept —
including the building block definition uuid that the runner puts in its per-run service account name
(`workspace.<workspace>.buildingblockdefinition.<bbd-uuid>`). A prefix that stopped at
`buildingblockdefinition` would let every other building block definition in the same workspace
federate into this service account.

The `roles/iam.workloadIdentityUser` binding is pool-wide
(`principalSet://iam.googleapis.com/.../workloadIdentityPools/POOL_ID/*`), which is not wider than
that condition: the pool holds one provider and the provider accepts one subject. Naming the subject
in the binding as well would make the binding depend on the building block definition uuid, and
`credentials_json` waits on that binding for IAM propagation — closing a dependency cycle.

#### Why the subjects are their own variable

The subjects name the building block definition, and that same definition carries `credentials_json`
as an input, so nothing on the credential path may depend on them. OpenTofu tracks module input
dependencies **per variable**, not per attribute: a `subjects` field inside
`workload_identity_federation` would taint every resource that reads any other field of that object,
including the pool, the `roles/iam.workloadIdentityUser` binding and — through the propagation wait —
the credentials themselves. `workload_identity_subjects` confines the dependency to the pool provider,
the one resource that needs it.

For the same reason `credentials_json` assembles its `audience` from the project number and the pool
identifier rather than reading `google_iam_workload_identity_pool_provider.meshstack.name`: reading
it back would put the pool provider on the credential path.

`tofu validate` on this module alone surfaces neither cycle. Only a root that wires
`meshstack_integration.tf` against a local `backplane/` does.

### Roles granted to the building block's service account

The service account the backplane creates is the identity the **building block** runs as. It
Expand Down Expand Up @@ -117,7 +149,8 @@ No modules.
| <a name="input_backplane_service_account_name"></a> [backplane\_service\_account\_name](#input\_backplane\_service\_account\_name) | The name of the service account to be created for the backplane | `string` | `"building-block-budget-alert"` | no |
| <a name="input_billing_account_id"></a> [billing\_account\_id](#input\_billing\_account\_id) | The billing account ID where budget permissions will be granted | `string` | n/a | yes |
| <a name="input_iam_propagation_delay_seconds"></a> [iam\_propagation\_delay\_seconds](#input\_iam\_propagation\_delay\_seconds) | Seconds to wait after granting the building block's IAM roles before publishing its credentials. GCP IAM is eventually consistent, and billing-account grants are among the slower ones. Set to 0 if the backplane is always provisioned well before any building block run. | `number` | `180` | no |
| <a name="input_workload_identity_federation"></a> [workload\_identity\_federation](#input\_workload\_identity\_federation) | Workload identity federation settings, sourced from data.meshstack\_integrations. | <pre>object({<br/> workload_identity_pool_identifier = string<br/> audience = string<br/> issuer = string<br/> subjects = list(string)<br/> subject_token_file_path = string<br/> })</pre> | n/a | yes |
| <a name="input_workload_identity_federation"></a> [workload\_identity\_federation](#input\_workload\_identity\_federation) | Workload identity federation settings, sourced from data.meshstack\_integrations. The accepted subjects are a separate variable — see workload\_identity\_subjects. | <pre>object({<br/> workload_identity_pool_identifier = string<br/> audience = string<br/> issuer = string<br/> subject_token_file_path = string<br/> })</pre> | n/a | yes |
| <a name="input_workload_identity_subjects"></a> [workload\_identity\_subjects](#input\_workload\_identity\_subjects) | Full `sub` claims of the OIDC tokens the pool provider accepts, matched exactly. Each must name the building block definition that authenticates with these credentials. | `list(string)` | n/a | yes |

## Outputs

Expand Down
14 changes: 11 additions & 3 deletions modules/gcp/budget-alert/backplane/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -35,16 +35,24 @@ resource "google_iam_workload_identity_pool_provider" "meshstack" {
"google.subject" = "assertion.sub"
}

# Exact match, not a prefix: the subjects carry the building block definition uuid, and a
# startsWith() against a prefix would admit every other building block definition whose runner
# service account name happens to share it.
attribute_condition = join(" || ", [
for subject in var.workload_identity_federation.subjects :
"google.subject.startsWith('${subject}')"
for subject in var.workload_identity_subjects :
"google.subject == '${subject}'"
])
}

resource "google_service_account_iam_binding" "workload_identity" {
service_account_id = google_service_account.backplane.name
role = "roles/iam.workloadIdentityUser"
members = ["principalSet://iam.googleapis.com/${google_iam_workload_identity_pool.meshstack.name}/*"]

# Pool-wide, and that is as narrow as it gets here: the pool holds exactly one provider and that
# provider admits exactly one subject, so this set has exactly one member. Naming the subject here
# instead would make this binding depend on the building block definition uuid, and the
# credentials output waits on this binding — see the comment on the audience in outputs.tf.
members = ["principalSet://iam.googleapis.com/${google_iam_workload_identity_pool.meshstack.name}/*"]
}

# Grant billing account permissions to create budgets
Expand Down
12 changes: 11 additions & 1 deletion modules/gcp/budget-alert/backplane/outputs.tf
Original file line number Diff line number Diff line change
@@ -1,3 +1,13 @@
locals {
# Deliberately assembled from the provider's own identifiers rather than read off

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

d: a short "workaround: construct pool identifier to break dependency cycle, see ./agents"

is sufficient

# google_iam_workload_identity_pool_provider.meshstack.name, to keep the credentials output free of
# any dependency on the pool provider — the one resource that consumes
# var.workload_identity_subjects. Those subjects name the building block definition, the definition
# carries these credentials, and reading the attribute back would close that loop into a cycle.
# The format is the documented resource name of a pool provider.
workload_identity_pool_provider_name = "projects/${data.google_project.backplane.number}/locations/global/workloadIdentityPools/${var.workload_identity_federation.workload_identity_pool_identifier}/providers/${var.workload_identity_federation.workload_identity_pool_identifier}"
}

output "service_account_email" {
description = "Email address of the backplane service account"
value = google_service_account.backplane.email
Expand All @@ -18,7 +28,7 @@ output "credentials_json" {
value = jsonencode({
universe_domain = "googleapis.com"
type = "external_account"
audience = "//iam.googleapis.com/${google_iam_workload_identity_pool_provider.meshstack.name}"
audience = "//iam.googleapis.com/${local.workload_identity_pool_provider_name}"
subject_token_type = "urn:ietf:params:oauth:token-type:jwt"
token_url = "https://sts.googleapis.com/v1/token"
service_account_impersonation_url = "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/${google_service_account.backplane.email}:generateAccessToken"
Expand Down
14 changes: 12 additions & 2 deletions modules/gcp/budget-alert/backplane/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,21 @@ variable "workload_identity_federation" {
workload_identity_pool_identifier = string
audience = string
issuer = string
subjects = list(string)
subject_token_file_path = string
})
nullable = false
description = "Workload identity federation settings, sourced from data.meshstack_integrations."
description = "Workload identity federation settings, sourced from data.meshstack_integrations. The accepted subjects are a separate variable — see workload_identity_subjects."
}

# Deliberately not a field of workload_identity_federation. Its value names the building block
# definition, which in turn carries the credentials_json output — and OpenTofu tracks module input
# dependencies per variable, not per attribute. Folding the subjects into that object would make
# every resource that reads any of its fields depend on the building block definition, and the
# credentials would depend on themselves.
variable "workload_identity_subjects" {
type = list(string)
nullable = false
description = "Full `sub` claims of the OIDC tokens the pool provider accepts, matched exactly. Each must name the building block definition that authenticates with these credentials."
}

variable "iam_propagation_delay_seconds" {
Expand Down
Loading
Loading