Skip to content

RFC: report compute providers and release stage per Namespace - #867

Closed
rossnelson wants to merge 4 commits into
mainfrom
propose-compute-provider-status
Closed

rossnelson wants to merge 4 commits into
mainfrom
propose-compute-provider-status

Conversation

@rossnelson

@rossnelson rossnelson commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

RFC — not for merge as-is

Proposes one source of truth for which compute providers a Namespace can use, whether each is enabled, and what release stage each is in — replacing the pattern of adding a boolean to Capabilities per provider.

Revised after review. The first draft put this on GetSystemInfoResponse. @Quinn-Klassen pointed out that will not work, because Cloud serves that endpoint as a fixed response at the reverse proxy layer, and suggested a Namespace-scoped surface. That is what this now does, and the objection turned out to be right on the mechanism and on the correct scope. History is kept so the argument is reviewable.

The AgentCore work this came out of does not depend on this and is not blocked by it.

The problem with a boolean per provider

server_scaled_provider_cloud_run = 13 (#799) works, but it only answers "is this one provider usable, Service-wide". Three costs follow.

A new provider needs a full release train before a client can even gate on it. Proto field → api release → server release. Cloud Run is the worked example: the UI shipped the option in temporalio/ui#3518 on 2026-06-12, the capability field landed 2026-07-06, and the UI could not consume it until an api bump on 2026-08-13. Two months, for a boolean.

It carries no release stage, so every client hardcodes its own. temporalio/ui has this table today:

export const defaultReleaseStage = {
  lambda: 'public-preview',
  agentcore: 'pre-release',
  'cloud-run': 'pre-release',
};

Moving Lambda to GA is a UI release. cloud-ui keeps a parallel list, the CLI keeps one for scaler pairing, and temporal-auto-scaled-workers — the actual authority — keeps the real one.

Service scope cannot express what is actually true. Provider availability varies per Namespace: a provider is tied to the cloud its Namespace runs in. cloud-ui already encodes this, deriving the list from the Namespace's RegionID_CloudProvider — an AWS Namespace offers Lambda and AgentCore, a GCP Namespace offers Cloud Run. A single Service-wide boolean has no way to say that.

All three are live right now:

Proposed shape

message ComputeProviderStatus {
    // Matches ComputeProvider.type — the provider's identity, not a display name.
    string type = 1;
    // Whether a ComputeConfig naming this provider will be accepted here.
    bool enabled = 2;
    temporal.api.enums.v1.ComputeProviderReleaseStage release_stage = 3;
}

on NamespaceInfo, alongside the existing capabilities and limits:

repeated temporal.api.compute.v1.ComputeProviderStatus compute_providers = 9;

A Service adds a provider without a proto change. A provider moves from pre-release to GA without a client release. Availability can differ per Namespace. And clients drop their hardcoded tables.

Why NamespaceInfo, concretely

Both halves of Quinn's point check out in the client code:

  • GetSystemInfo does not reach Cloud. temporalio/ui's fetchSystemInfo returns {} outright when isCloud, and cloud-ui synthesizes the entire systemInfo object client-side from account feature flags rather than calling the endpoint. A per-provider boolean there is not a reachable signal.
  • NamespaceInfo does. cloud-ui already reads namespace.namespaceInfo?.capabilities in standalone-nexus-guard.svelte. NamespaceInfo.Capabilities is a live, actively growing surface — 17 fields, most recently standalone_activity_operator_commands.

Cloud already has a working model for this, and it is two-key

Worth spelling out, because it decides whether the server populating this is
useful or academic.

NamespaceInfo.capabilities is not synthesized by cloud-ui the way
systemInfo is. cloud-ui fetches the real namespace from the data plane and
reads the capabilities off the wire (src/lib/services/settings.ts):

`${webUrl}/api/v1/namespaces/${namespace?.namespace}`
 dataplaneNamespace?.namespaceInfo?.capabilities

It then overlays account state on top, with two distinct idioms:

capabilities: {
  ...dataplaneNamespace?.namespaceInfo?.capabilities,
  workerHeartbeats: dataplane...?.workerHeartbeats ?? true,
  // TODO: Update these to use the dataplaneNamespace value when feature flag is removed
  standaloneActivityStartDelay: standaloneActivitiesGAEnabled,
  // dataplaneNamespace?.namespaceInfo?.capabilities?.standaloneActivityStartDelay ?? true,
}

?? true for capabilities Cloud knows are universally on, so an older data
plane that does not report one is treated as capable. Feature-flag
substitution for a capability still rolling out, with the real read commented
out beneath it and a TODO to swap at GA.

And consumers AND the two keys rather than choosing between them
(standalone-nexus-guard.svelte):

namespace.namespaceInfo?.capabilities?.standaloneNexusOperation &&
  $CurrentUser.hasFeatureFlag('enable_standalone_nexus_operations')

The namespace capability answers is this possible here; the account flag
answers is this account permitted. That is the shape a provider list wants
too, and it already works.

The standaloneActivityStartDelay lines are also, by hand, exactly the
lifecycle this proposal describes: flag first, server-reported value later,
with the swap tracked in a comment. release_stage is what lets the server say
"this is generally available now" instead of a client releasing to delete a
TODO.

Prior art: this package started typed and moved away from it

api#704 introduced compute.v1 on the serverless branch in February, and
#752 reshaped it when that branch merged to master. Two things did not survive,
and both bear on this proposal.

Per-provider config was typed, then deliberately untyped. #704 carried a
ProviderDetailAWSLambda message holding the function ARN and an optional role
ARN. Today ComputeProvider.details is an opaque Payload. So a reviewer
should ask why this proposes a typed message after the package moved the other
way.

The answer is that those two things are different in kind. Provider config is
open-ended and genuinely provider-specific — a Lambda ARN, a Cloud Run worker
pool, an AgentCore endpoint ARN share no shape, and every new provider would
otherwise cost a proto message. Provider status is uniform: every provider has
an identity, an enabled bit, and a release stage, and no provider needs a field
the others do not. Untyping config was right for the same reason typing status
is: the shape either varies per provider or it does not.

ComputeConfig.task_queues was removed. #704 declared task queues as
name-and-type tuples; main keeps only task_queue_types. That is why a
serverless Version has no task queue until something teaches matching one: the
first invoke starts a Worker, it polls with versioning, and that registration
is the association. Nothing in the current API can state it up front. Not this
proposal's problem, but it is the reason create-version takes no task queue,
which surprises everyone who meets it.

Two other deliberate choices

It reports disabled providers too, rather than omitting them. That lets a client distinguish "this Namespace has never heard of AgentCore" from "it knows AgentCore but it is off here" and say so, instead of silently dropping the option. It is the difference between a "Coming Soon" badge and a provider that appears not to exist.

A repeated message rather than more booleans in NamespaceInfo.Capabilities. Booleans there would inherit the same two problems — a proto field per provider, and no release stage.

server_scaled_provider_cloud_run is kept for wire compatibility and marked superseded, with a note recording why no further per-provider booleans belong on that response.

Open questions for reviewers

  1. NamespaceInfo, or a dedicated RPC? NamespaceInfo is where namespace capability data already lives and already reaches Cloud, so it is the cheap and consistent choice. A ListComputeProviders would be more discoverable and could carry more per-provider detail later.

  2. Should release_stage live in the api at all? It is arguably product metadata, not a capability. The counter-argument is that it is already in every client, just duplicated and going stale.

  3. Where does the value come from? temporal-auto-scaled-workers has the real registry — iface.ComputeProviderType and RegisterComputeProvider. The server should derive this from it rather than maintaining a third list. Worth confirming that is exposable.

  4. Does Cloud populate this per Namespace, or keep overlaying account flags in cloud-ui? Resolved: neither is a lift. The data plane populates NamespaceInfo.capabilities today and Cloud passes it through; saas-control-plane writes no namespace capabilities at all, so nothing new is asked of the control plane. cloud-ui keeps its account-flag overlay during rollout and drops it at GA, which is what it already does for standaloneActivities. See "Cloud already has a working model" above.

  5. Should the enabled bit be per Namespace, or is per Service enough in practice? Availability varies per Namespace because of the cloud a Namespace runs in, which is static. If nothing varies it dynamically, a Service-level list plus the Namespace's cloud provider would also work and be cheaper to populate.

Verification

  • make http-api-docs run; openapi/openapiv2.json and openapi/openapiv3.yaml regenerated and committed.
  • make buf-lint clean.
  • make api-linter clean.
  • protoc -I. --descriptor_set_out=/dev/null $(find temporal -name '*.proto') clean across the whole tree.
  • make buf-breaking reports 18 errors, all pre-existingExecutionType in enums/v1/common.proto, CallbackInfo.request_id, and several Nexus fields. Clean main reports the same 18. None involve compute/v1 or namespace/v1; this change adds no breaking change.

Nothing else already does this

Checked before proposing, on current main of each repository:

  • NamespaceInfo has no compute, provider, agentcore, or serverless field, and no capability has been added to it since this RFC opened.
  • The server's namespace_handler.go sets five worker-related capabilities and nothing provider-related.
  • saas-proto mentions serverless only as infrastructure plumbing — an isServerless flag on internal service-account provisioning, and serverless_worker as a Key Vault cert purpose. Cloud has built certs and identity for server-scaled Workers; it has not built a surface for which providers a Namespace may use.

What is not done

  • No server implementation. Nothing populates compute_providers yet. This PR is the contract argument only.

Context

RFC, not for merge as-is. Proposes reporting compute providers as a
repeated message on GetSystemInfoResponse instead of adding one boolean to
Capabilities per provider.

- enums/v1/compute.proto: ComputeProviderReleaseStage
- compute/v1/status.proto: ComputeProviderStatus (type, enabled, release_stage)
- GetSystemInfoResponse.compute_providers = 3
- Marks server_scaled_provider_cloud_run as superseded, kept for wire
  compatibility, and closes the door on further per-provider booleans

The boolean pattern answers only "is this one provider usable". Every new
provider costs a proto field, an api release, and a server release before a
client can gate on it, and it carries no release stage, so each client
hardcodes its own table. Both problems are live: Cloud Run's capability
landed a month after the UI shipped the option, the server still does not
set it, and AgentCore is merged in temporal-auto-scaled-workers and the CLI
with no capability field at all.

OpenAPI specs are not regenerated here; buf could not be installed in this
environment. Regenerate before this leaves draft.
Quinn Klassen pointed out on #867 that GetSystemInfo is the wrong carrier:
Cloud serves that endpoint as a fixed response at the reverse proxy layer,
so nothing per-account or per-Namespace can reach a client through it, and
suggested a Namespace-scoped surface instead.

Both halves check out in the client code. temporalio/ui's fetchSystemInfo
returns {} outright when isCloud, and cloud-ui synthesizes the whole
systemInfo object client-side from account feature flags rather than
calling the endpoint. Meanwhile cloud-ui already reads
namespace.namespaceInfo.capabilities in standalone-nexus-guard.svelte, so
NamespaceInfo demonstrably does reach Cloud.

Namespace scope is also the correct scope on the merits, not just a way
around the proxy. A compute provider is tied to the cloud its Namespace
runs in, and cloud-ui already derives the provider list from the
Namespace's RegionID_CloudProvider: an AWS Namespace offers Lambda and
AgentCore, a GCP Namespace offers Cloud Run. One Service-wide list cannot
say that.

- NamespaceInfo.compute_providers = 9
- GetSystemInfoResponse otherwise reverted; the note on
  server_scaled_provider_cloud_run now records why no further per-provider
  booleans belong there
- OpenAPI specs regenerated
@rossnelson rossnelson changed the title RFC: report compute providers and release stage in GetSystemInfo RFC: report compute providers and release stage per Namespace Sep 8, 2026
api#704 carried a typed ProviderDetailAWSLambda before #752 replaced it with
an opaque Payload, so proposing a typed message here invites the obvious
objection. The distinction is that provider config is open-ended and
genuinely provider-specific, while provider status is uniform across every
provider, and the comment now says so where a reviewer reads the message.

Specs regenerated.
rossnelson added a commit to temporalio/ui that referenced this pull request Sep 9, 2026
…in Cloud

Capability gating was the wrong mechanism, and it produced the visible
nonsense: a provider selected, disabled, and badged "Coming Soon" at once.

Self-hosted has no per-account entitlement to express, and a Service that
cannot run a provider rejects the Version with a reason, so gating the picker
only hid a choice behind a badge nobody could act on. The default list is now
every provider, selectable.

Restriction belongs to the caller that has grounds for it. cloud-ui already
passes `providers` derived from the Namespace's own cloud, so an AWS Namespace
offers Lambda and AgentCore and a GCP Namespace offers Cloud Run. That path is
untouched: it always passes the prop, so it never used these defaults.

A Version's provider cannot be changed, so the edit form shows only the
provider in use rather than alternatives that cannot be applied. The previous
commit had this backwards, reading an accepted update-mask path as product
behaviour.

This also removes the local serverScaledProviderAgentCore augmentation on
Capabilities. Nothing gates on it now, so the UI no longer depends on a
capability field that does not exist and that temporalio/api#867 no longer
proposes.
rossnelson added a commit to temporalio/ui that referenced this pull request Sep 9, 2026
…in Cloud

Capability gating was the wrong mechanism, and it produced the visible
nonsense: a provider selected, disabled, and badged "Coming Soon" at once.

Self-hosted has no per-account entitlement to express, and a Service that
cannot run a provider rejects the Version with a reason, so gating the picker
only hid a choice behind a badge nobody could act on. The default list is now
every provider, selectable.

Restriction belongs to the caller that has grounds for it. cloud-ui already
passes `providers` derived from the Namespace's own cloud, so an AWS Namespace
offers Lambda and AgentCore and a GCP Namespace offers Cloud Run. That path is
untouched: it always passes the prop, so it never used these defaults.

A Version's provider cannot be changed, so the edit form shows only the
provider in use rather than alternatives that cannot be applied. The previous
commit had this backwards, reading an accepted update-mask path as product
behaviour.

This also removes the local serverScaledProviderAgentCore augmentation on
Capabilities. Nothing gates on it now, so the UI no longer depends on a
capability field that does not exist and that temporalio/api#867 no longer
proposes.
@rossnelson

Copy link
Copy Markdown
Contributor Author

Withdrawing this. The problem it described has since been solved server-side rather than in the API, and the half that remains is not an API concern.

Enablement now has a real source of truth. workercontroller.compute_providers.enabled is a namespace-scoped setting — a namespace-level value overrides the cell-wide one, and namespaces without an entry inherit the cell value. The Worker Controller enforces it when instantiating a provider. temporalio/temporal-auto-scaled-workers#129 made it authoritative by dropping the implicit allow-all when the list is null, so "which providers can this namespace use" is now a single configured value rather than something a client reconstructs from per-provider capability booleans.

That covers the first and stronger motivation here: a new provider no longer needs a proto field, an api release and a server release before anyone can gate on it.

Release stage stays with the client. That was this proposal's other job, and it is the weaker one. Whether something reads Public Preview or Pre-release is a product and presentation decision that changes on a different schedule than the server's, and the UI is a reasonable owner. Encoding it in the API buys less than it costs.

With both motivations addressed, there is nothing left for ComputeProviderStatus to add.

One thing stays true, deliberately. The enabled list is not readable by a client, so a self-hosted UI still offers every provider and the server rejects a config it cannot run, with a reason. That is the accepted trade: the alternative was gating on capabilities the server never advertised, which is what left Cloud Run unselectable outside Cloud for two months. If reading that list ever becomes worth exposing, it is a much smaller proposal now — surfacing one existing value, rather than introducing a concept.

Thanks @Quinn-Klassen for the correction on GetSystemInfoResponse earlier; the history is left in place since the reasoning is still the useful part.

@rossnelson rossnelson closed this Sep 14, 2026
rossnelson added a commit to temporalio/ui that referenced this pull request Sep 14, 2026
)

* feat(workers): add Amazon Bedrock AgentCore as a compute provider

Adds AgentCore to the serverless Worker forms, following the sequence Cloud
Run used: the UI ships gated behind a capability the api protos do not carry
yet, so OSS shows Coming Soon until a server advertises it, and cloud-ui can
enable it earlier by mapping an account feature flag into its synthesized
systemInfo capabilities.

AgentCore is invoke-based in temporal-auto-scaled-workers, so it pairs with
the no-sync scaler and takes the same assumed-role Access fields as Lambda.
It differs in one payload key: `endpoint_arn` rather than `arn`. The value is
the Runtime *Endpoint* ARN, since the provider parses the runtime id and
endpoint name out of it and rejects a bare Runtime ARN.

- Capability gate via `serverScaledProviderAgentCore`, declared as a local
  intersection on Capabilities until the proto field lands.
- Resource field validates the four-part runtime-endpoint ARN shape.
- The CloudFormation/Terraform role helper stays Lambda-only: it grants
  lambda:InvokeFunction and would hand out a role that cannot invoke a runtime.
- buildComputeConfigFromForm replaces the per-page provider ternaries, so
  mapping a provider to a ComputeConfig is one branch rather than three.
- Adds a feature-demo scenario recording what a reviewer checks and what the
  enabled path additionally requires from api and server.

* fix(workers): add decodeAgentCoreProviderDetails to the service test double

version-compute-details.svelte calls it, and deployment.svelte.test.ts
swaps the real deployments-service for the client test double, so the
missing export surfaced as an unhandled rejection during that test rather
than a failed assertion.

* fix(workers): stop gating the provider a Version already uses

Editing a Version whose provider the Service does not advertise rendered that
provider selected, disabled, and badged "Coming Soon" all at once, which says
the Version is running on something unavailable.

Capability gating exists to stop somebody choosing a provider the Service
cannot run. A Version already running on one is proof it works, so gating must
not describe it as unavailable. lockProvidersTo makes the provider in use
visible and selectable, hides the rest because a Version's provider cannot be
changed, and keeps the release stage, which stays true either way.

Two causes, both fixed:

The edit page never locked at all. lockComputeProvider was only wired to the
create page, so editing fell through to the capability-gated default list.

lockComputeProvider itself failed closed when the configured provider was
disabled or hidden, which produced the same fallback on the create page. It
now only declines when configuration does not know the provider at all.

Not AgentCore-specific: Cloud Run hits this on OSS today, since no server
advertises server_scaled_provider_cloud_run.

* fix(workers): keep the alternative providers offered when editing a Version

The previous commit ungated the provider a Version uses, but also hid every
alternative on the reasoning that a Version's provider cannot be changed. It
can: provider.type is an accepted update path on
UpdateWorkerDeploymentVersionComputeConfig, so switching a Version to another
provider is a real choice, and hiding them removed a capability the edit form
had before.

allowProviderInUse now ungates only the provider in use and leaves the
alternatives exactly as configured, so one the Service can run stays offered
and one it cannot stays refused. lockProvidersTo keeps the hiding behaviour and
is used where the provider is inherited rather than chosen: creating a new
Version in an existing Deployment.

* fix(workers): select any provider in self-hosted, block by Namespace in Cloud

Capability gating was the wrong mechanism, and it produced the visible
nonsense: a provider selected, disabled, and badged "Coming Soon" at once.

Self-hosted has no per-account entitlement to express, and a Service that
cannot run a provider rejects the Version with a reason, so gating the picker
only hid a choice behind a badge nobody could act on. The default list is now
every provider, selectable.

Restriction belongs to the caller that has grounds for it. cloud-ui already
passes `providers` derived from the Namespace's own cloud, so an AWS Namespace
offers Lambda and AgentCore and a GCP Namespace offers Cloud Run. That path is
untouched: it always passes the prop, so it never used these defaults.

A Version's provider cannot be changed, so the edit form shows only the
provider in use rather than alternatives that cannot be applied. The previous
commit had this backwards, reading an accepted update-mask path as product
behaviour.

This also removes the local serverScaledProviderAgentCore augmentation on
Capabilities. Nothing gates on it now, so the UI no longer depends on a
capability field that does not exist and that temporalio/api#867 no longer
proposes.

* feat(demo): a runnable AgentCore serverless Worker scenario (#3892)

* feat(demo): model the AgentCore serverless Worker as a runnable scenario

Adds the scenario, plus three capabilities the demo tool needed to express it.

requires.serverModules
  serverCommit cannot describe a feature that reaches the server through a
  dependency bump, because the commit lives in another repository. This checks
  the go.mod of the checkout being built. Pseudo-versions order by their
  embedded commit timestamp, and a tagged release satisfies a pseudo-version
  floor outright.

requires.commands, and a preflight
  A missing tool is cheap to detect and expensive to hit halfway through a
  build, or worse after a provider has been handed an address. Stages imply
  their own tools, so the tunnel stage requires ngrok without the definition
  restating it, and a workspace build requires go and git.

A tunnel stage
  A server-scaled Worker dials the frontend back to poll, and localhost is not
  reachable from a cloud provider. The outbound leg to the provider was never
  the problem. Scenarios read the address as context.publicAddress.

Failures carry remedies. remedy.ts formats what was attempted, the tool's
verbatim output, what to try, and where to look. ngrok error codes map to
concrete fixes where we know them, and to their documentation URL where we do
not: an invented remedy is worse than none.

The scenario resolves its own CLI rather than trusting PATH. The
--aws-agentcore-* flags arrived in 1.8.3, and a package-manager CLI is easily
older than the repository pins; this machine had 1.8.2 on PATH against 1.8.3
in bin/cli, which would have failed on an unknown flag.

It does not provision AWS. An AgentCore runtime bills while it exists, and
starting a demo should not create billable cloud resources as a side effect, so
it takes an endpoint ARN and the README covers provisioning.

* fix(demo): derive the workspace Go directive, and repoint runtimes correctly

Three problems the first real run of agentcore-serverless-worker exposed.

The go.work directive was a literal `go 1.26.4`. Go refuses a workspace whose
directive is below a member module's, and temporalio/cli has since moved to
1.26.5, so every source: 'workspace' scenario was one Go bump away from
failing. It now follows the higher of the two checkouts.

update-agent-runtime replaces a runtime rather than patching it, so sending
only --environment-variables fails argument validation. The scenario now reads
the current role, artifact, and network configuration and sends them back
unchanged, instead of telling the reader to assemble the command by hand.

'tunnel' was missing from STAGES, so --skip tunnel and --only tunnel were
rejected and the stage never appeared in a definition's stage list.

Also fixes a preflight test helper that passed at runtime while failing to
typecheck, which had hidden the STAGES errors from tsc.

* fix(demo): make the tunnel prove its address before handing it out

Children append to their log, and the tunnel's public address is read back out
of that log, so a previous run's url satisfied the wait before this run's ngrok
had written anything. The stage reported a plausible address, the scenario
configured a provider with it, and the failure surfaced inside an AWS container
as a connection refused, far from its cause. A stale address is syntactically
perfect, so it read as success.

Two fixes, because the second is the one that matters:

- The log is removed before the child starts, so the address can only come from
  this run.
- The address is proved with a TCP connect before it is returned. A tunnel that
  reports a url and carries no traffic is indistinguishable from a working one
  until something remote fails to dial it, and that covers more than this bug:
  a tunnel that never established reports a url too.

Found by running the scenario twice, which its own preview notes call for.

* feat(demo): optionally provision the AgentCore runtime

The scenario can now create what it needs instead of refusing when no
endpoint ARN is given. Off by default: an AgentCore runtime bills while it
exists, and starting a demo should not create billable cloud resources by
surprise.

Provisioning is idempotent by name, so repeated runs reuse one runtime rather
than adding another, and it checks that the calling identity can do every part
before creating anything, so a missing permission does not leave an account
half provisioned.

When access is missing it names the policies to attach rather than passing the
raw AccessDenied along. That includes the trap this cost us: the ECR managed
policies are named AmazonEC2ContainerRegistry*, so searching the IAM console
for "ecr" returns nothing useful, and the guidance says to search
ContainerRegistry instead. It also offers the alternative of asking someone for
an endpoint ARN rather than granting the permissions at all.

Nothing is torn down at the end. A scenario's shutdown gets a three second
grace, which is not enough to delete a runtime, and a reviewer wants the demo
to still exist when the run finishes, so the summary lists what was created and
the commands to remove it.

Creating a runtime immediately after creating its execution role fails with an
error that blames ECR permissions rather than IAM propagation, which sends the
reader to the wrong place, so that call retries for a minute.

The Worker image source ships with the scenario, including its own README on
the two requirements that are easy to miss: AgentCore only accepts arm64, and
a Worker with UseVersioning must give every workflow a VersioningBehavior or it
panics before it polls.

* feat(demo): apply unmet module requirements instead of reporting them

requires.serverModules declared what the server needed and then handed the
work back, telling a person to go and edit a checkout this tool had fetched
into its own cache. The point of a requirement is that the tool satisfies it.

A checkout this tool fetched now gets go get and go mod tidy run on it. A
server ref would not solve this: the bump is a one-line go.mod change nobody
has pushed to temporalio/temporal, and requiring a pushed branch to demo a
feature is a heavier prerequisite than bumping a throwaway checkout.
serverModules already names the exact version, so nothing else was needed.

A checkout somebody pointed TEMPORAL_SERVER_REPO at is their working tree and
is never modified. RepoSource carries whether the tool owns it, and the
unowned case fails with advice to bump it there or to unset the variable.

Two ways this could have gone quietly wrong. The bump changes checkout
content, which the build cache is keyed on, so the key folds in the module
requirements or a pre-bump binary is reused against a bumped go.mod. And go
mod tidy can lower a requirement another module constrains, so it is
re-verified rather than assumed.

Verified with TEMPORAL_SERVER_REPO unset: the tool fetched temporalio/temporal
at main (0405f547a), bumped auto-scaled-workers, built, and ran through to a
workflow completing on the AgentCore-hosted Worker.

* feat(demo): let a scenario refuse before anything expensive runs

Running the AgentCore scenario cold with nothing configured reported the
missing endpoint ARN from the scenarios stage, which is after fetching two
repositories, bumping a module, and compiling a server. A person waited five
minutes to be told something a string comparison knew at once.

Scenario gains an optional preflight, called before any stage. The AgentCore
scenario uses it for both of its prerequisites: that it has somewhere to point,
and, when provisioning, that the AWS identity can do every part. Neither needs
a running server, so neither should cost a build. Cold with nothing set now
fails in about two seconds.

AGENTCORE_PROVISION=1 opts in for a single run, so a reviewer does not edit a
tracked file and have to remember to revert it. The source resolution is shared
between preflight and run so they cannot disagree about what the scenario
needs.

Verified from nothing: no AWS resources, no .feature-demo, no environment. The
run fetched both repositories, bumped auto-scaled-workers, built the server,
opened a tunnel, created the ECR repository, built and pushed the arm64 Worker
image, created the execution role, created the runtime, and ran a workflow to
completion on the Worker inside AgentCore. The IAM propagation retry fired,
so it was not defensive.

* fix(demo): refuse before provisioning, and default provisioning on

Three things a fresh run exposed.

Provisioning is on by default for this scenario. It cannot run without a
runtime, and a demo whose first run fails is not a demo. The option remains,
AGENTCORE_PROVISION=1 turns it on for a single run, and an explicit endpoint
ARN still wins.

The tunnel guard moved above provisioning. Running with --only scenarios
created an ECR repository, an IAM role, and an AgentCore runtime, and then
refused because the tunnel stage had not run. Nothing billable should be
created for a run that cannot finish.

Preflight now checks that the runtime an endpoint ARN names actually exists.
An ARN is a string, so having one proves nothing about whether it resolves: a
stale one from a previous run looks entirely valid, silently shadows
provisioning because an explicit ARN takes precedence, and previously failed
only after a server build and a tunnel. It now fails in about two seconds and
says to unset AGENTCORE_ENDPOINT_ARN.

* fix(demo): create the AgentCore runtime with its environment already set

create-agent-runtime returns as soon as the request is accepted, and the
service rejects updates and invokes until the runtime leaves CREATING. The
scenario then immediately pointed the new runtime at the tunnel and got a
ConflictException. Whether that happened depended on how fast the rest of the
run was, so an earlier cold run passed on timing luck.

The tunnel is already open when provisioning runs, so the address goes in at
creation instead. That removes the racing update rather than timing around it,
and saves waiting for a second rollout. A reused runtime carries whatever a
previous run left on it, so that path still updates, and provisioning now
waits for READY either way.

An update that loses a race anyway reports the runtime as busy and says to run
again, rather than listing permissions that are not the problem.

* fix(demo): build the embedded frontend assets before the ui-server

The ui-server embeds the built frontend with //go:embed all:assets, and a Go
embed of a missing directory is a compile error, not an empty filesystem. A
worktree that has never built them cannot build the binary at all:

  ui/embed.go:8:12: pattern all:assets: no matching files found

The stage now runs pnpm build:server, which writes them to server/ui/assets,
when that directory is absent. Any fresh worktree hit this, so it belongs in
the stage rather than in setup instructions.

Both failures also explain themselves now instead of forwarding a compiler
error whose text does not mention the frontend.

* fix(demo): apply the runtime environment through update, not create

create-agent-runtime accepts --environment-variables and then ignores it. The
runtime comes back with environmentVariables null, and the Worker starts with
an empty TEMPORAL_ADDRESS and task queue:

  WORKER starting addr= ns=default tq= deployment=agentcore-demo

So provisioning a runtime from scratch produced a Worker that could not dial
anything, which is worse than the ConflictException it was meant to avoid.

The previous commit conflated two changes. Waiting for READY is what fixes the
race; moving the environment into create was an unnecessary optimisation on
top, and it does not work. Only the wait is kept, and the caller now always
applies the environment through update, which is the path that has been
exercised repeatedly.

This survived because every run after that change reused an existing runtime
and took the update path. The create path was not exercised until a teardown
forced a fresh provision.

* feat(demo): create the AgentCore Version through the UI, not the CLI

The scenario shelled out to `temporal worker deployment create-version`,
which proves the server accepts an aws-agentcore compute config. That was
never in question: the CLI could do this before any of this work existed.
What is under review is the UI, and a run that never opened the form said
nothing about it.

The Version is now created by driving the real create-version form with
Playwright: select Amazon Bedrock AgentCore, fill the Agent Runtime
Endpoint ARN and the Access fields, submit. `createVersion: 'cli'` keeps
the old path for a machine with no browser, and `headed: true` shows the
browser doing it. Screenshots land per step, so a failure leaves a picture
of where it stopped.

Preflight launches a browser before any stage starts, because Playwright
downloads its browsers separately from the package and finding that out
after a server build costs minutes.

The run also records what it had to type to get there. The form requires
IAM Role ARN and External ID; this server sets
require_role_and_external_id false and discards both. The UI has no
equivalent of the CLI's --aws-agentcore-skip-role-and-external-id, so a
self-hosted operator who turned the requirement off can create a Version
by CLI but not by form. FE-675 tracks that.

Adds `uiUrl` to the scenario context, set to the UI the ui stage started
so a scenario drives the checkout's UI rather than the server's bundled
one.

* fix(demo): drive the real form, and provision the role it needs

Proving the UI can create an AgentCore Version took more than pointing a
browser at the form.

The Access fields are not decorative. Setting
require_role_and_external_id to false makes the role optional, not
ignored: the server assumes whatever role the compute config carries. The
CLI can omit it with --aws-agentcore-skip-role-and-external-id; the form
requires both fields and always sends them, so a placeholder does not get
discarded, it fails at sts:AssumeRole after the form has done its job.
The scenario now provisions TemporalDemoAgentCoreInvoke, trusting this
account under an sts:ExternalId condition and granting
bedrock-agentcore:InvokeAgentRuntime, and waits for it to actually assume
before going on, because IAM is eventually consistent and the server
tries seconds later. The earlier comments claiming the server discards
these fields were wrong and are corrected here.

Each run also gets its own deployment. Versions from earlier runs go
INACTIVE, the server stops reporting their ComputeConfig (FE-672), and
lockComputeProvider cannot name a provider it cannot see, so the
create-version page renders an error where the form should be. Those
Versions cannot be deleted either: AgentCore sessions from earlier runs
keep polling, and the server refuses to delete a Version with active
pollers.

The driver itself had three faults, all found by running it. locator
count() does not wait, and the page renders the form only after it
fetches the deployment, so it looked before the form existed. The page
keeps hidden [role=alert] containers and a closed confirm modal whose
button is also a form submit, so first() waited on elements never shown.
And fullPage screenshots evaluate in the page, which trips over the
__name wrappers esno leaves behind.

Verified end to end against Bedrock AgentCore in us-west-2: the form
created the Version, the Worker Controller invoked the runtime, the
Worker dialed back through the tunnel and registered agentcore-tq, and
Greet returned "hello demo from AgentCore worker on localhost".

* docs(demo): keep internal references out of a public repo

This repository is public. The demo notes named an internal repository
and described how hosted Temporal maps an account entitlement into the
capabilities it synthesizes, and several comments cited internal tracker
ids that mean nothing to anyone reading this here. The observations they
supported are worth keeping, so they are restated without the pointers.

Also corrects a note that this branch had already made false: the
compute-provider scenario still said AgentCore must render disabled and
badged "Coming Soon" on a self-hosted server. Self-hosted now offers
every provider, and a reviewer following that note would have reported a
bug against correct behaviour.

* docs(demo): point the picker scenario at what this branch changed

The scenario listed the fields AgentCore renders but never sent a
reviewer to the two stories that show the actual behaviour change:
every provider selectable on a self-hosted server, and a Version locked
to the provider it already uses.

Those are the cases worth a human looking at. The first is where a
self-hosted server previously gated providers on capabilities it never
advertised, which left Cloud Run permanently unselectable. The second is
where a card used to render selected and disabled at once.

* feat(workers): AgentCore IAM setup material, for Cloud and self-hosted

The Access section required an IAM role and offered no way to create one
for AgentCore. Its CloudFormation and Terraform helpers grant
lambda:InvokeFunction, so sharing them would hand out a role that cannot
invoke a runtime, and the helper was hidden rather than reused.

Adds the AgentCore equivalents and follows the selected provider:

- temporal-agentcore-role.yaml grants bedrock-agentcore:InvokeAgentRuntime
  and GetAgentRuntimeEndpoint. Validation calls GetAgentRuntimeEndpoint
  first, so a role without it fails when the Version is created rather
  than when a Worker is wanted.
- serverless-worker-agentcore.tf points at the aws/agentcore module and
  takes agent_runtime_arns, the same four-part Endpoint ARN the Version
  takes.
- The Launch Stack link, the download filename, and the module link now
  follow the provider rather than always naming Lambda.

The trust policy is the part that differs by deployment. Cloud assumes
the role as a service principal, so it names temporal.io. A self-hosted
server assumes it as whatever IAM identity it runs as, which is an AWS
principal rather than a service and so a different key entirely. The
template takes a TemporalPrincipal parameter and switches on it, since
this PR makes the provider selectable outside Cloud for the first time.

The policy is the one proven by the demo scenario, which provisions the
same permissions and trust shape and only works because the role really
assumes.

Note the aws/agentcore Terraform module does not exist in
temporalio/terraform-modules yet. The snippet names where it will live,
as the Lambda and Cloud Run snippets do for theirs.

* fix(holocene): round the radio card to match the Io system

The card drew a square border while everything around it — buttons,
accordions, inputs — is rounded, so the compute provider picker read as
unfinished next to the rest of the form.

The optional panel beneath a selected card shares its bottom edge, so the
card drops its bottom corners when that panel is showing and the panel
takes them instead. No consumer passes that snippet today, but leaving it
unhandled would have put the sharp corner back the moment one did.

* fix(workers): restore the provider brand colours and round their tile

Two things in the provider picker's logo tile.

The marks lost their colour. Before the icon registry moved to Io, the
picker rendered holocene's aws.svelte, which carried fill="#FF9900". The
refactor remapped it to IconAws, the monochrome glyph, so the AWS and GCP
logos have rendered flat white on dark since 20 Aug. IconAwsColor and
IconGcpColor already existed and were used nowhere outside a Badge story.
These are vendor logos rather than UI icons, so they should keep their own
colour on either theme.

The tile was also explicitly rounded-none, predating the Io work, which
left a square inside a form where everything else is rounded.

* fix(demo): enable the compute provider the scenario needs

The provider allowlist inverted. `workercontroller.compute_providers.enabled`
used to mean "restrict to these", and an unset value allowed everything;
in the auto-scaled-workers release main now pins it means "only these",
and an unset value enables nothing at all.

The scenario never set it, because it never had to. The server now
reports that it could not instantiate the provider, which reads like the
build is missing it — the provider is there and switched off, and the
metric behind that branch says so where the message does not.

Worth knowing beyond this scenario: the same upgrade silently disables
Lambda and Cloud Run for any self-hosted operator who has not set the
list.

Dynamic config values may now be string arrays. The flag already passes
JSON, so only the schema stood in the way.

* fix(deployments): use the brand marks in the compute badge

The last place still rendering the monochrome glyphs. Same cause as the
provider picker: the icon registry move mapped these to IconAws and
IconGcp, so the badge has drawn a flat white AWS smile since August.

These identify a vendor rather than convey UI state, so currentColor is
the wrong default for them.

* fix(workers): use the small badge for the provider stage

The stage badge sits beside a provider name at text-sm, and at the
default size its mono uppercase label read heavier than the name it
qualifies. #3908 added a size variant, so take it rather than override
the padding locally.

The disabled-reason badge occupies the same slot and only ever renders
instead of this one, so it takes the same size.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant