Skip to content

Add the Agent Factories authoring surface - #2114

Open
MRayermannMSFT wants to merge 4 commits into
mainfrom
dev/mrayermannmsft/other/agent-factories-sdk
Open

Add the Agent Factories authoring surface#2114
MRayermannMSFT wants to merge 4 commits into
mainfrom
dev/mrayermannmsft/other/agent-factories-sdk

Conversation

@MRayermannMSFT

@MRayermannMSFT MRayermannMSFT commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

What

Adds the Agent Factories authoring surface to the Node.js SDK. A factory is a trusted extension-authored closure that orchestrates a fleet of subagents and is invoked by name: an extension declares one with defineFactory, registers it through joinSession({ factories }), and runs it with session.factory.run(...). Only metadata crosses to the runtime — the closure stays in the extension process and is invoked back over a reverse RPC, where it gets primitives for spawning subagents, journaling results so a resumed run replays completed work for free, composing work in parallel or as a pipeline, and reporting progress.

Runs resolve with a durable envelope for every outcome rather than throwing, and waitForRun settles once a run reaches a terminal status, so callers stay correct when execution later moves to background-only. The one generator change narrows an over-broad opaque-JSON conversion; without it a factory result generates as an object alone, which a non-object or void result cannot satisfy.

Everything is dark behind the runtime's agent_factories flag plus its billing gate, and every public type is marked @experimental.

Why

The runtime half of Agent Factories already merged (github/copilot-agent-runtime#12953 and #13077), but the SDK half has never landed — the original PR was closed unmerged — so the shipped runtime currently has no authoring client and the feature cannot work end to end for any SDK consumer. Until this lands, the factory E2E suite can only run against a locally injected SDK build rather than the published package.

Copilot AI review requested due to automatic review settings July 28, 2026 19:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds the experimental Agent Factories authoring surface to the Node.js SDK, including registration, execution, orchestration, persistence, observability, and documentation.

Changes:

  • Adds factory definition, registration, execution, resume, cancellation, and observability APIs.
  • Generates supporting JSON-RPC and session-event types with recursive JsonValue support.
  • Adds extensive factory tests and authoring documentation.
Show a summary per file
File Description
scripts/codegen/typescript.ts Generates recursive opaque JSON types.
nodejs/test/typescript-codegen.test.ts Tests opaque JSON generation.
nodejs/test/session-event-types.test.ts Checks factory and JSON exports.
nodejs/test/factory.test.ts Tests factory behavior and APIs.
nodejs/test/extension.test.ts Updates extension resume tests.
nodejs/src/types.ts Adds factory metadata and limit types.
nodejs/src/sessionFsProvider.ts Adds SQLite transaction support.
nodejs/src/session.ts Implements factory execution and orchestration.
nodejs/src/index.ts Exports factory APIs.
nodejs/src/generated/session-events.ts Adds factory events and JSON types.
nodejs/src/generated/rpc.ts Adds generated factory RPC contracts.
nodejs/src/factory.ts Defines the public factory authoring surface.
nodejs/src/extension.ts Registers factories through joinSession.
nodejs/src/client.ts Sends registration metadata and installs handlers.
nodejs/docs/factories.md Documents factory authoring and operation.
nodejs/docs/extensions.md Links factory documentation.

Review details

  • Files reviewed: 14/16 changed files
  • Comments generated: 7
  • Review effort level: Medium

Comment thread nodejs/src/sessionFsProvider.ts Outdated
Comment thread nodejs/src/session.ts
Comment thread nodejs/src/session.ts
Comment thread nodejs/src/factory.ts Outdated
Comment thread nodejs/src/index.ts
Comment thread nodejs/src/extension.ts
Comment thread nodejs/src/types.ts Outdated
Copilot AI review requested due to automatic review settings July 28, 2026 20:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Still open (7)
Comments suppressed due to low confidence (2)

nodejs/src/sessionFsProvider.ts:56

  • Making transaction required breaks every existing SessionFsProvider SQLite implementation in this repository: for example nodejs/test/session_fs_adapter.test.ts:62, nodejs/test/e2e/session_fs.e2e.test.ts:287, and nodejs/test/e2e/session_fs_sqlite.e2e.test.ts:203 implement only query and exists, so TypeScript compilation fails. Update all implementations (and transaction behavior tests), or make this capability optional with an adapter fallback to preserve compatibility.
    transaction(
        statements: SessionFsSqliteTransactionStatement[]
    ): Promise<SessionFsSqliteQueryResult[]>;

nodejs/src/types.ts:840

  • This broadens elicitation responses beyond the primitive schema declared immediately above: ElicitationSchemaField can only produce strings, numbers, booleans, or string arrays, while JsonValue also permits null, objects, and mixed/nested arrays. Those values cannot satisfy the advertised MCP primitive form contract, so invalid handler responses now type-check. Keep the primitive value union here; factory JSON values should not alter this unrelated API.
 * JSON field value in an elicitation result.
 */
export type ElicitationFieldValue = JsonValue;
  • Files reviewed: 15/17 changed files
  • Comments generated: 4
  • Review effort level: Medium

Comment thread scripts/codegen/typescript.ts Outdated
Comment thread nodejs/src/session.ts Outdated
Comment thread nodejs/src/session.ts Outdated
Comment thread nodejs/src/session.ts Outdated
Copilot AI review requested due to automatic review settings July 28, 2026 20:40
@MRayermannMSFT
MRayermannMSFT force-pushed the dev/mrayermannmsft/other/agent-factories-sdk branch from e2d54fa to 44d9159 Compare July 28, 2026 20:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Files not reviewed (6)
  • go/rpc/zrpc.go: Generated file
  • go/rpc/zrpc_encoding.go: Generated file
  • go/rpc/zsession_encoding.go: Generated file
  • go/rpc/zsession_events.go: Generated file
  • go/zsession_events.go: Generated file
  • nodejs/package-lock.json: Generated file
Resolved since last review (6)
  • nodejs/src/session.ts#L182 — Resolved by Copilot - pipeline() similarly converts hard RPC failures from a stage's agent() call into null,…
  • nodejs/src/session.ts#L154 — Resolved by Copilot - parallel() swallows every non-abort rejection, including ResponseError/ConnectionError from…
  • scripts/codegen/typescript.ts#L293 — Resolved by Copilot - This blanket replacement collapses every marked schema to JsonValue, including already-structured…
  • nodejs/src/extension.ts#L56 — Resolved by Copilot - The extension entry point has the same incomplete public surface: SessionFactoryApi.listRuns(),…
  • nodejs/src/index.ts#L181 — Resolved by Copilot - The new observability methods expose named DTOs (FactoryRunSummary, FactoryRunDetail,…
  • nodejs/src/sessionFsProvider.ts#L56 — Resolved by Copilot - Making transaction required is a breaking change to the existing public SessionFsSqliteProvider
Still open (5)
Comments suppressed due to low confidence (6)

nodejs/src/session.ts:1215

  • Overlapping factory.execute attempts can share a run ID (the new overlapping-attempts test explicitly exercises this), but this assignment overwrites the previous controller. A subsequent factory.abort therefore aborts only the newest attempt while the stale factory body can continue running and performing local side effects. Track all controllers per run ID and abort every active attempt.
                const controller = new AbortController();
                self.factoryAbortControllers.set(params.runId, controller);

nodejs/src/factory.ts:348

  • Validation is only performed here, but the returned handle exposes the same mutable definition.meta object. A caller can mutate phases or limits after defineFactory and before joinSession, bypassing these checks; registration then serializes the invalid metadata. Snapshot/freeze the validated metadata, or revalidate it during registration.
    validateLimits(definition.meta);
    validatePhases(definition.meta);

    const stored: StoredFactory = {
        meta: definition.meta,
        run: definition.run,
    };
    const handle = Object.freeze({ meta: definition.meta }) as FactoryHandle<TArgs, TResult>;

nodejs/src/session.ts:1954

  • The numeric branch accepts negative zero, but JSON serialization converts -0 to 0. That violates the stated lossless validation guarantee for factory results and journal replay. Detect Object.is(current, -0) and either reject it with a dedicated category or define an explicit normalization policy.
        if (typeof current === "number") {
            if (!Number.isFinite(current)) {

scripts/codegen/typescript.ts:326

  • This replacement discards non-structural schema metadata such as description, deprecated, and visibility tags. The generated diff already loses documentation from opaque fields such as CitationReference.providerMetadata and HookStartData.input. Preserve the rewritten node and only replace its type semantics after removing the opaque marker.
    nodejs/src/factory.ts:197
  • FactoryResumeErrorCode is exported as part of the new public authoring API but lacks the @experimental marker promised for every public factory type in the PR description. Add the same experimental notice used by the surrounding factory types.
/** Machine-readable pre-execution factory resume failure. */
export type FactoryResumeErrorCode =

nodejs/src/sessionFsProvider.ts:240

  • The adapter now has several new behavioral branches—unsupported providers, parameter normalization, successful transactions, and three error classifications—but none are covered in the existing nodejs/test/session_fs_adapter.test.ts suite that tests the other adapter operations. Add focused tests, especially for busy/locked versus post-commit-ambiguous classification, to prevent unsafe retry behavior.
        sqliteTransaction: async ({ statements }) => {
            if (!provider.sqlite?.transaction) {
                return {
                    results: [],
                    error: {
  • Files reviewed: 17/32 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

Copilot AI review requested due to automatic review settings July 28, 2026 20:57
Comment thread nodejs/test/factory.test.ts Fixed
Comment thread nodejs/test/factory.test.ts Fixed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Files not reviewed (6)
  • go/rpc/zrpc.go: Generated file
  • go/rpc/zrpc_encoding.go: Generated file
  • go/rpc/zsession_encoding.go: Generated file
  • go/rpc/zsession_events.go: Generated file
  • go/zsession_events.go: Generated file
  • nodejs/package-lock.json: Generated file
Still open (5)
Comments suppressed due to low confidence (4)

nodejs/src/session.ts:1296

  • Overlapping factory.execute calls for the same run are explicitly supported, but this assignment replaces the earlier controller. A later factory.abort (and session cleanup) therefore aborts only the newest attempt; an older extension body can keep running and performing side effects indefinitely. Track all controllers per run (or key them by execution token and abort every controller for the run), and remove only the completing attempt.
                const controller = new AbortController();
                self.factoryAbortControllers.set(params.runId, controller);

nodejs/src/sessionFsProvider.ts:296

  • SQLite extended busy/locked result codes are misclassified as fatal. Standard codes such as SQLITE_BUSY_SNAPSHOT (numeric 517) and SQLITE_LOCKED_SHAREDCACHE (262) retain 5 or 6 only in the low byte, so providers exposing extended names or numbers will skip the intended retry path. Match the SQLITE_BUSY*/SQLITE_LOCKED* families and mask numeric extended codes to their base code.
    const code = "code" in err ? err.code : undefined;
    const errcode = "errcode" in err ? err.errcode : undefined;
    return code === "SQLITE_BUSY" ||
        code === "SQLITE_LOCKED" ||
        code === "EBUSY" ||
        errcode === 5 ||
        errcode === 6
        ? "busyOrLocked"
        : "fatal";

nodejs/src/factory.ts:223

  • FactoryResumeErrorCode is a newly public Agent Factories type but is the only factory-specific public declaration here without @experimental, contrary to the PR's stated contract that every public type is marked experimental. Add the same experimental JSDoc used by the surrounding factory API types.
/** Machine-readable pre-execution factory resume failure. */
export type FactoryResumeErrorCode =

go/rpc/zrpc_encoding.go:1157

  • Clear Failure when the property is absent. As written, unmarshalling a terminal without failure into a reused FactoryRunTerminal leaves the previous failure attached, while the other optional fields are reset from raw; callers can therefore observe a stale failure from an earlier payload. Please update scripts/codegen/go.ts to initialize this union field to nil before conditionally decoding it, then regenerate this file.
  • Files reviewed: 17/32 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

Copilot AI review requested due to automatic review settings July 28, 2026 21:18
@MRayermannMSFT
MRayermannMSFT force-pushed the dev/mrayermannmsft/other/agent-factories-sdk branch from 2f86165 to 07a25bc Compare July 28, 2026 21:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Files not reviewed (6)
  • go/rpc/zrpc.go: Generated file
  • go/rpc/zrpc_encoding.go: Generated file
  • go/rpc/zsession_encoding.go: Generated file
  • go/rpc/zsession_events.go: Generated file
  • go/zsession_events.go: Generated file
  • nodejs/package-lock.json: Generated file
Resolved since last review (4)
  • nodejs/src/session.ts#L1192 — Resolved by Copilot - Indexing controllers only by runId loses the previous controller when overlapping execution…
  • nodejs/src/factory.ts#L352 — Resolved by Copilot - The validated metadata is retained by reference and only the outer handle is frozen. Callers can…
  • nodejs/src/session.ts#L1939 — Resolved by Copilot - This strict validator accepts -0, but JSON serialization changes it to 0
  • nodejs/src/session.ts#L274 — Resolved by Copilot - The RPC promise has already been created—and therefore the request already sent—before this…
Still open (1)
  • 🟠 nodejs/src/types.ts#L840 — This broadens elicitation responses beyond the values that the adjacent ElicitationSchemaField
Comments suppressed due to low confidence (3)

scripts/codegen/typescript.ts:325

  • Replacing an opaque node with a fresh object drops all schema metadata on that node, including its description and marker-derived JSDoc tags. This is already removing public field documentation such as CitationReference.providerMetadata and the @experimental text on ToolExecutionCompleteData.mcpMeta from the regenerated output. Preserve the rewritten node, remove only x-opaque-json, and add tsType so code generation retains those annotations.
    nodejs/src/sessionFsProvider.ts:236
  • The new transaction adapter path is not exercised by nodejs/test/session_fs_adapter.test.ts, although that suite covers the other adapter operations. Please add cases for successful parameter normalization, unsupported providers, and each retry classification; a regression here can incorrectly retry a possibly committed transaction or mishandle a rolled-back busy error.
        sqliteTransaction: async ({ statements }) => {

nodejs/src/factory.ts:223

  • This exported Agent Factories type is the only factory-specific public type here without the promised @experimental annotation, so generated API documentation will present it as stable while the related class and options are experimental.
/** Machine-readable pre-execution factory resume failure. */
export type FactoryResumeErrorCode =
  • Files reviewed: 17/32 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

Copilot AI review requested due to automatic review settings July 28, 2026 21:33
@MRayermannMSFT
MRayermannMSFT force-pushed the dev/mrayermannmsft/other/agent-factories-sdk branch from 07a25bc to d188f14 Compare July 28, 2026 21:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Files not reviewed (6)
  • go/rpc/zrpc.go: Generated file
  • go/rpc/zrpc_encoding.go: Generated file
  • go/rpc/zsession_encoding.go: Generated file
  • go/rpc/zsession_events.go: Generated file
  • go/zsession_events.go: Generated file
  • nodejs/package-lock.json: Generated file
Still open (1)
  • 🟠 nodejs/src/types.ts#L840 — This broadens elicitation responses beyond the values that the adjacent ElicitationSchemaField
Comments suppressed due to low confidence (3)

scripts/codegen/typescript.ts:324

  • The new test exercises only the branch that collapses an unconstrained opaque node. The preservation branch here protects existing public contracts, but has no regression coverage. Add cases where x-opaque-json accompanies type/properties and anyOf, and assert those schemas retain their structure rather than becoming JsonValue.
    nodejs/src/factory.ts:249
  • For a typed handle, args remains optional through RunOptions<TArgs>. A factory declared as defineFactory<{ files: string[] }, ...>(...) can therefore be invoked as session.factory.run(handle); the wrapper sends {}, while the factory body is typed as though args.files always exists. This makes the handle overload unsound and can turn an accepted call into a runtime failure. Require args for handle invocations unless TArgs explicitly permits the default empty object (the name-based overload can remain permissive).
    run<TArgs extends JsonValue>(
        factory: FactoryHandle<TArgs, JsonValue | void>,
        options?: RunOptions<TArgs>
    ): Promise<FactoryRunResult>;

nodejs/src/sessionFsProvider.ts:240

  • The new transaction adapter path is untested even though this module already has SQLite adapter coverage. Add tests for successful statement/parameter forwarding, the no-transaction fallback, and error classification (busyOrLocked, postCommitAmbiguous, and fatal) so protocol errors cannot be silently misclassified.
        sqliteTransaction: async ({ statements }) => {
            if (!provider.sqlite?.transaction) {
                return {
                    results: [],
                    error: {
  • Files reviewed: 20/35 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

Copilot AI review requested due to automatic review settings July 28, 2026 21:40
@MRayermannMSFT
MRayermannMSFT force-pushed the dev/mrayermannmsft/other/agent-factories-sdk branch from d188f14 to eb17d19 Compare July 28, 2026 21:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Files not reviewed (6)
  • go/rpc/zrpc.go: Generated file
  • go/rpc/zrpc_encoding.go: Generated file
  • go/rpc/zsession_encoding.go: Generated file
  • go/rpc/zsession_events.go: Generated file
  • go/zsession_events.go: Generated file
  • nodejs/package-lock.json: Generated file
Still open (1)
  • 🟠 nodejs/src/types.ts#L840 — This broadens elicitation responses beyond the values that the adjacent ElicitationSchemaField
Comments suppressed due to low confidence (4)

scripts/codegen/typescript.ts:327

  • Returning a new { tsType } object drops the original node's description and other annotations. As a result, this regeneration removes JSDoc from every unconstrained opaque field (for example, the arguments documentation in AssistantMessageToolRequest). Set tsType on rewritten and then strip the marker so the public documentation and annotations survive.
    nodejs/src/factory.ts:249
  • The typed-handle overload permits omitting options.args, but the implementation then sends {}. For a factory declared with TArgs = { files: string[] }, session.factory.run(handle) therefore compiles while context.args is typed as that object and actually receives {}, causing ordinary typed factory bodies to fail at runtime. Require args when the handle's TArgs does not admit the documented {} default, or reflect that default in the context type.
    run<TArgs extends JsonValue>(
        factory: FactoryHandle<TArgs, JsonValue | void>,
        options?: RunOptions<TArgs>
    ): Promise<FactoryRunResult>;

go/rpc/zrpc_encoding.go:1163

  • When failure is absent, this custom unmarshaller leaves any previous r.Failure value intact. Reusing a FactoryRunTerminal for successive decodes can therefore report a stale failure on a later successful/cancelled terminal value. Clear the destination union field before conditionally decoding it; the Go generator should be fixed and this file regenerated.
    nodejs/src/factory.ts:228
  • This newly exported factory-specific type is the exception to the PR's stated rule that every public Agent Factories type is marked @experimental. Add the same experimental JSDoc used by the surrounding public factory types so generated API documentation does not present this error-code contract as stable.
export type FactoryResumeErrorCode =
    | "not_found"
    | "non_resumable"
    | "already_active"
    | "reapproval_declined"
    | "no_approval_provider";
  • Files reviewed: 21/36 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

@MRayermannMSFT
MRayermannMSFT force-pushed the dev/mrayermannmsft/other/agent-factories-sdk branch from eb17d19 to 8697a23 Compare July 28, 2026 22:04
@MRayermannMSFT
MRayermannMSFT changed the base branch from main to dev/mrayermannmsft/other/copilot-1.0.75-bump July 28, 2026 22:09
Agent Factories let a trusted extension declare a JavaScript closure that
orchestrates a fleet of subagents, and invoke it by name. The runtime half
shipped in copilot-agent-runtime#12953 and #13077; this is the SDK half.
The feature is dark behind the runtime's `agent_factories` flag plus its
billing gate, and every public type is marked `@experimental`.

Authoring
- `defineFactory({ meta, run })` validates limits and phases at the
  authoring boundary and returns an opaque handle. The metadata is deep
  snapshotted and frozen, so mutating the caller's object afterwards cannot
  desynchronize what was validated from what is advertised.
- `joinSession({ factories })` registers handles for a session; names must
  be unique within the call.

Running
- `session.factory.run()` and `.resume()` resolve with the run envelope
  (`FactoryRunResult`) for every outcome — `completed`, `error`, `halted`,
  and `cancelled` alike. A declined fresh run resolves with a terminal
  `cancelled` envelope, because the run row already exists by the time the
  prompt is answered; only failures that occur before a run exists reject.
- `session.factory.waitForRun()` settles once a run reaches a terminal
  status. It subscribes to `factory.run_updated` before its first read so a
  transition cannot be missed, collapses a burst of invalidations into a
  single re-read, and re-reads periodically so a dropped event degrades into
  a slightly late resolution rather than an unbounded wait.

Inside a run
- `context.agent()` spawns a subagent, `step()` journals a result so a
  resumed run replays it for free, and `phase()`/`log()` report progress.
- `parallel()` and `pipeline()` compose subagent work; both propagate hard
  RPC failures rather than folding them into per-branch results.
- Journaled values are validated as strict JSON, because a step result has
  to survive serialization unchanged to be replayable. Negative zero is
  rejected for that reason: it serializes to `0`.
- Cancellation is checked before an operation is dispatched, so an aborted
  run does not start new runtime work. Abort controllers are keyed by run ID
  and execution token so overlapping attempts stay individually addressable.

Codegen
- Narrows the `x-opaque-json` conversion to definitions carrying no
  structural keywords. Only one of the annotated definitions is genuinely
  unconstrained; blanket-converting the rest discarded the structure of
  `McpServerConfig`, `ExternalToolResult`, `EventLogTypes`, and
  `UIElicitationSchemaProperty`. Without this, a factory result generates as
  an object alone, which a non-object or void result cannot satisfy.
- Genuinely opaque nodes emit `JsonValue` rather than `unknown`, a global
  tightening. `ToolTelemetry`, the hooks-invoke response, and one
  session-event assertion are realigned to match.

Docs are in `nodejs/docs/factories.md` and `nodejs/docs/factory-patterns.md`.
@github github deleted a comment from github-actions Bot Jul 30, 2026
@github github deleted a comment from github-actions Bot Jul 30, 2026
@github github deleted a comment from github-actions Bot Jul 30, 2026
@github github deleted a comment from github-actions Bot Jul 30, 2026
@github github deleted a comment from github-actions Bot Jul 30, 2026
@github github deleted a comment from github-actions Bot Jul 30, 2026
@github github deleted a comment from github-actions Bot Jul 30, 2026
@MRayermannMSFT
MRayermannMSFT marked this pull request as ready for review July 30, 2026 07:06
@MRayermannMSFT
MRayermannMSFT requested a review from a team as a code owner July 30, 2026 07:06
Copilot AI review requested due to automatic review settings July 30, 2026 07:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Resolved since last review (1)
  • nodejs/src/types.ts#L840 — Resolved by @​MRayermannMSFT - This broadens elicitation responses beyond the values that the adjacent ElicitationSchemaField
Comments suppressed due to low confidence (7)

scripts/codegen/typescript.ts:325

  • Replacing the entire schema node drops non-structural metadata such as description and visibility/stability annotations. This is why generated declarations such as FactoryAgentOptions.schema, FactoryExecuteRequest.args, and existing opaque event fields lose their JSDoc in this diff. Strip only the opaque marker and retain the rest of the node when adding tsType.
        if (isOpaqueJson(rewritten as JSONSchema7) && !hasStructuralConstraints(rewritten)) {
            return { tsType: "JsonValue" };

nodejs/src/factory.ts:200

  • The typed handle overload is unsound because args is optional for every TArgs. A caller can define FactoryHandle<{ files: string[] }, ...> and invoke run(handle) without options; the implementation then supplies {}, while the factory body is typed as if ctx.args.files always exists. Make fresh typed-handle runs require args when the declared argument shape requires it, while preserving the deprecated resume branch and no-argument factories.
export interface RunOptions<TArgs extends JsonValue = JsonValue> {
    /** Input surfaced as `context.args`. */
    args?: TArgs;

nodejs/src/factory.ts:140

  • This signature contradicts the implemented volatile-step behavior. The new test intentionally returns a function from { volatile: true }, but the public API still requires and returns JsonValue, forcing authors to use an unsafe cast and losing the producer's result type. Add a generic overload for options: { volatile: true }, while keeping the durable overload JSON-only.
    step(
        key: string,
        producer: () => Promise<JsonValue> | JsonValue,
        options?: FactoryStepOptions
    ): Promise<JsonValue>;

nodejs/docs/factories.md:224

  • This says no timer polling occurs, but waitForFactoryRun explicitly polls every five seconds (session.ts:547) as a fallback for missed invalidations. Document that periodic reread so users have accurate expectations about RPC traffic and settlement behavior.
It watches `factory.run_updated` and re-reads the durable envelope rather than polling on a timer, and it collapses a burst of invalidation events into a single in-flight read. Pass a `signal` to stop waiting:

nodejs/src/client.ts:1773

  • The new tests invoke mocked handlers directly, so they do not verify that factory metadata registration, reverse factory.execute callbacks, and the session-scoped RPC methods interoperate with the replay runtime. This PR's purpose is to make the feature work end to end; add a Node E2E scenario and snapshot that registers a factory and completes at least one run through the JSON-RPC boundary.
                factories: factories?.map((factory) => factory.meta),

nodejs/src/factory.ts:223

  • FactoryResumeErrorCode is part of the newly exported Agent Factories surface, but unlike the surrounding public factory types it lacks the promised @experimental marker. Add the marker so generated API documentation and consumers do not treat this alias as stable.
/** Machine-readable pre-execution factory resume failure. */
export type FactoryResumeErrorCode =

nodejs/src/factory.ts:60

  • This newly exported JsonValue alias also lacks the @experimental annotation promised for the public surface. Mark the exported alias consistently with the other Agent Factories types (or stop presenting the blanket experimental guarantee in the PR/API documentation).
/** A value that can be represented losslessly on the SDK JSON wire. */
export type JsonValue =
  • Files reviewed: 14/16 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

Preserve documentation metadata when a schema node collapses to `JsonValue`.
Returning a bare `{ tsType }` replaced the whole node, so genuinely opaque
declarations lost their JSDoc while their siblings kept theirs — visible on
`FactoryAgentOptions.schema` among others. Strip only the opaque marker and
the now-meaningless type keywords, keeping the description in place.

Correct the `waitForRun` description in the factories guide: it claimed no
timer polling, but a low-frequency periodic re-read runs alongside the
subscription so a dropped invalidation cannot leave the wait pending.

Mark `FactoryResumeErrorCode` `@experimental`, consistent with the rest of
the factory surface.
Copilot AI review requested due to automatic review settings July 30, 2026 07:28
@MRayermannMSFT

Copy link
Copy Markdown
Contributor Author

Thanks — went through all seven low-confidence items. Took three, declined four with reasoning.

Taken (CCR 1)

  • Opaque-JSON conversion dropped JSDoc (scripts/codegen/typescript.ts). Correct, and it was broader than the factory types: returning a bare { tsType } replaced the whole schema node, so every genuinely opaque declaration lost its description while its siblings kept theirs. Now strips only the opaque marker and the now-meaningless type keywords. Regeneration restores 262 lines of documentation across the generated surface — FactoryAgentOptions.schema, CanvasJsonSchema, provider action results, and others.
  • waitForRun doc was factually wrong (nodejs/docs/factories.md). It claimed no timer polling, but a low-frequency periodic re-read does run alongside the subscription — deliberately, so a dropped invalidation degrades into a slightly late resolution instead of an unbounded wait. Reworded to describe the actual behaviour.
  • FactoryResumeErrorCode was missing @experimental, inconsistent with the rest of the surface. Added.

Declined

  • Typed-handle args optionality and a generic step() overload for { volatile: true }: both are real ergonomic gaps, but both are public-API redesigns (new overloads and conditional generics) rather than fixes. The surface is @experimental precisely so shape changes like these can be made deliberately, with the runtime contract in view, rather than folded into a review pass.
  • "Add a Node E2E that registers a factory and completes a run through the JSON-RPC boundary." This already exists — it just isn't in this repo. The runtime's agent-factories.test.ts suite runs 22 tests against a real CLI build with this branch injected, exercising registration, reverse factory.execute, resume, limits, AI-credit accounting, observability, and agent isolation across the process boundary. A cross-repo advisory job runs it on every copilot-agent-runtime main commit; the latest run is green at 22/22 against this branch.
  • @experimental on JsonValue: it is a general "value representable on the JSON wire" alias, not part of the factory surface. Marking it experimental would wrongly signal that JSON representability itself is unstable.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Comments suppressed due to low confidence (5)

nodejs/src/session.ts:1366

  • An abort can occur while progress.flush() is pending; the volatile branch then starts its producer without rechecking the signal. This allows new extension work to begin after factory.abort has already cancelled the attempt, unlike the journaled branch whose next operation checks the signal.
                            await progress.flush();
                            if (options.volatile) {
                                return producer();

nodejs/src/factory.ts:255

  • The typed-handle overload does not preserve the handle's contract. options and args remain optional even when TArgs is a required shape, so run(handle) type-checks but the implementation supplies {} to a callback that expects TArgs. It also replaces the handle's TResult with JsonValue | void, making the result type carried by FactoryHandle unusable. Require arguments conditionally for typed handles and preserve TResult in the returned envelope.
    run<TArgs extends JsonValue>(
        factory: FactoryHandle<TArgs, JsonValue | void>,
        options?: RunOptions<TArgs>
    ): Promise<FactoryRunResult>;

nodejs/src/factory.ts:140

  • The runtime deliberately lets a volatile step return a non-JSON intermediate value, but this public signature still restricts every producer and result to JsonValue (the new test must cast the function result to never). Add a { volatile: true } overload generic over an unrestricted result type, while retaining the JSON-constrained overload for journaled steps.
    step(
        key: string,
        producer: () => Promise<JsonValue> | JsonValue,
        options?: FactoryStepOptions
    ): Promise<JsonValue>;

nodejs/src/factory.ts:59

  • JsonValue is newly exported from both package entry points but is the only hand-authored public type in this surface without @experimental, contradicting the PR's guarantee that every public type is marked experimental.
/** A value that can be represented losslessly on the SDK JSON wire. */
export type JsonValue =

nodejs/src/factory.ts:30

  • The exported run/detail types expose several generated supporting types that are not re-exported from either package entry point, including FactoryRunFailure, FactoryRunFailureKind, FactoryDurableOperation, FactoryCurrentPhase, FactoryDeclaredLimits, FactoryRunConsumed, and FactoryRunTerminal. Consumers therefore cannot directly name the typed failures and observations advertised by this API. Re-export these supporting types alongside the envelopes.
    FactoryProgressPage,
    FactoryRunDetail,
    FactoryRunResult,
    FactoryRunStatus,
    FactoryRunSummary,
  • Files reviewed: 14/16 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

@github-actions

This comment has been minimized.

Check the abort signal before running a volatile step producer.

`step()` awaits a progress flush before dispatching, which is an await point,
so `factory.abort` can land between entering the call and running the
producer. The journaled path is covered because its next operation goes
through `awaitFactoryOperation`, which checks the signal before dispatching;
the volatile path returned `producer()` directly and so could start new
extension work on an already-cancelled run.

The regression test aborts mid-run and asserts the producer never runs.
Removing the guard fails it.
Copilot AI review requested due to automatic review settings July 30, 2026 07:46
@MRayermannMSFT

Copy link
Copy Markdown
Contributor Author

Second pass — took the cancellation finding, re-declined the typed-handle one.

Taken (CCR 2) — session.ts:1366, volatile step could start after abort. Correct, and it broke an invariant this PR otherwise holds. step() awaits a progress flush before dispatching, which is an await point, so factory.abort can land between entering the call and running the producer. The journaled path is covered because its next operation goes through awaitFactoryOperation, which checks the signal before dispatching — but the volatile path returned producer() directly, so a cancelled run could still start new extension work.

Now guarded with throwIfFactoryAborted before the producer runs.

The regression test aborts mid-run and asserts the producer never executes. I verified it is real by removing the guard and re-running: the test fails. It is not a test that would pass either way.

Re-declined — typed-handle args optionality and TResult preservation. Same position as the previous pass: this is a public-API redesign (conditional generics to require args only when TArgs demands it, plus threading TResult through the envelope), not a defect fix. The surface is @experimental so this can be done deliberately, with the runtime contract in view, rather than folded into a review round. Worth doing — just not here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Comments suppressed due to low confidence (2)

scripts/codegen/typescript.ts:324

  • This conversion also catches opaque nodes that are explicitly non-JSON in-process values. The regenerated output now types McpRegisterExternalClientRequest.client/transport and callback fields such as SessionsOpenHandoff.onConfirm as JsonValue, even though their own generated docs say they hold MCP instances or functions. Those APIs can no longer be called with the values they are designed to accept. Please preserve unknown for internal non-wire opaque nodes, or introduce a distinct schema marker so only genuinely JSON-valued opaque nodes become JsonValue.
        if (isOpaqueJson(rewritten as JSONSchema7) && !hasStructuralConstraints(rewritten)) {

nodejs/src/client.ts:1773

  • The new registration path and reverse RPC are only tested with a mocked sendRequest and by directly invoking clientSessionApis.factory.execute; no Node E2E test exercises joinSession({ factories }) through the replaying runtime. That leaves the wire method names, metadata shape, handler registration, and runtime callback flow unverified—the exact end-to-end gap this PR is intended to close. Please add a replay snapshot and E2E covering registration and one factory run.
                factories: factories?.map((factory) => factory.meta),
  • Files reviewed: 14/16 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

@github-actions

This comment has been minimized.

@MRayermannMSFT

Copy link
Copy Markdown
Contributor Author

Third pass — checked the codegen finding carefully, declining both with evidence.

scripts/codegen/typescript.ts:324 — "those APIs can no longer be called with the values they are designed to accept." This one does not hold up. The cited fields carry their own documentation saying the opposite:

onProgress / onConfirm"Marked internal because a function reference cannot cross the JSON-RPC boundary."

Both are @internal, and neither SessionsOpenHandoff nor McpRegisterExternalClientRequest is exported from the public SDK surface. More to the point, on main these generate as { [k: string]: unknown }, which does not accept a function either — a function type has no implicit index signature. So the type was never callable with a callback through this generated declaration; this change swaps one non-function type for another and does not regress callability.

There is a real kernel here, though, and it is worth recording: JsonValue is narrower than unknown for nested values, so an opaque object holding non-JSON members is typed more strictly than before. The right fix is the one suggested — a schema marker that distinguishes "opaque but JSON-valued" from "opaque in-process value" — but that marker lives in the runtime's schema, so it is a cross-repo change rather than something to fold in here. The narrowing is also load-bearing for this PR: without it a factory result generates as an object alone, which non-object and void results cannot satisfy.

client.ts:1773 — Node E2E for joinSession({ factories }). Same as the earlier pass: that coverage exists, it just is not in this repo. The runtime's agent-factories.test.ts runs 22 tests against a real CLI build with this branch injected, covering registration, the reverse factory.execute callback, wire method names, metadata shape, resume, limits, AI-credit accounting, and observability across the JSON-RPC boundary. A cross-repo advisory job runs it on every copilot-agent-runtime main commit, and the latest run is green at 22/22 against this branch.

No code change for either. CI is green at 35/35 with 0 unresolved threads.

Copilot AI review requested due to automatic review settings July 30, 2026 12:47
@github-actions

Copy link
Copy Markdown
Contributor

Cross-SDK Consistency Review

Summary

This PR adds the Agent Factories authoring surface exclusively to the Node.js/TypeScript SDK (defineFactory, joinSession({ factories }), session.factory.*, SessionFactoryApi, FactoryContext, FactoryHandle, etc.).

Status in other SDKs

SDK Generated RPC types (Factory*) Authoring surface (defineFactory / SessionFactoryApi)
Node.js (this PR) ✅ Added here
Python ✅ (in generated/rpc.py) ❌ Not present
Go ✅ (in rpc/zrpc.go) ❌ Not present
.NET ✅ (in Generated/Rpc.cs) ❌ Not present
Rust ✅ (in generated/rpc.rs) ❌ Not present
Java ✅ (generated types) ❌ Not present

Assessment

This is an expected, intentional gap for the initial landing — the PR description explicitly notes the feature is dark behind the runtime's agent_factories flag and all public types are marked @experimental. The generated RPC types already exist in all SDKs from schema codegen, so the underlying wire protocol is covered everywhere.

The higher-level authoring surface (the defineFactory wrapper, the FactoryContext with spawnAgent/journal/parallel/pipeline helpers, and waitForRun) is Node.js-only after this PR. Follow-up work to reach cross-SDK parity would involve:

  • Python: define_factory() + FactoryContext dataclass/protocol, registration in join_session()
  • Go: DefineFactory() + FactoryContext interface, registration in JoinSession()
  • .NET: DefineFactory() + IFactoryContext / delegate, registration in JoinSession()
  • Java: defineFactory() + FactoryContext interface, registration in joinSession()
  • Rust: define_factory() + FactoryContext trait, registration in join_session()

This review is informational only — no blocking concerns for this PR. Tracking the follow-up implementations in remaining SDKs is recommended.

Generated by SDK Consistency Review Agent for #2114 · sonnet46 37.5 AIC · ⌖ 5.51 AIC · ⊞ 6.6K ·

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Comments suppressed due to low confidence (4)

nodejs/test/factory.test.ts:1630

  • This missing-factory invocation also omits the required executionToken, so the test request does not match FactoryExecuteRequest. Include a token even on the error path to ensure the failure is specifically about the factory name.
                args: {},

nodejs/src/factory.ts:140

  • The volatile branch intentionally bypasses JSON validation and the new test returns a function from it, but this public signature still restricts every producer/result to JsonValue. Consumers therefore need the same unsafe cast used by the test. Add a volatile: true generic overload that preserves arbitrary local result types, while keeping the journaled overload restricted to JsonValue, and update the implementation signature accordingly.
    step(
        key: string,
        producer: () => Promise<JsonValue> | JsonValue,
        options?: FactoryStepOptions
    ): Promise<JsonValue>;

nodejs/src/extension.ts:110

  • This is the integration boundary for the feature, but the added tests mock session.resume and reverse RPCs rather than exercising registration and execution through the replay runtime. Add an E2E case (and snapshot) that joins with factory metadata, receives factory.execute, and completes session.factory.run; otherwise protocol naming, feature-gate behavior, and runtime compatibility can all regress while the unit suite stays green.
    return client.resumeSessionForExtension(
        sessionId,
        {
            ...rest,
            onPermissionRequest: config.onPermissionRequest ?? defaultJoinSessionPermissionHandler,
            suppressResumeEvent: config.suppressResumeEvent ?? true,
        },
        factories

nodejs/test/factory.test.ts:1167

  • FactoryExecuteRequest.executionToken is required, but this test invokes the handler without it. Add a token so the test uses the actual reverse-RPC contract and does not silently exercise map/RPC behavior with an undefined token.

This issue also appears on line 1630 of the same file.

                args: {},
  • Files reviewed: 14/16 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants