Skip to content

test: run the JS SDK examples on the local runner - #745

Open
yaythomas wants to merge 1 commit into
mainfrom
feat/js-examples-harness
Open

yaythomas wants to merge 1 commit into
mainfrom
feat/js-examples-harness

Conversation

@yaythomas

@yaythomas yaythomas commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Adds .github/scripts/js_examples/run.py, which runs the JS SDK's example tests against the local runner in the testing package. No AWS account is involved; everything runs on the loopback.

hatch run dev-testing:js-examples                 # all examples
JS_SDK_DIR=~/github/aws-durable-execution-sdk-js \
  hatch run dev-testing:js-examples invoke        # your own JS checkout, one group
  • run.py uses a JS SDK checkout from --js-dir or JS_SDK_DIR, or clones JS main. It builds the examples, starts the servers, runs jest, and exits non-zero on any failure. It refuses to run unless its interpreter has this checkout's testing package.
  • lambda-shim.cjs hosts the built JS handlers, as the Lambda runtime does. It runs each function on reusable worker threads. When an invocation passes the function's Timeout, it returns Sandbox.Timedout and terminates the worker. step/interrupted-no-retry depends on this.
  • run.py serves WebRunner in a child process (multiprocessing), so each server session starts with no runner state. invoke_proxy.py, on a thread, turns the Lambda Invoke call the JS tests use to start an execution into the runner's POST /start-durable-execution, and forwards everything else unchanged.
  • .github/workflows/js-examples.yml runs the harness on PRs and pushes that touch the testing package. A manual run can choose another JS branch, tag or commit with js_ref. The harness unit tests run in test-parser.yml.

Locally and in CI: 121 of 121 test files with cloud tests pass in about 2.5 minutes on 4 CPUs. Usage is in .github/scripts/js_examples/README.md.

Canary: #746 dropped the chained-invoke result in the runner, and its run failed on invoke/simple as expected (run).

@yaythomas
yaythomas force-pushed the feat/js-examples-harness branch 3 times, most recently from 8d7ad48 to 63d202a Compare September 30, 2026 06:42
@yaythomas
yaythomas marked this pull request as ready for review September 30, 2026 15:34
@yaythomas
yaythomas deployed to ai-pr-review-runtime September 30, 2026 15:34 — with GitHub Actions Active
@yaythomas
yaythomas deployed to ai-pr-review-runtime September 30, 2026 15:38 — with GitHub Actions Active
Comment thread .github/scripts/js_examples/run.py Outdated
Comment thread .github/scripts/js_examples/run.py Outdated
Comment thread .github/scripts/js_examples/lambda-shim.cjs
Comment thread .github/scripts/js_examples/run.py Outdated
Comment thread .github/scripts/js_examples/run.py Outdated
Comment thread .github/scripts/js_examples/invoke_proxy.py
Comment thread .github/scripts/js_examples/run.py Outdated
@github-actions

This comment has been minimized.

@yaythomas
yaythomas force-pushed the feat/js-examples-harness branch from 63d202a to ecfaf35 Compare September 30, 2026 16:19
@yaythomas
yaythomas deployed to ai-pr-review-runtime September 30, 2026 16:28 — with GitHub Actions Active
Comment thread .github/scripts/js_examples/run.py
Comment thread .github/scripts/js_examples/run.py
Comment thread .github/scripts/js_examples/invoke_proxy.py
Comment thread .github/scripts/js_examples/run.py
@github-actions

This comment has been minimized.

Add .github/scripts/js_examples/run.py. It runs the JS SDK's example
tests against the testing package's local runner, and exits non-zero
on any failure. It uses a JS SDK checkout from --js-dir or JS_SDK_DIR,
or clones JS main. Run it with hatch run dev-testing:js-examples.

run.py serves WebRunner in a child process, so each server session
starts with no runner state. A small proxy on a thread turns the
tests' Lambda Invoke into POST /start-durable-execution.

The Lambda shim runs each function on reusable worker threads. When an
invocation passes the function's Timeout, the shim terminates the
worker, as Lambda does. step/interrupted-no-retry depends on this.

The js-examples workflow runs the harness on changes to the testing
package. All 121 test files with cloud tests pass in about 2.5 minutes
on 4 CPUs.
@yaythomas
yaythomas force-pushed the feat/js-examples-harness branch from ecfaf35 to 043d3de Compare October 1, 2026 19:22
@yaythomas
yaythomas deployed to ai-pr-review-runtime October 1, 2026 20:36 — with GitHub Actions Active
Comment on lines +296 to +299
durable = props.get("DurableConfig")
function_configs[function_name] = (
{"DurableConfig": dict(durable)} if durable else {}
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex AI review · Finding arf_v1_2q3t5r6bt6vqonbfkuf3wraohv

[P2] Preserve an empty DurableConfig as durable

The runner treats {"DurableConfig": {}} as a durable target with default timeout and retention, but this truthiness check emits {} and classifies it as plain. A chained invoke then runs it as a one-shot Lambda invocation instead of a child durable execution. Check for None or key presence and add an empty-config test.

Suggested change
durable = props.get("DurableConfig")
function_configs[function_name] = (
{"DurableConfig": dict(durable)} if durable else {}
)
durable = props.get("DurableConfig")
function_configs[function_name] = (
{"DurableConfig": dict(durable)} if durable is not None else {}
)

Comment on lines +128 to +129
"FunctionName": function_name,
"FunctionQualifier": "$LATEST",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex AI review · Finding arf_v1_l56oh7o542gv5lkojghfxre2xz

[P2] Split qualified identifiers before starting the runner

For name:7 or a function ARN, this stores the entire identifier as FunctionName and hard-codes $LATEST. The runner expects the bare name and qualifier separately, so execution details report malformed ARNs and the wrong version even though the shim happens to find the handler. Parse path and query qualifiers into separate fields and assert the complete start payload in the proxy tests.

functionName,
// An alias names no version the shim knows, so only a numeric
// qualifier is reported as the version.
functionVersion: /^\d+$/.test(qualifier ?? "") ? qualifier : "$LATEST",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex AI review · Finding arf_v1_dijv4e2didn4fvufp3ensmydyy

[P2] Report $LATEST.PUBLISHED as the executed version

The testing runner explicitly preserves $LATEST.PUBLISHED as an executed version, but the shim maps every nonnumeric qualifier to $LATEST. Handlers invoked with this supported qualifier therefore observe a context inconsistent with the runner's execution details. Preserve this special qualifier and add it to the qualified-identifier tests.

Suggested change
functionVersion: /^\d+$/.test(qualifier ?? "") ? qualifier : "$LATEST",
functionVersion:
qualifier === "$LATEST.PUBLISHED" || /^\d+$/.test(qualifier ?? "")
? qualifier
: "$LATEST",

Comment on lines +994 to +995
for directory in old:
shutil.rmtree(directory, ignore_errors=True)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex AI review · Finding arf_v1_airpqn5cq3y57oztz6av4w7kyf

[P3] Do not prune output directories that are still active

OUT_MARKER is created when a run starts, so an active directory becomes eligible for deletion here. Once ten newer runs start across other JS checkouts, this can remove the older run's logs, maps, and Jest output while they are still in use. Track completion or hold a per-run lock and prune only completed or unlocked directories, with a concurrent-pruning test.

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Codex AI review

Four actionable correctness issues remain, primarily around durable-target classification and qualified invocation semantics.

Reviewed commit 043d3decfcd46faf521f8cddcc42c0e71b110cc5. Workflow run

This branch was successfully deployed

1 active deployment
ai-pr-review-runtime — 043d3dec Deployed Oct 1, 2026 by yaythomas via ai-pr-review / Codex review / Generate Codex review #1125
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant