Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions .github/workflows/windows-render.yml
Original file line number Diff line number Diff line change
Expand Up @@ -353,6 +353,11 @@ jobs:
# invocations, CRLF, file URLs, etc.) in existing vitest suites.
# The producer package is skipped because its tests require Docker /
# Linux-only tooling (Dockerfile.test, LFS golden MP4 baselines).
# The aws-lambda package is skipped because it targets the AWS Lambda
# Linux runtime exclusively (`@sparticuz/chromium` is Linux-only; the
# ZIP layout is for `/var/task` on AL2023). Its handler.test.ts also
# trips a bun-on-Windows workspace-symlink quirk reading the
# producer's transitive `hono` dep.
# -------------------------------------------------------------------
test-windows:
name: Tests on windows-latest
Expand Down Expand Up @@ -407,9 +412,15 @@ jobs:
shell: pwsh
run: bun run build

- name: Run tests (all packages except producer)
- name: Run tests (all packages except producer + aws-lambda)
shell: pwsh
run: bun run --filter "!@hyperframes/producer" test
# Enumerate the packages we want to test instead of negating —
# `bun run --filter "!a" --filter "!b"` composes as a union (any
# package matching either negation runs), not an intersection.
# That meant `@hyperframes/producer` was effectively still being
# tested on Windows, which is what we explicitly want skipped
# (Docker / LFS-baseline tooling).
run: bun run --filter @hyperframes/core --filter @hyperframes/engine --filter @hyperframes/player --filter @hyperframes/cli --filter @hyperframes/studio --filter @hyperframes/shader-transitions test

- name: Run runtime contract test
shell: pwsh
Expand Down
1 change: 1 addition & 0 deletions Dockerfile.test
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ COPY packages/producer/package.json packages/producer/package.json
COPY packages/cli/package.json packages/cli/package.json
COPY packages/studio/package.json packages/studio/package.json
COPY packages/shader-transitions/package.json packages/shader-transitions/package.json
COPY packages/aws-lambda/package.json packages/aws-lambda/package.json
RUN bun install --frozen-lockfile

# Copy source
Expand Down
207 changes: 196 additions & 11 deletions bun.lock

Large diffs are not rendered by default.

113 changes: 113 additions & 0 deletions packages/aws-lambda/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# @hyperframes/aws-lambda

AWS Lambda adapter for HyperFrames distributed rendering. Wraps the OSS
`plan` / `renderChunk` / `assemble` primitives into a single Lambda handler
that Step Functions can dispatch on, plus a build pipeline that bundles
the handler + Chrome runtime + ffmpeg into a deployable ZIP.

The Lambda adapter ships in two parts: the foundation (this package + the
SAM example) validates the architecture end-to-end on real AWS; the
user-facing surface (CLI, CDK construct, migration guide) lands in
follow-up PRs.

## Architecture

```
┌──────────────────────────────────────────────────────────────────┐
│ Step Functions state machine │
│ Plan → Map(N) RenderChunk → Assemble │
└──────────────────────────────────────────────────────────────────┘
│ dispatches by event.Action
┌──────────────────────────────────────────────────────────────────┐
│ One Lambda function (this package's `dist/handler.zip`) │
│ handler.mjs │
│ ├─ Action="plan" → @hyperframes/producer/distributed │
│ ├─ Action="renderChunk" → @hyperframes/producer/distributed │
│ └─ Action="assemble" → @hyperframes/producer/distributed │
│ bin/ffmpeg — ffmpeg-static │
│ node_modules/@sparticuz/chromium/ — Lambda-optimised Chromium │
└──────────────────────────────────────────────────────────────────┘
│ pure functions over local paths
┌──────────────────────────────────────────────────────────────────┐
│ S3 bucket — plan tarball + per-chunk outputs + final mp4 │
└──────────────────────────────────────────────────────────────────┘
```

The handler downloads inputs from S3 into `/tmp`, calls the OSS primitive,
uploads outputs back to S3, and returns a small JSON result that fits
inside Step Functions' history budget (under 200 bytes per chunk).

## Chrome runtime

The package supports two Chromium sources:

| Source | Default | Size | When to pick it |
| ------------------------------- | ------- | ------------------ | --------------------------------------------------------------------------------------------------------------------- |
| `@sparticuz/chromium` | yes | ~70 MiB compressed | Lambda. Decompresses into `/tmp` at runtime; the rest of the ecosystem already uses it for headless-Chrome-in-Lambda. |
| Bundled `chrome-headless-shell` | no | ~140 MiB | Fallback. Used if `@sparticuz/chromium` ever drops `HeadlessExperimental.beginFrame` support. |

Pick the source at build time:

```bash
bun run --cwd packages/aws-lambda build:zip
bun run --cwd packages/aws-lambda build:zip -- --source=chrome-headless-shell
```

The handler reads `HYPERFRAMES_LAMBDA_CHROME_SOURCE` at boot. The build
script sets that env var via Lambda function configuration in
`examples/aws-lambda/template.yaml`.

## BeginFrame regression guard

HyperFrames' renderer drives Chrome via the CDP
`HeadlessExperimental.beginFrame` command — same path the K8s deploy uses.
The Lambda adapter assumes that `@sparticuz/chromium`'s
chrome-headless-shell build honours BeginFrame. To prove it (and re-prove
it on every release), the package ships a Docker probe:

```bash
# Build the Lambda-like container and run the probe.
bun run --cwd packages/aws-lambda probe:beginframe:docker
```

The probe boots `@sparticuz/chromium` inside
`public.ecr.aws/lambda/nodejs:22` and asserts CDP `beginFrame` with
`screenshot: true` returns a PNG buffer. Exit code 0 = green; non-zero =
fall back to bundling chrome-headless-shell directly via `--source=chrome-headless-shell`.

## Building the ZIP

```bash
bun install # at the monorepo root
bun run --cwd packages/aws-lambda build:zip # → packages/aws-lambda/dist/handler.zip
bun run --cwd packages/aws-lambda verify:zip-size # CI gate
```

The build script bundles `src/handler.ts` via esbuild, stages
`@sparticuz/chromium` and `puppeteer-core` under `node_modules/`, copies
ffmpeg-static into `bin/`, and zips the result. The unzipped layout is
designed to extract cleanly into Lambda's `/var/task/`.

`verify:zip-size` enforces:

- Unzipped ≤ 248 MiB (in-house budget; Lambda hard ceiling is 250 MiB unzipped — AWS docs label this "250 MB" but use binary mebibytes)
- Zipped ≤ 150 MiB (in-house budget; Lambda has no hard zipped cap for S3-deployed functions)

CI fails the PR if either is exceeded.

## Running tests

```bash
bun run --cwd packages/aws-lambda test # unit tests (no Chrome)
bun run --cwd packages/aws-lambda probe:beginframe # local probe (Linux only)
```

## What's NOT in this PR

- `examples/aws-lambda/template.yaml` (SAM template — separate PR).
- Real-AWS deploy + smoke workflow (separate PR).
- `npx hyperframes lambda deploy` CLI — follow-up.
- CDK construct (`HyperframesRenderStack`) — follow-up.
- Migration guide — follow-up.
55 changes: 55 additions & 0 deletions packages/aws-lambda/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
{
"name": "@hyperframes/aws-lambda",
"version": "0.0.1",
"description": "AWS Lambda adapter for HyperFrames distributed rendering — Plan/RenderChunk/Assemble handler + ZIP bundling.",
"repository": {
"type": "git",
"url": "https://github.com/heygen-com/hyperframes",
"directory": "packages/aws-lambda"
},
"files": [
"src/",
"scripts/",
"README.md"
],
"type": "module",
"main": "./src/index.ts",
"types": "./src/index.ts",
"exports": {
".": "./src/index.ts",
"./handler": "./src/handler.ts"
},
"publishConfig": {
"access": "public",
"registry": "https://registry.npmjs.org/"
},
"scripts": {
"build": "tsc --noEmit",
"build:zip": "tsx scripts/build-zip.ts",
"probe:beginframe": "tsx scripts/probe-beginframe.ts",
"probe:beginframe:docker": "docker build -f scripts/probe-beginframe.dockerfile -t hyperframes-lambda-probe:local ../.. && docker run --rm hyperframes-lambda-probe:local",
"test": "bun test",
"typecheck": "tsc --noEmit",
"verify:zip-size": "tsx scripts/verify-zip-size.ts"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.700.0",
"@hyperframes/producer": "workspace:^",
"@sparticuz/chromium": "148.0.0",
"ffmpeg-static": "^5.2.0",
"ffprobe-static": "^3.1.0",
"puppeteer-core": "^24.39.1",
"tar": "^7.4.3"
},
"devDependencies": {
"@types/aws-lambda": "^8.10.146",
"@types/node": "^25.0.10",
"@types/tar": "^6.1.13",
"esbuild": "^0.25.12",
"tsx": "^4.21.0",
"typescript": "^5.7.2"
},
"engines": {
"node": ">=22"
}
}
15 changes: 15 additions & 0 deletions packages/aws-lambda/scripts/_formatBytes.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
/**
* Shared binary-unit byte formatter for the build/verify scripts.
*
* The Lambda ZIP-size budget is in mebibytes (Lambda's 250 MB / 248 MiB
* gate is binary, not decimal), so logs and CI failure messages use
* KiB / MiB / GiB. This is intentionally a different unit system from
* `packages/cli/src/ui/format.ts`'s `formatBytes` (KB / MB, decimal) —
* don't conflate them.
*/
export function formatBytes(bytes: number): string {
if (bytes < 1024) return `${bytes} B`;
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KiB`;
if (bytes < 1024 * 1024 * 1024) return `${(bytes / (1024 * 1024)).toFixed(1)} MiB`;
return `${(bytes / (1024 * 1024 * 1024)).toFixed(2)} GiB`;
}
Loading
Loading