Merge remote-tracking branch 'origin/staging' into fix/csv-import-tx-and-schema

# Conflicts:
#	apps/sim/lib/table/service.ts

Author: TheodoreSpeaks

.agents/skills/emcn-design-review/SKILL.md

@@ -234,7 +234,7 @@ Use for context menus and action menus:
 <DropdownMenu>
   <DropdownMenuTrigger asChild>
     <Button variant="ghost">
-      <MoreHorizontal className="h-[14px] w-[14px]" />
+      <MoreHorizontal className="size-[14px]" />
     </Button>
   </DropdownMenuTrigger>
   <DropdownMenuContent align="end">
@@ -281,19 +281,21 @@ Rules:
 - Stack multiple skeletons for lists
 
 ### Icons
-Standard sizing — `h-[14px] w-[14px]` is the dominant pattern (400+ uses):
+Standard sizing — use the `size-*` shorthand. `size-[14px]` is the dominant pattern:
 
 ```tsx
-<Icon className="h-[14px] w-[14px] text-[var(--text-icon)]" />
+<Icon className="size-[14px] text-[var(--text-icon)]" />
 ```
 
-Size scale by frequency:
-1. `h-[14px] w-[14px]` — default for inline icons (most common)
-2. `h-[16px] w-[16px]` — slightly larger inline icons
-3. `h-3 w-3` (12px) — compact/tight spaces
-4. `h-4 w-4` (16px) — Tailwind equivalent, also common
-5. `h-3.5 w-3.5` (14px) — Tailwind equivalent of 14px
-6. `h-5 w-5` (20px) — larger icons, section headers
+Always prefer `size-*` over the legacy `h-* w-*` pair. `size-[14px]` is canonical; treat any `h-[Npx] w-[Npx]` or `h-N w-N` pair as a refactor target.
+
+Size scale (most common first):
+1. `size-[14px]` — default for inline icons
+2. `size-[16px]` — slightly larger inline icons
+3. `size-3` (12px) — compact/tight spaces
+4. `size-4` (16px) — Tailwind equivalent
+5. `size-3.5` (14px) — Tailwind equivalent of 14px
+6. `size-5` (20px) — larger icons, section headers
 
 Use `text-[var(--text-icon)]` for icon color (113+ uses in codebase).
 
@@ -332,4 +334,5 @@ Use `text-[var(--text-icon)]` for icon color (113+ uses in codebase).
 - Importing from emcn subpaths instead of barrel export
 - Using arbitrary z-index (`z-50`, `z-[9999]`) instead of z-index tokens
 - Custom shadows instead of shadow tokens
-- Icon sizes that don't follow the established scale (default to `h-[14px] w-[14px]`)
+- Icon sizes that don't follow the established scale (default to `size-[14px]`)
+- Splitting equal height/width into `h-* w-*` pairs instead of the `size-*` shorthand

.agents/skills/memory-load-check/SKILL.md

@@ -0,0 +1,138 @@
+---
+name: memory-load-check
+description: Review PRs and diffs for unbounded memory loading, concurrency explosions, oversized payload materialization, and missing pagination or byte caps. Use when reviewing cleanup jobs, background jobs, data imports/exports, file parsing, API fan-out, workflow execution payloads, large arrays/files, or any change that reads many rows, files, responses, logs, or external API pages into process memory.
+---
+
+# Memory Load Check
+
+Use this skill when a PR or diff could load unbounded data into a Node/Bun process, especially in cron routes, background tasks, API routes, workflow execution, file parsing, cleanup jobs, migrations, import/export flows, and external API integrations.
+
+## Review Goal
+
+Prove each changed path has explicit bounds for:
+- rows held in memory
+- bytes held in memory
+- concurrent promises, DB queries, HTTP calls, storage operations, and jobs
+- number of pages, batches, chunks, retries, and retained intermediate objects
+
+If any bound depends only on current production size or "probably small" data, treat it as a finding.
+
+## References
+
+Read these when doing a deeper pass:
+- Node.js streams/backpressure: https://nodejs.org/learn/modules/backpressuring-in-streams
+- Node.js stream usage: https://nodejs.org/en/learn/modules/how-to-use-streams
+- Keyset/cursor pagination over offset scans: https://blog.sequinstream.com/keyset-cursors-not-offsets-for-postgres-pagination/
+- Postgres pagination tradeoffs: https://www.citusdata.com/blog/2016/03/30/five-ways-to-paginate/
+
+## Sim Helpers To Prefer
+
+- `apps/sim/lib/cleanup/batch-delete.ts`
+  - `chunkedBatchDelete`: bounded SELECT -> optional side effect -> DELETE loop.
+  - `batchDeleteByWorkspaceAndTimestamp`: common workspace/timestamp cleanup wrapper.
+  - `selectRowsByIdChunks`: chunks large ID sets and enforces an overall row cap.
+  - `chunkArray`: use only after the input set itself is already bounded.
+- `apps/sim/lib/core/utils/stream-limits.ts`
+  - `PayloadSizeLimitError`
+  - `assertKnownSizeWithinLimit`
+  - `assertContentLengthWithinLimit`
+  - `readStreamToBufferWithLimit`
+  - `readNodeStreamToBufferWithLimit`
+  - `readResponseToBufferWithLimit`
+  - `readResponseTextWithLimit`
+- Cleanup dispatcher pattern in `apps/sim/lib/billing/cleanup-dispatcher.ts`
+  - page active workspaces with `WHERE id > afterId ORDER BY id LIMIT N`
+  - dispatch concrete chunks (`workspaceIds`, retention, label) instead of one giant scope
+  - prefer Trigger.dev queue/concurrency keys when available
+  - execute inline fallback chunks sequentially, not with unbounded `Promise.all`
+- File parse route pattern in `apps/sim/app/api/files/parse/route.ts`
+  - cap downloads and parsed output separately
+  - preserve partial results when a later item exceeds the cap
+  - never read untrusted response bodies without a byte cap
+- Large workflow value payloads
+  - prefer durable references/manifests over inlining large arrays or files
+  - materialize refs only behind an explicit byte budget
+
+## Review Workflow
+
+1. Identify every changed data source:
+   - database queries
+   - storage lists/downloads/uploads
+   - external API pagination
+   - file reads and HTTP responses
+   - workflow logs, snapshots, payloads, arrays, and manifests
+   - queues, cron routes, and background jobs
+2. For each source, write down the maximum cardinality and maximum bytes. If the code does not enforce one, it is unbounded.
+3. Trace whether data is processed incrementally or accumulated:
+   - arrays from `select`, `findMany`, `Promise.all`, `map`, `filter`, `flatMap`
+   - maps/sets keyed by all users, workspaces, executions, files, or rows
+   - `Buffer.concat`, `response.arrayBuffer()`, `response.text()`, `JSON.stringify`, `JSON.parse`
+   - queues of promises or job payloads built before dispatch
+4. Check concurrency separately from memory:
+   - no `Promise.all(items.map(...))` unless `items` is already small and bounded
+   - use chunks, sequential loops, queue concurrency, or a concurrency limiter
+   - align concurrency with DB pool size, storage/API limits, and task queue semantics
+5. Verify SQL shape:
+   - every bulk query has `LIMIT`
+   - large pagination uses cursor/keyset style (`id > afterId`, timestamps plus unique ID), not deep `OFFSET`
+   - `IN (...)` lists are chunked
+   - side-effect rows selected before delete have per-batch and per-run caps
+6. Verify byte safety:
+   - check `Content-Length` when available
+   - stream with cumulative byte accounting
+   - cap both input bytes and expanded output bytes
+   - reject or reference oversized values before serializing large JSON responses
+7. Confirm failure behavior:
+   - exceeding a cap should stop before loading more data
+   - partial successful work should be preserved when the API contract expects it
+   - retries should not duplicate huge in-memory state
+   - cleanup jobs should make progress over future runs instead of widening one run
+
+## Red Flags
+
+- loads all active workspaces, users, executions, logs, files, messages, or subscriptions before filtering
+- builds a full `Map` or `Set` for a platform-wide scope
+- uses `Promise.all` over rows from an unbounded query
+- fetches all pages from an external API before processing
+- reads an entire file, HTTP response, or stream without a max byte budget
+- checks size only after `Buffer.concat`, `arrayBuffer`, `text`, `JSON.parse`, or parse expansion
+- chunks only after loading the complete dataset
+- paginates with unbounded/deep `OFFSET` on a mutable or large table
+- creates one queue job per row without batching or a queue-level concurrency key
+- accumulates per-row errors/results with no maximum
+- adds a cache, singleton, or module-level collection without eviction or size limits
+
+## Preferred Fixes
+
+- Move filters into SQL/API requests and select only needed columns.
+- Replace full-table loads with cursor/keyset pagination and a deterministic order.
+- Process one page/batch at a time; do not keep previous pages unless needed.
+- Add per-batch and per-run row caps so long backlogs drain across repeated jobs.
+- Split large ID lists with `selectRowsByIdChunks` or `chunkArray` after bounding the source.
+- Use `chunkedBatchDelete` for cleanup loops with row side effects.
+- Use stream-limit helpers for file/HTTP/body reads.
+- Store large workflow values as refs/manifests and materialize only within a caller budget.
+- Replace unbounded `Promise.all` with sequential chunk loops, queue concurrency, or a small limiter.
+- Include tests that prove caps stop work early and partial results or progress are preserved.
+
+## Findings Format
+
+Lead with concrete findings, ordered by risk:
+
+```markdown
+## Findings
+
+- **P1 Unbounded workspace load in cleanup dispatch** (`path/to/file.ts`)
+  The new path calls `select().from(workspace)` without a limit, then builds maps for every row before dispatch. In production this scales with all active workspaces and can exhaust the app process. Page by `workspace.id` with a fixed limit and dispatch bounded chunks.
+
+## Good Signals
+
+- Uses `readResponseToBufferWithLimit` for external downloads.
+- Inline fallback processes chunks sequentially.
+
+## Residual Risk
+
+- The row cap is explicit, but no test currently proves the loop stops at the cap.
+```
+
+Only say "good to go" when every changed source has explicit row, byte, and concurrency bounds or the boundedness is proven by a stable invariant.

.agents/skills/validate-integration/SKILL.md

@@ -102,8 +102,8 @@ For **every** tool file, check:
 - [ ] No fields are missing that the API provides and users would commonly need
 - [ ] No phantom fields defined that the API doesn't return
 - [ ] `optional: true` is set on fields that may not exist in all responses
-- [ ] When using `type: 'json'` and the shape is known, `properties` defines the inner fields
-- [ ] When using `type: 'array'`, `items` defines the item structure with `properties`
+- [ ] When using `type: 'json'` and the shape is known, `properties` defines the inner fields (tool outputs only — block outputs do not support `properties`)
+- [ ] When using `type: 'array'`, `items` defines the item structure with `properties` (tool outputs only)
 - [ ] Field descriptions are accurate and helpful
 
 ### Types (types.ts)
@@ -190,9 +190,8 @@ For **each tool** in `tools.access`:
 ### Block Outputs
 - [ ] Outputs cover the key fields returned by ALL tools (not just one operation)
 - [ ] Output types are correct (`'string'`, `'number'`, `'boolean'`, `'json'`)
-- [ ] `type: 'json'` outputs either:
-  - Describe inner fields in the description string (GOOD): `'User profile (id, name, username, bio)'`
-  - Use nested output definitions (BEST): `{ id: { type: 'string' }, name: { type: 'string' } }`
+- [ ] `type: 'json'` outputs describe inner fields in the description string: `'User profile (id, name, username, bio)'` or `'[{address, status, type}]'` for arrays
+- [ ] **Do NOT add a `properties: {...}` field on block outputs.** Block-level `OutputFieldDefinition` (from `@sim/workflow-types/blocks`) only accepts `{ type, description?, condition?, hiddenFromDisplay? }`. Nested `properties` is a tool-level construct (`OutputProperty`) — adding it to a block output will fail TypeScript at build time
 - [ ] No opaque `type: 'json'` with vague descriptions like `'Response data'`
 - [ ] Outputs that only appear for certain operations use `condition` if supported, or document which operations return them
 
@@ -232,13 +231,23 @@ If any tools support pagination:
 - [ ] Pagination response fields (`nextToken`, `cursor`, etc.) are included in tool outputs
 - [ ] Pagination subBlocks are set to `mode: 'advanced'`
 
-## Step 7: Validate Error Handling
+## Step 7: Validate Memory Load Safety
+
+If any tool lists, searches, exports, imports, downloads, uploads, paginates, batches, transforms arrays, or reads file/HTTP bodies, read `.agents/skills/memory-load-check/SKILL.md` and apply it to the integration.
+
+- [ ] List/search tools expose API limits and do not auto-fetch every page into memory
+- [ ] Transform logic does not build unbounded arrays, maps, sets, or `Promise.all` fan-outs
+- [ ] File and HTTP body reads use explicit byte caps or existing stream-limit helpers
+- [ ] Large result payloads are summarized, paginated, referenced, or capped rather than raw-dumped
+- [ ] Pagination and download tests cover caps, early stop behavior, or partial-result preservation when relevant
+
+## Step 8: Validate Error Handling
 
 - [ ] `transformResponse` checks for error conditions before accessing data
 - [ ] Error responses include meaningful messages (not just generic "failed")
 - [ ] HTTP error status codes are handled (check `response.ok` or status codes)
 
-## Step 8: Report and Fix
+## Step 9: Report and Fix
 
 ### Report Format
 
@@ -297,6 +306,7 @@ After fixing, confirm:
 - [ ] Validated OAuth scopes use centralized utilities (getScopesForService, getCanonicalScopesForProvider) — no hardcoded arrays
 - [ ] Validated scope descriptions exist in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` for all scopes
 - [ ] Validated pagination consistency across tools and block
+- [ ] Validated memory load safety using `.agents/skills/memory-load-check/SKILL.md` when tools list/search/download/import/export/batch data
 - [ ] Validated error handling (error checks, meaningful messages)
 - [ ] Validated registry entries (tools and block, alphabetical, correct imports)
 - [ ] Reported all issues grouped by severity