Skip to content

feat(stack): EQL v3 typed schema + strongly-typed client (@cipherstash/stack/schema/v3, /v3)#535

Open
tobyhede wants to merge 33 commits into
mainfrom
feat/eql-v3-text-search-schema
Open

feat(stack): EQL v3 typed schema + strongly-typed client (@cipherstash/stack/schema/v3, /v3)#535
tobyhede wants to merge 33 commits into
mainfrom
feat/eql-v3-text-search-schema

Conversation

@tobyhede

@tobyhede tobyhede commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

EQL v3 typed schema + strongly-typed client (@cipherstash/stack/schema/v3, @cipherstash/stack/v3)

Expands schema/v3 from the single text_search builder to every generated EQL v3 SQL domain — each a typed builder with explicit query capabilities and per-domain plaintext inference — and adds a strongly-typed v3 client (@cipherstash/stack/v3) that derives input/output types from the schema and rejects misuse at compile time. v2 is unchanged; pick the model by import path.

Usage

import { Encryption } from "@cipherstash/stack";
import { encryptedTable, encryptedTextSearchColumn } from "@cipherstash/stack/schema/v3";

// 1. Define your schema
const users = encryptedTable("users", {
  email: encryptedTextSearchColumn("email"),
});

// 2. Initialize the client
const client = await Encryption({ schemas: [users] });

// 3. Encrypt
const encryptResult = await client.encrypt("secret@example.com", {
  column: users.email,
  table: users,
});
if (encryptResult.failure) {
  // Handle errors your way
}

// 4. Decrypt
const decryptResult = await client.decrypt(encryptResult.data);
// decryptResult.data => "secret@example.com"

Mix any v3 domains in one table — each column declares its own type and query capabilities:

import {
  encryptedTable,
  encryptedTextEqColumn,
  encryptedInt4OrdColumn,
  encryptedTimestamptzColumn,
} from "@cipherstash/stack/schema/v3";

const events = encryptedTable("events", {
  actor:     encryptedTextEqColumn("actor"),           // equality
  weight:    encryptedInt4OrdColumn("weight"),         // order + range
  createdAt: encryptedTimestamptzColumn("created_at"), // storage only
});

Plaintext types are inferred per domain, and Date / bigint values work directly:

import type { InferPlaintext } from "@cipherstash/stack/schema/v3";

type Events = InferPlaintext<typeof events>;
// { actor: string; weight: number; createdAt: Date }

await client.encrypt(new Date(), { table: events, column: events.createdAt });

Queryability is enforced at compile time — storage-only columns can't be queried:

await client.encryptQuery(30, {
  table: events,
  column: events.weight,
  queryType: "orderAndRange",
});

await client.encryptQuery(new Date(), { table: events, column: events.createdAt });
//                                                            ^ type error: not queryable

Domains & capabilities

Every domain maps to an EQL v3 SQL type and exposes getQueryCapabilities() / isQueryable().

Suffix Example builder Capabilities
(none) encryptedInt4Column storage only
_eq encryptedTextEqColumn equality
_ord, _ord_ore encryptedInt4OrdColumn equality + order/range
text_match encryptedTextMatchColumn free-text
text_search encryptedTextSearchColumn equality + order/range + free-text

Covered: int2/4/8, float4/8, numeric, date, timestamptz, text*, bool → inferred as number / bigint / Date / string / boolean.

Strongly-typed client (@cipherstash/stack/v3)

A dedicated, definitively-typed client surface for v3 schemas. EncryptionV3 mirrors Encryption; typedClient retypes an existing client. Both re-export the v3 builders, so a single import provides everything needed to author and use a schema.

import { EncryptionV3, encryptedTable, encryptedTextSearchColumn } from "@cipherstash/stack/v3";

const users = encryptedTable("users", { email: encryptedTextSearchColumn("email") });
const client = await EncryptionV3({ schemas: [users] });

await client.encrypt("a@b.com", { table: users, column: users.email }); // ok
await client.encrypt(123,       { table: users, column: users.email }); // ✗ number ≠ string

Every method derives its types from the concrete table / column arguments:

  • encrypt / encryptQuery pin the plaintext to the column's domain type (text → string, int8 → bigint, timestamptz → Date, …). encryptQuery additionally constrains queryType to the column's capabilities and rejects storage-only columns at compile time.
  • encryptModel / bulkEncryptModels validate schema-column fields against their inferred plaintext type (passthrough fields like id are untouched) and return a precise encrypted model.
  • decryptModel / bulkDecryptModels return the precise plaintext model, reconstructing Date / bigint values from the encrypt-config cast_as.

Because the typed methods bind to the concrete branded v3 classes, a hand-rolled structural table/column — or a column borrowed from a different table's domain — is rejected. This closes the soundness gap where a non-branded structural table could be encrypted at runtime while typed as plaintext.

Also

  • encryptModel / bulkEncryptModels accept any v3 table (structural BuildableTable); v2 unchanged.
  • encrypt / encryptQuery value type widened to include Date | bigint (new Plaintext type) — the FFI declares these as cast targets but omits them from its JsPlaintext input union.
  • Runtime payloads and emitted config are unchanged; text_search stays byte-identical to a v2 equality().orderAndRange().freeTextSearch() column. The only runtime addition is the per-column Date / bigint reconstruction on the typed client's model-decrypt paths.

Notes

Pre-existing on this branch, not introduced here: pnpm build's dts step may fail on src/wasm-inline.ts (missing @cipherstash/auth dep) in some environments, and live client/pg tests require CipherStash credentials. v3 unit + type tests are green (test:types: 50/50, including the untouched v2 and v3 .test-d.ts files) and v3 dist artifacts build (dist/encryption/v3.*).

The typed client's decrypt reconstruction is gated on a live round-trip check (needs credentials): if the FFI already returns Date / bigint, the reconstruction can be dropped as a pure-type optimization — it is idempotent and safe either way.

@tobyhede tobyhede requested a review from a team as a code owner June 30, 2026 22:27
@changeset-bot

changeset-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 46f0071

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 6 packages
Name Type
@cipherstash/stack Minor
@cipherstash/bench Patch
@cipherstash/prisma-next Patch
@cipherstash/basic-example Patch
@cipherstash/prisma-next-example Patch
@cipherstash/e2e Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR adds a v3 text_search schema builder API, widens encryption/query types to accept structural builders, wires export and typecheck support, and updates CLI dotenv loading plus E2E test execution.

Changes

EQL v3 text_search schema DSL and client type widening

Layer / File(s) Summary
Planning and design documentation
.changeset/eql-v3-text-search.md, .changeset/eql-v3-typed-schema.md, docs/superpowers/plans/..., docs/superpowers/specs/...
Adds a minor changeset note plus implementation plan and design spec documents for the v3 text_search DSL, compatibility constraints, and testing expectations.
v3 text_search schema builders
packages/stack/src/schema/v3/index.ts
Implements EncryptedTextSearchColumn, encryptedTextSearchColumn, v3 EncryptedTable/encryptedTable, buildEncryptConfig, and v3 inference helpers with deep-cloned build output and reserved-key collision checks.
Public client and operation type widening
packages/stack/src/types.ts, packages/stack/src/schema/index.ts, packages/stack/src/encryption/helpers/infer-index-type.ts, packages/stack/src/encryption/helpers/model-helpers.ts, packages/stack/src/encryption/index.ts, packages/stack/src/encryption/operations/*
Adds BuildableColumn, BuildableQueryColumn, and BuildableTable, updates client config and query/encrypt option types, and retypes the encrypt, bulk-encrypt, schema, and index-inference helpers to accept structural builders.
Structural column-name resolution
packages/stack/src/wasm-inline.ts, packages/stack/__tests__/wasm-inline-column-name.test.ts
Exports getColumnName and changes it to validate columns structurally through getName(), removing the previous instanceof-based checks for v2-only encrypted column classes.
Export wiring and typecheck pipeline
packages/stack/package.json, packages/stack/tsup.config.ts, packages/stack/tsconfig.typecheck.json, packages/stack/vitest.config.ts, .github/workflows/tests.yml, packages/stack/scripts/install-eql-v3.ts
Adds the ./schema/v3 package export and type mappings, includes the v3 entry in tsup, and wires scoped typecheck config, Vitest typecheck settings, a test:types script, and the CI step that runs it.
Runtime and type-level acceptance tests
packages/stack/__tests__/schema-v3.test.ts, packages/stack/__tests__/schema-v3.test-d.ts, packages/stack/__tests__/schema-v3-client.test.ts, packages/stack/__tests__/schema-v3-pg.test.ts, packages/stack/__tests__/cjs-require.test.ts, packages/stack/__tests__/helpers/*
Adds runtime tests for the v3 schema builders and type-level tests for v3 schema inference and client integration, plus live Postgres/CJS helper coverage and the negative queryability check.

CLI dotenv loading and non-PTY test helper

Layer / File(s) Summary
CLI dotenv loading and helper
packages/cli/src/bin/main.ts, packages/cli/tests/helpers/run.ts
Loads .env* files with quiet: true, adds the pipe-based run helper, and exports result types for CLI test execution.
CLI E2E updates
packages/cli/tests/e2e/runner-aware-help.e2e.test.ts, packages/cli/tests/e2e/smoke.e2e.test.ts
Switches runner-aware help tests to the new run helper and adds a smoke-test assertion that the dotenv banner is absent from help output.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related issues

Possibly related PRs

  • cipherstash/stack#493: Shares the same encryption query/bulk typing surface and the Plaintext widening work.
  • cipherstash/stack#496: Closely related to the structural getColumnName() change in packages/stack/src/wasm-inline.ts.
  • cipherstash/stack#497: Shares the same encryption operation code paths and lock-context typing changes.

Suggested reviewers: calvinbrewer

Poem

A bunny hopped through schemas bright,
With v3 text search taking flight 🐇
New builders bloom, the types align,
And CLI banners settle fine.
Thump! The tests now run with cheer,
Quiet envs and clean paths here.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 78.57% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: adding EQL v3 typed schema support and a strongly typed client surface.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/eql-v3-text-search-schema

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
packages/stack/src/types.ts (1)

111-113: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Possible redundant union arm.

If EncryptedColumn already has a public getEqlType() method and no private/protected members forcing nominal typing, the explicit EncryptedColumn arm is structurally subsumed by BuildableColumn & { getEqlType(): string } and could be dropped for simplicity. Not a functional issue either way; only worth simplifying if confirmed redundant.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/stack/src/types.ts` around lines 111 - 113, The BuildableQueryColumn
union appears to have a redundant arm if EncryptedColumn already satisfies
BuildableColumn & { getEqlType(): string } structurally. Check the
EncryptedColumn type and, if it does not rely on nominal typing via
private/protected members, simplify the BuildableQueryColumn alias in types.ts
by removing the explicit EncryptedColumn union member and keeping only the
shared structural form.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/stack/src/types.ts`:
- Around line 111-113: The BuildableQueryColumn union appears to have a
redundant arm if EncryptedColumn already satisfies BuildableColumn & {
getEqlType(): string } structurally. Check the EncryptedColumn type and, if it
does not rely on nominal typing via private/protected members, simplify the
BuildableQueryColumn alias in types.ts by removing the explicit EncryptedColumn
union member and keeping only the shared structural form.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 01405e99-c5e9-4153-8af3-10759fd8ebbd

📥 Commits

Reviewing files that changed from the base of the PR and between 93fc5f9 and efe4cc0.

📒 Files selected for processing (16)
  • .changeset/eql-v3-text-search.md
  • .github/workflows/tests.yml
  • docs/superpowers/plans/2026-06-30-eql-v3-text-search-schema-plan.md
  • docs/superpowers/specs/2026-06-30-eql-v3-text-search-schema-design.md
  • packages/stack/__tests__/schema-v3.test-d.ts
  • packages/stack/__tests__/schema-v3.test.ts
  • packages/stack/package.json
  • packages/stack/src/encryption/helpers/infer-index-type.ts
  • packages/stack/src/encryption/operations/bulk-encrypt.ts
  • packages/stack/src/encryption/operations/encrypt.ts
  • packages/stack/src/schema/index.ts
  • packages/stack/src/schema/v3/index.ts
  • packages/stack/src/types.ts
  • packages/stack/tsconfig.typecheck.json
  • packages/stack/tsup.config.ts
  • packages/stack/vitest.config.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
packages/cli/tests/helpers/run.ts (1)

44-87: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

No timeout/kill safeguard for a hung child.

If the spawned CLI process hangs (e.g., waiting on unexpected input despite stdio: ['ignore', ...]), nothing here kills it — the test will eventually time out via vitest, but the orphaned child process keeps running. Consider an optional timeout that calls child.kill() and rejects.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/tests/helpers/run.ts` around lines 44 - 87, The run helper
currently waits indefinitely for the spawned CLI in run() and only resolves on
close, so a hung child can outlive the test. Update
packages/cli/tests/helpers/run.ts by adding an optional timeout to RunOptions
and wiring it in run() to call child.kill() and reject if the process does not
exit in time, while preserving the existing stdout/stderr capture and cleanup in
the child.on('close') path.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/cli/tests/helpers/run.ts`:
- Around line 44-87: The run helper currently waits indefinitely for the spawned
CLI in run() and only resolves on close, so a hung child can outlive the test.
Update packages/cli/tests/helpers/run.ts by adding an optional timeout to
RunOptions and wiring it in run() to call child.kill() and reject if the process
does not exit in time, while preserving the existing stdout/stderr capture and
cleanup in the child.on('close') path.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8faee085-09ef-4411-822e-50addd54c10c

📥 Commits

Reviewing files that changed from the base of the PR and between 32707e2 and 30cd5f4.

📒 Files selected for processing (5)
  • packages/cli/src/bin/main.ts
  • packages/cli/tests/e2e/runner-aware-help.e2e.test.ts
  • packages/cli/tests/e2e/smoke.e2e.test.ts
  • packages/cli/tests/helpers/run.ts
  • packages/stack/src/schema/v3/index.ts
✅ Files skipped from review due to trivial changes (1)
  • packages/cli/tests/e2e/smoke.e2e.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/stack/src/schema/v3/index.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/stack/src/schema/v3/index.ts (1)

460-470: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Snapshot nested freeTextSearch() options when you store them.

Lines 465-466 keep opts.tokenizer and opts.token_filters by reference, so mutating the caller’s options object after configuration silently changes this builder’s later build() output. The rest of this class is explicitly avoiding shared nested state, so this should clone on write too.

Suggested fix
   freeTextSearch(opts?: MatchIndexOpts): this {
     // A fresh defaults object per call supplies the `?? ` fallbacks, so no
     // nested default object is ever shared into `this.matchOpts` by reference.
     const defaults = defaultMatchOpts()
 
     this.matchOpts = {
-      tokenizer: opts?.tokenizer ?? defaults.tokenizer,
-      token_filters: opts?.token_filters ?? defaults.token_filters,
+      tokenizer: opts?.tokenizer
+        ? { ...opts.tokenizer }
+        : { ...defaults.tokenizer },
+      token_filters: opts?.token_filters
+        ? opts.token_filters.map((f) => ({ ...f }))
+        : defaults.token_filters.map((f) => ({ ...f })),
       k: opts?.k ?? defaults.k,
       m: opts?.m ?? defaults.m,
       include_original: opts?.include_original ?? defaults.include_original,
     }
     return this
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/stack/src/schema/v3/index.ts` around lines 460 - 470, The
freeTextSearch method in the v3 schema builder is still storing nested options
by reference, so later mutations to the caller’s MatchIndexOpts can leak into
build() output. Update freeTextSearch in packages/stack/src/schema/v3/index.ts
to clone the tokenizer and token_filters values when assigning this.matchOpts,
matching the class’s existing clone-on-write approach and avoiding shared nested
state.
🧹 Nitpick comments (6)
docs/superpowers/plans/2026-07-01-eql-v3-typed-schema.md (3)

510-510: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Fix redundant phrasing.

"Repeat the same exact pattern" → "Repeat the same pattern" or "Repeat this exact pattern".

- Repeat the same exact pattern for:
+ Repeat the same pattern for:
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/superpowers/plans/2026-07-01-eql-v3-typed-schema.md` at line 510, Update
the phrasing in the plans document to remove redundancy: change the “Repeat the
same exact pattern for:” text to either “Repeat the same pattern for:” or
“Repeat this exact pattern for:”. Locate the sentence in the document section
containing that exact phrase and keep the rest unchanged.

Source: Linters/SAST tools


117-117: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider rewording for readability.

Three successive list items begin with "Schemas with". While this is a list format where parallelism is expected, consider varying the structure if the static analysis tool flagged it as an issue.

- - Schemas with required `hm` support equality.
- - Schemas with required `ob` support order/range.
- - Schemas with required `bf` support free-text search.
+ - `hm` required → equality support.
+ - `ob` required → order/range support.
+ - `bf` required → free-text search support.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/superpowers/plans/2026-07-01-eql-v3-typed-schema.md` at line 117, Reword
the list item about storage-only schemas to avoid repeating the same “Schemas
with” sentence pattern as the surrounding bullets. Update the wording in the
schema documentation section so it still conveys that schemas containing only v,
i, and c are storage-only, but uses a different sentence structure for
readability and to satisfy the static analysis warning.

Source: Linters/SAST tools


27-27: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider rewording for readability.

Three successive sentences beginning with "In" - though this appears to be in the file structure list where parallelism is intentional. Given the context is a structured plan document, this is acceptable but could be tightened.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/superpowers/plans/2026-07-01-eql-v3-typed-schema.md` at line 27, The
plan document wording is a bit repetitive in the file-structure list, where
several consecutive bullets/sentences start with the same “In” pattern. Rephrase
the affected entries in the schema-plan section to improve readability while
preserving the parallel structure, keeping the “BuildableTable” bullet clear and
concise.

Source: Linters/SAST tools

packages/stack/scripts/install-eql-v3.ts (1)

1-1: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Bare dotenv/config re-introduces the banner noise this cohort suppresses elsewhere.

The CLI entrypoint now loads env files with config({ path: '.env.local', quiet: true }) specifically to avoid dotenv v17's injected-env banner. This script imports dotenv/config directly with no options, so it will still print that banner whenever it runs (e.g. in CI logs).

♻️ Suggested fix
-import 'dotenv/config'
+import { config } from 'dotenv'
 import postgres from 'postgres'
 import { installEqlV3IfNeeded } from '../__tests__/helpers/eql-v3'
+
+config({ quiet: true })
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/stack/scripts/install-eql-v3.ts` at line 1, The install-eql-v3
script is importing dotenv in a way that re-enables the noisy injected-env
banner. Update the startup env loading in this script to use the same explicit
dotenv config pattern as the CLI entrypoint, with a fixed .env.local path and
quiet enabled, and remove the bare dotenv/config import so the script stays
silent in CI. Reference the script’s top-level env bootstrap in
install-eql-v3.ts.
packages/stack/__tests__/schema-v3.test.ts (1)

2-3: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Exercise this failure through a public API instead of resolveIndexType.

This test now imports @/encryption/helpers/infer-index-type directly and asserts the helper’s internal error strings, which makes the suite brittle to refactors inside the implementation rather than the supported contract. Please move this misuse coverage to the public entry point that surfaces the same runtime failure. As per coding guidelines, "Prefer testing via public API; avoid reaching into private internals in tests".

Also applies to: 661-677

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/stack/__tests__/schema-v3.test.ts` around lines 2 - 3, The test is
reaching into the private `resolveIndexType` helper and asserting its internal
error strings, which makes it brittle. Update
`packages/stack/__tests__/schema-v3.test.ts` to remove the direct
`@/encryption/helpers/infer-index-type` import and exercise the same failure
through the public `encryptConfigSchema`/`encryptedColumn` API instead. Keep the
coverage for the misuse case, but assert the runtime failure surfaced by the
supported schema entry point rather than helper internals.

Source: Coding guidelines

packages/stack/__tests__/schema-v3.test-d.ts (1)

229-301: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add a model-inference case with aliased column names.

This file already proves v3 domain inference for aliased builders like createdAtcreated_at, but the encryptModel / bulkEncryptModels acceptance cases only cover same-name keys. One typed assertion for an aliased encryptedTimestamptzColumn('created_at') model field would protect the exact v3 field mapping this PR is widening.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/stack/__tests__/schema-v3.test-d.ts` around lines 229 - 301, Add a
typed model-inference test for aliased v3 encrypted columns, since the current
`encryptModel` and `bulkEncryptModels` cases only cover same-name fields. Update
the `schema-v3.test-d.ts` assertions near the existing
`encryptModel`/`bulkEncryptModels` checks to include a model using
`encryptedTimestamptzColumn('created_at')` with an aliased property name. Verify
the inferred `EncryptionClient.encryptModel` and/or `bulkEncryptModels` result
type maps the aliased field correctly while still preserving unrelated fields.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/superpowers/plans/2026-07-01-eql-v3-typed-schema.md`:
- Around line 63-65: The task steps currently hardcode a developer-specific
absolute path in the read-only references, which will not work for other
environments. Update the instructions in the plan to use repository-relative
paths or an environment-agnostic placeholder, and if needed mention that the
base path must be configured by the user; keep the references to the
inventory.rs and schema/v3/*.json locations clear without embedding a local
machine path.

In `@packages/stack/__tests__/helpers/eql-v3.ts`:
- Around line 28-41: The advisory lock handling in installEqlV3IfNeeded is using
separate sql calls, so the lock and unlock may run on different pooled
connections. Update the function to run the entire check/install/unlock flow on
a reserved connection via sql.reserve(), or replace
pg_advisory_lock/pg_advisory_unlock with pg_advisory_xact_lock if the EQL v3
install path is transaction-safe, and keep the existing hasEqlV3TextSearch and
eqlV3Sql execution logic inside that reserved scope.

In `@packages/stack/__tests__/schema-v3-pg.test.ts`:
- Around line 133-149: The cleanup hooks in the schema-v3-pg test only remove
rows from protect_ci_v3_text_search, so the typed-domain fixture data in
protect_ci_v3_typed_domains is left behind. Update the existing beforeEach and
afterAll hooks in schema-v3-pg.test.ts to also delete rows for the typed-domain
table using the same TEST_RUN_ID guard, alongside the current cleanup logic. Use
the existing beforeEach, afterAll, and sql cleanup blocks as the place to add
the matching protect_ci_v3_typed_domains deletion.

In `@packages/stack/src/types.ts`:
- Around line 151-183: The public BuildableTable shape is too weak for the
encryption inference used by encryptModel() and bulkEncryptModels(), so
structurally accepted tables lose the literal column keys needed by
EncryptedFromBuildableTable. Fix this by either adding the column map
brand/_columnType to the BuildableTable contract and keeping
BuildableTableColumns aligned with it, or by narrowing the affected APIs/types
back to the branded table builder type so the return type reflects encrypted
fields correctly.

---

Outside diff comments:
In `@packages/stack/src/schema/v3/index.ts`:
- Around line 460-470: The freeTextSearch method in the v3 schema builder is
still storing nested options by reference, so later mutations to the caller’s
MatchIndexOpts can leak into build() output. Update freeTextSearch in
packages/stack/src/schema/v3/index.ts to clone the tokenizer and token_filters
values when assigning this.matchOpts, matching the class’s existing
clone-on-write approach and avoiding shared nested state.

---

Nitpick comments:
In `@docs/superpowers/plans/2026-07-01-eql-v3-typed-schema.md`:
- Line 510: Update the phrasing in the plans document to remove redundancy:
change the “Repeat the same exact pattern for:” text to either “Repeat the same
pattern for:” or “Repeat this exact pattern for:”. Locate the sentence in the
document section containing that exact phrase and keep the rest unchanged.
- Line 117: Reword the list item about storage-only schemas to avoid repeating
the same “Schemas with” sentence pattern as the surrounding bullets. Update the
wording in the schema documentation section so it still conveys that schemas
containing only v, i, and c are storage-only, but uses a different sentence
structure for readability and to satisfy the static analysis warning.
- Line 27: The plan document wording is a bit repetitive in the file-structure
list, where several consecutive bullets/sentences start with the same “In”
pattern. Rephrase the affected entries in the schema-plan section to improve
readability while preserving the parallel structure, keeping the
“BuildableTable” bullet clear and concise.

In `@packages/stack/__tests__/schema-v3.test-d.ts`:
- Around line 229-301: Add a typed model-inference test for aliased v3 encrypted
columns, since the current `encryptModel` and `bulkEncryptModels` cases only
cover same-name fields. Update the `schema-v3.test-d.ts` assertions near the
existing `encryptModel`/`bulkEncryptModels` checks to include a model using
`encryptedTimestamptzColumn('created_at')` with an aliased property name. Verify
the inferred `EncryptionClient.encryptModel` and/or `bulkEncryptModels` result
type maps the aliased field correctly while still preserving unrelated fields.

In `@packages/stack/__tests__/schema-v3.test.ts`:
- Around line 2-3: The test is reaching into the private `resolveIndexType`
helper and asserting its internal error strings, which makes it brittle. Update
`packages/stack/__tests__/schema-v3.test.ts` to remove the direct
`@/encryption/helpers/infer-index-type` import and exercise the same failure
through the public `encryptConfigSchema`/`encryptedColumn` API instead. Keep the
coverage for the misuse case, but assert the runtime failure surfaced by the
supported schema entry point rather than helper internals.

In `@packages/stack/scripts/install-eql-v3.ts`:
- Line 1: The install-eql-v3 script is importing dotenv in a way that re-enables
the noisy injected-env banner. Update the startup env loading in this script to
use the same explicit dotenv config pattern as the CLI entrypoint, with a fixed
.env.local path and quiet enabled, and remove the bare dotenv/config import so
the script stays silent in CI. Reference the script’s top-level env bootstrap in
install-eql-v3.ts.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4aa02b84-db93-4a9b-aa84-d185e2c884d9

📥 Commits

Reviewing files that changed from the base of the PR and between 30cd5f4 and b54e6d4.

📒 Files selected for processing (25)
  • .changeset/eql-v3-typed-schema.md
  • docs/query-api-walkthrough.md
  • docs/superpowers/plans/2026-07-01-eql-v3-typed-schema.md
  • packages/stack/__tests__/cjs-require.test.ts
  • packages/stack/__tests__/fixtures/eql-v3/cipherstash-encrypt-v3.sql
  • packages/stack/__tests__/helpers/eql-v3.ts
  • packages/stack/__tests__/helpers/stub-auth-wasm-inline.ts
  • packages/stack/__tests__/helpers/stub-protect-ffi-wasm-inline.ts
  • packages/stack/__tests__/schema-v3-client.test.ts
  • packages/stack/__tests__/schema-v3-pg.test.ts
  • packages/stack/__tests__/schema-v3.test-d.ts
  • packages/stack/__tests__/schema-v3.test.ts
  • packages/stack/__tests__/wasm-inline-column-name.test.ts
  • packages/stack/package.json
  • packages/stack/scripts/install-eql-v3.ts
  • packages/stack/src/encryption/helpers/infer-index-type.ts
  • packages/stack/src/encryption/helpers/model-helpers.ts
  • packages/stack/src/encryption/index.ts
  • packages/stack/src/encryption/operations/bulk-encrypt-models.ts
  • packages/stack/src/encryption/operations/encrypt-model.ts
  • packages/stack/src/encryption/operations/encrypt-query.ts
  • packages/stack/src/encryption/operations/encrypt.ts
  • packages/stack/src/schema/v3/index.ts
  • packages/stack/src/types.ts
  • packages/stack/vitest.config.ts
✅ Files skipped from review due to trivial changes (3)
  • docs/query-api-walkthrough.md
  • packages/stack/tests/helpers/stub-protect-ffi-wasm-inline.ts
  • .changeset/eql-v3-typed-schema.md
🚧 Files skipped from review as they are similar to previous changes (4)
  • packages/stack/tests/wasm-inline-column-name.test.ts
  • packages/stack/src/encryption/helpers/infer-index-type.ts
  • packages/stack/package.json
  • packages/stack/src/encryption/operations/encrypt.ts

Comment thread docs/superpowers/plans/2026-07-01-eql-v3-typed-schema.md Outdated
Comment thread packages/stack/__tests__/helpers/eql-v3.ts Outdated
Comment thread packages/stack/__tests__/schema-v3-pg.test.ts
Comment thread packages/stack/src/types.ts
tobyhede added 24 commits July 1, 2026 12:52
Column builders are copied onto the EncryptedTable instance for accessor
access (users.email). A column named build/tableName/columnBuilders/
_columnType would silently overwrite that member — worst case a 'build'
column breaks buildEncryptConfig's tb.build() call at runtime.

Throw a clear error at table-definition time instead. Scoped to v3; v2
retains its existing behavior.

Found by CodeRabbit review.
…ntry

The wasm-inline encrypt entry typed opts.column as the widened structural
BuildableColumn, but getColumnName still gated on instanceof EncryptedColumn
|| EncryptedField and threw for a v3 EncryptedTextSearchColumn — a runtime
break the type promise hid. Resolve the name structurally (typeof getName)
so v3 columns round-trip through WasmEncryptionClient.encrypt(); still throws
for non-builder JS input. getColumnName is the only instanceof gate on this
path; the rest reads table.tableName structurally.

Adds wasm-inline-column-name.test.ts exercising the seam (v2 column/field +
v3 column + non-builder). Like its sibling wasm-inline-normalize.test.ts the
suite cannot load in environments missing the @cipherstash/protect-ffi
/wasm-inline dep subpath.
Config tables are keyed by name, so two tables with the same tableName
silently dropped the earlier one. Add a v3-only additive guard that throws
on a duplicate (Object.hasOwn). v2's buildEncryptConfig keeps its existing
silent-overwrite behavior (no-v2-change constraint).
The RESERVED_TABLE_KEYS guard only covered own members (build, tableName,
columnBuilders, _columnType), so a column named constructor/toString/valueOf/
hasOwnProperty was assigned as an own property, shadowing the Object.prototype
member. Add an `in` check (isReservedTableKey) so any prototype-chain member
is also rejected, keeping the table object well-behaved for reflection.
…freeTextSearch' for match

A v3 text_search column emits unique+ore+match, and shared index inference
picks by priority unique > match > ore. So encryptQuery without an explicit
queryType builds an EQUALITY term (via unique) — a substring matches nothing.
Document on EncryptedTextSearchColumn + encryptedTextSearchColumn that callers
must pass queryType:'freeTextSearch' (FFI 'match') for free-text queries.

Addresses review finding #2 (naming footgun; doc-only, no runtime change).
The runner-aware `--help` E2E test flaked in CI, always dropping the tail of
the help (the Examples section with `bunx stash init` / `bunx stash db
install`) while the earlier Usage line survived.

Root cause is a Linux-only node-pty behaviour, not a race: `--help` emits ~5KB
in a single `console.log` and the process exits immediately. When the pty
slave closes with data still unread, Linux discards the pty buffer, so under
CI load node-pty never delivers the tail — `onData` fires for the head only.
macOS ptys retain that buffer, which is why it never reproduced locally.

Fix: add a pipe-based `run()` helper (`child_process.spawn`, piped stdio) for
non-interactive command assertions. Pipes buffer in-process and deliver every
byte before `close`, so large bursts are never truncated on any platform.
`run()` keeps `exitCode` as the real numeric code (null on signal) so a crash
can't masquerade as a clean exit. Migrate `runner-aware-help.e2e.test.ts` to
it; the PTY `render()` helper stays for interactive tests.

Also pass `quiet: true` to dotenv's `config()` calls: v17 prints an `injected
env (N) from …` banner to stdout on every call, so the CLI was emitting four
noisy, non-deterministic banner lines ahead of its own output on every
invocation. A smoke-test guard keeps the banner from returning.

Mirrors the fix on fix/cli-e2e-pty-drain (#537) so this branch's CI is green
independently.
Add table-driven runtime tests for all 40 EQL v3 domain builders (name,
eqlType, capabilities, config, queryability) plus type-level tests for
nominal domain distinctness, InferPlaintext mapping, queryability of
BuildableQueryColumn, and v3/v2 model inference.
Add a generic EncryptedV3Column base parameterised by a literal domain
definition (eqlType, castAs, capabilities), plus concrete builders for every
EQL v3 domain (int2/4/8, float4/8, numeric, date, timestamptz, text*, bool).
Each carries explicit getQueryCapabilities()/isQueryable() metadata, emits
capability-derived indexes, and drives precise InferPlaintext. Refactors
EncryptedTextSearchColumn onto the shared base while preserving its
byte-identical config and match-tuning override.
tobyhede added 4 commits July 1, 2026 12:54
Tighten BuildableQueryColumn so only capability-bearing, queryable columns
are accepted by encryptQuery. Widen encryptModel/bulkEncryptModels to any
BuildableTable via EncryptedFromBuildableTable (keyed off the _columnType
brand), covering both v2 and v3 tables. Introduce a Plaintext type
(JsPlaintext | Date | bigint) for the single-value encrypt/encryptQuery
entry points so v3 date/timestamptz/int8 domains accept their natural JS
values; cast to JsPlaintext only at the FFI boundary (the FFI declares these
as CastAs targets but omits them from its JsPlaintext input union).
Expand env-gated client and Postgres suites with representative v3 domains
(storage-only, equality, order, match, bigint, date) and a typed-domain PG
table. Add a v3 CJS export assertion. Fix wasm-inline test resolution by
aliasing the unresolvable @cipherstash/{protect-ffi,auth}/wasm-inline
specifiers to local stubs in vitest.config.ts. Includes the v3 install
script, EQL v3 SQL fixture, and eql-v3 test helper.
@tobyhede tobyhede force-pushed the feat/eql-v3-text-search-schema branch from b54e6d4 to ed78233 Compare July 1, 2026 03:13
tobyhede added 2 commits July 1, 2026 14:58
- types: BuildableTableColumns fallback never -> Record<never, never> so a
  structurally BuildableTable-typed value degrades to the model unchanged
  instead of over-encrypting every field (keyof never is string|number|symbol)
- schema/v3: clone-on-write in EncryptedTextSearchColumn.freeTextSearch so
  caller opts mutated before build() cannot leak into emitted config
- test helper: run EQL v3 advisory lock/check/install/unlock on one reserved
  connection (sql.reserve) instead of across pooled backends
- pg test: clean up protect_ci_v3_typed_domains rows in beforeEach/afterAll
- install script: quiet dotenv config (drop bare dotenv/config banner)
- tests: graceful-degradation, aliased-column, and freeTextSearch clone-on-write
  regression coverage
- docs: replace developer-specific absolute paths with placeholders; wording
…/v3)

Add EncryptionV3 / typedClient returning a TypedEncryptionClient<S> that
derives types from the concrete table/column builder arguments:

- encrypt/encryptQuery pin plaintext to the column domain; encryptQuery
  constrains queryType to the column capabilities and rejects storage-only
  columns at compile time
- encryptModel/bulkEncryptModels validate schema fields against inferred
  plaintext (passthrough fields untouched), precise encrypted output
- decryptModel/bulkDecryptModels return precise plaintext, reconstructing
  Date/bigint from the encrypt-config cast_as

Binding to the concrete branded v3 classes closes the soundness gap where a
non-branded structural table could encrypt at runtime while typed as plaintext.
Runtime is unchanged except a per-column Date/bigint reconstruction on model
decrypt. v2 client surface is untouched.

New @cipherstash/stack/v3 subpath re-exports the v3 builders for a single import.
@tobyhede tobyhede changed the title feat(stack): EQL v3 text_search authoring DSL (@cipherstash/stack/schema/v3) feat(stack): EQL v3 typed schema + strongly-typed client (@cipherstash/stack/schema/v3, /v3) Jul 1, 2026
tobyhede added 3 commits July 1, 2026 16:59
The native protect-ffi marshals plaintext through JSON serialization,
which throws "Do not know how to serialize a BigInt" for JS bigint
values. int8/bigint v3 domains now accept/return lossless string
plaintext instead; cast_as stays big_int so server-side casting is
unchanged. Fixes the failing schema-v3-client live integration test.

Also hardens adjacent paths surfaced in review:
- encrypt: reject NaN / Infinity number plaintexts
- wasm-inline resolveStrategy: guard missing workspaceCrn/accessKey at
  runtime for JS callers that bypass the compile-time union
- docs: correct query API walkthrough lock-context description
- v3: thread optional lockContext through typed decryptModel/bulkDecryptModels
  so identity-bound models can be decrypted
- encryptQuery: only route to batch mode when no opts are supplied; explicit
  EncryptQueryOptions forces the single-plaintext path even for array inputs
- package.json: add "v3" typesVersions entry so the ./v3 subpath resolves
  types under node/node10 resolution
- identity: resolveLockContext uses a structural guard alongside instanceof
  for cross-realm LockContext safety
- test: correct cast_as comment (bigint, not big_int)
EncryptedTable.build() keyed the encrypt config by the JS property name,
but encrypt/decrypt look columns up by column.getName() (the DB name).
The two only match when the JS key equals the column string, so a
camelCase-to-snake_case mapping (e.g. externalId -> external_id) made the
native FFI report "column not found in Encrypt config" at encrypt time.

Key by builder.getName() instead. Adds a regression test asserting the
config keys by DB name. Surfaced by the schema-v3-client int8/date live
tests once the BigInt serialization failure was cleared.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant