From 84078d7d36bf5cf0fd16a479ce16c48c5d804f32 Mon Sep 17 00:00:00 2001 From: Nathaniel Tucker Date: Tue, 28 Apr 2026 01:10:04 -0400 Subject: [PATCH] feat(endpoint): add Scalar schema for lens-dependent entity fields (#3887) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(endpoint): add Scalar schema for lens-dependent entity fields Introduces Scalar + ScalarCell classes following the Union pattern: - Scalar (SchemaSimple, not entity-like) routes normalize/denormalize - ScalarCell (entity-like, internal) stores grouped cell data - EntityMixin.normalize: if/else to pass whole entity to Scalar - EntityMixin.denormalize: completely unchanged (Union-like wrapper) - Entity stores lens-independent {id,field} wrappers - Denormalize joins correct cell based on endpoint args Co-authored-by: natmaster * test(endpoint): add Scalar schema tests Tests cover: - normalize stores wrapper refs in entity, cell data in ScalarCell - multiple entities, different lenses produce separate cells - denormalize joins correct cell based on lens args - different lens args produce different results from same entity - missing lens returns undefined for scalar fields - column-only fetch via Values stores cells without modifying entities - column fetch cells joinable via denormalize with Company schema - merge accumulation updates existing cells - Scalar constructor and queryKey Co-authored-by: natmaster * docs(rest): add Scalar schema API documentation Covers usage (full entity + column-only endpoints), constructor options, normalize/denormalize flow, normalized storage model, and related APIs. Co-authored-by: natmaster * Remove dead _lastCpk field from Scalar class _lastCpk was declared and initialized but never read or written elsewhere in the codebase. Co-authored-by: Nathaniel Tucker * enchance: Better design * enhance(endpoint): clean up Scalar parent-entity context plumbing Replace the encoded-key hack with a direct `parentEntity` argument: - `EntityMixin.normalize` now dispatches schemas marked `acceptsPrimitives` directly (bypassing `visit`'s primitive short-circuit) and passes `this` as the 7th arg. - `Scalar.normalize` reads `parentEntity` to derive entity key and pk; no more parsing `'||'` out of the visit key. - `parent` is now the entity data row (standard `Visit` contract), not the Entity class. - `getVisit` and the `SchemaSimple` interface are unchanged — zero impact on the normalize hot path (verified at parity with HEAD across 8-run A/B benchmarks). Made-with: Cursor * enhance(normalizr): track parentEntity in visit walker Move parent-entity context tracking into `getVisit` itself, eliminating the per-schema-type dispatch in `EntityMixin`. The walker now: - Maintains a `currentEntity` closure variable, save/restored around every entity visit (schemas with `pk`). - Passes it as a 7th `parentEntity` arg to every `schema.normalize` call. - Honors a new `acceptsPrimitives` opt-in marker so schemas like `Scalar` receive primitive values instead of being short-circuited. `EntityMixin.normalize`'s field loop is now a single uniform `visit(...)` call — no more conditional branch for Scalar fields. `Scalar.normalize` reads `parentEntity` from the standard 7th arg; `parent` is the entity data row as the standard `Visit` contract specifies. Trade-off: ~3% normalize-throughput cost on the hot path (closure save/restore around every entity visit). Validated with 8-run A/B benchmarks. Buys a uniform schema contract — Scalar (and any future context-dependent schema) needs no special case in `EntityMixin`. Made-with: Cursor * refactor(normalizr): collapse entity/non-entity branches in visit walker Both branches called `schema.normalize` with the same args except for the parent-entity context. Snapshot `prev = currentEntity` first, then conditionally update `currentEntity = schema` for entities. Pass `prev` — which equals the prior entity for entities and the still-current entity for non-entities — and unconditionally restore. One call site instead of two, no behavior change. 8-run A/B benchmarks at parity with the prior version (within noise). Made-with: Cursor * internal: Update website types * fix: visit edge case * fix(normalizr): tighten table-resident schema check in unvisit The `else if` branch for table-resident schemas without `pk` was matching any schema exposing a `key` property. `Invalidate` extends PolymorphicSchema and exposes `key` via a getter, so it was being routed into `unvisitEntity` -> `unvisitEntityObject`, which calls `schema.createIfValid()` -- a method `Invalidate` does not implement. This caused `TypeError` on basic Invalidate denormalization and broke deletion/symbol propagation. Switch the discriminator to `typeof createIfValid === 'function'`, which is the precise capability `unvisitEntityObject` requires. This matches Scalar (which now implements Mergeable) and regular entities, but not Invalidate, Query, Union, etc. -- they continue falling through to their own `denormalize` methods. Made-with: Cursor * fix: edge cases * docs: updates * test(endpoint): cover Scalar merge, denormalize edge cases Add direct tests for merge/shouldReorder/mergeWithStore/mergeMetaWithStore and denormalize passthroughs (falsy, symbol, object, missing-lens, missing cell, cpk string + Values round-trip) to bring Scalar.ts to 100% coverage. Made-with: Cursor * docs: Fix * fix(core): handle Scalar/wrapper schemas in skip-denormalize check `getResponseMeta` short-circuits `denormalize()` when the endpoint schema doesn't transform the response. The previous check (`schemaHasEntity`) had two bugs surfaced by Scalar: 1. For `Values(Scalar)` it walked `Object.values(scalarInstance)`, recursed into the `lensSelector` function, and looped forever. 2. Scalar is table-resident without `pk`, so it was not detected as needing denormalize — the raw cpk strings were returned instead of the joined cell data. Replace with `requiresDenormalize`, which mirrors `getVisit.ts`: schemas that define `normalize` always need denormalize to reconstruct the response. This collapses the entity check, the Scalar special- case, the wrapper `.schema` recursion, and the self-loop guard into a single early-exit, so schema class instances never fall through to `Object.values()` traversal of their instance fields. Adds regression tests for both `Values(Scalar)` (column-only endpoint) and Entity-with-Scalar-fields, with a hard 2s timeout so a re- introduced infinite recursion fails fast rather than hanging Jest. Net: -89B minified, neutral-to-positive on `core ^get` benchmarks. Made-with: Cursor * docs: Fix types * internal: Update tests for check for memoization paths * fix: cache busting with args * fix: does not over-denormalize a schema map containing string values * docs: Release notes and migration for breaking changes * docs(blog): Conform v0.17 release post to blog style guide Add nonFilterArgumentKeys feature, embed a Scalar HooksPlayground demo (replacing dead imports), recategorize binary auto-detection under Other Improvements, and link PR #3887 for the Scalar / denormalize delegate work. Made-with: Cursor * internal: Bench and upgrade skill * refactor(normalizr): drop precomputed key from Dep, pass args to set Makes the `Dep` shape strictly monomorphic (`{path, entity}` only) by removing the optional `key` field and having `WeakDependencyMap.set` re-evaluate `path(args)` when the path is a `KeyFn`. Callers in `globalCache` now forward `this._args` to `set`. Benefits: - Eliminates the `wrong map` deopts observed on `WeakDependencyMap.set` and `GlobalCache#paths` caused by the previously polymorphic Dep shape. - Simpler, tighter interface -- one fewer field to keep in sync at each `argsKey` call site. - Slightly smaller gzipped esm bundle (-17 B); cjs flat. Macro throughput is statistically unchanged vs the prior shape across the denormalize/normalize suites (all deltas well within 95% CI over 5 runs). The change is a clarity + deopt-cleanup refactor, not a perf optimization. Made-with: Cursor * internal: Add WeakDependencyMap microbenchmarks Made-with: Cursor * bench: isolate Scalar MemoCache from shared priming Previously, the shared `memo` used by Project/User/AllProjects/Values benches was also primed with StockSchema (scalar) entries during suite setup. Cross-schema priming in a single MemoCache perturbs V8 hidden-class state for downstream cached-path benches — masking real deltas by ~15% on `denormalizeLong Values withCache` and adding systematic noise to other withCache benches. Move Stock priming to a dedicated `memoStock` MemoCache instance used only by the two Scalar withCache benches. Non-Scalar benches now see the same `memo` shape they did prior to the Scalar PR, so measurements are comparable against master. Verified with 5x full suite runs: denormalizeLong Values withCache: 7273 -> 8674 ops/sec (+19%) other benches within run-to-run noise. Made-with: Cursor * perf: restore entity-only fast paths in denormalize cache Recovers the 3–7% regressions introduced by "fix: cache busting with args" (acdb4b161c) on cached denormalize benches. Root cause: `argsKey` support added unconditional `typeof === 'function'` branches, dynamic `push`-based path materialization, and post-hoc filter scans to the hot `WeakDependencyMap.get` / `GlobalCache.paths` / `GlobalCache.getResults` paths — every caller paid the cost, even entity-only chains. Changes ------- normalizr/memo/WeakDependencyMap * Sticky `hasStr` flag: set true only when a `KeyFn` dep is stored. * `get` uses the pre-acdb entity-only loop when `hasStr` is false (common case), and a separate `_getMixed` slow path otherwise. * Expose `hasStringDeps` for consumers to gate their own work. normalizr/memo/globalCache * Per-frame `_hasArgsKey` flag set in `argsKey()`. * `paths()` restores pre-allocated `new Array(n)` + indexed writes when no function-typed dep was pushed this frame. * `getResults` skips the function-strip scan on cache hit unless the result WDM has ever stored a string dep. normalizr/denormalize/unvisit + schemas/{Array,Object}, endpoint/schemas/ {Array,Object,Values,EntityMixin} * Hoist `delegate.args` / `delegate.unvisit` out of per-entity and per-array-element loops so hot denormalize walks read a closure local instead of doing a property load per call. Measurements (5-run medians, ops/sec, vs a9e9… pre-acdb baseline) ----------------------------------------------------------------- pre at-acdb HEAD denormalizeShort 500x 1234 1198 1583 +28.3% denormalize bidirectional 50 8549 7922 10801 +26.3% denormalizeLong 437 424 552 +26.3% denormalizeLong with mixin Entity 411 396 515 +25.3% denormalizeLong All withCache 10479 10401 12242 +16.8% denormalizeLong Values 380 359 439 +15.5% denormalizeLong Query-sorted withCache 10858 10763 12305 +13.3% query All withCache 11071 10619 12387 +11.9% denormalizeLong withCache 12119 11708 12514 + 3.3% denormalizeLong Values withCache 8879 8692 8875 0.0% queryShort 500x withCache 4792 4556 4494 - 6.2% denormalizeShort 500x withCache 13126 12364 12397 - 5.6% The short 500x benches amplify per-call overhead ~500×; the residual regression there reflects the unavoidable delegate-object indirection still required for `argsKey` support. Aggregate across the suite is strongly net-positive vs pre-acdb. Tests: packages/normalizr + packages/endpoint — all 680 pass. Made-with: Cursor * docs: Update docs * internal: More tests * fix(normalizr): propagate argsKey flag on entity-cache hit When the result cache missed (new input ref) but every entity ref was unchanged, `GlobalCache.getEntity` replayed cached deps without running `computeValue`, leaving `_hasArgsKey` false. `paths()` then took its fast path and leaked function-typed (`argsKey`) deps from the replayed chain into the returned `EntityPath[]` subscription list. Set `_hasArgsKey` on cache hit when the per-entity `WeakDependencyMap` has ever stored a function dep (`hasStringDeps`), keeping the single branch outside the push loop to preserve hidden-class stability on the hot path. Made-with: Cursor * internal: TODO on Scalar pk context mismatch Scalar.normalize re-derives the enclosing entity's pk without the `parent`/`key` context that EntityMixin.normalize uses, so any custom pk() reading those args would key the Scalar cell under a different id than the entity is stored under. Made-with: Cursor * internal: clarify intent of Scalar.denormalize falsy guard Made-with: Cursor * internal: trim Scalar.denormalize guard comment Made-with: Cursor * docs: Tuning * docs: Update agents to latest design * fix(endpoint): guard Scalar.denormalize against truthy non-string primitives Truthy non-string primitives (e.g. `0.5`, `true`, `42`) previously fell through the falsy/symbol guard and into `delegate.unvisit(this, input)`. Since Scalar has no `pk`, `unvisit`'s `createIfValid` fast path only matches string inputs, so non-string primitives re-dispatched to `Scalar.denormalize` — infinite recursion / stack overflow. This can surface during schema migration when Scalar is added to an entity that still has cached raw numeric or boolean field values in the store. Tighten the guard to pass through any non-string, non-object input so stale values degrade gracefully instead of crashing. Made-with: Cursor * enhance: entityPk + queryKey * fix(endpoint): scope Scalar.entityPk surrounding-key heuristic to authoritative map keys Previously `entityPk` returned any non-undefined `key`, but `Array.normalize` forwards the *parent's* field name as `key` to every element. When `[Scalar]` or `Collection([Scalar])` was nested under a plain object schema like `{ stock: [Scalar] }`, every item received the same field-name pk, collapsing all cells onto one compound pk and silently corrupting data. Trust `key` only when the enclosing container literally maps it to the cell — `parent[key] === input` — which holds for `Values(Scalar)` (the intended use of the surrounding-key heuristic) but not for arrays. Co-authored-by: Nathaniel Tucker --------- Co-authored-by: Cursor Agent --- .changeset/denormalize-delegate.md | 53 + .changeset/scalar-entity-binding.md | 70 + .cursor/skills/data-client-schema/SKILL.md | 9 + .../references/EntityMixin.md | 1 + .../data-client-schema/references/Lazy.md | 1 + .../data-client-schema/references/Scalar.md | 1 + .../data-client-v0.17-migration/SKILL.md | 163 ++ AGENTS.md | 2 +- README.md | 7 + docs/core/api/useQuery.md | 4 +- docs/core/shared/_schema_table.mdx | 7 + docs/rest/api/Collection.md | 2 +- docs/rest/api/Lazy.md | 2 +- docs/rest/api/Query.md | 2 +- docs/rest/api/Scalar.md | 274 ++++ docs/rest/api/resource.md | 2 +- docs/rest/api/schema.md | 2 +- docs/rest/shared/_ScalarDemo.mdx | 123 ++ examples/benchmark/filter.js | 6 +- examples/benchmark/micro.js | 74 +- examples/benchmark/normalizr.js | 99 ++ examples/benchmark/schemas.js | 68 + examples/benchmark/src/index.ts | 1 + packages/core/src/controller/Controller.ts | 33 +- .../src/controller/__tests__/getResponse.ts | 145 +- packages/endpoint/README.md | 7 + packages/endpoint/src/index.ts | 1 + packages/endpoint/src/interface.ts | 50 +- packages/endpoint/src/schema.d.ts | 22 +- packages/endpoint/src/schema.js | 1 + packages/endpoint/src/schemas/Array.ts | 12 +- packages/endpoint/src/schemas/Collection.ts | 13 +- packages/endpoint/src/schemas/Entity.ts | 4 +- packages/endpoint/src/schemas/EntityMixin.ts | 6 +- packages/endpoint/src/schemas/EntityTypes.ts | 9 +- .../endpoint/src/schemas/ImmutableUtils.ts | 6 - packages/endpoint/src/schemas/Invalidate.ts | 11 +- packages/endpoint/src/schemas/Lazy.ts | 22 +- packages/endpoint/src/schemas/Object.ts | 17 +- packages/endpoint/src/schemas/Query.ts | 17 +- packages/endpoint/src/schemas/Scalar.ts | 285 ++++ packages/endpoint/src/schemas/Union.ts | 10 +- packages/endpoint/src/schemas/Values.ts | 10 +- .../src/schemas/__tests__/Collection.test.ts | 5 +- .../src/schemas/__tests__/Scalar.test.ts | 1357 +++++++++++++++++ packages/graphql/README.md | 7 + packages/normalizr/AGENTS.md | 28 +- packages/normalizr/README.md | 7 + .../src/__tests__/WeakDependencyMap.test.ts | 85 ++ .../src/__tests__/globalCache.test.ts | 258 ++++ .../src/__tests__/localCache.test.ts | 132 ++ packages/normalizr/src/buildQueryKey.ts | 1 + packages/normalizr/src/denormalize/cache.ts | 4 + .../normalizr/src/denormalize/denormalize.ts | 2 +- .../normalizr/src/denormalize/localCache.ts | 11 + packages/normalizr/src/denormalize/unvisit.ts | 47 +- packages/normalizr/src/interface.ts | 23 +- packages/normalizr/src/memo/MemoCache.ts | 2 +- .../normalizr/src/memo/WeakDependencyMap.ts | 97 +- packages/normalizr/src/memo/globalCache.ts | 70 +- packages/normalizr/src/normalize/getVisit.ts | 44 +- packages/normalizr/src/schemas/Array.ts | 7 +- .../normalizr/src/schemas/ImmutableUtils.ts | 11 +- packages/normalizr/src/schemas/Object.ts | 7 +- .../__tests__/useSuspense-scalar.web.tsx | 190 +++ packages/react/src/state/GCPolicy.ts | 4 +- packages/rest/README.md | 7 + ...-content-property-binary-auto-detection.md | 87 -- ...2026-04-24-v0.17-scalar-typed-downloads.md | 298 ++++ website/sidebars-endpoint.json | 4 + .../editor-types/@data-client/core.d.ts | 44 +- .../editor-types/@data-client/endpoint.d.ts | 199 ++- .../editor-types/@data-client/graphql.d.ts | 199 ++- .../editor-types/@data-client/normalizr.d.ts | 46 +- .../editor-types/@data-client/rest.d.ts | 199 ++- .../Playground/editor-types/globals.d.ts | 199 ++- website/src/fixtures/companies.ts | 57 + .../static/codemods/__tests__/v0.17.test.js | 253 +++ website/static/codemods/v0.17.js | 432 ++++++ 79 files changed, 5663 insertions(+), 414 deletions(-) create mode 100644 .changeset/denormalize-delegate.md create mode 100644 .changeset/scalar-entity-binding.md create mode 120000 .cursor/skills/data-client-schema/references/EntityMixin.md create mode 120000 .cursor/skills/data-client-schema/references/Lazy.md create mode 120000 .cursor/skills/data-client-schema/references/Scalar.md create mode 100644 .cursor/skills/data-client-v0.17-migration/SKILL.md create mode 100644 docs/rest/api/Scalar.md create mode 100644 docs/rest/shared/_ScalarDemo.mdx create mode 100644 packages/endpoint/src/schemas/Scalar.ts create mode 100644 packages/endpoint/src/schemas/__tests__/Scalar.test.ts create mode 100644 packages/normalizr/src/__tests__/globalCache.test.ts create mode 100644 packages/normalizr/src/__tests__/localCache.test.ts create mode 100644 packages/react/src/hooks/__tests__/useSuspense-scalar.web.tsx delete mode 100644 website/blog/2026-04-05-v0.17-content-property-binary-auto-detection.md create mode 100644 website/blog/2026-04-24-v0.17-scalar-typed-downloads.md create mode 100644 website/src/fixtures/companies.ts create mode 100644 website/static/codemods/__tests__/v0.17.test.js create mode 100644 website/static/codemods/v0.17.js diff --git a/.changeset/denormalize-delegate.md b/.changeset/denormalize-delegate.md new file mode 100644 index 000000000000..c79488d52dad --- /dev/null +++ b/.changeset/denormalize-delegate.md @@ -0,0 +1,53 @@ +--- +'@data-client/endpoint': minor +'@data-client/rest': minor +'@data-client/graphql': minor +'@data-client/normalizr': minor +'@data-client/core': minor +'@data-client/react': minor +'@data-client/vue': minor +--- + +**BREAKING**: `Schema.denormalize()` is now `(input, delegate)` instead +of the previous `(input, args, unvisit)` 3-parameter signature. + +```ts +// before +denormalize(input, args, unvisit) { + return unvisit(this.schema, input); +} + +// after +denormalize(input, delegate) { + return delegate.unvisit(this.schema, input); +} +``` + +The new [`IDenormalizeDelegate`](https://dataclient.io/docs/api/Schema) +exposes `unvisit`, `args`, and a new `argsKey(fn)` helper that registers +a memoization dimension when output varies with endpoint args. Reading +`delegate.args` directly does *not* contribute to cache invalidation — +schemas that branch on args must call `argsKey`. The `fn` reference +doubles as the cache path key, so it must be **referentially stable** +— define it on the instance or at module scope, not inline per call: + +```ts +class LensSchema { + constructor({ lens }) { + this.lensSelector = lens; // stable reference across calls + } + denormalize(input, delegate) { + const portfolio = delegate.argsKey(this.lensSelector); + return this.lookup(input, portfolio); + } +} +``` + +All built-in schemas (`Array`, `Object`, `Values`, `Union`, `Query`, +`Invalidate`, `Lazy`, `Collection`) have been updated. Custom schemas +implementing `SchemaSimple` must update their `denormalize` signature. + +`Schema.normalize()` and the `visit()` callback also gain an optional +trailing `parentEntity` argument tracking the nearest enclosing +entity-like schema. This is additive — existing schemas don't need +changes unless they want to use it. diff --git a/.changeset/scalar-entity-binding.md b/.changeset/scalar-entity-binding.md new file mode 100644 index 000000000000..6ae363a785ba --- /dev/null +++ b/.changeset/scalar-entity-binding.md @@ -0,0 +1,70 @@ +--- +'@data-client/endpoint': minor +'@data-client/rest': minor +'@data-client/graphql': minor +'@data-client/normalizr': minor +'@data-client/core': minor +'@data-client/react': minor +'@data-client/vue': minor +--- + +Add [Scalar](https://dataclient.io/rest/api/Scalar) schema for lens-dependent entity fields. + +`Scalar` models entity fields whose values vary by a runtime "lens" (such as the +selected portfolio, currency, or locale). Multiple components can render the +same entity through different lenses simultaneously — each sees the correct +values without the entity itself ever being mutated. Lens-dependent values are +stored in a separate cell table and joined at denormalize time from endpoint +args. + +New exports: `Scalar`, `schema.Scalar`. + +A single `Scalar` instance can serve both as an `Entity.schema` field (parent +entity inferred from the visit) and standalone — inside `Values(Scalar)`, +`[Scalar]`, or `Collection([Scalar])` — for cheap column-only refreshes +(entity bound explicitly via `entity`). Cell pks are derived from the map key +or via `Scalar.entityPk()`, which defaults to `Entity.pk()` so custom and +composite primary keys work with no override: + +```ts +import { Collection, Entity, RestEndpoint, Scalar } from '@data-client/rest'; + +class Company extends Entity { + id = ''; + price = 0; + pct_equity = 0; + shares = 0; +} +const PortfolioScalar = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'portfolio', + entity: Company, +}); +Company.schema = { + pct_equity: PortfolioScalar, + shares: PortfolioScalar, +}; + +// Full load — Company rows + scalar cells for the current portfolio +export const getCompanies = new RestEndpoint({ + path: '/companies', + searchParams: {} as { portfolio: string }, + schema: new Collection([Company], { argsKey: () => ({}) }), +}); +// Lens-only refresh — writes to the same Scalar(portfolio) cell table +export const getPortfolioColumns = new RestEndpoint({ + path: '/companies/columns', + searchParams: {} as { portfolio: string }, + schema: new Collection([PortfolioScalar], { + argsKey: ({ portfolio }) => ({ portfolio }), + }), +}); +``` + +`useSuspense(getCompanies, { portfolio: 'A' })` and +`useSuspense(getCompanies, { portfolio: 'B' })` resolve to different +`pct_equity` / `shares` while sharing the same `Company` row. + +`Scalar.queryKey` enumerates cells in its table for the current lens, so +endpoints that use `Scalar` directly as their top-level schema reconstruct +from cache without a network round-trip once the cells are present. diff --git a/.cursor/skills/data-client-schema/SKILL.md b/.cursor/skills/data-client-schema/SKILL.md index a131941bb884..dd252d29c3cc 100644 --- a/.cursor/skills/data-client-schema/SKILL.md +++ b/.cursor/skills/data-client-schema/SKILL.md @@ -12,9 +12,11 @@ to represent the data expected. ### Object - [Entity](references/Entity.md) - represents a single unique object (denormalized) +- [EntityMixin](references/EntityMixin.md) - turn any pre-existing class into an Entity - [new Union(Entity)](references/Union.md) - polymorphic objects (A | B) - `{[key:string]: Schema}` - immutable objects - [new Invalidate(Entity|Union)](references/Invalidate.md) - to delete an Entity +- [new Lazy(() => Schema)](references/Lazy.md) - break circular imports / defer deep recursive denormalization ### List @@ -27,6 +29,10 @@ to represent the data expected. - `new Collection(Values(Schema))` - mutable/growable maps - [new Values(Schema)](references/Values.md) - immutable maps +### Lens-dependent entity fields + +- [new Scalar({ lens, key, entity? })](references/Scalar.md) - per-field cells that vary by a runtime "lens" (portfolio, currency, locale) without mutating the underlying entity + ### Derived / selector pattern - [new Query(Queryable)](references/Query.md) - memoized programmatic selectors @@ -156,10 +162,13 @@ See [partial-entities](references/partial-entities.md) for patterns and examples For detailed API documentation, see the [references](references/) directory: - [Entity](references/Entity.md) - Normalized data class +- [EntityMixin](references/EntityMixin.md) - Turn any class into an Entity - [Collection](references/Collection.md) - Mutable/growable lists - [Union](references/Union.md) - Polymorphic schemas - [Query](references/Query.md) - Programmatic selectors - [Invalidate](references/Invalidate.md) - Delete entities +- [Lazy](references/Lazy.md) - Deferred / circular schemas +- [Scalar](references/Scalar.md) - Lens-dependent entity fields - [Values](references/Values.md) - Map schemas - [All](references/All.md) - List all entities of a kind - [Array](references/Array.md) - Immutable list schema diff --git a/.cursor/skills/data-client-schema/references/EntityMixin.md b/.cursor/skills/data-client-schema/references/EntityMixin.md new file mode 120000 index 000000000000..dd3773c826ba --- /dev/null +++ b/.cursor/skills/data-client-schema/references/EntityMixin.md @@ -0,0 +1 @@ +../../../../docs/rest/api/EntityMixin.md \ No newline at end of file diff --git a/.cursor/skills/data-client-schema/references/Lazy.md b/.cursor/skills/data-client-schema/references/Lazy.md new file mode 120000 index 000000000000..e40397b7975c --- /dev/null +++ b/.cursor/skills/data-client-schema/references/Lazy.md @@ -0,0 +1 @@ +../../../../docs/rest/api/Lazy.md \ No newline at end of file diff --git a/.cursor/skills/data-client-schema/references/Scalar.md b/.cursor/skills/data-client-schema/references/Scalar.md new file mode 120000 index 000000000000..f68bf486582f --- /dev/null +++ b/.cursor/skills/data-client-schema/references/Scalar.md @@ -0,0 +1 @@ +../../../../docs/rest/api/Scalar.md \ No newline at end of file diff --git a/.cursor/skills/data-client-v0.17-migration/SKILL.md b/.cursor/skills/data-client-v0.17-migration/SKILL.md new file mode 100644 index 000000000000..4c316ec93a7f --- /dev/null +++ b/.cursor/skills/data-client-v0.17-migration/SKILL.md @@ -0,0 +1,163 @@ +--- +name: data-client-v0.17-migration +description: Migrate custom @data-client schemas from the v0.16 denormalize(input, args, unvisit) signature to the v0.17 denormalize(input, delegate) signature. Use when upgrading to v0.17, when seeing TS errors about unvisit not being callable, or when adapting custom Schema implementations. +--- + +# @data-client v0.17 Migration + +Applies to anyone implementing a custom [`Schema`](https://dataclient.io/docs/api/Schema) — `SchemaSimple`, `SchemaClass`, polymorphic wrappers, or types that subclass `EntityMixin` directly. Built-in schemas (`Entity`, `resource()`, `Collection`, `Union`, `Values`, `Array`, `Object`, `Query`, `Invalidate`, `Lazy`) are migrated by the library. + +The automated codemod handles the common cases: + +```bash +npx jscodeshift -t https://dataclient.io/codemods/v0.17.js --extensions=ts,tsx,js,jsx src/ +``` + +This skill describes what it does and how to handle the cases it can't. + +## What changed + +`Schema.denormalize()` now takes a single `delegate` instead of `(args, unvisit)`. Reading `delegate.args` does **not** contribute to cache invalidation — schemas whose output varies with endpoint args must register that dependency through `delegate.argsKey(fn)`. + +```ts +// before +denormalize(input, args, unvisit) { + return unvisit(this.schema, input); +} + +// after +denormalize(input, delegate) { + return delegate.unvisit(this.schema, input); +} +``` + +Full delegate surface ([`IDenormalizeDelegate`](https://dataclient.io/docs/api/Schema)): + +```ts +interface IDenormalizeDelegate { + unvisit(schema: any, input: any): any; + readonly args: readonly any[]; + argsKey(fn: (args: readonly any[]) => string | undefined): string | undefined; +} +``` + +## Migration rules + +### Class methods + +`(input, args, unvisit)` → `(input, delegate)`. Inside the body: + +- `unvisit(schema, value)` → `delegate.unvisit(schema, value)` +- bare `args` references (including spreads) → `delegate.args` +- pass-through `someSchema.denormalize(input, args, unvisit)` → `someSchema.denormalize(input, delegate)` + +```ts +// before +class Wrapper { + denormalize(input: {}, args: readonly any[], unvisit: any) { + const value = unvisit(this.schema, input); + return this.process(value, ...args); + } +} + +// after +class Wrapper { + denormalize(input: {}, delegate: IDenormalizeDelegate) { + const value = delegate.unvisit(this.schema, input); + return this.process(value, ...delegate.args); + } +} +``` + +### TypeScript signatures + +Update method signatures and `declare` fields the same way: + +```ts +// before +interface MySchema { + denormalize(input: {}, args: readonly any[], unvisit: (s: any, v: any) => any): any; +} + +class Lazy { + declare _denormalizeNullable: ( + input: {}, + args: readonly any[], + unvisit: (s: any, v: any) => any, + ) => any; +} + +// after — the codemod adds `IDenormalizeDelegate` to your existing +// `@data-client/{rest,endpoint,normalizr,...}` import as an inline +// `type` specifier. Only when no such import exists does it create a +// new `import type { IDenormalizeDelegate } from '@data-client/endpoint'`. +import { Entity, type IDenormalizeDelegate } from '@data-client/rest'; + +interface MySchema { + denormalize(input: {}, delegate: IDenormalizeDelegate): any; +} + +class Lazy { + declare _denormalizeNullable: (input: {}, delegate: IDenormalizeDelegate) => any; +} +``` + +The codemod matches `denormalize`, `_denormalize`, and `_denormalizeNullable` on type declarations. + +### args-dependent output (manual) + +The codemod will rewrite `args` to `delegate.args`, but if your schema's *return value* depends on those args, you must also register an [`argsKey`](https://dataclient.io/docs/api/Schema) so memoization invalidates correctly. The codemod cannot do this for you. + +`argsKey` returns `fn(args)` for convenience **and** the function reference doubles as the cache path key on `WeakDependencyMap` — so `fn` must be **referentially stable**. Bind it in the constructor or at module scope; an inline arrow creates a new reference per call and misses the cache every time. + +```ts +// before — args read directly +class LensSchema { + denormalize(input, args, unvisit) { + const lens = args[0]?.portfolio; + return this.lookup(input, lens); + } +} + +// after — stable instance field + declared memo dimension +class LensSchema { + constructor({ lens }) { + this.lensSelector = lens; + } + denormalize(input, delegate) { + const lens = delegate.argsKey(this.lensSelector); + return this.lookup(input, lens); + } +} +``` + +See [`Scalar`](https://dataclient.io/rest/api/Scalar) for a real-world example. + +## What the codemod skips + +These are rare; do them by hand: + +- **Computed/string-keyed methods**: only literal `denormalize` keys are matched. +- **Methods reassigned dynamically** (`obj.denormalize = function(input, args, unvisit) { ... }`). +- **Custom helper functions** that wrap `(args, unvisit)` and are passed around — you'll need to update both the helper and its callers. +- **`argsKey` registration** for schemas whose output varies with `args` (see above). + +## New (additive, no migration needed) + +`Schema.normalize()` and the `visit()` callback gain an optional trailing `parentEntity` parameter — the nearest enclosing entity-like schema, tracked automatically by the visit walker. Existing schemas don't need changes; new schemas can opt in. + +## Where to find affected code + +Search for these patterns in your codebase: + +- `denormalize(input` followed by 3 params — both class methods and bare functions +- `unvisit(` calls inside a `denormalize` body +- Spread `...args` inside a `denormalize` body +- TS interfaces / `declare` fields with the 3-param signature +- Custom `Schema` / `SchemaClass` / `SchemaSimple` implementations + +## Reference + +- Changeset: `.changeset/denormalize-delegate.md` +- Built-in schema diffs: `packages/endpoint/src/schemas/{Array,Object,Values,Union,Query,Invalidate,Lazy,Collection}.ts` +- New interface: [`IDenormalizeDelegate`](https://dataclient.io/docs/api/Schema) diff --git a/AGENTS.md b/AGENTS.md index a90478f10d33..f72fbff4a350 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,6 @@ # Reactive Data Client -Monorepo for `@data-client` npm packages. +Monorepo for `@data-client` high performance npm packages. ## Architecture diff --git a/README.md b/README.md index fbd5d25c570f..b31d2abec0bf 100644 --- a/README.md +++ b/README.md @@ -437,6 +437,13 @@ For the small price of 9kb gziped.    [🏁Get started now](https://da 🛑 +Scalar +✅ +Scalar +lens-dependent entity fields +✅ + + any Query(Queryable) diff --git a/docs/core/api/useQuery.md b/docs/core/api/useQuery.md index 1fdc5edfb1c6..f794d373d643 100644 --- a/docs/core/api/useQuery.md +++ b/docs/core/api/useQuery.md @@ -16,7 +16,7 @@ import VoteDemo from '../shared/\_VoteDemo.mdx'; Data rendering without the fetch. Access any [Queryable Schema](/rest/api/schema#queryable)'s store value; like [Entity](/rest/api/Entity), [All](/rest/api/All), [Collection](/rest/api/Collection), [Query](/rest/api/Query), -and [Union](/rest/api/Union). [Lazy](/rest/api/Lazy) fields also work via their [`.query`](/rest/api/Lazy#query) accessor. +[Union](/rest/api/Union), and [Scalar](/rest/api/Scalar). [Lazy](/rest/api/Lazy) fields also work via their [`.query`](/rest/api/Lazy#query) accessor. If the value does not exist, returns `undefined`. `useQuery()` is reactive to data [mutations](../getting-started/mutations.md); rerendering only when necessary. Returns `undefined` @@ -60,7 +60,7 @@ function useQuery( [Queryable](/rest/api/schema#queryable) schemas require an `queryKey()` method that returns something. These include [Entity](/rest/api/Entity), [All](/rest/api/All), [Collection](/rest/api/Collection), [Query](/rest/api/Query), -and [Union](/rest/api/Union). [Lazy](/rest/api/Lazy) fields produce a Queryable via their [`.query`](/rest/api/Lazy#query) accessor. +[Union](/rest/api/Union), and [Scalar](/rest/api/Scalar). [Lazy](/rest/api/Lazy) fields produce a Queryable via their [`.query`](/rest/api/Lazy#query) accessor. ```ts interface Queryable { diff --git a/docs/core/shared/_schema_table.mdx b/docs/core/shared/_schema_table.mdx index 19d378410c54..6b0da04f3751 100644 --- a/docs/core/shared/_schema_table.mdx +++ b/docs/core/shared/_schema_table.mdx @@ -66,6 +66,13 @@ 🛑 +Scalar +✅ +[Scalar](/rest/api/Scalar) +lens-dependent entity fields +✅ + + any [Query(Queryable)](/rest/api/Query) diff --git a/docs/rest/api/Collection.md b/docs/rest/api/Collection.md index c21ee2950943..548ff0a78703 100644 --- a/docs/rest/api/Collection.md +++ b/docs/rest/api/Collection.md @@ -309,7 +309,7 @@ class User extends Entity { In this case, `user.todos` and getTodos() response (from the argsKey example) will always be the same (referentially equal) Array. -### nonFilterArgumentKeys? +### nonFilterArgumentKeys? {#nonFilterArgumentKeys} A convenient alternative to [argsKey](#argsKey) diff --git a/docs/rest/api/Lazy.md b/docs/rest/api/Lazy.md index e8b1e50447a1..04411b4749de 100644 --- a/docs/rest/api/Lazy.md +++ b/docs/rest/api/Lazy.md @@ -109,7 +109,7 @@ const buildings = useQuery( Returns a `LazyQuery` instance suitable for [useQuery](/docs/api/useQuery). The `LazyQuery`: - **`queryKey(args)`** — If the inner schema has a `queryKey` (Entity, Collection, etc.), delegates to it. Otherwise returns `args[0]` directly (for array/object schemas where you pass the raw normalized value). -- **`denormalize(input, args, unvisit)`** — Delegates to the inner schema, resolving IDs into full entity instances. +- **`denormalize(input, delegate)`** — Delegates to the inner schema, resolving IDs into full entity instances. The `.query` getter always returns the same instance (cached). diff --git a/docs/rest/api/Query.md b/docs/rest/api/Query.md index 1f305840e970..3ee56881599c 100644 --- a/docs/rest/api/Query.md +++ b/docs/rest/api/Query.md @@ -24,7 +24,7 @@ the same high performance and referential equality guarantees expected of Reacti [Schema](./schema.md) used to retrieve/denormalize data from the Reactive Data Client cache. This accepts any [Queryable](/rest/api/schema#queryable) schema: [Entity](./Entity.md), [All](./All.md), [Collection](./Collection.md), [Query](./Query.md), -[Union](./Union.md), and [Object](./Object.md) schemas for joining multiple entities. +[Union](./Union.md), [Scalar](./Scalar.md), and [Object](./Object.md) schemas for joining multiple entities. [Lazy](./Lazy.md) fields produce a Queryable via their [`.query`](./Lazy.md#query) accessor. ### process(entries, ...args) {#process} diff --git a/docs/rest/api/Scalar.md b/docs/rest/api/Scalar.md new file mode 100644 index 000000000000..079de1187a70 --- /dev/null +++ b/docs/rest/api/Scalar.md @@ -0,0 +1,274 @@ +--- +title: Scalar Schema - Lens-Dependent Entity Fields +sidebar_label: Scalar +--- + +import ScalarDemo from '../shared/\_ScalarDemo.mdx'; + +# Scalar + +`Scalar` describes [Entity](./Entity.md) fields whose values depend on endpoint args, +such as portfolio-, currency-, or locale-specific columns on the same row. + +Use `Scalar` when the field belongs to an entity, but its value changes based on a +"lens" selected by the request. Multiple components can render the same entity with +different lens args at the same time, each receiving the correct scalar values. + +- `lens`: **required** Selects the lens value from endpoint args. +- `key`: **required** Namespaces this scalar's internal table. +- `entity`: Binds the scalar to an `Entity` when it is used outside of an + `Entity.schema` field. + +::::note + +`Scalar` is for scalar values like numbers, strings, booleans, or date-derived values. +Use normal nested [schemas](./schema.md) for relationships to other entities. + +:::: + +## Usage + +In this example, `pct_equity` and `shares` depend on the selected portfolio, while +`name` and `price` are stable properties of the `Company` entity. + + + +On first render, `getCompanies` fetches once to populate the Company entities and +the initial `Scalar(portfolio)` cells. Every later portfolio switch re-denormalizes +from the existing `Collection` entity with the new lens — no network fetch — and +`getPortfolioColumns` fetches only the lens-dependent cells for portfolios the +user actually visits. Revisit a portfolio already in cache and neither endpoint +fires again. + +Wrapping lists in [Collection](./Collection.md) is what makes this work: +`Array` has no `queryKey`, so `useSuspense(getCompanies, { portfolio: 'B' })` +would miss the endpoint cache and trigger a refetch. `Collection.queryKey()` +returns its pk when the `Collection` entity is in the store, so the reuse path +fires as long as the pk is stable across the cases you want to share. + +Here [`argsKey: () => ({})`](./Collection.md#argsKey) forces every portfolio to +the same `pk`, so one Collection entity serves all lenses. When an endpoint has +real filter args alongside the lens, keep the filters in the pk and drop only +the lens: + +```typescript +new Collection([Company], { + argsKey: ({ portfolio, ...filters }) => filters, +}); +``` + +[`nonFilterArgumentKeys`](./Collection.md#nonFilterArgumentKeys) is a separate +concern — it controls which args are ignored when a mutation like `push` or +`assign` matches existing collections — and does _not_ collapse pks. Use it for +sort or pagination args where results differ per value (distinct pks) but +creates should still reach every variant. + +`getPortfolioColumns` also uses `Collection`, but keeps `portfolio` in its pk +with `argsKey: ({ portfolio }) => ({ portfolio })` because each portfolio has a +distinct column response. `Scalar.entityPk()` derives each cell's Company id +from the array item (delegating to `Company.pk()` by default), so the endpoint +can use the natural REST shape: + +```typescript +[ + { id: '1', pct_equity: 0.5, shares: 10000 }, + { id: '2', pct_equity: 0.2, shares: 4000 }, +] +``` + +### Entity Fields + +Use `Scalar` in an `Entity.schema` field when lens-dependent values arrive as part +of the entity response. + +```typescript +import { Collection, Entity, RestEndpoint, Scalar } from '@data-client/rest'; + +const PortfolioScalar = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'portfolio', +}); + +class Company extends Entity { + id = ''; + price = 0; + pct_equity = 0; + shares = 0; + + static schema = { + pct_equity: PortfolioScalar, + shares: PortfolioScalar, + }; +} + +const getCompanies = new RestEndpoint({ + path: '/companies', + searchParams: {} as { portfolio: string }, + schema: new Collection([Company], { argsKey: () => ({}) }), +}); +``` + +A single unbound `Scalar` instance can be shared across multiple entity classes. +When used as an `Entity.schema` field, the parent entity is inferred during +normalization. + +### Values Endpoint + +Use [Values](./Values.md) when an endpoint returns only the scalar columns, keyed by +entity pk. Since this response has no enclosing entity schema, pass `entity` when +constructing the `Scalar`. + +```typescript +import { Entity, RestEndpoint, Scalar, Values } from '@data-client/rest'; + +const CompanyPortfolioScalar = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'portfolio', + entity: Company, +}); + +const getPortfolioColumns = new RestEndpoint({ + path: '/companies/columns', + searchParams: {} as { portfolio: string }, + schema: new Values(CompanyPortfolioScalar), +}); + +// Response: { '1': { pct_equity: 0.5, shares: 32342 }, '2': { ... } } +``` + +Column-only endpoints write `Scalar(portfolio)` cells without modifying the +`Company` entities. A bound `Scalar` can still be used as an `Entity.schema` field; +the inferred parent entity takes precedence there. + +## Options + +```typescript +new Scalar({ lens, key, entity? }) +``` + +### lens(args): string | undefined {#lens} + +Selects the lens value from endpoint args, such as a portfolio ID. + +The lens value must be present when normalizing a response. Returning `undefined` +during normalize throws because the scalar cell cannot be stored under a retrievable +key. During denormalize, a missing lens returns `undefined` for that field. + +The returned value becomes part of the stored cell key and is also used for +cell lookup during [queryKey](#queryKey). It must be a string that does not +contain `|` — the `|` character is the cpk delimiter +(`entityKey|entityPk|lens`), and a lens containing `|` would collide with +other lenses that share the same trailing segment. + +### key: string {#key} + +Unique name for this scalar type. This namespaces the internal `Scalar` entity table. + +For example, `key: 'portfolio'` stores cells in `Scalar(portfolio)`. + +### entity?: Entity {#entity} + +Entity class this `Scalar` stores cells for. + +This is optional when the scalar is used as a field on `Entity.schema`, where the +parent entity is inferred. It is required for standalone usage such as +`new Values(PortfolioScalar)`. + +### entityPk(input, parent, key, args): string | number | undefined {#entityPk} + +Derives the bound Entity's primary key when `Scalar` is used standalone, such as +inside `Values`, `[Scalar]`, or `Collection([Scalar])`. The cell's actual pk +stored under `Scalar(key)` is the compound `entityKey|entityPk|lens` — this +method only supplies the `entityPk` piece. + +By default `entityPk()`: + +- returns the surrounding map `key` when it authoritatively addresses the + cell — i.e. `parent[key] === input`, as in `Values(Scalar)` where the map + key is the entity pk and the cell may not carry the pk fields — then +- delegates to the bound `Entity.pk(input, parent, key, args)` static so + `[Scalar]` and `Collection([Scalar])` array responses — including arrays + nested under a parent object schema like `{ stock: [Scalar] }`, and + custom or composite Entity pks — work out of the box. + +Override `entityPk()` in a subclass only when the response uses an id field the +`Entity.pk()` does not read: + +```typescript +class CompanyIdScalar extends Scalar { + entityPk(input: any) { + return input.companyId; + } +} +``` + +## Behavior + +### Normalize + +When normalizing an entity response, `Scalar` stores the field value in a separate +cell keyed by: + +```text +entityKey|entityPk|lensValue +``` + +The entity row keeps a lens-independent reference to that cell. This lets one +entity row point to different scalar values depending on the current endpoint args. + +When normalizing a `Values` response, each top-level key is treated as the entity pk, +and the response value is stored as that entity's scalar cell for the current lens. + +### Denormalize + +During denormalization, `Scalar` reads the current lens from endpoint args and looks +up the matching cell. If no matching lens or cell exists, the field denormalizes to +`undefined`. + +Because the lens participates in denormalization memoization, separate portfolio, +currency, or locale views cache independently while sharing the same base entity +data. + +### queryKey {#queryKey} + +`Scalar` is a [Queryable](/rest/api/schema#queryable) schema. When used as a +top-level endpoint schema — or passed to [useQuery](/docs/api/useQuery), +[Controller.get](/docs/api/Controller#get), [schema.Query](./Query.md), or any +other Queryable consumer — it reports the cpks of all cells whose lens matches +the current args: + +- Returns an array of compound pks on hit. +- Returns `undefined` when the lens is `undefined`, the table is missing, or + no cell matches the current lens. + +The common case — `Scalar` nested as an `Entity.schema` field — never reaches +this method. Denormalization goes through the parent entity, so `queryKey` +is only consulted when `Scalar` is itself the root schema being queried. + +### Normalized Storage + +```typescript +entities['Company']['1'] = { + id: '1', + price: 100, + pct_equity: ['1', 'pct_equity', 'Company'], + shares: ['1', 'shares', 'Company'], +} + +entities['Scalar(portfolio)']['Company|1|portfolioA'] = { + pct_equity: 0.5, + shares: 32342, +} + +entities['Scalar(portfolio)']['Company|1|portfolioB'] = { + pct_equity: 0.3, + shares: 323, +} +``` + +## Related + +- [Entity](/rest/api/Entity) — defines the base entity that scalar fields attach to +- [Values](./Values.md) — used for column-only endpoints (dictionary keyed by entity pk) +- [Union](./Union.md) — similar wrapper pattern for polymorphic entities +- [Queryable](/rest/api/schema#queryable) — Scalar participates in [useQuery](/docs/api/useQuery), [Controller.get](/docs/api/Controller#get), and [schema.Query](./Query.md) diff --git a/docs/rest/api/resource.md b/docs/rest/api/resource.md index 074b1017c3bd..f22393226461 100644 --- a/docs/rest/api/resource.md +++ b/docs/rest/api/resource.md @@ -149,7 +149,7 @@ If specified, will add [Resource.getList.getPage](#getpage) method on the `Resou ### nonFilterArgumentKeys -Pass-through option to [Collection.nonFilterArgumentKeys](./Collection.md#nonfilterargumentkeys) +Pass-through option to [Collection.nonFilterArgumentKeys](./Collection.md#nonFilterArgumentKeys) for [getList](#getlist) schema. ```ts diff --git a/docs/rest/api/schema.md b/docs/rest/api/schema.md index 8a54cec6e5cc..80f430e45f93 100644 --- a/docs/rest/api/schema.md +++ b/docs/rest/api/schema.md @@ -244,7 +244,7 @@ This enables their use in these additional cases: - Improve performance of [useSuspense](/docs/api/useSuspense), [useDLE](/docs/api/useDLE) by rendering before endpoint resolution `Querables` include [Entity](./Entity.md), [All](./All.md), [Collection](./Collection.md), [Query](./Query.md), -and [Union](./Union.md). [Lazy](./Lazy.md) fields produce a Queryable via their [`.query`](./Lazy.md#query) accessor. +[Union](./Union.md), and [Scalar](./Scalar.md). [Lazy](./Lazy.md) fields produce a Queryable via their [`.query`](./Lazy.md#query) accessor. ```ts interface Queryable { diff --git a/docs/rest/shared/_ScalarDemo.mdx b/docs/rest/shared/_ScalarDemo.mdx new file mode 100644 index 000000000000..56cd0de40cbd --- /dev/null +++ b/docs/rest/shared/_ScalarDemo.mdx @@ -0,0 +1,123 @@ +import HooksPlayground from '@site/src/components/HooksPlayground'; +import { companyFixtures } from '@site/src/fixtures/companies'; + + + +```ts title="api/Company" {11-20,26-28,34-36} collapsed +import { Collection, Entity, RestEndpoint, Scalar } from '@data-client/rest'; + +export class Company extends Entity { + id = ''; + name = ''; + price = 0; + pct_equity = 0; + shares = 0; +} + +const PortfolioScalar = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'portfolio', + entity: Company, +}); +Company.schema = { + pct_equity: PortfolioScalar, + shares: PortfolioScalar, +}; + +export const getCompanies = new RestEndpoint({ + path: '/companies', + searchParams: {} as { portfolio: string }, + // `portfolio` is a lens, not a filter — the returned Company list is the same + // regardless of lens. Dropping it from `argsKey` collapses every portfolio to + // one Collection pk, so `Collection.queryKey()` finds the list on every + // switch and `useSuspense` reuses it without refetching. + schema: new Collection([Company], { argsKey: () => ({}) }), +}); + +export const getPortfolioColumns = new RestEndpoint({ + path: '/companies/columns', + searchParams: {} as { portfolio: string }, + schema: new Collection([PortfolioScalar], { + argsKey: ({ portfolio }) => ({ portfolio }), + }), +}); +``` + +```tsx title="CompanyGrid" collapsed +import { type Company } from './api/Company'; + +export default function CompanyGrid({ companies }: { companies: Company[] }) { + return ( + + + + + + + + + + + {companies.map(c => ( + + + + + + + ))} + +
NamePrice% EquityShares
{c.name}${c.price.toFixed(2)}{formatPercent(c.pct_equity)}{formatShares(c.shares)}
+ ); +} + +function formatPercent(value: number | undefined) { + return value === undefined ? 'loading...' : `${(value * 100).toFixed(1)}%`; +} + +function formatShares(value: number | undefined) { + return value === undefined ? 'loading...' : value.toLocaleString(); +} +``` + +```tsx title="PortfolioGrid" +import { useSuspense, useFetch } from '@data-client/react'; +import { getCompanies, getPortfolioColumns } from './api/Company'; +import CompanyGrid from './CompanyGrid'; + +function PortfolioGrid() { + const [portfolio, setPortfolio] = React.useState('A'); + // Fetches on first render, then re-denormalizes from cache on every + // portfolio switch. The Collection's `queryKey()` ignores `portfolio`, + // so there is no endpoint refetch on switch. + const companies = useSuspense(getCompanies, { portfolio }); + // The first render's `useSuspense` already populated `Scalar(portfolio)` + // for `firstPortfolio`, so we only fetch columns when the user switches + // away. `useFetch` then dedupes later revisits via its endpoint cache. + const firstPortfolio = React.useRef(portfolio).current; + useFetch( + getPortfolioColumns, + portfolio === firstPortfolio ? null : { portfolio }, + ); + + return ( +
+ + +
+ ); +} + +render(); +``` + +
diff --git a/examples/benchmark/filter.js b/examples/benchmark/filter.js index a177c58bd831..3848e83b1fc5 100644 --- a/examples/benchmark/filter.js +++ b/examples/benchmark/filter.js @@ -7,13 +7,13 @@ * * @param {Benchmark.Suite} suite * @param {string} [filter] - * @returns {(name: string, fn: () => void) => void} + * @returns {(name: string, fn: () => void, options?: object) => void} */ export function createAdd(suite, filter) { const match = createMatcher(filter); - return (name, fn) => { + return (name, fn, options) => { if (match(name)) { - suite.add(name, fn); + suite.add(name, fn, options); } }; } diff --git a/examples/benchmark/micro.js b/examples/benchmark/micro.js index 498392c9b0d7..457a9d74b9af 100644 --- a/examples/benchmark/micro.js +++ b/examples/benchmark/micro.js @@ -1,16 +1,18 @@ +import { WeakDependencyMap } from './dist/index.js'; import { createAdd } from './filter.js'; import { printStatus } from './printStatus.js'; /** * Microbenchmark suite for testing very specific, isolated operations. * - * Tests 6 optimization patterns: + * Tests optimization patterns: * 1. forEach vs indexed for loop * 2. reduce+spread vs direct mutation * 3. array.map vs pre-allocated loop * 4. repeated getter vs cached * 5. slice+map vs pre-allocated extraction * 6. Map double-get vs single-get + * 7. WeakDependencyMap get/set (entity vs argsKey/scalar paths) * * @param {import('benchmark').Suite} suite * @param {string} [filter] @@ -302,6 +304,76 @@ export default function addMicroSuite(suite, filter) { } }); + // ============================================================ + // Optimization 7: WeakDependencyMap get/set hot path + // ============================================================ + // Targeted at the recent WeakDependencyMap refactor (drop `key`, + // pass args to set, lazy-allocate `next`). Allocates fresh entities + // each iteration to avoid weak-ref cleanup races. + const wdmEntities = Array.from({ length: 30 }, (_, i) => ({ id: i })); + const wdmGetDep = path => wdmEntities[path[1]]; + const lensFn = args => args[0]?.lens; + + function buildEntityDeps() { + const deps = new Array(30); + for (let i = 0; i < 30; i++) { + deps[i] = { path: ['Project', i], entity: wdmEntities[i] }; + } + return deps; + } + + function buildScalarDeps() { + const deps = new Array(30); + for (let i = 0; i < 28; i++) { + deps[i] = { path: ['Project', i], entity: wdmEntities[i] }; + } + // Include `key` so this works against pre-refactor `set` signatures too + // (variant A/A+B `set` recomputes from `path(args)`; `key` is harmless). + deps[28] = { path: lensFn, entity: undefined, key: 'portfolio-A' }; + deps[29] = { path: ['Project', 29], entity: wdmEntities[29] }; + return deps; + } + + const entityDeps = buildEntityDeps(); + const scalarDeps = buildScalarDeps(); + const wdmArgs = [{ lens: 'portfolio-A' }]; + + // Pre-built map for get-only benchmarks (steady-state cache hit) + const primedEntityMap = new WeakDependencyMap(); + primedEntityMap.set(entityDeps, 'cached', wdmArgs); + const primedScalarMap = new WeakDependencyMap(); + primedScalarMap.set(scalarDeps, 'cached', wdmArgs); + const entityRoot = entityDeps[0].entity; + const scalarRoot = scalarDeps[0].entity; + + add('7-WDM get entity (30 chain)', () => { + for (let i = 0; i < 1000; i++) { + primedEntityMap.get(entityRoot, wdmGetDep, wdmArgs); + } + }); + + add('7-WDM get scalar (30 chain w/ argsKey)', () => { + for (let i = 0; i < 1000; i++) { + primedScalarMap.get(scalarRoot, wdmGetDep, wdmArgs); + } + }); + + add('7-WDM set+get entity (30 chain, fresh)', () => { + const m = new WeakDependencyMap(); + m.set(entityDeps, 'value', wdmArgs); + for (let i = 0; i < 100; i++) { + m.get(entityRoot, wdmGetDep, wdmArgs); + } + }); + + add('7-WDM set+get scalar (30 chain, fresh)', () => { + const m = new WeakDependencyMap(); + m.set(scalarDeps, 'value', wdmArgs); + for (let i = 0; i < 100; i++) { + m.get(scalarRoot, wdmGetDep, wdmArgs); + } + }); + // ============================================================ // Completion handler with V8 optimization status // ============================================================ diff --git a/examples/benchmark/normalizr.js b/examples/benchmark/normalizr.js index f8da921f0a3d..ea5479177034 100644 --- a/examples/benchmark/normalizr.js +++ b/examples/benchmark/normalizr.js @@ -19,6 +19,11 @@ import { User, Department, buildBidirectionalChain, + Stock, + StockSchema, + StockScalarValuesSchema, + buildStockData, + buildStockScalarUpdate, } from './schemas.js'; import userData from './user.json' with { type: 'json' }; @@ -42,6 +47,31 @@ const actionMeta = { date, expiresAt: date + 10000000, }; +const stockUpdateMeta = { + ...actionMeta, + fetchedAt: date + 1, + date: date + 1, +}; + +const stockData = buildStockData(700); +const stockScalarUpdate = buildStockScalarUpdate(700); +const stockArgs = [{ portfolio: 'portfolioA' }]; +const stockState = normalize( + StockSchema, + stockData, + stockArgs, + initialState, + actionMeta, +); +const stockUpdatedState = normalize( + StockScalarValuesSchema, + stockScalarUpdate, + stockArgs, + stockState, + stockUpdateMeta, +); +globalThis.__scalarChurn = () => + denormalize(Stock, 's-0', stockState.entities, stockArgs); export default function addNormlizrSuite(suite, filter) { const memo = new MemoCache(); @@ -55,7 +85,27 @@ export default function addNormlizrSuite(suite, filter) { [], ); + // Scalar/Stock benches use a dedicated MemoCache so they do not pollute the + // shared `memo` used by the Project/User/AllProjects benches above. Sharing + // a MemoCache across unrelated schemas perturbs V8 hidden-class state for + // cached-path benches (masked real deltas by ~15% on Values withCache). + const memoStock = new MemoCache(); + memoStock.denormalize( + StockSchema, + stockState.result, + stockState.entities, + stockArgs, + ); + memoStock.denormalize( + StockSchema, + stockState.result, + stockUpdatedState.entities, + stockArgs, + ); + let curState = initialState; + let scalarCurState = initialState; + let scalarUpdateCurState = stockState; const add = createAdd(suite, filter); @@ -67,6 +117,20 @@ export default function addNormlizrSuite(suite, filter) { normalize(ProjectSchemaValues, dataValues, [], curState, actionMeta); curState = { ...initialState, entities: {}, endpoints: {} }; }); + add('normalizeLong Scalar', () => { + normalize(StockSchema, stockData, stockArgs, scalarCurState, actionMeta); + scalarCurState = { ...initialState, entities: {}, endpoints: {} }; + }); + add('normalizeLong Scalar update', () => { + normalize( + StockScalarValuesSchema, + stockScalarUpdate, + stockArgs, + scalarUpdateCurState, + stockUpdateMeta, + ); + scalarUpdateCurState = stockState; + }); add('denormalizeLong', () => { return new MemoCache().denormalize(ProjectSchema, result, entities); }); @@ -87,6 +151,14 @@ export default function addNormlizrSuite(suite, filter) { valuesState.entities, ); }); + add('denormalizeLong Scalar donotcache', () => { + return denormalize( + StockSchema, + stockState.result, + stockState.entities, + stockArgs, + ); + }); add('denormalizeShort donotcache 500x', () => { for (let i = 0; i < 500; ++i) { denormalize(User, 'gnoff', githubState.entities); @@ -119,6 +191,17 @@ export default function addNormlizrSuite(suite, filter) { add('denormalizeLong withCache', () => { return memo.denormalize(ProjectSchema, result, entities, []); }); + add( + 'denormalizeLong withCache (Scalar churn)', + () => { + return memo.denormalize(ProjectSchema, result, entities, []); + }, + { + setup: function () { + globalThis.__scalarChurn(); + }, + }, + ); add('denormalizeLong Values withCache', () => { return memo.denormalize( ProjectSchemaValues, @@ -127,6 +210,22 @@ export default function addNormlizrSuite(suite, filter) { [], ); }); + add('denormalizeLong Scalar withCache', () => { + return memoStock.denormalize( + StockSchema, + stockState.result, + stockState.entities, + stockArgs, + ); + }); + add('denormalizeLong Scalar update withCache', () => { + return memoStock.denormalize( + StockSchema, + stockState.result, + stockUpdatedState.entities, + stockArgs, + ); + }); add('denormalizeLong All withCache', () => { return memo.denormalize( AllProjects, diff --git a/examples/benchmark/schemas.js b/examples/benchmark/schemas.js index 2ed3c3991f8f..2809541349e8 100644 --- a/examples/benchmark/schemas.js +++ b/examples/benchmark/schemas.js @@ -5,6 +5,7 @@ import { Collection, All, Query, + Scalar, } from './dist/index.js'; export class BuildTypeDescription extends Entity { @@ -107,6 +108,73 @@ export const getSortedProjects = new Query( }, ); +export class Stock extends Entity { + id = ''; + ticker = ''; + exchange = ''; + sector = ''; + industry = ''; + name = ''; + marketCap = 0; + price = 0; + pct_equity = 0; + shares = 0; + + static key = 'Stock'; + pk() { + return this.id; + } +} + +export const PortfolioScalar = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'portfolio', + entity: Stock, +}); +Stock.schema = { + price: PortfolioScalar, + pct_equity: PortfolioScalar, + shares: PortfolioScalar, +}; + +export const StockSchema = { + stock: [Stock], +}; +export const StockScalarValuesSchema = { + stock: new Values(PortfolioScalar), +}; + +export function buildStockData(count = 700) { + const stock = []; + for (let i = 0; i < count; i++) { + stock.push({ + id: `s-${i}`, + ticker: `TKR${i}`, + exchange: i % 2 ? 'NASDAQ' : 'NYSE', + sector: `sector-${i % 11}`, + industry: `industry-${i % 23}`, + name: `Stock Number ${i}`, + marketCap: 1_000_000 + i * 137, + price: 10 + (i % 500) * 0.13, + pct_equity: ((i % 100) + 1) / 1000, + shares: 100 + i * 7, + }); + } + return { stock }; +} + +export function buildStockScalarUpdate(count = 700) { + const stock = {}; + for (let i = 0; i < count; i++) { + stock[`s-${i}`] = { + price: 10 + (i % 500) * 0.17, + pct_equity: ((i % 100) + 2) / 1000, + shares: 100 + i * 8, + }; + } + return { stock }; +} + // Degenerate bidirectional chain for #3822 stack overflow testing export class Department extends Entity { id = ''; diff --git a/examples/benchmark/src/index.ts b/examples/benchmark/src/index.ts index 61e1a11114bb..81d21c901479 100644 --- a/examples/benchmark/src/index.ts +++ b/examples/benchmark/src/index.ts @@ -15,5 +15,6 @@ export { Collection, All, Query, + Scalar, Invalidate, } from '@data-client/endpoint'; diff --git a/packages/core/src/controller/Controller.ts b/packages/core/src/controller/Controller.ts index 4efdde25d54d..dc6d75f0cc7c 100644 --- a/packages/core/src/controller/Controller.ts +++ b/packages/core/src/controller/Controller.ts @@ -14,7 +14,6 @@ import { DenormalizeNullable, EntityPath, MemoCache, - isEntity, denormalize, validateQueryKey, } from '@data-client/normalizr'; @@ -526,7 +525,7 @@ export default class Controller< if (shouldQuery) { isInvalid = !validateQueryKey(input); // endpoint without entities - } else if (!schema || !schemaHasEntity(schema)) { + } else if (!schema || !requiresDenormalize(schema)) { return { data: cacheEndpoints, expiryStatus: this.getExpiryStatus( @@ -645,23 +644,25 @@ function entityExpiresAt( return expiresAt; } -/** Determine whether the schema has any entities. +/** Whether `denormalize()` must run to reconstruct the response. * - * Without entities, denormalization is not needed, and results should not be queried. + * True iff some node in the schema tree defines `normalize` — meaning it + * transforms the response when written (entity write, polymorphic hoist, + * lazy delegation, etc.), so the cached value differs from the response. + * Mirrors the dispatch in `getVisit.ts`: schema classes are detected here + * by the same hook the visit walker uses, so we never need to walk their + * instance fields. */ -function schemaHasEntity(schema: Schema): boolean { - if (isEntity(schema)) return true; +function requiresDenormalize(schema: Schema): boolean { + if (!schema) return false; if (Array.isArray(schema)) - return schema.length !== 0 && schemaHasEntity(schema[0]); - if (schema && (typeof schema === 'object' || typeof schema === 'function')) { - const nestedSchema = - 'schema' in schema ? (schema.schema as Record) : schema; - if (typeof nestedSchema === 'function') { - return schemaHasEntity(nestedSchema); - } - return Object.values(nestedSchema).some(x => schemaHasEntity(x)); - } - return false; + return schema.length !== 0 && requiresDenormalize(schema[0]); + // Must reject primitives before probing `.normalize` — `String.prototype.normalize` exists. + const t = typeof schema; + if (t !== 'object' && t !== 'function') return false; + if (typeof (schema as any).normalize === 'function') return true; + // Plain-object schema map (e.g. `{ data: [Tacos], page: { ... } }`). + return Object.values(schema as object).some(x => requiresDenormalize(x)); } export type { ErrorTypes }; diff --git a/packages/core/src/controller/__tests__/getResponse.ts b/packages/core/src/controller/__tests__/getResponse.ts index e5c349c572c5..d9ea3eb446b1 100644 --- a/packages/core/src/controller/__tests__/getResponse.ts +++ b/packages/core/src/controller/__tests__/getResponse.ts @@ -1,4 +1,4 @@ -import { Endpoint, Entity } from '@data-client/endpoint'; +import { Endpoint, Entity, Scalar, schema } from '@data-client/endpoint'; import { ExpiryStatus } from '../..'; import { initialState } from '../../state/reducer/createReducer'; @@ -121,6 +121,26 @@ describe('Controller.getResponse()', () => { `); }); + it('does not over-denormalize a schema map containing string values', () => { + // Regression: `String.prototype.normalize` made string leaves look like schemas. + const controller = new Contoller(); + const ep = new Endpoint(() => Promise.resolve(), { + key() { + return 'string-only-schema'; + }, + schema: { label: 'hello', count: 0, nested: { value: 'world' } }, + }); + const cached = { label: 'hi', count: 5, nested: { value: 'there' } }; + const state = { + ...initialState, + endpoints: { [ep.key()]: cached }, + }; + const { data, expiryStatus } = controller.getResponse(ep, state); + expect(expiryStatus).toBe(ExpiryStatus.Valid); + // Reference equality — denormalize must be skipped for entity-free schemas. + expect(data).toBe(cached); + }); + it('infers schema with extra members but not set', () => { const controller = new Contoller(); class Tacos extends Entity { @@ -175,6 +195,129 @@ describe('Controller.getResponse()', () => { }); }); +describe('Controller.getResponse() with Scalar', () => { + // Regression: `requiresDenormalize` (formerly `schemaHasEntity`) previously + // walked `Object.values(scalar)` and recursed into the `lensSelector` + // function, causing infinite recursion (RangeError). The table-resident + // Scalar schema (no `pk`) was also not recognized, so `Values(Scalar)` + // returned false and Controller skipped denormalization — returning raw + // cpk strings instead of the joined cell data. See + // packages/core/src/controller/Controller.ts `requiresDenormalize`. + + class Company extends Entity { + id = ''; + price = 0; + pct_equity = 0; + shares = 0; + + static key = 'Company'; + } + const PortfolioScalar = new Scalar({ + lens: (args: readonly any[]) => args[0]?.portfolio, + key: 'portfolio', + entity: Company, + }); + Company.schema = { + pct_equity: PortfolioScalar, + shares: PortfolioScalar, + } as any; + + // Hard cap so a regression that re-introduces infinite recursion fails + // immediately rather than appearing as a generic Jest timeout. + const FAST_TIMEOUT = 2000; + + it( + 'denormalizes Values(Scalar) cells without infinite recursion', + () => { + const controller = new Contoller(); + const ep = new Endpoint( + ({ portfolio }: { portfolio: string }) => Promise.resolve(), + { + key: ({ portfolio }) => `getColumns ${portfolio}`, + schema: new schema.Values(PortfolioScalar), + }, + ); + + const state = { + ...initialState, + entities: { + 'Scalar(portfolio)': { + 'Company|1|A': { pct_equity: 0.5, shares: 100 }, + 'Company|2|A': { pct_equity: 0.2, shares: 40 }, + }, + }, + endpoints: { + [ep.key({ portfolio: 'A' })]: { + '1': 'Company|1|A', + '2': 'Company|2|A', + }, + }, + }; + + let result: ReturnType; + expect(() => { + result = controller.getResponse(ep, { portfolio: 'A' }, state); + }).not.toThrow(); + + expect(result!.expiryStatus).toBe(ExpiryStatus.Valid); + // Critical: cells are joined, not raw cpk strings — proves + // `requiresDenormalize` returned true so denormalize ran. + expect(result!.data).toEqual({ + '1': { pct_equity: 0.5, shares: 100 }, + '2': { pct_equity: 0.2, shares: 40 }, + }); + }, + FAST_TIMEOUT, + ); + + it( + 'denormalizes Entity with Scalar field schema without infinite recursion', + () => { + const controller = new Contoller(); + const ep = new Endpoint( + ({ portfolio }: { portfolio: string }) => Promise.resolve(), + { + key: ({ portfolio }) => `getCompanies ${portfolio}`, + schema: [Company], + }, + ); + + const state = { + ...initialState, + entities: { + Company: { + '1': { + id: '1', + price: 100, + pct_equity: ['1', 'pct_equity', 'Company'], + shares: ['1', 'shares', 'Company'], + }, + }, + 'Scalar(portfolio)': { + 'Company|1|A': { pct_equity: 0.5, shares: 100 }, + }, + }, + endpoints: { + [ep.key({ portfolio: 'A' })]: ['1'], + }, + }; + + let result: ReturnType; + expect(() => { + result = controller.getResponse(ep, { portfolio: 'A' }, state); + }).not.toThrow(); + + expect(result!.expiryStatus).toBe(ExpiryStatus.Valid); + const company = (result!.data as any[])[0]; + expect(company.id).toBe('1'); + expect(company.price).toBe(100); + expect(company.pct_equity).toBe(0.5); + expect(company.shares).toBe(100); + }, + FAST_TIMEOUT, + ); +}); + describe('Snapshot.getResponseMeta()', () => { it('denormalizes schema with extra members but not set', () => { const controller = new Contoller(); diff --git a/packages/endpoint/README.md b/packages/endpoint/README.md index ff73650e511c..eee61bdb8023 100644 --- a/packages/endpoint/README.md +++ b/packages/endpoint/README.md @@ -277,6 +277,13 @@ Networking definition: [Endpoints](https://dataclient.io/rest/api/Endpoint) 🛑 +Scalar +✅ +Scalar +lens-dependent entity fields +✅ + + any Query(Queryable) diff --git a/packages/endpoint/src/index.ts b/packages/endpoint/src/index.ts index 694c029d87cd..dae98e3acb5f 100644 --- a/packages/endpoint/src/index.ts +++ b/packages/endpoint/src/index.ts @@ -19,6 +19,7 @@ export { Values, All, Lazy, + Scalar, unshift, } from './schema.js'; // Without this we get 'cannot be named without a reference to' for resource()....why is this? diff --git a/packages/endpoint/src/interface.ts b/packages/endpoint/src/interface.ts index 650de6e25d17..c4b08645e319 100644 --- a/packages/endpoint/src/interface.ts +++ b/packages/endpoint/src/interface.ts @@ -24,6 +24,19 @@ export type Serializable< > = (value: any) => T; export interface SchemaSimple { + /** + * Normalize a value into entity table form. + * + * @param input The value being normalized. + * @param parent The parent object/array/dictionary containing `input`. + * @param key The key under which `input` lives on `parent`. + * @param args The endpoint args for this normalize call. + * @param visit Recursive visitor for nested schemas. + * @param delegate Store accessors for reading/writing entities. + * @param parentEntity Nearest enclosing entity-like schema (one with `pk`), + * tracked automatically by the visit walker. `Scalar` + * uses this to discover its entity binding. + */ normalize( input: any, parent: any, @@ -31,12 +44,9 @@ export interface SchemaSimple { args: any[], visit: (...args: any) => any, delegate: { getEntity: any; setEntity: any }, + parentEntity?: any, ): any; - denormalize( - input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, - ): T; + denormalize(input: {}, delegate: IDenormalizeDelegate): T; queryKey( args: Args, unvisit: (...args: any) => any, @@ -111,7 +121,21 @@ export interface EntityTable { | undefined; } -/** Visits next data + schema while recurisvely normalizing */ +/** + * Visits next data + schema while recursively normalizing. + * + * @param schema The schema to apply to `value`. + * @param value The value being visited. + * @param parent The parent object/array/dictionary that holds `value`. + * Schemas that recurse via `visit` should pass their own + * `input` (or the surrounding container) here. + * @param key The key under which `value` lives on `parent`. + * @param args The endpoint args for this normalize call. + * + * The walker internally tracks the nearest enclosing entity-like schema and + * forwards it to `schema.normalize` as a trailing `parentEntity` argument — + * see `SchemaSimple.normalize`. Consumers of `visit` don't pass it. + */ export interface Visit { (schema: any, value: any, parent: any, key: any, args: readonly any[]): any; creating?: boolean; @@ -159,6 +183,20 @@ export interface IQueryDelegate { INVALID: symbol; } +/** Helpers during schema.denormalize() */ +export interface IDenormalizeDelegate { + /** Recursive denormalize of nested schemas */ + unvisit(schema: any, input: any): any; + /** Raw endpoint args. Reading this does NOT contribute to cache + * invalidation — if your output varies with args, register an `argsKey` + * so the cache buckets correctly. */ + readonly args: readonly any[]; + /** Adds a memoization dimension to the surrounding cache frame. + * `fn` must be referentially stable (it doubles as the cache path key). + * Returns `fn(args)` for convenience. */ + argsKey(fn: (args: readonly any[]) => string | undefined): string | undefined; +} + /** Helpers during schema.normalize() */ export interface INormalizeDelegate { /** Action meta-data for this normalize call */ diff --git a/packages/endpoint/src/schema.d.ts b/packages/endpoint/src/schema.d.ts index 5a0c06507d3d..9ff83f3a1812 100644 --- a/packages/endpoint/src/schema.d.ts +++ b/packages/endpoint/src/schema.d.ts @@ -1,6 +1,7 @@ import type { Schema, EntityInterface, + IDenormalizeDelegate, PolymorphicInterface, SchemaClass, IQueryDelegate, @@ -27,6 +28,7 @@ import { import { default as Invalidate } from './schemas/Invalidate.js'; import { default as Lazy } from './schemas/Lazy.js'; import { default as Query } from './schemas/Query.js'; +import { default as Scalar } from './schemas/Scalar.js'; import type { CollectionConstructor, DefaultArgs, @@ -35,7 +37,7 @@ import type { UnionResult, } from './schemaTypes.js'; -export { EntityMap, Invalidate, Query, Lazy, EntityMixin, Entity }; +export { EntityMap, Invalidate, Query, Lazy, Scalar, EntityMixin, Entity }; export type { SchemaClass }; @@ -82,8 +84,7 @@ export class Array implements SchemaClass { denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): (S extends EntityMap ? T : Denormalize)[]; queryKey( @@ -137,8 +138,7 @@ export class All< denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): (S extends EntityMap ? T : Denormalize)[]; queryKey( @@ -176,11 +176,7 @@ export class Object< _denormalizeNullable(): DenormalizeNullableObject; - denormalize( - input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, - ): DenormalizeObject; + denormalize(input: {}, delegate: IDenormalizeDelegate): DenormalizeObject; queryKey( args: ObjectArgs, @@ -268,8 +264,7 @@ export interface UnionInstance< denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): AbstractInstanceType; queryKey( @@ -351,8 +346,7 @@ export class Values implements SchemaClass { denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): Record< string, Choices extends EntityMap ? T : Denormalize diff --git a/packages/endpoint/src/schema.js b/packages/endpoint/src/schema.js index 03b5ccaca386..26dced46e42a 100644 --- a/packages/endpoint/src/schema.js +++ b/packages/endpoint/src/schema.js @@ -12,3 +12,4 @@ export { } from './schemas/EntityMixin.js'; export { default as Query } from './schemas/Query.js'; export { default as Lazy } from './schemas/Lazy.js'; +export { default as Scalar } from './schemas/Scalar.js'; diff --git a/packages/endpoint/src/schemas/Array.ts b/packages/endpoint/src/schemas/Array.ts index aef07c2bc023..728d02643f32 100644 --- a/packages/endpoint/src/schemas/Array.ts +++ b/packages/endpoint/src/schemas/Array.ts @@ -1,6 +1,6 @@ import PolymorphicSchema from './Polymorphic.js'; import { filterEmpty, getValues } from './utils.js'; -import { Visit } from '../interface.js'; +import { IDenormalizeDelegate, Visit } from '../interface.js'; /** * Represents arrays @@ -15,14 +15,12 @@ export default class ArraySchema extends PolymorphicSchema { ); } - denormalize( - input: any, - args: any[], - unvisit: (schema: any, input: any) => any, - ) { + denormalize(input: any, delegate: IDenormalizeDelegate) { return input.map ? input - .map((entityOrId: any) => this.denormalizeValue(entityOrId, unvisit)) + .map((entityOrId: any) => + this.denormalizeValue(entityOrId, delegate.unvisit), + ) .filter(filterEmpty) : input; } diff --git a/packages/endpoint/src/schemas/Collection.ts b/packages/endpoint/src/schemas/Collection.ts index 551276f91003..5df1d25dbba4 100644 --- a/packages/endpoint/src/schemas/Collection.ts +++ b/packages/endpoint/src/schemas/Collection.ts @@ -2,6 +2,7 @@ import ArraySchema from './Array.js'; import { consistentSerialize } from './consistentSerialize.js'; import Values from './Values.js'; import { + IDenormalizeDelegate, INormalizeDelegate, PolymorphicInterface, IQueryDelegate, @@ -276,10 +277,9 @@ export default class CollectionSchema< denormalize( input: any, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): ReturnType { - return this.schema.denormalize(input, args, unvisit) as any; + return this.schema.denormalize(input, delegate) as any; } } @@ -525,12 +525,11 @@ function createIfValid(value: object): any | undefined { function denormalize( this: CollectionSchema, input: any, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): any { return Array.isArray(input) ? - (this.schema.denormalize(input, args, unvisit) as any) - : (this.schema.denormalize([input], args, unvisit)[0] as any); + (this.schema.denormalize(input, delegate) as any) + : (this.schema.denormalize([input], delegate)[0] as any); } /** * We call schema.denormalize and schema.normalize directly diff --git a/packages/endpoint/src/schemas/Entity.ts b/packages/endpoint/src/schemas/Entity.ts index 35e55c4361e3..3337db9d9711 100644 --- a/packages/endpoint/src/schemas/Entity.ts +++ b/packages/endpoint/src/schemas/Entity.ts @@ -1,3 +1,4 @@ +import type { IDenormalizeDelegate } from '../interface.js'; import { AbstractInstanceType } from '../normal.js'; import { Entity as EntityMixin } from '../schema.js'; @@ -94,7 +95,6 @@ First three members: ${JSON.stringify(input.slice(0, 3), null, 2)}`; declare static denormalize: ( this: T, input: any, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ) => AbstractInstanceType; } diff --git a/packages/endpoint/src/schemas/EntityMixin.ts b/packages/endpoint/src/schemas/EntityMixin.ts index 221f4c282d78..bf5afe2ecaa8 100644 --- a/packages/endpoint/src/schemas/EntityMixin.ts +++ b/packages/endpoint/src/schemas/EntityMixin.ts @@ -3,6 +3,7 @@ import type { Visit, IQueryDelegate, INormalizeDelegate, + IDenormalizeDelegate, } from '../interface.js'; import { AbstractInstanceType } from '../normal.js'; import type { @@ -344,8 +345,7 @@ export default function EntityMixin( static denormalize( this: T, input: any, - args: any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): AbstractInstanceType { if (typeof input === 'symbol') { return input as any; @@ -354,7 +354,7 @@ export default function EntityMixin( // note: iteration order must be stable for (const key of Object.keys(this.schema)) { const schema = this.schema[key]; - const value = unvisit(schema, input[key]); + const value = delegate.unvisit(schema, input[key]); if (typeof value === 'symbol') { // if default is not 'falsy', then this is required, so propagate INVALID symbol diff --git a/packages/endpoint/src/schemas/EntityTypes.ts b/packages/endpoint/src/schemas/EntityTypes.ts index 8e5f0d0179d5..a4669cc628d3 100644 --- a/packages/endpoint/src/schemas/EntityTypes.ts +++ b/packages/endpoint/src/schemas/EntityTypes.ts @@ -1,4 +1,8 @@ -import type { Schema, IQueryDelegate } from '../interface.js'; +import type { + Schema, + IDenormalizeDelegate, + IQueryDelegate, +} from '../interface.js'; import { AbstractInstanceType } from '../normal.js'; /** @@ -185,8 +189,7 @@ export interface IEntityClass { >( this: T, input: any, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): AbstractInstanceType; /** All instance defaults set */ readonly defaults: any; diff --git a/packages/endpoint/src/schemas/ImmutableUtils.ts b/packages/endpoint/src/schemas/ImmutableUtils.ts index fa4e7fd40886..8df19febb739 100644 --- a/packages/endpoint/src/schemas/ImmutableUtils.ts +++ b/packages/endpoint/src/schemas/ImmutableUtils.ts @@ -24,12 +24,6 @@ export function isImmutable(object: {}): object is { /** * Denormalize an immutable entity. - * - * @param {Schema} schema - * @param {Immutable.Map|Immutable.Record} input - * @param {function} unvisit - * @param {function} getDenormalizedEntity - * @return {Immutable.Map|Immutable.Record} */ export function denormalizeImmutable( schema: any, diff --git a/packages/endpoint/src/schemas/Invalidate.ts b/packages/endpoint/src/schemas/Invalidate.ts index 007f9dbc3baa..789170eb2d48 100644 --- a/packages/endpoint/src/schemas/Invalidate.ts +++ b/packages/endpoint/src/schemas/Invalidate.ts @@ -1,5 +1,9 @@ import PolymorphicSchema from './Polymorphic.js'; -import type { EntityInterface, INormalizeDelegate } from '../interface.js'; +import type { + EntityInterface, + IDenormalizeDelegate, + INormalizeDelegate, +} from '../interface.js'; import type { AbstractInstanceType } from '../normal.js'; type ProcessableEntity = EntityInterface & { process: any }; @@ -108,12 +112,11 @@ export default class Invalidate< denormalize( id: string | { id: string; schema: string }, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): E extends ProcessableEntity ? AbstractInstanceType : AbstractInstanceType { // denormalizeValue handles both single entity and polymorphic cases - return this.denormalizeValue(id, unvisit) as any; + return this.denormalizeValue(id, delegate.unvisit) as any; } /* istanbul ignore next */ diff --git a/packages/endpoint/src/schemas/Lazy.ts b/packages/endpoint/src/schemas/Lazy.ts index 65a4d44fef09..38149db4f98c 100644 --- a/packages/endpoint/src/schemas/Lazy.ts +++ b/packages/endpoint/src/schemas/Lazy.ts @@ -1,4 +1,8 @@ -import type { Schema, SchemaSimple } from '../interface.js'; +import type { + IDenormalizeDelegate, + Schema, + SchemaSimple, +} from '../interface.js'; import type { Denormalize, DenormalizeNullable, @@ -57,7 +61,7 @@ export default class Lazy implements SchemaSimple { return visit(this.schema, input, parent, key, args); } - denormalize(input: {}, _args: readonly any[], _unvisit: any): any { + denormalize(input: {}, _delegate: IDenormalizeDelegate): any { // If we could figure out we're processing while nested vs from queryKey, then can can get rid of LazyQuery and just use this in both contexts. return input; } @@ -83,8 +87,7 @@ export default class Lazy implements SchemaSimple { declare _denormalizeNullable: ( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ) => any; declare _normalizeNullable: () => NormalizeNullable; @@ -103,12 +106,8 @@ export class LazyQuery> { this.schema = schema; } - denormalize( - input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, - ): Denormalize { - return unvisit(this.schema, input); + denormalize(input: {}, delegate: IDenormalizeDelegate): Denormalize { + return delegate.unvisit(this.schema, input); } queryKey( @@ -125,7 +124,6 @@ export class LazyQuery> { declare _denormalizeNullable: ( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ) => DenormalizeNullable; } diff --git a/packages/endpoint/src/schemas/Object.ts b/packages/endpoint/src/schemas/Object.ts index 5d7be8eb730e..f6686c4a6ca3 100644 --- a/packages/endpoint/src/schemas/Object.ts +++ b/packages/endpoint/src/schemas/Object.ts @@ -1,5 +1,5 @@ import { isImmutable, denormalizeImmutable } from './ImmutableUtils.js'; -import { Visit } from '../interface.js'; +import { IDenormalizeDelegate, Visit } from '../interface.js'; export const normalize = ( schema: any, @@ -26,11 +26,10 @@ export const normalize = ( export function denormalize( schema: any, input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): any { if (isImmutable(input)) { - return denormalizeImmutable(schema, input, unvisit); + return denormalizeImmutable(schema, input, delegate.unvisit); } const object: Record = { ...input }; @@ -38,7 +37,7 @@ export function denormalize( for (let i = 0; i < keys.length; i++) { const key = keys[i]; - const item = unvisit(schema[key], object[key]); + const item = delegate.unvisit(schema[key], object[key]); if (object[key] !== undefined) { object[key] = item; } @@ -93,12 +92,8 @@ export default class ObjectSchema { return normalize(this.schema, ...args); } - denormalize( - input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, - ): any { - return denormalize(this.schema, input, args, unvisit); + denormalize(input: {}, delegate: IDenormalizeDelegate): any { + return denormalize(this.schema, input, delegate); } queryKey(args: any, unvisit: any) { diff --git a/packages/endpoint/src/schemas/Query.ts b/packages/endpoint/src/schemas/Query.ts index d82f071f6866..e794e08a0075 100644 --- a/packages/endpoint/src/schemas/Query.ts +++ b/packages/endpoint/src/schemas/Query.ts @@ -1,4 +1,8 @@ -import type { Queryable, SchemaSimple } from '../interface.js'; +import type { + IDenormalizeDelegate, + Queryable, + SchemaSimple, +} from '../interface.js'; import type { Denormalize, NormalizeNullable, SchemaArgs } from '../normal.js'; /** @@ -27,10 +31,12 @@ export default class Query< return (this.schema as any).normalize(...args); } - denormalize(input: {}, args: any, unvisit: any): ReturnType

{ - const value = unvisit(this.schema, input); + denormalize(input: {}, delegate: IDenormalizeDelegate): ReturnType

{ + const value = delegate.unvisit(this.schema, input); return ( - typeof value === 'symbol' ? value : this.process(value, ...args)) as any; + typeof value === 'symbol' ? value : ( + this.process(value, ...delegate.args) + )) as any; } queryKey( @@ -42,8 +48,7 @@ export default class Query< declare _denormalizeNullable: ( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ) => ReturnType

| undefined; declare _normalizeNullable: () => NormalizeNullable; diff --git a/packages/endpoint/src/schemas/Scalar.ts b/packages/endpoint/src/schemas/Scalar.ts new file mode 100644 index 000000000000..d980b4960bc4 --- /dev/null +++ b/packages/endpoint/src/schemas/Scalar.ts @@ -0,0 +1,285 @@ +import type { + IDenormalizeDelegate, + INormalizeDelegate, + IQueryDelegate, + Mergeable, + Visit, +} from '../interface.js'; + +interface ScalarOptions { + /** + * Selects the lens value from Endpoint args. + * + * The returned value is part of the stored cell key, so it must be stable + * for a given lens selection. + */ + lens: (args: readonly any[]) => string | undefined; + /** + * Unique namespace for this Scalar's internal entity table. + */ + key: string; + /** + * Entity class this Scalar stores cells for. + * + * Optional when used as a field on `Entity.schema`, where the parent Entity + * is inferred. Required for standalone usage such as `schema.Values`. + */ + entity?: { + key: string; + pk?: (...args: any[]) => string | number | undefined; + }; +} + +/** + * Represents lens-dependent scalar fields on entities. + * + * Scalar stores values that belong to an Entity but vary by Endpoint args, + * such as portfolio-, currency-, or locale-specific columns. Use it as an + * `Entity.schema` field, or bind `entity` when using it standalone in + * `schema.Values`. + * + * @see https://dataclient.io/rest/api/Scalar + */ +export default class Scalar implements Mergeable { + readonly key: string; + readonly lensSelector: (args: readonly any[]) => string | undefined; + readonly entity: ScalarOptions['entity']; + readonly entityKey: string | undefined; + /** + * Allow normalize to receive primitive field values. + * + * Scalar stores per-cell values like `0.5`, so the visit walker must not + * apply its primitive short-circuit before dispatching to `normalize()`. + */ + readonly acceptsPrimitives = true; + + /** + * Represents lens-dependent scalar fields on entities. + * + * @see https://dataclient.io/rest/api/Scalar + */ + constructor(options: ScalarOptions) { + this.key = `Scalar(${options.key})`; + this.lensSelector = options.lens; + this.entity = options.entity; + this.entityKey = options.entity?.key; + } + + /** + * The bound Entity's pk for a standalone scalar cell. + * + * Prefers the surrounding map key (authoritative for `Values(Scalar)`, + * where `parent[key] === input`), then falls back to the bound + * `Entity.pk(...)`. Other shapes — `[Scalar]` top-level (where `key` is + * `undefined`) or nested under a plain object schema like + * `{ stock: [Scalar] }` (where `Array.normalize` forwards the parent + * object's field name as `key`, but `parent[key]` is the enclosing array, + * not the item) — must derive pk from the item itself. + * + * @see https://dataclient.io/rest/api/Scalar#entityPk + * @param [input] the scalar cell input + * @param [parent] When normalizing, the object which included the cell + * @param [key] When normalizing, the surrounding map key (if any) + * @param [args] ...args sent to Endpoint + */ + entityPk( + input: any, + parent: any, + key: string | undefined, + args: readonly any[], + ): string | number | undefined { + // Only trust `key` when the enclosing container literally maps it to this + // cell — i.e. `Values(Scalar)`, where the map key is the entity pk by + // construction. `Array.normalize` forwards the *outer* object's field name + // as `key` for every element (see Array.ts), so `key !== undefined` alone + // would collapse every item onto the same compound pk. + if (key !== undefined && parent != null && parent[key] === input) { + return key; + } + return this.entity?.pk?.(input, parent, key, args); + } + + createIfValid(props: any) { + return { ...props }; + } + + merge(existing: any, incoming: any) { + return { ...existing, ...incoming }; + } + + /** + * Determines whether an incoming write is older than the stored cell. + * + * Defaults to comparing `fetchedAt`, matching Entity behavior so older + * responses do not overwrite newer values. + */ + shouldReorder( + existingMeta: { date: number; fetchedAt: number }, + incomingMeta: { date: number; fetchedAt: number }, + existing: any, + incoming: any, + ) { + return incomingMeta.fetchedAt < existingMeta.fetchedAt; + } + + mergeWithStore( + existingMeta: { date: number; fetchedAt: number }, + incomingMeta: { date: number; fetchedAt: number }, + existing: any, + incoming: any, + ) { + return this.shouldReorder(existingMeta, incomingMeta, existing, incoming) ? + this.merge(incoming, existing) + : this.merge(existing, incoming); + } + + mergeMetaWithStore( + existingMeta: { fetchedAt: number; date: number; expiresAt: number }, + incomingMeta: { fetchedAt: number; date: number; expiresAt: number }, + existing: any, + incoming: any, + ) { + return this.shouldReorder(existingMeta, incomingMeta, existing, incoming) ? + existingMeta + : incomingMeta; + } + + normalize( + input: any, + parent: any, + key: any, + args: any[], + visit: Visit, + delegate: INormalizeDelegate, + parentEntity: any, + ): any { + const lensValue = this.lensSelector(args); + + // Without a lens we cannot derive a retrievable cell key — writing to + // `${…}|undefined` would silently corrupt the Scalar table (literal + // "undefined" collides across rows) and denormalize would never find + // the data. A missing lens during normalize is a configuration bug; + // throw rather than mask it. + if (lensValue === undefined) { + throw new Error( + `${this.key}: lens returned undefined during normalize. Ensure endpoint args contain the lens value.`, + ); + } + + // Entity-field path: the visit walker supplies the enclosing Entity + // class as `parentEntity`. `input` is the per-cell scalar value + // (e.g. 0.5), `parent` is the entity data row, `key` is the field name + // (always a string from Object.keys, no coercion needed). + if (typeof parentEntity === 'function' && parentEntity.pk) { + const entityKey: string = parentEntity.key; + // TODO: this re-derives the enclosing entity's pk without the enclosing + // `parent`/`key` context that `EntityMixin.normalize` used to compute + // the authoritative id (see EntityMixin.ts around `this.pk(processedEntity, parent, key, args)`). + // If a custom `pk()` reads its 2nd/3rd args, we will key the Scalar cell + // differently than the entity is stored under and denormalize will miss. + // Fix by threading the enclosing parent/key through the visit walker + // (e.g. via a parentContext param on `visit` / INormalizeDelegate) and + // forwarding them here instead of `undefined, undefined`. + const id = `${parentEntity.pk(parent, undefined, undefined, args)}`; + // Merge only this field's value — EntityMixin's loop calls us once + // per scalar field, and `merge` accumulates them into the cell. + delegate.mergeEntity(this, `${entityKey}|${id}|${lensValue}`, { + [key]: input, + }); + // Wrapper is a tuple: `[entityPk, fieldName, entityKey]`. Using an + // array (not an object) lets denormalize distinguish it from cell + // data via `Array.isArray` without a brand property — and produces + // smaller serialized state for SSR. `entityKey` is recorded so a + // single Scalar shared across entity types resolves the correct cell. + return [id, key, entityKey]; + } + + // Standalone path: input is scalar cell data, and pk is derived from either + // the item itself (`[Scalar]`) or the surrounding key (`Values(Scalar)`). + // No parent entity to infer from — require explicit `entity` binding. + if (this.entityKey === undefined) { + throw new Error( + 'Scalar used standalone (e.g. inside `schema.Values`) requires an `entity` option to bind it to an Entity class.', + ); + } + const id = this.entityPk(input, parent, key, args); + if (id === undefined) { + throw new Error( + `${this.key}: cannot derive entity pk for cell. Provide items with an ` + + '`id` field, a pk-keyed map key, or override Scalar.entityPk().', + ); + } + // cpk = compound pk: `"entityKey|entityPk|lensValue"`; used throughout this file. + const cpk = `${this.entityKey}|${id}|${lensValue}`; + delegate.mergeEntity(this, cpk, { ...input }); + return cpk; + } + + denormalize(input: any, delegate: IDenormalizeDelegate): any { + // Legit inputs are wrapper array, cpk string, cell object, or symbol. + // Any other primitive — falsy (`0`, `''`, `false`, `null`, `undefined`) + // or truthy (`0.5`, `true`, `42`, `bigint`, `symbol`, etc.) — cannot be + // resolved via the entity table and must short-circuit here. Without + // this, non-string truthy primitives would infinite-recurse through the + // Values branch below: `unvisit` re-dispatches back to us because + // `isEntity(this)` is false and its string-only fast path misses. This + // can happen during schema migration when a Scalar is added to an entity + // with cached raw numeric/boolean field values still in the store. + if (!input || (typeof input !== 'object' && typeof input !== 'string')) { + return input; + } + + // Entity-field wrapper `[entityPk, fieldName, entityKey]`: cpk is derived + // from current `args`, so record `lensSelector` on the enclosing Entity's + // cache frame. `lensSelector` must be ctor-bound (stable ref); see WeakDependencyMap. + if (Array.isArray(input)) { + const lensValue = delegate.argsKey(this.lensSelector); + if (lensValue === undefined) return undefined; + const cellData = delegate.unvisit( + this, + `${input[2]}|${input[0]}|${lensValue}`, + ); + return cellData && typeof cellData !== 'symbol' ? + cellData[input[1]] + : undefined; + } + + // Cell data passes through unchanged (recursive call from `unvisit` + // when looking up the cell via its compound pk). + if (typeof input === 'object') return input; + + // String-cpk fallback. Walker-driven Values(Scalar) never reaches this — + // `unvisit.ts` intercepts string+createIfValid before `schema.denormalize`. + // No `argsKey`: cpk is fixed at normalize-time; per-entity WDM keys on cpk. + return delegate.unvisit(this, input); + } + + /** + * Returns the cpks of cells matching the current lens, or undefined. + * + * Only consulted when `Scalar` is an endpoint's top-level schema; field + * usage resolves through the parent entity. Relies on `lens` not + * containing the cpk delimiter `|`. + */ + queryKey( + args: readonly any[], + unvisit: any, + delegate: IQueryDelegate, + ): string[] | undefined { + const lens = this.lensSelector(args); + if (lens === undefined) return undefined; + + const cells = delegate.getEntities(this.key); + if (!cells) return undefined; + + const cpks: string[] = []; + for (const [cpk, entity] of cells.entries()) { + if (!entity || typeof entity === 'symbol') continue; + const lastPipe = cpk.lastIndexOf('|'); + if (lastPipe !== -1 && cpk.slice(lastPipe + 1) === lens) { + cpks.push(cpk); + } + } + return cpks.length ? cpks : undefined; + } +} diff --git a/packages/endpoint/src/schemas/Union.ts b/packages/endpoint/src/schemas/Union.ts index ef91dcaa8b34..810c3395f029 100644 --- a/packages/endpoint/src/schemas/Union.ts +++ b/packages/endpoint/src/schemas/Union.ts @@ -1,5 +1,5 @@ import PolymorphicSchema from './Polymorphic.js'; -import { Visit } from '../interface.js'; +import { IDenormalizeDelegate, Visit } from '../interface.js'; /** * Represents polymorphic values. @@ -22,12 +22,8 @@ export default class UnionSchema extends PolymorphicSchema { return this.normalizeValue(input, parent, key, args, visit); } - denormalize( - input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, - ) { - return this.denormalizeValue(input, unvisit); + denormalize(input: {}, delegate: IDenormalizeDelegate) { + return this.denormalizeValue(input, delegate.unvisit); } queryKey(args: any, unvisit: (schema: any, args: any) => any) { diff --git a/packages/endpoint/src/schemas/Values.ts b/packages/endpoint/src/schemas/Values.ts index dc36237d00f7..60fb84ce8177 100644 --- a/packages/endpoint/src/schemas/Values.ts +++ b/packages/endpoint/src/schemas/Values.ts @@ -1,5 +1,5 @@ import PolymorphicSchema from './Polymorphic.js'; -import { Visit } from '../interface.js'; +import { IDenormalizeDelegate, Visit } from '../interface.js'; /** * Represents variably sized objects @@ -26,17 +26,13 @@ export default class ValuesSchema extends PolymorphicSchema { return output; } - denormalize( - input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, - ): any { + denormalize(input: {}, delegate: IDenormalizeDelegate): any { const output: Record = {}; const keys = Object.keys(input); for (let i = 0; i < keys.length; i++) { const k = keys[i]; const entityOrId = (input as any)[k]; - const value = this.denormalizeValue(entityOrId, unvisit); + const value = this.denormalizeValue(entityOrId, delegate.unvisit); // remove empty or deleted values if (value && typeof value !== 'symbol') { diff --git a/packages/endpoint/src/schemas/__tests__/Collection.test.ts b/packages/endpoint/src/schemas/__tests__/Collection.test.ts index 937a34d66598..1464eff028e4 100644 --- a/packages/endpoint/src/schemas/__tests__/Collection.test.ts +++ b/packages/endpoint/src/schemas/__tests__/Collection.test.ts @@ -67,12 +67,11 @@ test('key works with custom schema', () => { denormalize( input: any, - args: any[], - unvisit: (schema: any, input: any) => any, + delegate: { unvisit: (schema: any, input: any) => any }, ) { return input.map ? input.map((entityOrId: any) => - this.denormalizeValue(entityOrId, unvisit), + this.denormalizeValue(entityOrId, delegate.unvisit), ) : input; } diff --git a/packages/endpoint/src/schemas/__tests__/Scalar.test.ts b/packages/endpoint/src/schemas/__tests__/Scalar.test.ts new file mode 100644 index 000000000000..81e99517b578 --- /dev/null +++ b/packages/endpoint/src/schemas/__tests__/Scalar.test.ts @@ -0,0 +1,1357 @@ +// eslint-env jest +import { normalize } from '@data-client/normalizr'; +import { IDEntity } from '__tests__/new'; + +import { SimpleMemoCache } from './denormalize'; +import { Collection, schema, Scalar } from '../..'; + +/** Minimal IDenormalizeDelegate for direct schema.denormalize() unit tests. */ +function makeDelegate( + args: readonly any[], + unvisit: (schema: any, input: any) => any, +) { + return { + args, + unvisit, + argsKey(fn: (args: readonly any[]) => string | undefined) { + return fn(args); + }, + }; +} + +let dateSpy: jest.Spied; +beforeAll(() => { + dateSpy = jest + .spyOn(global.Date, 'now') + .mockImplementation(() => new Date('2019-05-14T11:01:58.135Z').valueOf()); +}); +afterAll(() => { + dateSpy.mockRestore(); +}); + +class Company extends IDEntity { + price = 0; + pct_equity = 0; + shares = 0; + + static key = 'Company'; +} +// Bound to Company so the Values-only column-fetch tests below also work. +// (When a Scalar is only ever used as an Entity field, `entity` can be omitted.) +const PortfolioScalar = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'portfolio', + entity: Company, +}); +Company.schema = { + pct_equity: PortfolioScalar, + shares: PortfolioScalar, +} as any; + +describe('Scalar', () => { + describe('normalize', () => { + it('stores scalar fields in Scalar and wrapper refs in entity', () => { + const result = normalize( + [Company], + [{ id: '1', price: 100, pct_equity: 0.5, shares: 32342 }], + [{ portfolio: 'portfolioA' }], + ); + + // Entity stores wrapper refs, not raw scalar values + const companyEntity = result.entities['Company']?.['1']; + expect(companyEntity).toBeDefined(); + expect(companyEntity.id).toBe('1'); + expect(companyEntity.price).toBe(100); + expect(companyEntity.pct_equity).toEqual(['1', 'pct_equity', 'Company']); + expect(companyEntity.shares).toEqual(['1', 'shares', 'Company']); + + // Scalar stores the grouped data + const cellKey = 'Scalar(portfolio)'; + const cpk = 'Company|1|portfolioA'; + const cell = result.entities[cellKey]?.[cpk]; + expect(cell).toBeDefined(); + expect(cell.pct_equity).toBe(0.5); + expect(cell.shares).toBe(32342); + }); + + it.each([ + ['zero', 0], + ['empty string', ''], + ['false', false], + ])('normalizes falsy scalar value (%s) into the cell', (_label, value) => { + const result = normalize( + [Company], + [{ id: '1', price: 100, pct_equity: value, shares: 32342 }], + [{ portfolio: 'portfolioA' }], + ); + + // Entity must store a wrapper, not the raw falsy value. + const companyEntity = result.entities['Company']?.['1']; + expect(companyEntity.pct_equity).toEqual(['1', 'pct_equity', 'Company']); + + // Scalar cell must record the falsy value verbatim. + const cell = + result.entities['Scalar(portfolio)']?.['Company|1|portfolioA']; + expect(cell).toBeDefined(); + expect(cell.pct_equity).toBe(value); + expect(cell.shares).toBe(32342); + }); + + it('round-trips falsy scalar values through denormalize', () => { + const state = normalize( + [Company], + [{ id: '1', price: 0, pct_equity: 0, shares: 0 }], + [{ portfolio: 'portfolioA' }], + ); + + const memo = new SimpleMemoCache(); + const result = memo.denormalize([Company], state.result, state.entities, [ + { portfolio: 'portfolioA' }, + ]) as any[]; + + expect(result[0].pct_equity).toBe(0); + expect(result[0].shares).toBe(0); + expect(result[0].price).toBe(0); + }); + + it('handles multiple entities in array', () => { + const result = normalize( + [Company], + [ + { id: '1', price: 100, pct_equity: 0.5, shares: 32342 }, + { id: '2', price: 200, pct_equity: 0.3, shares: 1000 }, + ], + [{ portfolio: 'portfolioA' }], + ); + + const cellKey = 'Scalar(portfolio)'; + expect(result.entities[cellKey]?.['Company|1|portfolioA']).toEqual({ + pct_equity: 0.5, + shares: 32342, + }); + expect(result.entities[cellKey]?.['Company|2|portfolioA']).toEqual({ + pct_equity: 0.3, + shares: 1000, + }); + }); + + it('normalizes different lenses into separate cells', () => { + const stateA = normalize( + [Company], + [{ id: '1', price: 100, pct_equity: 0.5, shares: 32342 }], + [{ portfolio: 'portfolioA' }], + ); + const stateB = normalize( + [Company], + [{ id: '1', price: 100, pct_equity: 0.3, shares: 323 }], + [{ portfolio: 'portfolioB' }], + stateA, + ); + + const cellKey = 'Scalar(portfolio)'; + expect(stateB.entities[cellKey]?.['Company|1|portfolioA']).toEqual({ + pct_equity: 0.5, + shares: 32342, + }); + expect(stateB.entities[cellKey]?.['Company|1|portfolioB']).toEqual({ + pct_equity: 0.3, + shares: 323, + }); + }); + }); + + describe('denormalize', () => { + it('joins scalar cell data based on lens args', () => { + const state = normalize( + [Company], + [{ id: '1', price: 100, pct_equity: 0.5, shares: 32342 }], + [{ portfolio: 'portfolioA' }], + ); + + const memo = new SimpleMemoCache(); + const result = memo.denormalize([Company], state.result, state.entities, [ + { portfolio: 'portfolioA' }, + ]) as any[]; + + expect(result).toBeDefined(); + expect(result.length).toBe(1); + expect(result[0].id).toBe('1'); + expect(result[0].price).toBe(100); + expect(result[0].pct_equity).toBe(0.5); + expect(result[0].shares).toBe(32342); + }); + + it('different lens args produce different results from same entity', () => { + const stateA = normalize( + [Company], + [{ id: '1', price: 100, pct_equity: 0.5, shares: 32342 }], + [{ portfolio: 'portfolioA' }], + ); + const stateB = normalize( + [Company], + [{ id: '1', price: 100, pct_equity: 0.3, shares: 323 }], + [{ portfolio: 'portfolioB' }], + stateA, + ); + + // A single shared MemoCache must produce different results when only + // the lens args change — the entity-level cache must bucket by lens. + const memo = new SimpleMemoCache(); + + const resultA = memo.denormalize( + [Company], + stateB.result, + stateB.entities, + [{ portfolio: 'portfolioA' }], + ) as any[]; + expect(resultA[0].pct_equity).toBe(0.5); + expect(resultA[0].shares).toBe(32342); + + const resultB = memo.denormalize( + [Company], + stateB.result, + stateB.entities, + [{ portfolio: 'portfolioB' }], + ) as any[]; + expect(resultB[0].pct_equity).toBe(0.3); + expect(resultB[0].shares).toBe(323); + + // Non-scalar fields are the same + expect(resultA[0].id).toBe('1'); + expect(resultB[0].id).toBe('1'); + expect(resultA[0].price).toBe(100); + expect(resultB[0].price).toBe(100); + }); + + it('returns undefined for scalar fields when lens is missing from args', () => { + const state = normalize( + [Company], + [{ id: '1', price: 100, pct_equity: 0.5, shares: 32342 }], + [{ portfolio: 'portfolioA' }], + ); + + const memo = new SimpleMemoCache(); + const result = memo.denormalize([Company], state.result, state.entities, [ + {}, + ]) as any[]; + + expect(result[0].id).toBe('1'); + expect(result[0].price).toBe(100); + expect(result[0].pct_equity).toBeUndefined(); + expect(result[0].shares).toBeUndefined(); + }); + }); + + describe('column-only fetch via Values', () => { + it('stores cells without modifying Company entities', () => { + // First: full entity fetch to establish Company entities and _entityKey + const initial = normalize( + [Company], + [ + { id: '1', price: 100, pct_equity: 0.5, shares: 32342 }, + { id: '2', price: 200, pct_equity: 0.3, shares: 1000 }, + ], + [{ portfolio: 'portfolioA' }], + ); + + // Column-only fetch for portfolioB + const columnResult = normalize( + new schema.Values(PortfolioScalar), + { + '1': { pct_equity: 0.7, shares: 555 }, + '2': { pct_equity: 0.1, shares: 999 }, + }, + [{ portfolio: 'portfolioB' }], + initial, + ); + + // Company entities should not have changed + expect(columnResult.entities['Company']?.['1'].price).toBe(100); + expect(columnResult.entities['Company']?.['2'].price).toBe(200); + + // New Scalar entries for portfolioB + const cellKey = 'Scalar(portfolio)'; + expect(columnResult.entities[cellKey]?.['Company|1|portfolioB']).toEqual({ + pct_equity: 0.7, + shares: 555, + }); + expect(columnResult.entities[cellKey]?.['Company|2|portfolioB']).toEqual({ + pct_equity: 0.1, + shares: 999, + }); + + // Original portfolioA cells still intact + expect(columnResult.entities[cellKey]?.['Company|1|portfolioA']).toEqual({ + pct_equity: 0.5, + shares: 32342, + }); + }); + + it('column fetch cells are joinable via denormalize', () => { + const initial = normalize( + [Company], + [{ id: '1', price: 100, pct_equity: 0.5, shares: 32342 }], + [{ portfolio: 'portfolioA' }], + ); + + const withB = normalize( + new schema.Values(PortfolioScalar), + { '1': { pct_equity: 0.7, shares: 555 } }, + [{ portfolio: 'portfolioB' }], + initial, + ); + + const memo = new SimpleMemoCache(); + + // Denormalize with portfolioB lens → sees column fetch data + const result = memo.denormalize( + [Company], + initial.result, + withB.entities, + [{ portfolio: 'portfolioB' }], + ) as any[]; + + expect(result[0].pct_equity).toBe(0.7); + expect(result[0].shares).toBe(555); + expect(result[0].price).toBe(100); + }); + }); + + describe('array-of-records standalone', () => { + it('normalizes [Scalar] using each item id as the entity pk', () => { + const state = normalize( + [PortfolioScalar], + [ + { id: '1', pct_equity: 0.7, shares: 555 }, + { id: '2', pct_equity: 0.1, shares: 999 }, + ], + [{ portfolio: 'portfolioB' }], + ); + + expect(state.result).toEqual([ + 'Company|1|portfolioB', + 'Company|2|portfolioB', + ]); + expect(state.entities['Scalar(portfolio)']).toMatchObject({ + 'Company|1|portfolioB': { + id: '1', + pct_equity: 0.7, + shares: 555, + }, + 'Company|2|portfolioB': { + id: '2', + pct_equity: 0.1, + shares: 999, + }, + }); + }); + + it('normalizes Collection([Scalar]) and denormalizes the array round-trip', () => { + const columns = new Collection([PortfolioScalar], { + argsKey: ({ portfolio }: { portfolio: string }) => ({ portfolio }), + }); + const state = normalize( + columns, + [ + { id: '1', pct_equity: 0.7, shares: 555 }, + { id: '2', pct_equity: 0.1, shares: 999 }, + ], + [{ portfolio: 'portfolioB' }], + ); + + expect(state.result).toEqual('{"portfolio":"portfolioB"}'); + expect(state.entities['[Scalar(portfolio)]']).toEqual({ + '{"portfolio":"portfolioB"}': [ + 'Company|1|portfolioB', + 'Company|2|portfolioB', + ], + }); + + const memo = new SimpleMemoCache(); + const result = memo.denormalize(columns, state.result, state.entities, [ + { portfolio: 'portfolioB' }, + ]) as any[]; + expect(result).toEqual([ + { id: '1', pct_equity: 0.7, shares: 555 }, + { id: '2', pct_equity: 0.1, shares: 999 }, + ]); + }); + + it('joins array-fetched cells onto existing entities', () => { + const initial = normalize( + [Company], + [{ id: '1', price: 100, pct_equity: 0.5, shares: 32342 }], + [{ portfolio: 'portfolioA' }], + ); + const withB = normalize( + [PortfolioScalar], + [{ id: '1', pct_equity: 0.7, shares: 555 }], + [{ portfolio: 'portfolioB' }], + initial, + ); + + const memo = new SimpleMemoCache(); + const result = memo.denormalize( + [Company], + initial.result, + withB.entities, + [{ portfolio: 'portfolioB' }], + ) as any[]; + + expect(result[0].pct_equity).toBe(0.7); + expect(result[0].shares).toBe(555); + expect(result[0].price).toBe(100); + }); + + it('supports subclass entityPk overrides', () => { + class CompanyIdScalar extends Scalar { + entityPk(input: any): string | undefined { + return input?.companyId; + } + } + const CustomScalar = new CompanyIdScalar({ + lens: args => args[0]?.portfolio, + key: 'custom-portfolio', + entity: Company, + }); + + const state = normalize( + [CustomScalar], + [{ companyId: '1', pct_equity: 0.7, shares: 555 }], + [{ portfolio: 'portfolioB' }], + ); + + expect( + state.entities['Scalar(custom-portfolio)']?.['Company|1|portfolioB'], + ).toEqual({ companyId: '1', pct_equity: 0.7, shares: 555 }); + }); + + it('throws when the standalone path cannot derive an entity pk', () => { + expect(() => + normalize( + [PortfolioScalar], + [{ pct_equity: 0.7, shares: 555 }], + [{ portfolio: 'portfolioB' }], + ), + ).toThrow(/cannot derive entity pk/); + }); + }); + + describe('nested array-of-records under a plain object schema (regression)', () => { + // `[Scalar]` (or `Collection([Scalar])`) nested inside a plain object schema + // — `{ stock: [Scalar] }` — must derive each item's entity pk from the item + // itself, not from the parent's field name. Previously `Array.normalize` + // forwarded its enclosing field name (e.g. `'stock'`) as `key` to each + // child; `Scalar.entityPk` returned that key (since it was non-undefined), + // collapsing every cell onto compound pk `Company|stock|` and + // silently corrupting the data. + it('keys cells by item id, not by parent field name', () => { + const objSchema = { stock: [PortfolioScalar] }; + const state = normalize( + objSchema, + { + stock: [ + { id: '1', pct_equity: 0.7, shares: 555 }, + { id: '2', pct_equity: 0.1, shares: 999 }, + ], + }, + [{ portfolio: 'portfolioB' }], + ); + + expect(state.result).toEqual({ + stock: ['Company|1|portfolioB', 'Company|2|portfolioB'], + }); + expect(state.entities['Scalar(portfolio)']).toMatchObject({ + 'Company|1|portfolioB': { + id: '1', + pct_equity: 0.7, + shares: 555, + }, + 'Company|2|portfolioB': { + id: '2', + pct_equity: 0.1, + shares: 999, + }, + }); + // Must not key any cell by the parent field name. + expect( + Object.keys(state.entities['Scalar(portfolio)'] ?? {}).some(k => + k.includes('|stock|'), + ), + ).toBe(false); + }); + + it('keys cells by item id when nested as Collection([Scalar])', () => { + const columns = new Collection([PortfolioScalar], { + nestKey: (parent: any, key: string) => ({ portfolio: key }), + }); + const objSchema = { stock: columns }; + const state = normalize( + objSchema, + { + stock: [ + { id: '1', pct_equity: 0.7, shares: 555 }, + { id: '2', pct_equity: 0.1, shares: 999 }, + ], + }, + [{ portfolio: 'portfolioB' }], + ); + + // Cells are keyed by item id, not by the parent field name. + expect(state.entities['Scalar(portfolio)']).toMatchObject({ + 'Company|1|portfolioB': { + id: '1', + pct_equity: 0.7, + shares: 555, + }, + 'Company|2|portfolioB': { + id: '2', + pct_equity: 0.1, + shares: 999, + }, + }); + }); + }); + + describe('composite primary keys containing "|"', () => { + class CompositeCompany extends IDEntity { + type = ''; + num = ''; + price = 0; + pct_equity = 0; + shares = 0; + + pk(): string { + return `${this.type}|${this.num}`; + } + + static key = 'CompositeCompany'; + } + const CompositeScalar = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'portfolio-composite', + entity: CompositeCompany, + }); + CompositeCompany.schema = { + pct_equity: CompositeScalar, + shares: CompositeScalar, + } as any; + + it('entity-path cpk is consistent with Values-path cpk', () => { + const initial = normalize( + [CompositeCompany], + [ + { + id: '1', + type: 'type', + num: '123', + price: 100, + pct_equity: 0.5, + shares: 32342, + }, + ], + [{ portfolio: 'portfolioA' }], + ); + + const cellKey = 'Scalar(portfolio-composite)'; + const expectedCpk = 'CompositeCompany|type|123|portfolioA'; + expect(initial.entities[cellKey]?.[expectedCpk]).toEqual({ + pct_equity: 0.5, + shares: 32342, + }); + + const wrapper = + initial.entities['CompositeCompany']?.['type|123']?.pct_equity; + expect(wrapper).toEqual(['type|123', 'pct_equity', 'CompositeCompany']); + + const withB = normalize( + new schema.Values(CompositeScalar), + { 'type|123': { pct_equity: 0.7, shares: 555 } }, + [{ portfolio: 'portfolioB' }], + initial, + ); + + expect( + withB.entities[cellKey]?.['CompositeCompany|type|123|portfolioB'], + ).toEqual({ + pct_equity: 0.7, + shares: 555, + }); + + const memo = new SimpleMemoCache(); + const result = memo.denormalize( + [CompositeCompany], + initial.result, + withB.entities, + [{ portfolio: 'portfolioB' }], + ) as any[]; + + expect(result[0].pct_equity).toBe(0.7); + expect(result[0].shares).toBe(555); + expect(result[0].price).toBe(100); + }); + }); + + describe('merge accumulation', () => { + it('merges new scalar fields into existing cells', () => { + const state1 = normalize( + [Company], + [{ id: '1', price: 100, pct_equity: 0.5, shares: 32342 }], + [{ portfolio: 'portfolioA' }], + ); + + // Re-normalize same entity with updated scalar values + const state2 = normalize( + [Company], + [{ id: '1', price: 105, pct_equity: 0.55, shares: 33000 }], + [{ portfolio: 'portfolioA' }], + state1, + ); + + const cellKey = 'Scalar(portfolio)'; + const cell = state2.entities[cellKey]?.['Company|1|portfolioA']; + expect(cell.pct_equity).toBe(0.55); + expect(cell.shares).toBe(33000); + + // Company entity updated too (price changed) + expect(state2.entities['Company']?.['1'].price).toBe(105); + }); + }); + + describe('Scalar constructor', () => { + it('sets key, lensSelector, and entityKey when bound', () => { + const s = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'test', + entity: Company, + }); + expect(s.key).toBe('Scalar(test)'); + expect(s.lensSelector([{ portfolio: 'abc' }])).toBe('abc'); + expect(s.entityKey).toBe('Company'); + }); + + it('leaves entityKey undefined when `entity` is omitted', () => { + const s = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'test', + }); + expect(s.entityKey).toBeUndefined(); + }); + + it('throws when used inside Values without an `entity` binding', () => { + const Unbound = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'unbound', + }); + expect(() => + normalize(new schema.Values(Unbound), { '1': { x: 1 } }, [ + { portfolio: 'p' }, + ]), + ).toThrow(/entity/); + }); + }); + + describe('queryKey', () => { + // Minimal IQueryDelegate that serves cells from an in-memory map keyed by + // schema key -> cpk -> cell. Scalar.queryKey only reads getEntities. + function makeQueryDelegate( + cells: Record> = {}, + ) { + return { + getEntities(key: string) { + const table = cells[key]; + if (!table) return undefined; + return { + keys: () => Object.keys(table)[Symbol.iterator](), + entries: () => Object.entries(table)[Symbol.iterator](), + }; + }, + getEntity: (key: string, pk: string) => cells[key]?.[pk], + getIndex: () => undefined, + INVALID: Symbol('INVALID'), + } as any; + } + + const LensScalar = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'portfolio-q', + entity: Company, + }); + + it('returns undefined when lens is undefined and does not touch delegate', () => { + const delegate = { + getEntities: jest.fn(() => undefined), + } as any; + expect(LensScalar.queryKey([], undefined, delegate)).toBeUndefined(); + expect(LensScalar.queryKey([{}], undefined, delegate)).toBeUndefined(); + expect(delegate.getEntities).not.toHaveBeenCalled(); + }); + + it('returns undefined when the Scalar table is absent', () => { + const delegate = makeQueryDelegate({}); + expect( + LensScalar.queryKey([{ portfolio: 'A' }], undefined, delegate), + ).toBeUndefined(); + }); + + it('returns undefined when the table is empty', () => { + const delegate = makeQueryDelegate({ 'Scalar(portfolio-q)': {} }); + expect( + LensScalar.queryKey([{ portfolio: 'A' }], undefined, delegate), + ).toBeUndefined(); + }); + + it('returns undefined when no cell matches the current lens', () => { + const delegate = makeQueryDelegate({ + 'Scalar(portfolio-q)': { + 'Company|1|B': { pct_equity: 0.1 }, + 'Company|2|C': { pct_equity: 0.2 }, + }, + }); + expect( + LensScalar.queryKey([{ portfolio: 'A' }], undefined, delegate), + ).toBeUndefined(); + }); + + it('returns the cpk array for cells matching the current lens', () => { + const delegate = makeQueryDelegate({ + 'Scalar(portfolio-q)': { + 'Company|1|A': { pct_equity: 0.5 }, + 'Company|2|A': { pct_equity: 0.3 }, + 'Company|1|B': { pct_equity: 0.9 }, + }, + }); + const result = LensScalar.queryKey( + [{ portfolio: 'A' }], + undefined, + delegate, + ); + expect(result).toEqual(['Company|1|A', 'Company|2|A']); + }); + + it('skips invalid (symbol / falsy) entries when enumerating', () => { + const INVALID = Symbol('INVALID'); + const delegate = makeQueryDelegate({ + 'Scalar(portfolio-q)': { + 'Company|1|A': { pct_equity: 0.5 }, + 'Company|2|A': INVALID, + 'Company|3|A': undefined as any, + 'Company|4|A': { pct_equity: 0.4 }, + }, + }); + const result = LensScalar.queryKey( + [{ portfolio: 'A' }], + undefined, + delegate, + ); + expect(result).toEqual(['Company|1|A', 'Company|4|A']); + }); + + it('returns cells across multiple entity types when a Scalar is shared', () => { + const delegate = makeQueryDelegate({ + 'Scalar(portfolio-q)': { + 'Company|1|A': { pct_equity: 0.5 }, + 'Fund|1|A': { pct_equity: 0.9 }, + 'Fund|2|B': { pct_equity: 0.1 }, + }, + }); + const result = LensScalar.queryKey( + [{ portfolio: 'A' }], + undefined, + delegate, + ); + expect(result).toEqual(['Company|1|A', 'Fund|1|A']); + }); + + it('handles composite entity pks containing `|`', () => { + const delegate = makeQueryDelegate({ + 'Scalar(portfolio-q)': { + 'Company|a|b|A': { pct_equity: 0.5 }, + 'Company|c|d|B': { pct_equity: 0.3 }, + }, + }); + const result = LensScalar.queryKey( + [{ portfolio: 'A' }], + undefined, + delegate, + ); + expect(result).toEqual(['Company|a|b|A']); + }); + + it('over-matches when another lens ends with the current lens after `|` (documented limitation)', () => { + // `lens` must not contain `|`. If a caller violates this, a lookup + // for lens 'A' matches cells whose lens is 'X|A' because + // lastIndexOf('|') picks the delimiter inside the offending lens + // value rather than the cpk delimiter. Captured here so the behavior + // is explicit; see the `lens` option doc. + const delegate = makeQueryDelegate({ + 'Scalar(portfolio-q)': { + 'Company|1|A': { pct_equity: 0.5 }, + 'Company|2|X|A': { pct_equity: 0.3 }, + }, + }); + const result = LensScalar.queryKey( + [{ portfolio: 'A' }], + undefined, + delegate, + ); + expect(result).toEqual(['Company|1|A', 'Company|2|X|A']); + }); + + it('supports subclass overrides of queryKey', () => { + class MyScalar extends Scalar { + queryKey(_args: readonly any[], _unvisit: any, _delegate: any) { + return ['custom|1|A']; + } + } + const s = new MyScalar({ + lens: args => args[0]?.portfolio, + key: 'portfolio-sub', + entity: Company, + }); + const delegate = makeQueryDelegate({}); + expect(s.queryKey([{ portfolio: 'A' }], undefined, delegate)).toEqual([ + 'custom|1|A', + ]); + }); + }); + + describe('multi-entity sharing (regression)', () => { + // A single Scalar instance shared across multiple entity types: the entity + // key is inferred at normalize time from the parent Entity class and stored + // on the wrapper, so denormalize resolves to the correct cell regardless of + // normalization order. + class Fund extends IDEntity { + price = 0; + pct_equity = 0; + + static key = 'Fund'; + } + const SharedPortfolio = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'portfolio-shared', + }); + class Co extends IDEntity { + price = 0; + pct_equity = 0; + static key = 'Co'; + static schema = { pct_equity: SharedPortfolio }; + } + Fund.schema = { pct_equity: SharedPortfolio } as any; + + it('normalizes Co and Fund into distinct cells without cross-contamination', () => { + const state = normalize( + [Co], + [{ id: '1', price: 100, pct_equity: 0.5 }], + [{ portfolio: 'portfolioA' }], + ); + const state2 = normalize( + [Fund], + [{ id: '1', price: 200, pct_equity: 0.9 }], + [{ portfolio: 'portfolioA' }], + state, + ); + const cellKey = 'Scalar(portfolio-shared)'; + expect(state2.entities[cellKey]?.['Co|1|portfolioA']).toEqual({ + pct_equity: 0.5, + }); + expect(state2.entities[cellKey]?.['Fund|1|portfolioA']).toEqual({ + pct_equity: 0.9, + }); + }); + + it('denormalizes Co correctly even after Fund was normalized last', () => { + const state = normalize( + [Co], + [{ id: '1', price: 100, pct_equity: 0.5 }], + [{ portfolio: 'portfolioA' }], + ); + const state2 = normalize( + [Fund], + [{ id: '1', price: 200, pct_equity: 0.9 }], + [{ portfolio: 'portfolioA' }], + state, + ); + + const memo = new SimpleMemoCache(); + const coResult = memo.denormalize([Co], state.result, state2.entities, [ + { portfolio: 'portfolioA' }, + ]) as any[]; + expect(coResult[0].pct_equity).toBe(0.5); + + const memo2 = new SimpleMemoCache(); + const fundResult = memo2.denormalize( + [Fund], + state2.result, + state2.entities, + [{ portfolio: 'portfolioA' }], + ) as any[]; + expect(fundResult[0].pct_equity).toBe(0.9); + }); + }); + + describe('field name collision with wrapper shape (regression)', () => { + // Cell data is passed back through `Scalar.denormalize` by `unvisitEntity` + // (which calls `schema.denormalize(entityCopy, args, unvisit)` after + // `createIfValid`). The wrapper is an array `[id, field, entityKey]` and + // cell data is always a plain object, so `Array.isArray` distinguishes + // them — even when the user names a Scalar-mapped field `field`/`id`. + class Reading extends IDEntity { + label = ''; + field = 0; + id_ = ''; + + static key = 'Reading'; + } + const ReadingScalar = new Scalar({ + lens: args => args[0]?.sensor, + key: 'sensor', + entity: Reading, + }); + Reading.schema = { + field: ReadingScalar, + id_: ReadingScalar, + } as any; + + it('round-trips when cell data has a property named `field`', () => { + const state = normalize( + [Reading], + [{ id: '1', label: 'Site A', field: 42, id_: 'collision' }], + [{ sensor: 'sensorA' }], + ); + + // Cell stores the user-defined `field` and `id_` columns verbatim. + const cell = state.entities['Scalar(sensor)']?.['Reading|1|sensorA']; + expect(cell).toEqual({ field: 42, id_: 'collision' }); + + const memo = new SimpleMemoCache(); + const result = memo.denormalize([Reading], state.result, state.entities, [ + { sensor: 'sensorA' }, + ]) as any[]; + + expect(result[0].id).toBe('1'); + expect(result[0].label).toBe('Site A'); + // Cell data `{ field: 42, ... }` is a plain object; the wrapper is an + // array. `Array.isArray` keeps them distinct so `field` resolves to 42 + // rather than being misidentified as a wrapper and returning `undefined`. + expect(result[0].field).toBe(42); + expect(result[0].id_).toBe('collision'); + }); + }); + + describe('lens returning undefined during normalize (regression)', () => { + // A missing lens at normalize time is a configuration bug: the cpk would + // collapse to `…|undefined` (literal "undefined" colliding across rows) + // and denormalize would never find the data again. We throw rather than + // silently corrupt the Scalar table or drop the value. + it('throws on the entity-field path when the lens is missing', () => { + expect(() => + normalize( + [Company], + [{ id: '1', price: 100, pct_equity: 0.5, shares: 32342 }], + // No `portfolio` key on args → lensSelector returns undefined. + [{}], + ), + ).toThrow(/lens returned undefined/); + }); + + it('throws on the Values path when the lens is missing', () => { + expect(() => + normalize( + new schema.Values(PortfolioScalar), + { '1': { pct_equity: 0.7, shares: 555 } }, + [{}], + ), + ).toThrow(/lens returned undefined/); + }); + }); + + describe('denormalize passthroughs and edge cases', () => { + const s = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'edge', + entity: Company, + }); + + it.each([ + ['null', null], + ['undefined', undefined], + ['zero', 0], + ['empty string', ''], + ['false', false], + ])( + 'returns the falsy input (%s) without further lookup', + (_label, value) => { + const unvisit = jest.fn(); + expect( + s.denormalize(value, makeDelegate([{ portfolio: 'p' }], unvisit)), + ).toBe(value); + expect(unvisit).not.toHaveBeenCalled(); + }, + ); + + // Regression: truthy non-string primitives previously fell through to + // `delegate.unvisit`, which re-dispatched back to `Scalar.denormalize` + // (no `pk`, string fast path misses) and stack-overflowed. This can + // occur during schema migration when a Scalar is added to an entity + // with cached raw numeric/boolean field values still in the store. + it.each([ + ['positive number', 0.5], + ['integer', 42], + ['negative number', -1], + ['true', true], + ['NaN', NaN], + ['Infinity', Infinity], + ['bigint', 10n], + ])( + 'returns the truthy non-string primitive input (%s) without recursion', + (_label, value) => { + const unvisit = jest.fn(); + expect( + s.denormalize(value, makeDelegate([{ portfolio: 'p' }], unvisit)), + ).toBe(value); + expect(unvisit).not.toHaveBeenCalled(); + }, + ); + + it('returns symbol inputs unchanged (suspense/INVALID propagation)', () => { + const sym = Symbol('INVALID'); + const unvisit = jest.fn(); + expect( + s.denormalize(sym, makeDelegate([{ portfolio: 'p' }], unvisit)), + ).toBe(sym); + expect(unvisit).not.toHaveBeenCalled(); + }); + + it('returns plain-object inputs unchanged (cell data passthrough)', () => { + // Cell data flows back through `Scalar.denormalize` from + // `unvisitEntityObject` after a cpk lookup; it must pass through so the + // wrapper-resolving branch above can index the named field. + const cell = { pct_equity: 0.5, shares: 10 }; + const unvisit = jest.fn(); + expect( + s.denormalize(cell, makeDelegate([{ portfolio: 'p' }], unvisit)), + ).toBe(cell); + expect(unvisit).not.toHaveBeenCalled(); + }); + + it('returns undefined for entity-field wrapper when lens is missing at denormalize', () => { + const wrapper: [string, string, string] = ['1', 'pct_equity', 'Company']; + const unvisit = jest.fn(); + expect( + s.denormalize(wrapper, makeDelegate([{}], unvisit)), + ).toBeUndefined(); + // No lookup attempted when we cannot derive the cell key. + expect(unvisit).not.toHaveBeenCalled(); + }); + + it('returns undefined when the cell lookup yields undefined', () => { + const wrapper: [string, string, string] = [ + 'missing', + 'pct_equity', + 'Company', + ]; + const unvisit = jest.fn(() => undefined); + expect( + s.denormalize(wrapper, makeDelegate([{ portfolio: 'p' }], unvisit)), + ).toBeUndefined(); + expect(unvisit).toHaveBeenCalledWith(s, 'Company|missing|p'); + }); + + it('returns undefined when the cell lookup yields a symbol (INVALID)', () => { + const INVALID = Symbol('INVALID'); + const wrapper: [string, string, string] = ['1', 'pct_equity', 'Company']; + const unvisit = jest.fn(() => INVALID); + expect( + s.denormalize(wrapper, makeDelegate([{ portfolio: 'p' }], unvisit)), + ).toBeUndefined(); + }); + + it('does not stack-overflow when an entity holds stale raw primitive values (schema migration)', () => { + // Reproduce the real-world scenario: an entity was previously stored + // with raw numeric scalar fields, and now its schema declares those + // fields as a Scalar. EntityMixin.denormalize calls + // `delegate.unvisit(Scalar, 0.5)` per field; without the primitive + // guard, `unvisit` dispatches back into `Scalar.denormalize(0.5)` + // (string fast path misses, `isEntity(this)` is false) and recurses + // until the stack overflows. + const staleState = { + Company: { + '1': { id: '1', price: 100, pct_equity: 0.5, shares: 32342 }, + }, + }; + const memo = new SimpleMemoCache(); + const result = memo.denormalize([Company], ['1'], staleState, [ + { portfolio: 'portfolioA' }, + ]) as any[]; + + // Stale raw values pass through unchanged — no crash, no recursion. + expect(result[0].id).toBe('1'); + expect(result[0].price).toBe(100); + expect(result[0].pct_equity).toBe(0.5); + expect(result[0].shares).toBe(32342); + }); + + it('looks up by cpk string and returns the cell when called via Values path', () => { + // Recursive entry from `unvisitEntity` for string inputs (e.g. when a + // Scalar appears as the value type in `schema.Values`). The Scalar should + // delegate to `unvisit` so the cache layer can resolve and memoize. + const cell = { pct_equity: 0.7 }; + const unvisit = jest.fn(() => cell); + const out = s.denormalize( + 'Company|1|portfolioB', + makeDelegate([{ portfolio: 'portfolioB' }], unvisit), + ); + expect(unvisit).toHaveBeenCalledWith(s, 'Company|1|portfolioB'); + expect(out).toBe(cell); + }); + }); + + describe('Values denormalize round-trip', () => { + it('returns cell objects keyed by entity pk', () => { + const state = normalize( + new schema.Values(PortfolioScalar), + { + '1': { pct_equity: 0.7, shares: 555 }, + '2': { pct_equity: 0.1, shares: 999 }, + }, + [{ portfolio: 'portfolioB' }], + ); + + const memo = new SimpleMemoCache(); + const result = memo.denormalize( + new schema.Values(PortfolioScalar), + state.result, + state.entities, + [{ portfolio: 'portfolioB' }], + ) as Record; + + expect(result['1']).toEqual({ pct_equity: 0.7, shares: 555 }); + expect(result['2']).toEqual({ pct_equity: 0.1, shares: 999 }); + }); + }); + + describe('createIfValid', () => { + it('returns a shallow copy of the input', () => { + const s = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'cif', + }); + const input = { pct_equity: 0.5, shares: 10 }; + const out = s.createIfValid(input); + expect(out).toEqual(input); + // Must not mutate or alias the original — denormalize relies on a fresh + // object so cache writes don't leak back into the source. + expect(out).not.toBe(input); + }); + }); + + describe('merge', () => { + it('combines existing and incoming, with incoming winning on overlap', () => { + const s = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'm', + }); + expect(s.merge({ a: 1, b: 2 }, { b: 3, c: 4 })).toEqual({ + a: 1, + b: 3, + c: 4, + }); + }); + + it('accumulates per-field writes when called repeatedly (EntityMixin loop)', () => { + const s = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'm2', + }); + // EntityMixin invokes Scalar once per field; the store applies `merge` + // between the existing cell and each one-key write. + const cell = s.merge({}, { pct_equity: 0.5 }); + const cell2 = s.merge(cell, { shares: 32342 }); + expect(cell2).toEqual({ pct_equity: 0.5, shares: 32342 }); + }); + }); + + describe('shouldReorder / mergeWithStore / mergeMetaWithStore', () => { + const s = new Scalar({ + lens: args => args[0]?.portfolio, + key: 'reorder', + }); + const olderMeta = { date: 1, fetchedAt: 1, expiresAt: 100 }; + const newerMeta = { date: 2, fetchedAt: 2, expiresAt: 200 }; + + it('reorders only when incoming is older than existing', () => { + // Stale (older) write should be reordered behind the newer existing. + expect( + s.shouldReorder(newerMeta, olderMeta, { x: 'new' }, { x: 'old' }), + ).toBe(true); + // Fresh (newer) write should NOT be reordered. + expect( + s.shouldReorder(olderMeta, newerMeta, { x: 'old' }, { x: 'new' }), + ).toBe(false); + // Equal timestamps: not strictly older, so do not reorder. + expect( + s.shouldReorder(olderMeta, olderMeta, { x: 'a' }, { x: 'b' }), + ).toBe(false); + }); + + it('mergeWithStore: newer incoming overrides existing fields', () => { + const result = s.mergeWithStore( + olderMeta, + newerMeta, + { a: 1, b: 2 }, + { b: 3, c: 4 }, + ); + expect(result).toEqual({ a: 1, b: 3, c: 4 }); + }); + + it('mergeWithStore: stale incoming yields to existing on overlap', () => { + // Reorder branch: merge(incoming, existing) so existing wins. + const result = s.mergeWithStore( + newerMeta, + olderMeta, + { a: 1, b: 2 }, + { b: 99, c: 4 }, + ); + expect(result).toEqual({ a: 1, b: 2, c: 4 }); + }); + + it('mergeMetaWithStore: keeps newer meta, drops stale incoming meta', () => { + // Newer incoming → use incomingMeta. + expect(s.mergeMetaWithStore(olderMeta, newerMeta, {}, {})).toBe( + newerMeta, + ); + // Stale incoming → keep existingMeta. + expect(s.mergeMetaWithStore(newerMeta, olderMeta, {}, {})).toBe( + newerMeta, + ); + }); + }); + + describe('Values-only fetch without prior entity fetch (regression)', () => { + it('builds correct compound pks when no entity has been normalized yet', () => { + const result = normalize( + new schema.Values(PortfolioScalar), + { + '1': { pct_equity: 0.7, shares: 555 }, + '2': { pct_equity: 0.1, shares: 999 }, + }, + [{ portfolio: 'portfolioB' }], + ); + + const cellKey = 'Scalar(portfolio)'; + expect(result.entities[cellKey]).toBeDefined(); + expect(result.entities[cellKey]?.['Company|1|portfolioB']).toEqual({ + pct_equity: 0.7, + shares: 555, + }); + expect(result.entities[cellKey]?.['Company|2|portfolioB']).toEqual({ + pct_equity: 0.1, + shares: 999, + }); + // Never write a cell with an `undefined` entity-key prefix. + expect( + Object.keys(result.entities[cellKey] ?? {}).some(k => + k.startsWith('undefined|'), + ), + ).toBe(false); + }); + }); + + describe('shared MemoCache across lens changes (regression)', () => { + // Reproduces the docs scenario at docs/rest/api/Scalar.md: after both + // portfolio A and B are loaded, switching back used to return stale + // values. The entity-level memo had no `args` dimension, so the chain + // recorded for the first call would replay against the original Scalar + // cell (e.g. CellA) regardless of the current lens — re-renders with the + // new portfolio kept returning the previously-computed entity. + it('A → B → A on the same state and entity yields the correct lens result each time', () => { + const stateA = normalize( + [Company], + [{ id: '1', price: 100, pct_equity: 0.5, shares: 32342 }], + [{ portfolio: 'portfolioA' }], + ); + const stateB = normalize( + [Company], + [{ id: '1', price: 100, pct_equity: 0.3, shares: 323 }], + [{ portfolio: 'portfolioB' }], + stateA, + ); + + // One MemoCache shared across all denormalize calls — mirrors how the + // controller reuses a single MemoCache across re-renders. + const memo = new SimpleMemoCache(); + + const first = memo.denormalize( + [Company], + stateB.result, + stateB.entities, + [{ portfolio: 'portfolioA' }], + ) as any[]; + expect(first[0].pct_equity).toBe(0.5); + expect(first[0].shares).toBe(32342); + + const second = memo.denormalize( + [Company], + stateB.result, + stateB.entities, + [{ portfolio: 'portfolioB' }], + ) as any[]; + expect(second[0].pct_equity).toBe(0.3); + expect(second[0].shares).toBe(323); + + // The third switch is what was "stuck" in the docs example: same + // entities/result, just the args change back. Must return A's values. + const third = memo.denormalize( + [Company], + stateB.result, + stateB.entities, + [{ portfolio: 'portfolioA' }], + ) as any[]; + expect(third[0].pct_equity).toBe(0.5); + expect(third[0].shares).toBe(32342); + }); + + it('alternating lens many times never returns the wrong cell', () => { + const state = normalize( + [Company], + [ + { id: '1', price: 100, pct_equity: 0.5, shares: 32342 }, + { id: '2', price: 200, pct_equity: 0.7, shares: 1000 }, + ], + [{ portfolio: 'portfolioA' }], + ); + const stateB = normalize( + [Company], + [ + { id: '1', price: 100, pct_equity: 0.3, shares: 323 }, + { id: '2', price: 200, pct_equity: 0.4, shares: 50 }, + ], + [{ portfolio: 'portfolioB' }], + state, + ); + + const memo = new SimpleMemoCache(); + const lenses = ['portfolioA', 'portfolioB', 'portfolioA', 'portfolioB']; + for (const portfolio of lenses) { + const result = memo.denormalize( + [Company], + stateB.result, + stateB.entities, + [{ portfolio }], + ) as any[]; + if (portfolio === 'portfolioA') { + expect(result[0].pct_equity).toBe(0.5); + expect(result[0].shares).toBe(32342); + expect(result[1].pct_equity).toBe(0.7); + expect(result[1].shares).toBe(1000); + } else { + expect(result[0].pct_equity).toBe(0.3); + expect(result[0].shares).toBe(323); + expect(result[1].pct_equity).toBe(0.4); + expect(result[1].shares).toBe(50); + } + } + }); + }); +}); diff --git a/packages/graphql/README.md b/packages/graphql/README.md index dbcb9b342d32..0c723bfd824b 100644 --- a/packages/graphql/README.md +++ b/packages/graphql/README.md @@ -167,6 +167,13 @@ return controller.fetch(createReview, data)} />; 🛑 +Scalar +✅ +Scalar +lens-dependent entity fields +✅ + + any Query(Queryable) diff --git a/packages/normalizr/AGENTS.md b/packages/normalizr/AGENTS.md index f355622d588f..2c12ea2f8fcf 100644 --- a/packages/normalizr/AGENTS.md +++ b/packages/normalizr/AGENTS.md @@ -5,7 +5,7 @@ Non-obvious constraints when changing `src/memo/`, `src/denormalize/`, or `src/i ## Correctness constraints - **Referential equality is a contract.** Successive `MemoCache.denormalize` calls with identical inputs must return the same object reference. `useSuspense`/`useCache`/`useQuery` skip re-renders on `===`, and many tests assert `toBe`. Probe-and-discard caching schemes (write to a throwaway cache, "real" cache repopulates next call) silently break this. -- **`WeakDependencyMap` is identity-keyed.** Cache hits depend on every entity ref in the recorded chain being `===` to the live walk's refs. Anything that wraps/proxies refs on read defeats memoization. Variants that introduce new caches (buckets, scopes) must share the same `baseGetEntity` so chain refs stay coherent. +- **`WeakDependencyMap` entity links are identity-keyed.** Entity deps live on a `WeakMap` keyed by the entity object; `argsKey` deps live on a parallel `Map` keyed by `fn(args)` strings branching off a stable function ref (see `WeakDependencyMap.ts` `next` / `nextStr`). Cache hits on the entity segment depend on every ref in the recorded chain being `===` to the live walk's refs — anything that wraps/proxies refs on read defeats memoization. Variants that introduce new caches (buckets, scopes) must share the same `getEntity` (the `DenormGetEntity` from `IMemoPolicy.getEntities(entities)`) so chain refs stay coherent. - **Both MemoCache tiers must stay coherent** — `endpoints` (top-level) and `_getCache` (per-entity) walk against the same store. Any new caching dimension applies to both, or top-level hits return stale per-entity values. - **Cache lookup precedes traversal.** Pick `WeakDependencyMap` → look up → on miss, traverse and write. You can't decide which map to use *based on* what the traversal observes. Discover-then-lookup designs need either a separate pre-scan or a `setRaw`-style promotion primitive (doesn't exist yet). - **Storage shape is API.** `state.entities[key][pk]` is observed by snapshot tests, SSR payloads, and devtools. Shape changes are breaking even when normalize/denormalize round-trip correctly. @@ -20,7 +20,7 @@ When walking the schema *definition* (not data), the canonical recursion is used 3. Plain object literals (`Object.prototype` or `null` proto) — recurse values. 4. Otherwise terminate. -Carry a `WeakSet` for cycle detection (`User.posts → Post.author → User` is common). +Cycle detection during normalize uses `getCheckLoop` (`src/normalize/getCheckLoop.ts`), which tracks `(entityKey, pk) → Set`. Custom definition walkers that could loop (e.g. `User.posts → Post.author → User`) should carry their own visited set; there is no shared `WeakSet`. **Entity classes are functions** (`typeof === 'function'`). Walkers that early-return on `typeof !== 'object'` skip every Entity. The correct guard is `t === 'object' || t === 'function'`. @@ -35,14 +35,30 @@ Carry a `WeakSet` for cycle detection (`User.posts → Post.author → User` is ## Args-dependent schemas -Schemas whose denormalized output depends on per-call `args` (not just stored data) must register an args contributor in their constructor: +Schemas whose denormalized output depends on per-call `args` (not just stored data) must register that dependency inside `denormalize` via `delegate.argsKey(fn)`: ```ts -import { registerArgsContributor } from '../memo/argsContributors.js'; -registerArgsContributor(this, (args) => /* stable string fingerprint | undefined */); +denormalize(input, delegate) { + const lens = delegate.argsKey(this.lensSelector); // this.lensSelector is ctor-bound + if (lens === undefined) return undefined; + // Encode the bucket key in your stored pk at normalize time, then unvisit: + return delegate.unvisit(this, buildCompoundPk(input, lens)); +} ``` -The contributor must be pure of `args`. Storage shape must let `denormalize` recover the right cell from `args` (typically by encoding the fingerprint into `pk` at normalize time). Schemas without this dependency need no opt-in. +See `packages/endpoint/src/schemas/Scalar.ts` (`denormalize`) for the reference implementation. + +Contract: + +- `fn(args)` must be pure and return a `string | undefined` bucket key. +- `fn` must be **referentially stable** — it's the cache *path key* on `WeakDependencyMap`. Bind it in the constructor or module-level; never an inline arrow re-created per call. Inline functions make every cache lookup miss. +- `argsKey` returns `fn(args)` for convenience (same value `set` will recompute for the chain). +- Reading `delegate.args` directly does **not** contribute to memoization — only `argsKey` adds a dep. +- Storage shape must let `denormalize` recover the right cell from the bucket key (typically by encoding it into `pk` at normalize time). `Scalar` is the reference implementation. + +Under the hood: `argsKey` pushes `{path: fn, entity: undefined}` onto the current `GlobalCache` frame; `WeakDependencyMap` branches function-typed paths via a lazy `Map` alongside the usual `WeakMap`. See `src/memo/WeakDependencyMap.ts` (`KeyFn`, `hasStringDeps`) and `src/memo/globalCache.ts` (`argsKey`, `_hasArgsKey`) for the fast-path gating. + +Schemas without args-dependence need no opt-in. When no `argsKey` fires in a frame, `GlobalCache.paths`/`getResults` take the entity-only fast path (gated on `_hasArgsKey` and `WeakDependencyMap.hasStringDeps`). ## Tests diff --git a/packages/normalizr/README.md b/packages/normalizr/README.md index 66627726399e..968464c1238d 100644 --- a/packages/normalizr/README.md +++ b/packages/normalizr/README.md @@ -335,6 +335,13 @@ Available from [@data-client/endpoint](https://www.npmjs.com/package/@data-clien 🛑 +Scalar +✅ +Scalar +lens-dependent entity fields +✅ + + any Query(Queryable) diff --git a/packages/normalizr/src/__tests__/WeakDependencyMap.test.ts b/packages/normalizr/src/__tests__/WeakDependencyMap.test.ts index 53150894b169..21e945ed469e 100644 --- a/packages/normalizr/src/__tests__/WeakDependencyMap.test.ts +++ b/packages/normalizr/src/__tests__/WeakDependencyMap.test.ts @@ -99,4 +99,89 @@ describe('WeakDependencyMap', () => { expect(wem.get([], getEntity)[0]).toBeUndefined(); }); + + describe('argsKey (function-typed) deps', () => { + // `argFn` is the stable fn reference that doubles as the path key. + // `set()` stores `curLink.nextPath = dep.path` (the function itself), + // and bucket keys are derived each call from `dep.path(args)`. + const argFn = (args: readonly any[]) => args[0]?.userId; + const depFn = { path: argFn, entity: undefined }; + + it('hasStringDeps flips to true only after a function-typed set', () => { + const wem = new WeakDependencyMap(); + expect(wem.hasStringDeps).toBe(false); + // entity-only writes never flip the flag + wem.set([depA, depB], 'entity-only'); + expect(wem.hasStringDeps).toBe(false); + wem.set([depA, depFn], 'with-args', [{ userId: 'bob' }]); + expect(wem.hasStringDeps).toBe(true); + }); + + it('buckets values by the string returned from argsKey', () => { + const wem = new WeakDependencyMap(); + wem.set([depA, depFn], 'bob-value', [{ userId: 'bob' }]); + wem.set([depA, depFn], 'alice-value', [{ userId: 'alice' }]); + + expect(wem.get(a, getEntity, [{ userId: 'bob' }])[0]).toBe('bob-value'); + expect(wem.get(a, getEntity, [{ userId: 'alice' }])[0]).toBe( + 'alice-value', + ); + // args not previously seen are a miss + expect(wem.get(a, getEntity, [{ userId: 'carol' }])[0]).toBeUndefined(); + }); + + it('routes argsKey=undefined to a shared sentinel bucket', () => { + const wem = new WeakDependencyMap(); + // argFn returns undefined for both [] and [{}] (no userId field) — both + // must land in the same UNDEF_KEY bucket so a lookup-time miss doesn't + // mask a legitimately cached `undefined`-argsKey value. + wem.set([depA, depFn], 'no-user', []); + + expect(wem.get(a, getEntity, [])[0]).toBe('no-user'); + expect(wem.get(a, getEntity, [{}])[0]).toBe('no-user'); + expect(wem.get(a, getEntity, [{ other: 'x' }])[0]).toBe('no-user'); + // explicit userId goes to a different bucket → miss + expect(wem.get(a, getEntity, [{ userId: 'bob' }])[0]).toBeUndefined(); + }); + + it('walks mixed entity + function + entity chains on get', () => { + // Exercises _getMixed: the slow path must correctly interleave + // function-keyed lookups (nextStr.get) with entity-keyed lookups + // (next.get) while traversing one cached chain. + const wem = new WeakDependencyMap(); + wem.set([depA, depFn, depB], 'mixed', [{ userId: 'bob' }]); + + expect(wem.get(a, getEntity, [{ userId: 'bob' }])[0]).toBe('mixed'); + // different argsKey output hits the function branch and misses + expect(wem.get(a, getEntity, [{ userId: 'alice' }])[0]).toBeUndefined(); + + // removing entity `b` from the store makes the tail entity lookup + // resolve to the UNDEF sentinel in mixed mode, which doesn't match + // the originally-recorded reference → miss + const getEntityMissingB = MemoPolicy.getEntities({ A: { '1': a } }); + expect( + wem.get(a, getEntityMissingB, [{ userId: 'bob' }])[0], + ).toBeUndefined(); + }); + + it('records journey with function refs for later subscription filtering', () => { + // getResults() strips function-typed paths when returning subscription + // paths; this test guarantees the function reference is preserved in + // the journey so the caller can still detect and filter it. + const wem = new WeakDependencyMap(); + wem.set([depA, depFn], 'value', [{ userId: 'bob' }]); + const [value, journey] = wem.get(a, getEntity, [{ userId: 'bob' }]); + expect(value).toBe('value'); + expect(journey).toEqual([depA.path, argFn]); + }); + + it('overwrites the same (entity, argsKey) cell on re-set', () => { + // argsKey bucketing is keyed by fn(args), so repeating the same deps + // and args must land in the exact same Link — not leak a stale value. + const wem = new WeakDependencyMap(); + wem.set([depA, depFn], 'first', [{ userId: 'bob' }]); + wem.set([depA, depFn], 'second', [{ userId: 'bob' }]); + expect(wem.get(a, getEntity, [{ userId: 'bob' }])[0]).toBe('second'); + }); + }); }); diff --git a/packages/normalizr/src/__tests__/globalCache.test.ts b/packages/normalizr/src/__tests__/globalCache.test.ts new file mode 100644 index 000000000000..879aa306e9d2 --- /dev/null +++ b/packages/normalizr/src/__tests__/globalCache.test.ts @@ -0,0 +1,258 @@ +import type { EntityInterface } from '../interface'; +import { getEntityCaches } from '../memo/entitiesCache'; +import GlobalCache from '../memo/globalCache'; +import { MemoPolicy } from '../memo/Policy'; +import type { DenormGetEntity, EndpointsCache } from '../memo/types'; +import WeakDependencyMap from '../memo/WeakDependencyMap'; + +// GlobalCache is MemoCache's per-denormalize frame. It coordinates three +// caches: +// - localCache: dedupes entity walks within one denormalize call (cycles) +// - _getCache: per-entity WeakDependencyMap recording chains of deps +// - _resultCache: top-level endpoint cache keyed by input object +// These unit tests assert the contracts individual callers rely on: args +// defaulting, argsKey bucketing, paths()/subscription filtering, and +// result-cache hit/miss behavior. +describe('GlobalCache', () => { + const Foo = { key: 'Foo' } as EntityInterface; + + const makeDeps = (entities: Record> = {}) => { + const getEntity: DenormGetEntity = MemoPolicy.getEntities(entities); + const getCache = getEntityCaches(new Map()); + const resultCache: EndpointsCache = new WeakDependencyMap(); + return { getEntity, getCache, resultCache }; + }; + + describe('constructor', () => { + it('accepts an optional args list — defaulting to []', () => { + // Covers the default-arg path that MemoCache never triggers (it always + // forwards the endpoint's args). argsKey with no bound args must still + // produce a consistent value. + const { getEntity, getCache, resultCache } = makeDeps(); + const cache = new GlobalCache(getEntity, getCache, resultCache); + + expect(cache.argsKey(args => `len:${args.length}`)).toBe('len:0'); + // fn(undefined-indexed) returns undefined, which argsKey passes through + expect(cache.argsKey(args => args[0]?.x)).toBeUndefined(); + }); + }); + + describe('argsKey', () => { + it('returns fn(args) bound at construction', () => { + const { getEntity, getCache, resultCache } = makeDeps(); + const cache = new GlobalCache(getEntity, getCache, resultCache, [ + { portfolio: 'A' }, + ]); + expect(cache.argsKey(args => args[0]?.portfolio)).toBe('A'); + }); + + it('filters function-typed deps out of subscription paths', () => { + // When argsKey registers a function dep, paths() must NOT surface it in + // the subscription list — consumers subscribe to entity cells by + // {key, pk}, and a function is not an entity path. + const { getEntity, getCache, resultCache } = makeDeps({ + Foo: { '1': { id: '1' } }, + }); + const cache = new GlobalCache(getEntity, getCache, resultCache, [ + { portfolio: 'A' }, + ]); + const lens = (args: readonly any[]) => args[0]?.portfolio; + + // First simulate a denormalize frame: the walker calls argsKey then + // records an entity dep via getEntity. + const input = { id: '1' }; + const { data, paths } = cache.getResults(input, true, () => { + cache.argsKey(lens); + cache.getEntity('1', Foo, { id: '1' }, m => + m.set('1', { id: '1', resolved: true }), + ); + return [{ id: '1', resolved: true }]; + }); + + expect(data).toEqual([{ id: '1', resolved: true }]); + // Only entity paths — function dep is stripped by paths() + expect(paths).toEqual([{ key: 'Foo', pk: '1' }]); + expect(paths.every(p => typeof p !== 'function')).toBe(true); + }); + }); + + describe('getResults', () => { + it('when cachable=false: runs compute and returns paths without hitting resultCache', () => { + const { getEntity, getCache, resultCache } = makeDeps({ + Foo: { '1': { id: '1' } }, + }); + const cache = new GlobalCache(getEntity, getCache, resultCache); + const compute = jest.fn(() => { + cache.getEntity('1', Foo, { id: '1' }, m => m.set('1', { id: '1' })); + return { id: '1' }; + }); + + const r = cache.getResults(null, false, compute); + expect(compute).toHaveBeenCalledTimes(1); + expect(r.data).toEqual({ id: '1' }); + // paths still reflect the entities that were walked + expect(r.paths).toEqual([{ key: 'Foo', pk: '1' }]); + }); + + it('when cachable=true: caches the result by input identity across fresh frames', () => { + // MemoCache allocates a new GlobalCache per denormalize call, so the + // second call must hit the SHARED resultCache even though the frame + // object is new. Compute should only run once. + const entities = { Foo: { '1': { id: '1', name: 'first' } } }; + const { getEntity, getCache, resultCache } = makeDeps(entities); + const input = { id: '1' }; + + const firstFrame = new GlobalCache(getEntity, getCache, resultCache, []); + const compute1 = jest.fn(() => { + firstFrame.getEntity('1', Foo, entities.Foo['1'], m => + m.set('1', { ...entities.Foo['1'] }), + ); + return { value: 'computed' }; + }); + const r1 = firstFrame.getResults(input, true, compute1); + expect(compute1).toHaveBeenCalledTimes(1); + + const secondFrame = new GlobalCache(getEntity, getCache, resultCache, []); + const compute2 = jest.fn(() => ({ value: 'should-not-run' })); + const r2 = secondFrame.getResults(input, true, compute2); + expect(compute2).not.toHaveBeenCalled(); + // Value and paths survive across frames and maintain reference + expect(r2.data).toBe(r1.data); + expect(r2.paths).toEqual(r1.paths); + }); + + it('cache hit across frames also works when argsKey was used (paths filtered)', () => { + // Once the resultCache has stored any function-typed dep, future hits + // must strip them when returning paths. Ensures the on-hit filter at + // getResults branches correctly. + const entities = { Foo: { '1': { id: '1' } } }; + const { getEntity, getCache, resultCache } = makeDeps(entities); + const input = { id: '1' }; + const lens = (args: readonly any[]) => args[0]?.portfolio; + const args = [{ portfolio: 'A' }]; + + const frame1 = new GlobalCache(getEntity, getCache, resultCache, args); + frame1.getResults(input, true, () => { + frame1.argsKey(lens); + frame1.getEntity('1', Foo, entities.Foo['1'], m => + m.set('1', { ...entities.Foo['1'] }), + ); + return { v: 1 }; + }); + + // Second frame, same inputs/args — should hit the cache and still + // return only entity paths. + const frame2 = new GlobalCache(getEntity, getCache, resultCache, args); + const compute = jest.fn(() => ({ v: 'not-run' })); + const r = frame2.getResults(input, true, compute); + expect(compute).not.toHaveBeenCalled(); + expect(r.paths).toEqual([{ key: 'Foo', pk: '1' }]); + expect(r.paths.every(p => typeof p !== 'function')).toBe(true); + }); + }); + + describe('getEntity', () => { + it('memoizes the per-entity WeakDependencyMap lookup across sibling calls', () => { + // Two sibling references to the same entity inside one denormalize + // frame must re-use the cached walk: compute runs once. + const entities = { Foo: { '1': { id: '1' } } }; + const { getEntity, getCache, resultCache } = makeDeps(entities); + const cache = new GlobalCache(getEntity, getCache, resultCache); + const compute = jest.fn((m: Map) => + m.set('1', { id: '1', built: true }), + ); + + const a = cache.getEntity('1', Foo, entities.Foo['1'], compute); + const b = cache.getEntity('1', Foo, entities.Foo['1'], compute); + expect(a).toBe(b); + expect(compute).toHaveBeenCalledTimes(1); + }); + + it('reuses cached dependency chain from a previous denormalize frame', () => { + // Cross-frame entity cache: the second frame should NOT call + // computeValue at all because _getCache returns a cached chain for + // the same entity reference. + const entities = { Foo: { '1': { id: '1' } } }; + const { getEntity, getCache, resultCache } = makeDeps(entities); + + const frame1 = new GlobalCache(getEntity, getCache, resultCache); + const compute1 = jest.fn((m: Map) => + m.set('1', { id: '1', built: true }), + ); + const v1 = frame1.getEntity('1', Foo, entities.Foo['1'], compute1); + expect(compute1).toHaveBeenCalledTimes(1); + + const frame2 = new GlobalCache(getEntity, getCache, resultCache); + const compute2 = jest.fn((m: Map) => + m.set('1', { should: 'not-run' }), + ); + const v2 = frame2.getEntity('1', Foo, entities.Foo['1'], compute2); + expect(compute2).not.toHaveBeenCalled(); + expect(v2).toBe(v1); + }); + + it('preserves function-dep filtering across a result-cache miss + entity-cache hit', () => { + // Regression: when the result cache misses (new input ref) but every + // entity ref is unchanged, getEntity replays cached deps WITHOUT + // running computeValue — so argsKey is never called in this frame. + // The replayed deps may contain function-typed (`argsKey`) paths from + // the prior frame; `paths()` must still strip them from the + // subscription list, otherwise a function leaks into EntityPath[]. + const entities = { Foo: { '1': { id: '1' } } }; + const { getEntity, getCache, resultCache } = makeDeps(entities); + const lens = (args: readonly any[]) => args[0]?.portfolio; + const args = [{ portfolio: 'A' }]; + + // Frame 1: populate the per-entity cache with a chain that contains + // both the entity dep and a function (argsKey) dep, exactly as would + // happen when an Entity's computeValue walks a Scalar field. + const frame1 = new GlobalCache(getEntity, getCache, resultCache, args); + frame1.getResults({ id: '1' }, true, () => { + frame1.getEntity('1', Foo, entities.Foo['1'], m => { + frame1.argsKey(lens); + m.set('1', { id: '1', resolved: true }); + }); + return [{ id: '1', resolved: true }]; + }); + + // Frame 2: NEW input ref (resultCache miss) but same entity ref + // (entity cache hits, so computeValue / argsKey do NOT run). + const frame2 = new GlobalCache(getEntity, getCache, resultCache, args); + const { paths } = frame2.getResults({ id: '1' }, true, () => { + frame2.getEntity('1', Foo, entities.Foo['1'], () => { + throw new Error( + 'computeValue must not run — entity cache should hit', + ); + }); + return [{ id: '1', resolved: true }]; + }); + + expect(paths.every(p => typeof p !== 'function')).toBe(true); + expect(paths).toEqual([{ key: 'Foo', pk: '1' }]); + }); + + it('recomputes when the entity reference changes (WeakMap identity)', () => { + // Entity-keyed chains depend on === identity of the entity ref. A new + // object at entities[key][pk] busts the cache even if contents match. + const initial = { id: '1', name: 'old' }; + const entities = { Foo: { '1': initial } }; + const { getEntity, getCache, resultCache } = makeDeps(entities); + + const frame1 = new GlobalCache(getEntity, getCache, resultCache); + frame1.getEntity('1', Foo, initial, m => + m.set('1', { ...initial, denormed: true }), + ); + + // Swap entity for a new object (same pk, fresh ref). + const next = { id: '1', name: 'new' }; + entities.Foo['1'] = next; + + const frame2 = new GlobalCache(getEntity, getCache, resultCache); + const compute = jest.fn((m: Map) => + m.set('1', { ...next, denormed: true }), + ); + frame2.getEntity('1', Foo, next, compute); + expect(compute).toHaveBeenCalledTimes(1); + }); + }); +}); diff --git a/packages/normalizr/src/__tests__/localCache.test.ts b/packages/normalizr/src/__tests__/localCache.test.ts new file mode 100644 index 000000000000..258a86a8e138 --- /dev/null +++ b/packages/normalizr/src/__tests__/localCache.test.ts @@ -0,0 +1,132 @@ +import LocalCache from '../denormalize/localCache'; +import type { EntityInterface } from '../interface'; + +// LocalCache is the uncached-path cache used by the plain `denormalize()` +// entry point. Unlike GlobalCache it does NOT: +// - persist across denormalize calls +// - track dependency paths for subscriptions +// - bucket by argsKey (its argsKey is a pure passthrough) +// It exists only to (1) deduplicate intra-walk entity computations (so cycles +// terminate and a shared entity isn't rebuilt twice) and (2) provide an API- +// compatible `Cache` so schemas like Scalar can call `delegate.argsKey()` +// without special-casing the uncached path. +describe('LocalCache', () => { + const Foo = { key: 'Foo' } as EntityInterface; + const Bar = { key: 'Bar' } as EntityInterface; + + describe('getEntity', () => { + it('computes once per pk within a single cache instance', () => { + const cache = new LocalCache(); + const computeValue = jest.fn((m: Map) => + m.set('1', { id: '1', built: true }), + ); + + const first = cache.getEntity('1', Foo, { id: '1' }, computeValue); + const second = cache.getEntity('1', Foo, { id: '1' }, computeValue); + + expect(first).toEqual({ id: '1', built: true }); + expect(second).toBe(first); + expect(computeValue).toHaveBeenCalledTimes(1); + }); + + it('computes independently for distinct pks of the same key', () => { + const cache = new LocalCache(); + const compute1 = jest.fn((m: Map) => + m.set('1', { id: '1' }), + ); + const compute2 = jest.fn((m: Map) => + m.set('2', { id: '2' }), + ); + + cache.getEntity('1', Foo, { id: '1' }, compute1); + cache.getEntity('2', Foo, { id: '2' }, compute2); + + expect(compute1).toHaveBeenCalledTimes(1); + expect(compute2).toHaveBeenCalledTimes(1); + }); + + it('partitions cache by schema key so same pk under distinct keys never collides', () => { + // A real-world cycle can have Foo/1 and Bar/1 simultaneously in-flight. + // They must resolve to distinct values. + const cache = new LocalCache(); + const foo = cache.getEntity('1', Foo, { id: '1' }, m => + m.set('1', 'foo-1'), + ); + const bar = cache.getEntity('1', Bar, { id: '1' }, m => + m.set('1', 'bar-1'), + ); + + expect(foo).toBe('foo-1'); + expect(bar).toBe('bar-1'); + }); + + it('surfaces whatever value computeValue stores (including undefined-sentinels)', () => { + // unvisit writes `undefined` into the map to mark "resolved to undefined" + // (e.g. missing nested ref). The get() must still short-circuit compute + // on the next call — but LocalCache uses `!localCacheKey.get(pk)` as its + // has-check, so an `undefined` value intentionally causes recomputation. + const cache = new LocalCache(); + const compute = jest.fn((m: Map) => m.set('1', undefined)); + + expect(cache.getEntity('1', Foo, { id: '1' }, compute)).toBeUndefined(); + expect(cache.getEntity('1', Foo, { id: '1' }, compute)).toBeUndefined(); + // This is the documented behavior: a stored `undefined` re-triggers + // compute. Change detector — if this behavior changes, GlobalCache's + // same-shaped check must be revisited too (see globalCache.getEntity). + expect(compute).toHaveBeenCalledTimes(2); + }); + }); + + describe('getResults', () => { + it('always calls computeValue — no cross-call caching', () => { + const cache = new LocalCache(); + const compute = jest.fn(() => ({ v: 'result' })); + + const r1 = cache.getResults({}, true, compute); + const r2 = cache.getResults({}, true, compute); + + expect(r1.data).toEqual({ v: 'result' }); + expect(r2.data).toEqual({ v: 'result' }); + expect(compute).toHaveBeenCalledTimes(2); + }); + + it('returns empty paths regardless of cachable flag (no subscription tracking)', () => { + const cache = new LocalCache(); + expect(cache.getResults({}, false, () => 1).paths).toEqual([]); + expect(cache.getResults({}, true, () => 1).paths).toEqual([]); + }); + }); + + describe('argsKey', () => { + it('evaluates fn against the constructor-bound args', () => { + const cache = new LocalCache([{ userId: 'u1' }]); + expect(cache.argsKey(args => args[0]?.userId)).toBe('u1'); + }); + + it('returns undefined when fn does (no coercion)', () => { + const cache = new LocalCache([{}]); + expect(cache.argsKey(args => args[0]?.userId)).toBeUndefined(); + }); + + it('defaults to an empty args list when constructor args were omitted', () => { + const cache = new LocalCache(); + expect(cache.argsKey(args => `len:${args.length}`)).toBe('len:0'); + }); + + it('is a pure passthrough — does not affect entity caching', () => { + // In GlobalCache, argsKey registers a dep and buckets results per + // fn(args). LocalCache discards deps entirely, so calling argsKey must + // not alter how getEntity memoizes (compute still runs once per pk). + const cache = new LocalCache([{ userId: 'u1' }]); + const fn = (args: readonly any[]) => args[0]?.userId; + + cache.argsKey(fn); + cache.argsKey(fn); + + const compute = jest.fn((m: Map) => m.set('1', 'ok')); + cache.getEntity('1', Foo, { id: '1' }, compute); + cache.getEntity('1', Foo, { id: '1' }, compute); + expect(compute).toHaveBeenCalledTimes(1); + }); + }); +}); diff --git a/packages/normalizr/src/buildQueryKey.ts b/packages/normalizr/src/buildQueryKey.ts index bf0636fb8177..ddcfc716d3cb 100644 --- a/packages/normalizr/src/buildQueryKey.ts +++ b/packages/normalizr/src/buildQueryKey.ts @@ -14,6 +14,7 @@ export default function buildQueryKey(delegate: IQueryDelegate) { ): NormalizeNullable { // schema classes if (canQuery(schema)) { + if (args.length === 1 && args[0] === null) return undefined as any; return schema.queryKey(args as any[], queryKey, delegate); } diff --git a/packages/normalizr/src/denormalize/cache.ts b/packages/normalizr/src/denormalize/cache.ts index 594ff05f455e..c99cea74dde3 100644 --- a/packages/normalizr/src/denormalize/cache.ts +++ b/packages/normalizr/src/denormalize/cache.ts @@ -1,5 +1,6 @@ import type { EntityInterface, EntityPath } from '../interface.js'; import type { INVALID } from './symbol.js'; +import type { KeyFn } from '../memo/WeakDependencyMap.js'; export default interface Cache { getEntity( @@ -16,4 +17,7 @@ export default interface Cache { data: any; paths: EntityPath[]; }; + /** Records `fn(args)` as a memoization dimension for the surrounding + * entity-cache frame and returns the value. */ + argsKey(fn: KeyFn): string | undefined; } diff --git a/packages/normalizr/src/denormalize/denormalize.ts b/packages/normalizr/src/denormalize/denormalize.ts index c30654832e64..2067b4346a3c 100644 --- a/packages/normalizr/src/denormalize/denormalize.ts +++ b/packages/normalizr/src/denormalize/denormalize.ts @@ -18,7 +18,7 @@ export function denormalize( return getUnvisit( MemoPolicy.getEntities(entities), - new LocalCache(), + new LocalCache(args), args, )(schema, input).data; } diff --git a/packages/normalizr/src/denormalize/localCache.ts b/packages/normalizr/src/denormalize/localCache.ts index 52954a93f548..74e2635cec59 100644 --- a/packages/normalizr/src/denormalize/localCache.ts +++ b/packages/normalizr/src/denormalize/localCache.ts @@ -1,9 +1,15 @@ import type Cache from './cache.js'; import type { EntityInterface, EntityPath } from '../interface.js'; import type { INVALID } from './symbol.js'; +import type { KeyFn } from '../memo/WeakDependencyMap.js'; export default class LocalCache implements Cache { private localCache = new Map>(); + declare private _args: readonly any[]; + + constructor(args: readonly any[] = []) { + this._args = args; + } getEntity( pk: string, @@ -33,4 +39,9 @@ export default class LocalCache implements Cache { } { return { data: computeValue(), paths: [] }; } + + /** Uncached path simply evaluates and discards — no dependency tracking. */ + argsKey(fn: KeyFn): string | undefined { + return fn(this._args); + } } diff --git a/packages/normalizr/src/denormalize/unvisit.ts b/packages/normalizr/src/denormalize/unvisit.ts index d92a3201ddaf..9d46fb836773 100644 --- a/packages/normalizr/src/denormalize/unvisit.ts +++ b/packages/normalizr/src/denormalize/unvisit.ts @@ -1,7 +1,11 @@ import type Cache from './cache.js'; import { INVALID } from './symbol.js'; import { UNDEF } from './UNDEF.js'; -import type { EntityInterface, EntityPath } from '../interface.js'; +import type { + EntityInterface, + EntityPath, + IDenormalizeDelegate, +} from '../interface.js'; import { isEntity } from '../isEntity.js'; import type { DenormGetEntity } from '../memo/types.js'; import { denormalize as arrayDenormalize } from '../schemas/Array.js'; @@ -10,8 +14,7 @@ import { denormalize as objectDenormalize } from '../schemas/Object.js'; const getUnvisitEntity = ( getEntity: DenormGetEntity, cache: Cache, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ) => { return function unvisitEntity( schema: EntityInterface, @@ -21,7 +24,7 @@ const getUnvisitEntity = ( const entity = inputIsId ? getEntity({ key: schema.key, pk: entityOrId }) : entityOrId; if (typeof entity === 'symbol') { - return schema.denormalize(entity, args, unvisit); + return schema.denormalize(entity, delegate); } if ( @@ -46,14 +49,14 @@ const getUnvisitEntity = ( let pk: string | number | undefined = inputIsId ? entityOrId : ( - (schema.pk(entity, undefined, undefined, args) as any) + (schema.pk(entity, undefined, undefined, delegate.args) as any) ); // if we can't generate a working pk we cannot do cache lookups properly, // so simply denormalize without caching if (pk === undefined || pk === '' || pk === 'undefined') { return noCacheGetEntity(localCacheKey => - unvisitEntityObject(schema, entity, '', localCacheKey, args, unvisit), + unvisitEntityObject(schema, entity, '', localCacheKey, delegate), ); } @@ -62,7 +65,7 @@ const getUnvisitEntity = ( // last function computes if it is not in any caches return cache.getEntity(pk, schema, entity, localCacheKey => - unvisitEntityObject(schema, entity, pk, localCacheKey, args, unvisit), + unvisitEntityObject(schema, entity, pk, localCacheKey, delegate), ); }; }; @@ -72,8 +75,7 @@ function unvisitEntityObject( entity: object, pk: string, localCacheKey: Map, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): void { const entityCopy = schema.createIfValid(entity); @@ -84,7 +86,7 @@ function unvisitEntityObject( // set before we recurse to prevent cycles causing infinite loops localCacheKey.set(pk, entityCopy); // we still need to set in case denormalize recursively finds INVALID - localCacheKey.set(pk, schema.denormalize(entityCopy, args, unvisit)); + localCacheKey.set(pk, schema.denormalize(entityCopy, delegate)); } } @@ -106,8 +108,16 @@ const getUnvisit = ( ) => { let depth = 0; let depthLimitHit = false; + // Single delegate object reused for the whole denormalize tree. Recursive + // schemas call `delegate.unvisit(...)` for nested types and + // `delegate.argsKey(fn)` to register an args-derived cache dimension. + const delegate: IDenormalizeDelegate = { + args, + unvisit, + argsKey: fn => cache.argsKey(fn), + }; // we don't inline this as making this function too big inhibits v8's JIT - const unvisitEntity = getUnvisitEntity(getEntity, cache, args, unvisit); + const unvisitEntity = getUnvisitEntity(getEntity, cache, delegate); function unvisit(schema: any, input: any): any { if (!schema) return input; @@ -125,7 +135,7 @@ const getUnvisit = ( if (typeof schema === 'object') { const method = Array.isArray(schema) ? arrayDenormalize : objectDenormalize; - return method(schema, input, args, unvisit); + return method(schema, input, delegate); } } else { if (isEntity(schema)) { @@ -150,9 +160,20 @@ const getUnvisit = ( const result = unvisitEntity(schema, input); depth--; return result; + } else if ( + typeof (schema as any).createIfValid === 'function' && + typeof input === 'string' + ) { + // Fast path for string inputs to table-resident schemas without `pk` + // (e.g. Scalar): look up via `unvisitEntity` directly, bypassing + // `schema.denormalize`. Their pre-computed id bakes args-dependent info + // (Scalar: lens) into the pk, so the WDM keyed by (schema.key, pk, schema) + // segregates naturally — no `argsKey` layer needed. Discriminator is + // `createIfValid` (Invalidate has only `key` and falls through). + return unvisitEntity(schema as any, input); } - return schema.denormalize(input, args, unvisit); + return schema.denormalize(input, delegate); } return input; diff --git a/packages/normalizr/src/interface.ts b/packages/normalizr/src/interface.ts index e24a34f4bfd0..58dbf914fb23 100644 --- a/packages/normalizr/src/interface.ts +++ b/packages/normalizr/src/interface.ts @@ -27,12 +27,11 @@ export interface SchemaSimple { args: any[], visit: (...args: any) => any, delegate: { getEntity: any; setEntity: any }, + /** The nearest enclosing entity-like schema (one with `pk`), if any. + * Tracked automatically by the visit walker. */ + parentEntity?: any, ): any; - denormalize( - input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, - ): T; + denormalize(input: {}, delegate: IDenormalizeDelegate): T; queryKey( args: Args, unvisit: (...args: any) => any, @@ -145,6 +144,20 @@ export interface IQueryDelegate { INVALID: symbol; } +/** Helpers during schema.denormalize() */ +export interface IDenormalizeDelegate { + /** Recursive denormalize of nested schemas */ + unvisit(schema: any, input: any): any; + /** Raw endpoint args. Reading this does NOT contribute to cache + * invalidation — if your output varies with args, register an `argsKey` + * so the cache buckets correctly. */ + readonly args: readonly any[]; + /** Adds a memoization dimension to the surrounding cache frame. + * `fn` must be referentially stable (it doubles as the cache path key). + * Returns `fn(args)` for convenience. */ + argsKey(fn: (args: readonly any[]) => string | undefined): string | undefined; +} + /** Helpers during schema.normalize() */ export interface INormalizeDelegate { /** Action meta-data for this normalize call */ diff --git a/packages/normalizr/src/memo/MemoCache.ts b/packages/normalizr/src/memo/MemoCache.ts index 82e2d9846a77..67711e9e525f 100644 --- a/packages/normalizr/src/memo/MemoCache.ts +++ b/packages/normalizr/src/memo/MemoCache.ts @@ -58,7 +58,7 @@ export default class MemoCache { return getUnvisit( getEntity, - new GlobalCache(getEntity, this._getCache, this.endpoints), + new GlobalCache(getEntity, this._getCache, this.endpoints, args), args, )(schema, input); } diff --git a/packages/normalizr/src/memo/WeakDependencyMap.ts b/packages/normalizr/src/memo/WeakDependencyMap.ts index c6469443491b..cb01c139befe 100644 --- a/packages/normalizr/src/memo/WeakDependencyMap.ts +++ b/packages/normalizr/src/memo/WeakDependencyMap.ts @@ -1,11 +1,21 @@ import { UNDEF } from '../denormalize/UNDEF.js'; +/** Function path used by `argsKey` deps. Distinguished from object paths + * via `typeof === 'function'`. */ +export type KeyFn = (args: readonly any[]) => string | undefined; + +/** Sentinel string used in place of `undefined` for string-keyed deps so we + * can perform Map lookups consistently. */ +const UNDEF_KEY = '\0'; + /** Maps a (ordered) list of dependencies to a value. * * Useful as a memoization cache for flat/normalized stores. * - * All dependencies are only weakly referenced, allowing automatic garbage collection - * when any dependencies are no longer used. + * Object dependencies are weakly referenced (via `WeakMap`), allowing + * automatic garbage collection when the dependency is no longer used. + * String-keyed dependencies (used by `argsKey`) sit on a `Map` keyed by the + * value returned from `path(args)`, branching on a stable function reference. */ export default class WeakDependencyMap< Path, @@ -13,39 +23,88 @@ export default class WeakDependencyMap< V = any, > { private readonly next = new WeakMap>(); - private nextPath: Path | undefined = undefined; + private nextPath: Path | KeyFn | undefined = undefined; + /** Sticky: true once any function-typed (`argsKey`) dep has been stored. + * Lets `get` pick the entity-only fast path when no schema in this map + * uses `argsKey` — avoids a polymorphic `typeof` branch per walk step. */ + private hasStr = false; - get(entity: K, getDependency: GetDependency) { + get( + entity: K, + getDependency: GetDependency, + args: readonly any[] = [], + ) { let curLink = this.next.get(entity); if (!curLink) return EMPTY; + if (this.hasStr) return this._getMixed(curLink, getDependency, args); while (curLink.nextPath) { // we cannot perform lookups with `undefined`, so we use a special object to represent undefined - const nextDependency = getDependency(curLink.nextPath) ?? UNDEF; - curLink = curLink.next.get(nextDependency as any); - if (!curLink) return EMPTY; + const nextDependency = getDependency(curLink.nextPath as Path) ?? UNDEF; + const nextLink = curLink.next.get(nextDependency as any); + if (!nextLink) return EMPTY; + curLink = nextLink; } // curLink exists, but has no path - so must have a value return [curLink.value, curLink.journey] as readonly [V, Path[]]; } - set(dependencies: Dep[], value: V) { + /** Slow path: dep chain may interleave entity and `argsKey`-style deps. */ + private _getMixed( + curLink: Link, + getDependency: GetDependency, + args: readonly any[], + ) { + while (curLink.nextPath) { + let nextLink: Link | undefined; + if (typeof curLink.nextPath === 'function') { + const keyValue = (curLink.nextPath as KeyFn)(args) ?? UNDEF_KEY; + nextLink = curLink.nextStr?.get(keyValue); + } else { + const nextDependency = getDependency(curLink.nextPath as Path) ?? UNDEF; + nextLink = curLink.next.get(nextDependency as any); + } + if (!nextLink) return EMPTY; + curLink = nextLink; + } + return [curLink.value, curLink.journey] as readonly [V, Path[]]; + } + + set(dependencies: Dep[], value: V, args: readonly any[] = []) { if (dependencies.length < 1) throw new KeySize(); let curLink: Link = this as any; - for (const { path, entity } of dependencies) { - let nextLink = curLink.next.get(entity as K); - if (!nextLink) { - nextLink = new Link(); - // void members are represented as a symbol so we can lookup - curLink.next.set(entity ?? UNDEF, nextLink); + for (const dep of dependencies) { + let nextLink: Link | undefined; + if (typeof dep.path === 'function') { + this.hasStr = true; + if (!curLink.nextStr) curLink.nextStr = new Map(); + const k = (dep.path as KeyFn)(args) ?? UNDEF_KEY; + nextLink = curLink.nextStr.get(k); + if (!nextLink) { + nextLink = new Link(); + curLink.nextStr.set(k, nextLink); + } + } else { + nextLink = curLink.next.get(dep.entity as K); + if (!nextLink) { + nextLink = new Link(); + // void members are represented as a symbol so we can lookup + curLink.next.set((dep.entity ?? UNDEF) as K, nextLink); + } } - curLink.nextPath = path; + curLink.nextPath = dep.path as any; curLink = nextLink; } // in case there used to be more curLink.nextPath = undefined; curLink.value = value; // we could recompute this on get, but it would have a cost and we optimize for `get` - curLink.journey = dependencies.map(dep => dep.path); + curLink.journey = dependencies.map(d => d.path) as Path[]; + } + + /** True once any `argsKey`-style dep has been written. Consumers can use + * this to skip function-stripping work on the hit path when false. */ + get hasStringDeps(): boolean { + return this.hasStr; } } @@ -54,7 +113,7 @@ export type GetDependency = ( ) => K | undefined; export interface Dep { - path: Path; + path: Path | KeyFn; entity: K | undefined; } @@ -63,7 +122,9 @@ const EMPTY = [undefined, undefined] as const; /** Link in a chain */ class Link { next = new WeakMap>(); - nextPath: Path | undefined = undefined; + /** Lazily allocated branch for string-keyed (`argsKey`) deps. */ + nextStr: Map> | undefined = undefined; + nextPath: Path | KeyFn | undefined = undefined; value: V | undefined = undefined; journey: Path[] = []; } diff --git a/packages/normalizr/src/memo/globalCache.ts b/packages/normalizr/src/memo/globalCache.ts index 8bb7626af875..b63306aebf8a 100644 --- a/packages/normalizr/src/memo/globalCache.ts +++ b/packages/normalizr/src/memo/globalCache.ts @@ -1,6 +1,9 @@ import type { GetEntityCache } from './entitiesCache.js'; import { EndpointsCache } from './types.js'; -import WeakDependencyMap, { type Dep } from './WeakDependencyMap.js'; +import WeakDependencyMap, { + type Dep, + type KeyFn, +} from './WeakDependencyMap.js'; import type Cache from '../denormalize/cache.js'; import type { INVALID } from '../denormalize/symbol.js'; import type { EntityInterface, EntityPath } from '../interface.js'; @@ -21,15 +24,32 @@ export default class GlobalCache implements Cache { declare private _getEntity: DenormGetEntity; declare private _resultCache: EndpointsCache; + declare private _args: readonly any[]; + /** Set true once `argsKey` is called for this denormalize frame. Gates + * function-stripping fast paths in `paths()` / `getResults()`. */ + private _hasArgsKey = false; constructor( getEntity: DenormGetEntity, getCache: GetEntityCache, resultCache: EndpointsCache, + args: readonly any[] = [], ) { this._getEntity = getEntity; this._getCache = getCache; this._resultCache = resultCache; + this._args = args; + } + + /** Records `fn(args)` as a string-keyed dependency for the surrounding + * entity-cache frame and returns the value. The function reference is the + * cache path key (must be referentially stable); `set` later re-evaluates + * the function with the same `args` to derive the bucket key — keeps the + * Dep shape monomorphic with entity-style deps (`{path, entity}`). */ + argsKey(fn: KeyFn): string | undefined { + this._hasArgsKey = true; + this.dependencies.push({ path: fn as any, entity: undefined }); + return fn(this._args); } getEntity( @@ -49,7 +69,11 @@ export default class GlobalCache implements Cache { object, EntityCacheValue > = this._getCache(pk, schema); - const [cacheValue, cachePath] = globalCache.get(entity, this._getEntity); + const [cacheValue, cachePath] = globalCache.get( + entity, + this._getEntity, + this._args, + ); // TODO: what if this just returned the deps - then we don't need to store them if (cachePath) { @@ -61,6 +85,12 @@ export default class GlobalCache implements Cache { for (let i = 0; i < cdeps.length; i++) { this.dependencies.push(cdeps[i]); } + // Replayed deps may include function-typed (`argsKey`) paths from + // a prior frame's computeValue (e.g. Scalar.denormalize). Since + // computeValue didn't run here, `argsKey()` wasn't called and the + // flag would otherwise stay false — causing `paths()`'s fast path + // to leak function refs into the EntityPath subscription list. + if (globalCache.hasStringDeps) this._hasArgsKey = true; return cacheValue.value; } // if we don't find in denormalize cache then do full denormalize @@ -84,7 +114,7 @@ export default class GlobalCache implements Cache { dependencies: localKey, value: localCacheKey.get(pk), }; - globalCache.set(localKey, cacheValue); + globalCache.set(localKey, cacheValue, this._args); // start of cycle - reset cycle detection if (this.cycleIndex === trackingIndex) { @@ -135,7 +165,11 @@ export default class GlobalCache implements Cache { return { data: computeValue(), paths: this.paths() }; } - let [data, paths] = this._resultCache.get(input, this._getEntity); + let [data, paths] = this._resultCache.get( + input, + this._getEntity, + this._args, + ); if (paths === undefined) { data = computeValue(); @@ -143,18 +177,40 @@ export default class GlobalCache implements Cache { paths = this.paths(); // fill pre-allocated slot 0 with the input reference this.dependencies[0] = { path: { key: '', pk: '' }, entity: input }; - this._resultCache.set(this.dependencies, data); + this._resultCache.set(this.dependencies, data, this._args); } else { paths.shift(); + // strip any function-typed (`argsKey`) paths — not subscribable entities. + // Only possible when the result cache has ever stored such a dep. + if (this._resultCache.hasStringDeps) { + for (let i = 0; i < paths.length; i++) { + if (typeof paths[i] === 'function') { + paths = paths.filter(p => typeof p !== 'function') as EntityPath[]; + break; + } + } + } } return { data, paths }; } + /** Materialize the EntityPath subscription list. Function-typed + * (`argsKey`) deps are not subscribable entities and are filtered out. */ protected paths() { const deps = this.dependencies; - const paths = new Array(deps.length - 1); + // Fast path: when no `argsKey` was recorded this frame, `deps[1..]` are + // all entity paths — restore the pre-allocated indexed-write pattern. + if (!this._hasArgsKey) { + const paths = new Array(deps.length - 1) as EntityPath[]; + for (let i = 1; i < deps.length; i++) { + paths[i - 1] = deps[i].path as EntityPath; + } + return paths; + } + const paths: EntityPath[] = []; for (let i = 1; i < deps.length; i++) { - paths[i - 1] = deps[i].path; + const p = deps[i].path; + if (typeof p !== 'function') paths.push(p as EntityPath); } return paths; } diff --git a/packages/normalizr/src/normalize/getVisit.ts b/packages/normalizr/src/normalize/getVisit.ts index 1e371e56bc1a..6547f26389d0 100644 --- a/packages/normalizr/src/normalize/getVisit.ts +++ b/packages/normalizr/src/normalize/getVisit.ts @@ -3,6 +3,14 @@ import { normalize as arrayNormalize } from '../schemas/Array.js'; import { normalize as objectNormalize } from '../schemas/Object.js'; export const getVisit = (delegate: INormalizeDelegate) => { + // Tracks the nearest enclosing entity-like schema so nested schemas (e.g. + // Scalar) can read their entity context via the 7th arg passed to + // schema.normalize. A closure variable is used (rather than a property on + // `delegate`) because cycling distinct Entity subclasses through a single + // property pollutes its inline cache and propagates polymorphic access into + // inlined call sites, deoptimizing the normalize hot path. + let currentEntity: any = undefined; + const visit = ( schema: any, value: any, @@ -10,19 +18,37 @@ export const getVisit = (delegate: INormalizeDelegate) => { key: any, args: readonly any[], ) => { - if (!value || !schema) { - return value; - } + if (value == null || !schema) return value; - if (schema.normalize && typeof schema.normalize === 'function') { - if (typeof value !== 'object') { - if (schema.pk) return `${value}`; - return value; + // Primitive value: most schemas just pass it through. Two exceptions: + // - Entity-by-id references (`schema.pk`): coerce truthy ids to string; + // falsy ids pass through verbatim as "no reference" (legacy behavior). + // - Schemas opting in via `acceptsPrimitives` (e.g. Scalar) receive the + // value — including falsy primitives like `0` — via `.normalize`. + if (typeof value !== 'object') { + if (typeof schema.normalize !== 'function') return value; + if (!schema.acceptsPrimitives) { + return value && schema.pk ? `${value}` : value; } - return schema.normalize(value, parent, key, args, visit, delegate); } - if (typeof value !== 'object' || typeof schema !== 'object') return value; + if (typeof schema.normalize === 'function') { + const prev = currentEntity; + if (schema.pk) currentEntity = schema; + const result = schema.normalize( + value, + parent, + key, + args, + visit, + delegate, + prev, + ); + currentEntity = prev; + return result; + } + + if (typeof schema !== 'object') return value; const method = Array.isArray(schema) ? arrayNormalize : objectNormalize; return method(schema, value, parent, key, args, visit); diff --git a/packages/normalizr/src/schemas/Array.ts b/packages/normalizr/src/schemas/Array.ts index aff1e4f548d5..83d12d0054f0 100644 --- a/packages/normalizr/src/schemas/Array.ts +++ b/packages/normalizr/src/schemas/Array.ts @@ -39,12 +39,13 @@ export const normalize = ( export const denormalize = ( schema: any, input: any, - args: readonly any[], - unvisit: any, + delegate: { unvisit: any }, ): any => { schema = validateSchema(schema); return input.map ? - input.map(entityOrId => unvisit(schema, entityOrId)).filter(filterEmpty) + input + .map(entityOrId => delegate.unvisit(schema, entityOrId)) + .filter(filterEmpty) : input; }; diff --git a/packages/normalizr/src/schemas/ImmutableUtils.ts b/packages/normalizr/src/schemas/ImmutableUtils.ts index 85658c0b9057..21314a922a46 100644 --- a/packages/normalizr/src/schemas/ImmutableUtils.ts +++ b/packages/normalizr/src/schemas/ImmutableUtils.ts @@ -26,18 +26,11 @@ export function isImmutable(object: {}): object is { /** * Denormalize an immutable entity. - * - * @param {Schema} schema - * @param {Immutable.Map|Immutable.Record} input - * @param {function} unvisit - * @param {function} getDenormalizedEntity - * @return {Immutable.Map|Immutable.Record} */ export function denormalizeImmutable( schema: any, input: any, - args: readonly any[], - unvisit: any, + delegate: { unvisit: any }, ): any { let deleted = false; const obj = Object.keys(schema).reduce((object, key) => { @@ -45,7 +38,7 @@ export function denormalizeImmutable( // we're accessing them using string keys. const stringKey = `${key}`; - const item = unvisit(schema[stringKey], object.get(stringKey)); + const item = delegate.unvisit(schema[stringKey], object.get(stringKey)); if (typeof item === 'symbol') { deleted = true; } diff --git a/packages/normalizr/src/schemas/Object.ts b/packages/normalizr/src/schemas/Object.ts index 51440a2e8638..d3caa7a4ed4c 100644 --- a/packages/normalizr/src/schemas/Object.ts +++ b/packages/normalizr/src/schemas/Object.ts @@ -28,11 +28,10 @@ export const normalize = ( export const denormalize = ( schema: any, input: {}, - args: readonly any[], - unvisit: any, + delegate: { unvisit: any }, ): any => { if (isImmutable(input)) { - return denormalizeImmutable(schema, input, args, unvisit); + return denormalizeImmutable(schema, input, delegate); } const object: any = { ...input }; @@ -40,7 +39,7 @@ export const denormalize = ( const keys = Object.keys(schema); for (let i = 0; i < keys.length; i++) { const k = keys[i]; - const item = unvisit(schema[k], object[k]); + const item = delegate.unvisit(schema[k], object[k]); if (object[k] !== undefined) { object[k] = item; } diff --git a/packages/react/src/hooks/__tests__/useSuspense-scalar.web.tsx b/packages/react/src/hooks/__tests__/useSuspense-scalar.web.tsx new file mode 100644 index 000000000000..257f0b3e8b4f --- /dev/null +++ b/packages/react/src/hooks/__tests__/useSuspense-scalar.web.tsx @@ -0,0 +1,190 @@ +import { Collection, Entity, RestEndpoint, Scalar } from '@data-client/rest'; +import { jest } from '@jest/globals'; +import { waitFor } from '@testing-library/react'; +import React from 'react'; + +import { renderDataHook } from '../../../../test'; +import useFetch from '../useFetch'; +import useSuspense from '../useSuspense'; + +class Company extends Entity { + id = ''; + name = ''; + price = 0; + pct_equity = 0; + shares = 0; +} +const PortfolioScalar = new Scalar({ + lens: args => (args[0] as { portfolio: string })?.portfolio, + key: 'portfolio', + entity: Company, +}); +Company.schema = { + pct_equity: PortfolioScalar, + shares: PortfolioScalar, +} as any; + +const getCompanies = new RestEndpoint({ + path: '/companies', + searchParams: {} as { portfolio: string }, + schema: [Company], +}); + +const getPortfolioColumns = new RestEndpoint({ + path: '/companies/columns', + searchParams: {} as { portfolio: string }, + schema: new Collection([PortfolioScalar], { + argsKey: ({ portfolio }: { portfolio: string }) => ({ portfolio }), + }), +}); + +const portfolioRows: Record = { + A: [ + { + id: '1', + name: 'Acme Corp', + price: 145.2, + pct_equity: 0.5, + shares: 10000, + }, + { id: '2', name: 'Globex', price: 89.5, pct_equity: 0.2, shares: 4000 }, + ], + B: [ + { id: '1', name: 'Acme Corp', price: 145.2, pct_equity: 0.3, shares: 6000 }, + { id: '2', name: 'Globex', price: 89.5, pct_equity: 0.4, shares: 8000 }, + ], +}; +const portfolioColumns: Record = { + A: [ + { id: '1', pct_equity: 0.5, shares: 10000 }, + { id: '2', pct_equity: 0.2, shares: 4000 }, + ], + B: [ + { id: '1', pct_equity: 0.3, shares: 6000 }, + { id: '2', pct_equity: 0.4, shares: 8000 }, + ], +}; + +describe('Scalar lens — useSuspense across portfolio switches (regression)', () => { + // Mirrors the docs scenario at docs/rest/api/Scalar.md: with both + // `useSuspense(getCompanies)` and `useFetch(getPortfolioColumns)` running, + // toggling the portfolio (A → B → A → B) used to "stick" on the third + // switch because the entity-level memo cached `Company` per (entity ref) + // and replayed the original Scalar cell regardless of current args. + it('updates pct_equity/shares correctly on every portfolio change', async () => { + const { result, rerender, waitForNextUpdate } = renderDataHook( + ({ portfolio }: { portfolio: 'A' | 'B' }) => { + const companies = useSuspense(getCompanies, { portfolio }); + useFetch(getPortfolioColumns, { portfolio }); + return companies; + }, + { + initialProps: { portfolio: 'A' as 'A' | 'B' }, + resolverFixtures: [ + { + endpoint: getCompanies, + response: ({ portfolio }) => portfolioRows[portfolio], + }, + { + endpoint: getPortfolioColumns, + response: ({ portfolio }) => portfolioColumns[portfolio], + }, + ], + }, + ); + + expect(result.current).toBeUndefined(); + await waitForNextUpdate(); + expect(result.current[0].pct_equity).toBe(0.5); + expect(result.current[0].shares).toBe(10000); + expect(result.current[1].pct_equity).toBe(0.2); + + rerender({ portfolio: 'B' }); + await waitForNextUpdate(); + expect(result.current[0].pct_equity).toBe(0.3); + expect(result.current[0].shares).toBe(6000); + expect(result.current[1].pct_equity).toBe(0.4); + + // Third switch — the response is already in cache. Without the lensKey + // bucket, the entity memo returns the previously-computed Company-with-B + // even though args are now portfolio:'A'. + rerender({ portfolio: 'A' }); + // No suspense expected — data is cached. Give React a tick to re-render. + await new Promise(r => setTimeout(r, 0)); + expect(result.current[0].pct_equity).toBe(0.5); + expect(result.current[0].shares).toBe(10000); + expect(result.current[1].pct_equity).toBe(0.2); + + rerender({ portfolio: 'B' }); + await new Promise(r => setTimeout(r, 0)); + expect(result.current[0].pct_equity).toBe(0.3); + expect(result.current[0].shares).toBe(6000); + expect(result.current[1].pct_equity).toBe(0.4); + }); +}); + +describe('Scalar lens — Collection + useFetch fetch counts (docs demo)', () => { + // Mirrors docs/rest/api/Scalar.md: getCompanies uses + // `Collection([Company], { argsKey: () => ({}) })` so all portfolios share + // one list entity; getPortfolioColumns uses Collection([PortfolioScalar]) + // with per-portfolio pks to fill in scalar cells per portfolio. + const getCompaniesCollection = new RestEndpoint({ + path: '/companies', + searchParams: {} as { portfolio: string }, + schema: new Collection([Company], { argsKey: () => ({}) }), + }); + + // Matches the PortfolioGrid demo in docs/rest/api/Scalar.md: the initial + // portfolio's scalar cells come from `useSuspense(getCompanies)`, so + // `useFetch(getPortfolioColumns)` only needs to fire for portfolios the + // user switches to afterward. + function DemoHook({ portfolio }: { portfolio: 'A' | 'B' }) { + const companies = useSuspense(getCompaniesCollection, { portfolio }); + const firstPortfolio = React.useRef(portfolio).current; + useFetch( + getPortfolioColumns, + portfolio === firstPortfolio ? null : { portfolio }, + ); + return companies; + } + + it('skips columns on first load and refetches nothing when revisiting', async () => { + const companiesSpy = jest.fn( + ({ portfolio }: { portfolio: 'A' | 'B' }) => portfolioRows[portfolio], + ); + const columnsSpy = jest.fn( + ({ portfolio }: { portfolio: 'A' | 'B' }) => portfolioColumns[portfolio], + ); + const { result, rerender } = renderDataHook(DemoHook, { + initialProps: { portfolio: 'A' as 'A' | 'B' }, + resolverFixtures: [ + { endpoint: getCompaniesCollection, response: companiesSpy }, + { endpoint: getPortfolioColumns, response: columnsSpy }, + ], + }); + + // Initial render: `useSuspense(getCompanies)` fetches and populates + // Scalar(A). `useFetch(getPortfolioColumns)` is gated off for the + // initial portfolio, so no redundant columns fetch. + await waitFor(() => expect(result.current).not.toBeUndefined()); + expect(companiesSpy).toHaveBeenCalledTimes(1); + expect(columnsSpy).toHaveBeenCalledTimes(0); + expect(result.current[0].pct_equity).toBe(0.5); + + // Switch to B: the Company Collection entity is shared (pk excludes + // `portfolio`), so `useSuspense` reuses the list. `useFetch` fires once + // to populate Scalar(B) cells — companies stays at 1. + rerender({ portfolio: 'B' }); + await new Promise(r => setTimeout(r, 50)); + expect(companiesSpy).toHaveBeenCalledTimes(1); + expect(columnsSpy).toHaveBeenCalledTimes(1); + expect(result.current[0].pct_equity).toBe(0.3); + + // Switch back to A: neither endpoint refires. + rerender({ portfolio: 'A' }); + await new Promise(r => setTimeout(r, 50)); + expect(companiesSpy).toHaveBeenCalledTimes(1); + expect(columnsSpy).toHaveBeenCalledTimes(1); + expect(result.current[0].pct_equity).toBe(0.5); + }); +}); diff --git a/packages/react/src/state/GCPolicy.ts b/packages/react/src/state/GCPolicy.ts index a16154463bfa..fdf934073f97 100644 --- a/packages/react/src/state/GCPolicy.ts +++ b/packages/react/src/state/GCPolicy.ts @@ -1 +1,3 @@ -export { GCPolicy as default } from '@data-client/core'; +import { GCPolicy } from '@data-client/core'; + +export default GCPolicy; diff --git a/packages/rest/README.md b/packages/rest/README.md index fc81195183e8..5becbf1b63e4 100644 --- a/packages/rest/README.md +++ b/packages/rest/README.md @@ -231,6 +231,13 @@ supports inferring argument types from the path templates. 🛑 +Scalar +✅ +Scalar +lens-dependent entity fields +✅ + + any Query(Queryable) diff --git a/website/blog/2026-04-05-v0.17-content-property-binary-auto-detection.md b/website/blog/2026-04-05-v0.17-content-property-binary-auto-detection.md deleted file mode 100644 index 595fab055b11..000000000000 --- a/website/blog/2026-04-05-v0.17-content-property-binary-auto-detection.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: 'v0.17: Typed File Downloads, Binary Auto-Detection' -description: RestEndpoint content property for typed response parsing and automatic binary Content-Type detection -authors: [ntucker] -tags: [releases, rest] -draft: true ---- - -**New Features:** - -- [RestEndpoint `content` property](/blog/2026/04/05/v0.17-content-property-binary-auto-detection#content-property) - Typed file downloads, text responses, and streaming with a single property -- [Binary Content-Type auto-detection](/blog/2026/04/05/v0.17-content-property-binary-auto-detection#binary-auto-detection) - Images, PDFs, and other binary responses are automatically handled - -{/* truncate */} - -## RestEndpoint `content` property {#content-property} - -[RestEndpoint](/rest/api/RestEndpoint) now accepts a [`content`](/rest/api/RestEndpoint#content) property that controls -how the response body is parsed. The return type is inferred automatically from the value, and `schema` is -constrained to `undefined` at the type level for non-JSON content types. - -```typescript -import { RestEndpoint } from '@data-client/rest'; - -const downloadFile = new RestEndpoint({ - path: '/files/:id/download', - // highlight-next-line - content: 'blob', - dataExpiryLength: 0, -}); - -// Return type is Blob -- inferred from content, no explicit generics needed -const blob: Blob = await ctrl.fetch(downloadFile, { id: '123' }); -``` - -Previously, file downloads required a verbose `parseResponse` override. Now it's a single property. - -Accepted values: - -| Value | Parses via | Return type | -|---|---|---| -| `'json'` | `response.json()` | `any` | -| `'blob'` | `response.blob()` | `Blob` | -| `'text'` | `response.text()` | `string` | -| `'arrayBuffer'` | `response.arrayBuffer()` | `ArrayBuffer` | -| `'stream'` | `response.body` | `ReadableStream` | - -Setting `content` to a non-JSON value enforces `schema: undefined` at the type level, since binary data cannot be -normalized. A runtime check provides a clear error message if a normalizable schema is accidentally used. - -Works with `extend()` and subclasses: - -```typescript -// Extend to add blob support -const download = jsonEndpoint.extend({ - content: 'blob', - schema: undefined, -}); - -// Subclass pattern -class BlobEndpoint extends RestEndpoint { - content = 'blob' as const; -} -``` - -[#3868](https://github.com/reactive/data-client/pull/3868) - [`content` docs](/rest/api/RestEndpoint#content) - -## Binary Content-Type auto-detection {#binary-auto-detection} - -When `content` is not set, [`parseResponse`](/rest/api/RestEndpoint#parseResponse) now auto-detects binary -Content-Types and returns `response.blob()` instead of corrupting the data via `.text()`. - -Binary types like `image/*`, `audio/*`, `video/*`, `font/*`, `application/octet-stream`, `application/pdf`, and -others are now handled automatically. Text-like types (`text/*`, XML, HTML, JavaScript, CSS, etc.) continue to -use the existing text fallback with schema-aware error messages. - -This means endpoints that serve binary data with correct Content-Type headers work without any configuration: - -```typescript -const ep = new RestEndpoint({ - path: '/avatars/:id', -}); -// Server responds with Content-Type: image/png -// -> automatically returns a Blob (previously corrupted via .text()) -``` - -[#3868](https://github.com/reactive/data-client/pull/3868) diff --git a/website/blog/2026-04-24-v0.17-scalar-typed-downloads.md b/website/blog/2026-04-24-v0.17-scalar-typed-downloads.md new file mode 100644 index 000000000000..a91c6e42682e --- /dev/null +++ b/website/blog/2026-04-24-v0.17-scalar-typed-downloads.md @@ -0,0 +1,298 @@ +--- +title: 'v0.17: Scalar, Typed File Downloads, Filter-aware Collections' +description: Lens-dependent entity fields with Scalar, typed file downloads via RestEndpoint content, automatic binary handling, and resource() nonFilterArgumentKeys for sort-aware collections +authors: [ntucker] +tags: [releases, rest, schema, endpoint, collection] +draft: true +--- + +import ScalarDemo from '../../docs/rest/shared/\_ScalarDemo.mdx'; + +**New Features:** + +- [Scalar schema](/blog/2026/04/24/v0.17-scalar-typed-downloads#scalar) - Lens-dependent entity fields (e.g. portfolio-specific values) without ever mutating the underlying entity +- [RestEndpoint `content` property](/blog/2026/04/24/v0.17-scalar-typed-downloads#content-property) - Typed file downloads, text responses, and streaming with a single property +- [resource() `nonFilterArgumentKeys`](/blog/2026/04/24/v0.17-scalar-typed-downloads#non-filter-argument-keys) - Sort/pagination args don't fragment your [Collections](/rest/api/Collection) + +**Other Improvements:** + +- [Binary Content-Type auto-detection](/blog/2026/04/24/v0.17-scalar-typed-downloads#binary-auto-detection) - Images, PDFs, and other binary responses are handled automatically with no configuration ([#3868](https://github.com/reactive/data-client/pull/3868)) +- [Collection extender body types match HTTP method semantics](/rest/api/Collection) - PATCH extenders (`.move`, `.remove`) accept partial bodies; standalone `RestEndpoint` derives a typed body from the Collection's entity schema ([#3910](https://github.com/reactive/data-client/pull/3910)) +- Export [`CollectionOptions`](/rest/api/Collection) from `@data-client/endpoint` and `@data-client/rest` for typed Collection construction ([#3904](https://github.com/reactive/data-client/pull/3904)) + +**[Breaking Changes:](/blog/2026/04/24/v0.17-scalar-typed-downloads#migration-guide)** + +- [Schema.denormalize() takes a delegate](/blog/2026/04/24/v0.17-scalar-typed-downloads#denormalize-delegate) - `denormalize(input, args, unvisit)` → `denormalize(input, delegate)`. Affects custom [Schema](/docs/api/Schema) implementations only. + +Upgrade with the automated [codemod](/codemods/v0.17.js): + +```bash +npx jscodeshift -t https://dataclient.io/codemods/v0.17.js --extensions=ts,tsx,js,jsx src/ +``` + +{/* truncate */} + +import DiffEditor from '@site/src/components/DiffEditor'; +import PkgTabs from '@site/src/components/PkgTabs'; +import SkillTabs from '@site/src/components/SkillTabs'; + +## Scalar {#scalar} + +[Scalar](/rest/api/Scalar) handles entity fields whose values depend on a runtime "lens" — like the +selected portfolio, currency, or locale. Multiple components can render the **same** entity through +different lenses simultaneously, each seeing the correct values, while the entity itself never +changes. Lens-dependent values live in a separate cell table and are joined at denormalize time +from endpoint args. + + + +`name` and `price` references stay stable across portfolio switches because the `Company` entity +itself never changes — only the `Scalar` cell selected by the current lens does. A single `Scalar` +instance can serve both as an `Entity.schema` field (parent entity inferred from the visit) and +standalone inside [`Values`](/rest/api/Values), [`[Scalar]`](/rest/api/Array), +or [`Collection([Scalar])`](/rest/api/Collection) for cheap column-only refreshes (entity bound +explicitly via `entity`). Cell pks are derived from the map key or via +[`Scalar.entityPk()`](/rest/api/Scalar#entityPk), which defaults to `Entity.pk()` so custom and +composite primary keys work without an override. + +The normalized store keeps the entity stable and the lens cells separate: + +``` +entities['Company']['1'] = { + id: '1', + price: 145.20, + pct_equity: ['1', 'pct_equity', 'Company'], + shares: ['1', 'shares', 'Company'], +} + +entities['Scalar(portfolio)']['Company|1|A'] = { pct_equity: 0.50, shares: 10000 } +entities['Scalar(portfolio)']['Company|1|B'] = { pct_equity: 0.30, shares: 6000 } +``` + +The demo above pairs `getCompanies` with a cheap `getPortfolioColumns` lens-only refresh writing +to the same cell table — see the [Scalar documentation](/rest/api/Scalar) for a full walkthrough +of the caching behavior. + +`Scalar` also implements [queryKey](/rest/api/Scalar#queryKey) so it participates directly in +[useQuery](/docs/api/useQuery), [Controller.get](/docs/api/Controller#get), and [schema.Query](/rest/api/Query) — +the full [Queryable](/rest/api/schema#queryable) surface — enumerating its cells for the current lens. + +[#3887](https://github.com/reactive/data-client/pull/3887) - [Scalar docs](/rest/api/Scalar) + +## RestEndpoint `content` property {#content-property} + +[RestEndpoint](/rest/api/RestEndpoint) now accepts a [`content`](/rest/api/RestEndpoint#content) property that controls +how the response body is parsed. The return type is inferred automatically from the value, and `schema` is +constrained to `undefined` at the type level for non-JSON content types. + +```typescript +import { RestEndpoint } from '@data-client/rest'; + +const downloadFile = new RestEndpoint({ + path: '/files/:id/download', + // highlight-next-line + content: 'blob', + dataExpiryLength: 0, +}); + +// Return type is Blob — inferred from content, no explicit generics needed +const blob: Blob = await ctrl.fetch(downloadFile, { id: '123' }); +``` + +Previously, file downloads required a verbose `parseResponse` override. Now it's a single property. + +| Value | Parses via | Return type | +|---|---|---| +| `'json'` | `response.json()` | `any` | +| `'blob'` | `response.blob()` | `Blob` | +| `'text'` | `response.text()` | `string` | +| `'arrayBuffer'` | `response.arrayBuffer()` | `ArrayBuffer` | +| `'stream'` | `response.body` | `ReadableStream` | + +Setting `content` to a non-JSON value enforces `schema: undefined` at the type level, since binary data cannot be +normalized. A runtime check provides a clear error message if a normalizable schema is accidentally used. + +Works with `extend()` and subclasses: + +```typescript +// Extend a JSON endpoint into a blob downloader +const download = jsonEndpoint.extend({ + content: 'blob', + schema: undefined, +}); + +// Subclass for a reusable blob endpoint type +class BlobEndpoint extends RestEndpoint { + content = 'blob' as const; +} +``` + +[#3868](https://github.com/reactive/data-client/pull/3868) - [`content` docs](/rest/api/RestEndpoint#content) + +## resource() `nonFilterArgumentKeys` {#non-filter-argument-keys} + +[`nonFilterArgumentKeys`](/rest/api/Collection#nonfilterargumentkeys) lets [resource()](/rest/api/resource) +declare which argument keys are *not* used to filter the collection results — typically sort, +pagination, or display options. Without it, every distinct sort or page args would create a separate +[Collection](/rest/api/Collection) bucket, and newly created items wouldn't appear in lists with +different sort args even though the underlying data should match. + +```ts +import { Entity, resource } from '@data-client/rest'; + +class Post extends Entity { + id = ''; + title = ''; + group = ''; + author = ''; +} + +const PostResource = resource({ + path: '/:group/posts/:id', + searchParams: {} as { orderBy?: string; author?: string }, + schema: Post, + // highlight-next-line + nonFilterArgumentKeys: ['orderBy'], +}); +``` + +Now `group` and `author` are filter keys (they bucket separate Collections), but `orderBy` is +shared — `PostResource.getList.push()` adds the new post to *all* `orderBy` variants of a given +`{ group, author }` bucket. + +Accepts a `string[]`, `RegExp`, or predicate function: + +```ts +nonFilterArgumentKeys: /^(orderBy|page|cursor)$/, +``` + +[#3914](https://github.com/reactive/data-client/pull/3914) - [`nonFilterArgumentKeys` docs](/rest/api/Collection#nonfilterargumentkeys) + +## Binary Content-Type auto-detection {#binary-auto-detection} + +When `content` is not set, [`parseResponse`](/rest/api/RestEndpoint#parseResponse) now auto-detects binary +Content-Types and returns `response.blob()` instead of corrupting the data via `.text()`. + +Binary types like `image/*`, `audio/*`, `video/*`, `font/*`, `application/octet-stream`, `application/pdf`, and +others are now handled automatically. Text-like types (`text/*`, XML, HTML, JavaScript, CSS, etc.) continue to +use the existing text fallback with schema-aware error messages. + +This means endpoints that serve binary data with correct Content-Type headers work without any configuration: + +```typescript +const ep = new RestEndpoint({ + path: '/avatars/:id', +}); +// Server responds with Content-Type: image/png +// → automatically returns a Blob (previously corrupted via .text()) +``` + +[#3868](https://github.com/reactive/data-client/pull/3868) + +## Migration guide + +This upgrade requires updating all package versions simultaneously. + + + +The breaking change in this release affects only **custom [Schema](/docs/api/Schema) implementations**. +If you only use built-in schemas (`Entity`, `resource()`, `Collection`, `Union`, `Values`, `Array`, +`Object`, `Query`, `Invalidate`, `Lazy`, `Scalar`), the upgrade is drop-in. Otherwise, run the +codemod from the [hero](#scalar) above to migrate custom schemas, then read on for the manual cases. + +### Schema.denormalize() takes a delegate {#denormalize-delegate} + +Skip this section if you don't implement custom [Schema](/docs/api/Schema) classes. + +`Schema.denormalize()` is now `(input, delegate)` instead of the previous 3-parameter +`(input, args, unvisit)` signature. The new +[`IDenormalizeDelegate`](/docs/api/Schema) exposes `unvisit`, `args`, and a new `argsKey(fn)` helper. +[#3887](https://github.com/reactive/data-client/pull/3887) + + + +```ts title="Before" +import { Entity } from '@data-client/rest'; + +class Wrapper { + denormalize( + input: {}, + args: readonly any[], + unvisit: (s: any, v: any) => any, + ) { + const value = unvisit(this.schema, input); + return this.process(value, ...args); + } +} +``` + +```ts title="After" +import { Entity, type IDenormalizeDelegate } from '@data-client/rest'; + +class Wrapper { + denormalize(input: {}, delegate: IDenormalizeDelegate) { + const value = delegate.unvisit(this.schema, input); + return this.process(value, ...delegate.args); + } +} +``` + + + +The codemod handles class methods, object methods, function declarations, TypeScript interface +signatures, and pass-through `someSchema.denormalize(input, args, unvisit)` calls. It also adds +the `IDenormalizeDelegate` import as an inline `type` specifier on your existing +`@data-client/*` import (or creates a new `import type` line if none is found). + +#### args-dependent output + +Reading `delegate.args` directly does **not** contribute to memoization. If your schema's *output* +depends on those args (e.g. a lens), declare the dependency through `delegate.argsKey(fn)` so the +cache buckets correctly. The codemod cannot infer this — update by hand. + +`argsKey` returns `fn(args)` for convenience, and the function reference doubles as the cache path +key — so the **function must be referentially stable**. Store it on the instance (constructor) or +at module scope; never pass an inline arrow, or every call allocates a fresh reference and misses +the cache. + + + +```ts title="Before" +class LensSchema { + denormalize(input, args, unvisit) { + const lens = args[0]?.portfolio; + return this.lookup(input, lens); + } +} +``` + +```ts title="After" +class LensSchema { + constructor({ lens }) { + this.lensSelector = lens; // stable: set once in the constructor + } + denormalize(input, delegate) { + const lens = delegate.argsKey(this.lensSelector); + return this.lookup(input, lens); + } +} +``` + + + +See [Scalar](/rest/api/Scalar) for a worked example. + +AI-assisted migration is also available: + + + +#### Additive: `parentEntity` on normalize/visit + +`Schema.normalize()` and the `visit()` callback now receive an optional trailing `parentEntity` +argument tracking the nearest enclosing entity-like schema. This is **additive** — existing schemas +don't need changes. New schemas can opt in to discover their containing entity at normalize time +(used internally by [Scalar](/rest/api/Scalar)). + +### Upgrade support + +As usual, if you have any troubles or questions, feel free to join our [![Chat](https://img.shields.io/discord/768254430381735967.svg?style=flat-square&colorB=758ED3)](https://discord.gg/wXGV27xm6t) or [file a bug](https://github.com/reactive/data-client/issues/new/choose) diff --git a/website/sidebars-endpoint.json b/website/sidebars-endpoint.json index 2d4bcdc9c6cc..5a852ec27336 100644 --- a/website/sidebars-endpoint.json +++ b/website/sidebars-endpoint.json @@ -35,6 +35,10 @@ "type": "doc", "id": "api/Union" }, + { + "type": "doc", + "id": "api/Scalar" + }, { "type": "doc", "id": "api/Query" diff --git a/website/src/components/Playground/editor-types/@data-client/core.d.ts b/website/src/components/Playground/editor-types/@data-client/core.d.ts index 05353a23bc8b..333bd457db51 100644 --- a/website/src/components/Playground/editor-types/@data-client/core.d.ts +++ b/website/src/components/Playground/editor-types/@data-client/core.d.ts @@ -16,8 +16,11 @@ interface SchemaSimple { normalize(input: any, parent: any, key: any, args: any[], visit: (...args: any) => any, delegate: { getEntity: any; setEntity: any; - }): any; - denormalize(input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any): T; + }, + /** The nearest enclosing entity-like schema (one with `pk`), if any. + * Tracked automatically by the visit walker. */ + parentEntity?: any): any; + denormalize(input: {}, delegate: IDenormalizeDelegate): T; queryKey(args: Args, unvisit: (...args: any) => any, delegate: { getEntity: any; getIndex: any; @@ -83,6 +86,19 @@ interface IQueryDelegate { /** Return to consider results invalid */ INVALID: symbol; } +/** Helpers during schema.denormalize() */ +interface IDenormalizeDelegate { + /** Recursive denormalize of nested schemas */ + unvisit(schema: any, input: any): any; + /** Raw endpoint args. Reading this does NOT contribute to cache + * invalidation — if your output varies with args, register an `argsKey` + * so the cache buckets correctly. */ + readonly args: readonly any[]; + /** Adds a memoization dimension to the surrounding cache frame. + * `fn` must be referentially stable (it doubles as the cache path key). + * Returns `fn(args)` for convenience. */ + argsKey(fn: (args: readonly any[]) => string | undefined): string | undefined; +} /** Helpers during schema.normalize() */ interface INormalizeDelegate { /** Action meta-data for this normalize call */ @@ -205,22 +221,36 @@ type NormalizeNullable = S extends { declare const INVALID: unique symbol; +/** Function path used by `argsKey` deps. Distinguished from object paths + * via `typeof === 'function'`. */ +type KeyFn = (args: readonly any[]) => string | undefined; /** Maps a (ordered) list of dependencies to a value. * * Useful as a memoization cache for flat/normalized stores. * - * All dependencies are only weakly referenced, allowing automatic garbage collection - * when any dependencies are no longer used. + * Object dependencies are weakly referenced (via `WeakMap`), allowing + * automatic garbage collection when the dependency is no longer used. + * String-keyed dependencies (used by `argsKey`) sit on a `Map` keyed by the + * value returned from `path(args)`, branching on a stable function reference. */ declare class WeakDependencyMap { private readonly next; private nextPath; - get(entity: K, getDependency: GetDependency): readonly [undefined, undefined] | readonly [V, Path[]]; - set(dependencies: Dep[], value: V): void; + /** Sticky: true once any function-typed (`argsKey`) dep has been stored. + * Lets `get` pick the entity-only fast path when no schema in this map + * uses `argsKey` — avoids a polymorphic `typeof` branch per walk step. */ + private hasStr; + get(entity: K, getDependency: GetDependency, args?: readonly any[]): readonly [undefined, undefined] | readonly [V, Path[]]; + /** Slow path: dep chain may interleave entity and `argsKey`-style deps. */ + private _getMixed; + set(dependencies: Dep[], value: V, args?: readonly any[]): void; + /** True once any `argsKey`-style dep has been written. Consumers can use + * this to skip function-stripping work on the hit path when false. */ + get hasStringDeps(): boolean; } type GetDependency = (lookup: Path) => K | undefined; interface Dep { - path: Path; + path: Path | KeyFn; entity: K | undefined; } diff --git a/website/src/components/Playground/editor-types/@data-client/endpoint.d.ts b/website/src/components/Playground/editor-types/@data-client/endpoint.d.ts index 2692588bb433..8f5a410b9a66 100644 --- a/website/src/components/Playground/editor-types/@data-client/endpoint.d.ts +++ b/website/src/components/Playground/editor-types/@data-client/endpoint.d.ts @@ -202,11 +202,24 @@ type Serializable = (value: any) => T; interface SchemaSimple { + /** + * Normalize a value into entity table form. + * + * @param input The value being normalized. + * @param parent The parent object/array/dictionary containing `input`. + * @param key The key under which `input` lives on `parent`. + * @param args The endpoint args for this normalize call. + * @param visit Recursive visitor for nested schemas. + * @param delegate Store accessors for reading/writing entities. + * @param parentEntity Nearest enclosing entity-like schema (one with `pk`), + * tracked automatically by the visit walker. `Scalar` + * uses this to discover its entity binding. + */ normalize(input: any, parent: any, key: any, args: any[], visit: (...args: any) => any, delegate: { getEntity: any; setEntity: any; - }): any; - denormalize(input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any): T; + }, parentEntity?: any): any; + denormalize(input: {}, delegate: IDenormalizeDelegate): T; queryKey(args: Args, unvisit: (...args: any) => any, delegate: { getEntity: any; getIndex: any; @@ -248,7 +261,21 @@ interface EntityTable { [pk: string]: unknown; } | undefined; } -/** Visits next data + schema while recurisvely normalizing */ +/** + * Visits next data + schema while recursively normalizing. + * + * @param schema The schema to apply to `value`. + * @param value The value being visited. + * @param parent The parent object/array/dictionary that holds `value`. + * Schemas that recurse via `visit` should pass their own + * `input` (or the surrounding container) here. + * @param key The key under which `value` lives on `parent`. + * @param args The endpoint args for this normalize call. + * + * The walker internally tracks the nearest enclosing entity-like schema and + * forwards it to `schema.normalize` as a trailing `parentEntity` argument — + * see `SchemaSimple.normalize`. Consumers of `visit` don't pass it. + */ interface Visit { (schema: any, value: any, parent: any, key: any, args: readonly any[]): any; creating?: boolean; @@ -289,6 +316,19 @@ interface IQueryDelegate { /** Return to consider results invalid */ INVALID: symbol; } +/** Helpers during schema.denormalize() */ +interface IDenormalizeDelegate { + /** Recursive denormalize of nested schemas */ + unvisit(schema: any, input: any): any; + /** Raw endpoint args. Reading this does NOT contribute to cache + * invalidation — if your output varies with args, register an `argsKey` + * so the cache buckets correctly. */ + readonly args: readonly any[]; + /** Adds a memoization dimension to the surrounding cache frame. + * `fn` must be referentially stable (it doubles as the cache path key). + * Returns `fn(args)` for convenience. */ + argsKey(fn: (args: readonly any[]) => string | undefined): string | undefined; +} /** Helpers during schema.normalize() */ interface INormalizeDelegate { /** Action meta-data for this normalize call */ @@ -545,7 +585,7 @@ interface IEntityClass { * @see https://dataclient.io/rest/api/Entity#queryKey */ queryKey(args: readonly any[], unvisit: any, delegate: IQueryDelegate): any; - denormalize IEntityInstance & InstanceType) & IEntityClass & TBase>(this: T, input: any, args: readonly any[], unvisit: (schema: any, input: any) => any): AbstractInstanceType; + denormalize IEntityInstance & InstanceType) & IEntityClass & TBase>(this: T, input: any, delegate: IDenormalizeDelegate): AbstractInstanceType; /** All instance defaults set */ readonly defaults: any; } @@ -635,7 +675,7 @@ declare class Invalidate any): E extends ProcessableEntity ? AbstractInstanceType : AbstractInstanceType; + }, delegate: IDenormalizeDelegate): E extends ProcessableEntity ? AbstractInstanceType : AbstractInstanceType; _denormalizeNullable(): (E extends ProcessableEntity ? AbstractInstanceType : AbstractInstanceType) | undefined; _normalizeNullable(): string | undefined; } @@ -674,12 +714,12 @@ declare class Lazy implements SchemaSimple { */ constructor(schema: S); normalize(input: any, parent: any, key: any, args: any[], visit: (...args: any) => any, _delegate: any): any; - denormalize(input: {}, _args: readonly any[], _unvisit: any): any; + denormalize(input: {}, _delegate: IDenormalizeDelegate): any; queryKey(_args: readonly any[], _unvisit: (...args: any) => any, _delegate: any): undefined; /** Queryable schema for use with useQuery() to resolve lazy relationships */ get query(): LazyQuery; private _query; - _denormalizeNullable: (input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any) => any; + _denormalizeNullable: (input: {}, delegate: IDenormalizeDelegate) => any; _normalizeNullable: () => NormalizeNullable; } /** @@ -691,12 +731,12 @@ declare class Lazy implements SchemaSimple { declare class LazyQuery> { schema: S; constructor(schema: S); - denormalize(input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any): Denormalize; + denormalize(input: {}, delegate: IDenormalizeDelegate): Denormalize; queryKey(args: Args, unvisit: (...args: any) => any, delegate: { getEntity: any; getIndex: any; }): any; - _denormalizeNullable: (input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any) => DenormalizeNullable; + _denormalizeNullable: (input: {}, delegate: IDenormalizeDelegate) => DenormalizeNullable; } /** @@ -716,15 +756,126 @@ declare class Query; + denormalize(input: {}, delegate: IDenormalizeDelegate): ReturnType

; queryKey(args: ProcessParameters, unvisit: (schema: any, args: any) => any): any; - _denormalizeNullable: (input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any) => ReturnType

| undefined; + _denormalizeNullable: (input: {}, delegate: IDenormalizeDelegate) => ReturnType

| undefined; _normalizeNullable: () => NormalizeNullable; } type ProcessParameters = P extends (entries: any, ...args: infer Par) => any ? Par extends [] ? SchemaArgs : Par & SchemaArgs : SchemaArgs; +interface ScalarOptions { + /** + * Selects the lens value from Endpoint args. + * + * The returned value is part of the stored cell key, so it must be stable + * for a given lens selection. + */ + lens: (args: readonly any[]) => string | undefined; + /** + * Unique namespace for this Scalar's internal entity table. + */ + key: string; + /** + * Entity class this Scalar stores cells for. + * + * Optional when used as a field on `Entity.schema`, where the parent Entity + * is inferred. Required for standalone usage such as `schema.Values`. + */ + entity?: { + key: string; + pk?: (...args: any[]) => string | number | undefined; + }; +} +/** + * Represents lens-dependent scalar fields on entities. + * + * Scalar stores values that belong to an Entity but vary by Endpoint args, + * such as portfolio-, currency-, or locale-specific columns. Use it as an + * `Entity.schema` field, or bind `entity` when using it standalone in + * `schema.Values`. + * + * @see https://dataclient.io/rest/api/Scalar + */ +declare class Scalar implements Mergeable { + readonly key: string; + readonly lensSelector: (args: readonly any[]) => string | undefined; + readonly entity: ScalarOptions['entity']; + readonly entityKey: string | undefined; + /** + * Allow normalize to receive primitive field values. + * + * Scalar stores per-cell values like `0.5`, so the visit walker must not + * apply its primitive short-circuit before dispatching to `normalize()`. + */ + readonly acceptsPrimitives = true; + /** + * Represents lens-dependent scalar fields on entities. + * + * @see https://dataclient.io/rest/api/Scalar + */ + constructor(options: ScalarOptions); + /** + * The bound Entity's pk for a standalone scalar cell. + * + * Prefers the surrounding map key (authoritative for `Values(Scalar)`), + * then falls back to the bound `Entity.pk(...)`. + * + * @see https://dataclient.io/rest/api/Scalar#entityPk + * @param [input] the scalar cell input + * @param [parent] When normalizing, the object which included the cell + * @param [key] When normalizing, the surrounding map key (if any) + * @param [args] ...args sent to Endpoint + */ + entityPk(input: any, parent: any, key: string | undefined, args: readonly any[]): string | number | undefined; + createIfValid(props: any): any; + merge(existing: any, incoming: any): any; + /** + * Determines whether an incoming write is older than the stored cell. + * + * Defaults to comparing `fetchedAt`, matching Entity behavior so older + * responses do not overwrite newer values. + */ + shouldReorder(existingMeta: { + date: number; + fetchedAt: number; + }, incomingMeta: { + date: number; + fetchedAt: number; + }, existing: any, incoming: any): boolean; + mergeWithStore(existingMeta: { + date: number; + fetchedAt: number; + }, incomingMeta: { + date: number; + fetchedAt: number; + }, existing: any, incoming: any): any; + mergeMetaWithStore(existingMeta: { + fetchedAt: number; + date: number; + expiresAt: number; + }, incomingMeta: { + fetchedAt: number; + date: number; + expiresAt: number; + }, existing: any, incoming: any): { + fetchedAt: number; + date: number; + expiresAt: number; + }; + normalize(input: any, parent: any, key: any, args: any[], visit: Visit, delegate: INormalizeDelegate, parentEntity: any): any; + denormalize(input: any, delegate: IDenormalizeDelegate): any; + /** + * Returns the cpks of cells matching the current lens, or undefined. + * + * Only consulted when `Scalar` is an endpoint's top-level schema; field + * usage resolves through the parent entity. Relies on `lens` not + * containing the cpk delimiter `|`. + */ + queryKey(args: readonly any[], unvisit: any, delegate: IQueryDelegate): string[] | undefined; +} + type CollectionOptions = ({ /** Defines lookups for Collections nested in other schemas. * @@ -927,8 +1078,7 @@ declare class Array$1 implements SchemaClass { denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): (S extends EntityMap ? T : Denormalize)[]; queryKey( @@ -982,8 +1132,7 @@ declare class All< denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): (S extends EntityMap ? T : Denormalize)[]; queryKey( @@ -1021,11 +1170,7 @@ declare class Object$1< _denormalizeNullable(): DenormalizeNullableObject; - denormalize( - input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, - ): DenormalizeObject; + denormalize(input: {}, delegate: IDenormalizeDelegate): DenormalizeObject; queryKey( args: ObjectArgs, @@ -1113,8 +1258,7 @@ interface UnionInstance< denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): AbstractInstanceType; queryKey( @@ -1196,8 +1340,7 @@ declare class Values implements SchemaClass { denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): Record< string, Choices extends EntityMap ? T : Denormalize @@ -1252,6 +1395,8 @@ type schema_d_Query, ...args: any) => any> = Query; declare const schema_d_Query: typeof Query; +type schema_d_Scalar = Scalar; +declare const schema_d_Scalar: typeof Scalar; type schema_d_SchemaAttributeFunction = SchemaAttributeFunction; type schema_d_SchemaClass = SchemaClass; type schema_d_SchemaFunction = SchemaFunction; @@ -1269,7 +1414,7 @@ type schema_d_Values = Values; declare const schema_d_Values: typeof Values; declare const schema_d_unshift: typeof unshift; declare namespace schema_d { - export { schema_d_All as All, Array$1 as Array, schema_d_Collection as Collection, type schema_d_CollectionArrayAdder as CollectionArrayAdder, type schema_d_CollectionArrayOrValuesAdder as CollectionArrayOrValuesAdder, type schema_d_CollectionConstructor as CollectionConstructor, type schema_d_CollectionFromSchema as CollectionFromSchema, type schema_d_CollectionInterface as CollectionInterface, schema_d_CollectionRoot as CollectionRoot, type schema_d_CollectionValuesAdder as CollectionValuesAdder, type schema_d_DefaultArgs as DefaultArgs, EntityMixin as Entity, type schema_d_EntityInterface as EntityInterface, type schema_d_EntityMap as EntityMap, schema_d_EntityMixin as EntityMixin, schema_d_Invalidate as Invalidate, schema_d_Lazy as Lazy, type schema_d_MergeFunction as MergeFunction, Object$1 as Object, schema_d_Query as Query, type schema_d_SchemaAttributeFunction as SchemaAttributeFunction, type schema_d_SchemaClass as SchemaClass, type schema_d_SchemaFunction as SchemaFunction, type schema_d_StrategyFunction as StrategyFunction, schema_d_Union as Union, type schema_d_UnionConstructor as UnionConstructor, type schema_d_UnionInstance as UnionInstance, type schema_d_UnionResult as UnionResult, schema_d_UnionRoot as UnionRoot, schema_d_Values as Values, schema_d_unshift as unshift }; + export { schema_d_All as All, Array$1 as Array, schema_d_Collection as Collection, type schema_d_CollectionArrayAdder as CollectionArrayAdder, type schema_d_CollectionArrayOrValuesAdder as CollectionArrayOrValuesAdder, type schema_d_CollectionConstructor as CollectionConstructor, type schema_d_CollectionFromSchema as CollectionFromSchema, type schema_d_CollectionInterface as CollectionInterface, schema_d_CollectionRoot as CollectionRoot, type schema_d_CollectionValuesAdder as CollectionValuesAdder, type schema_d_DefaultArgs as DefaultArgs, EntityMixin as Entity, type schema_d_EntityInterface as EntityInterface, type schema_d_EntityMap as EntityMap, schema_d_EntityMixin as EntityMixin, schema_d_Invalidate as Invalidate, schema_d_Lazy as Lazy, type schema_d_MergeFunction as MergeFunction, Object$1 as Object, schema_d_Query as Query, schema_d_Scalar as Scalar, type schema_d_SchemaAttributeFunction as SchemaAttributeFunction, type schema_d_SchemaClass as SchemaClass, type schema_d_SchemaFunction as SchemaFunction, type schema_d_StrategyFunction as StrategyFunction, schema_d_Union as Union, type schema_d_UnionConstructor as UnionConstructor, type schema_d_UnionInstance as UnionInstance, type schema_d_UnionResult as UnionResult, schema_d_UnionRoot as UnionRoot, schema_d_Values as Values, schema_d_unshift as unshift }; } declare const Entity_base: IEntityClass { @@ -1312,7 +1457,7 @@ declare abstract class Entity extends Entity_base { * @see https://dataclient.io/rest/api/Entity#process */ static process(input: any, parent: any, key: string | undefined, args: any[]): any; - static denormalize: (this: T, input: any, args: readonly any[], unvisit: (schema: any, input: any) => any) => AbstractInstanceType; + static denormalize: (this: T, input: any, delegate: IDenormalizeDelegate) => AbstractInstanceType; } declare function validateRequired(processedEntity: any, requiredDefaults: Record): string | undefined; @@ -1320,4 +1465,4 @@ declare function validateRequired(processedEntity: any, requiredDefaults: Record /** https://www.typescriptlang.org/docs/handbook/release-notes/typescript-5-4.html#the-noinfer-utility-type */ type NI = NoInfer; -export { type AbstractInstanceType, All, Array$1 as Array, type CheckLoop, Collection, type CollectionOptions, type DefaultArgs, type Denormalize, type DenormalizeNullable, type DenormalizeNullableObject, type DenormalizeObject, Endpoint, type EndpointExtendOptions, type EndpointExtraOptions, type EndpointInstance, type EndpointInstanceInterface, type EndpointInterface, type EndpointOptions, type EndpointParam, type EndpointToFunction, type EntitiesInterface, type EntitiesPath, Entity, type EntityFields, type EntityInterface, type EntityMap, EntityMixin, type EntityPath, type EntityTable, type ErrorTypes, type ExpiryStatusInterface, ExtendableEndpoint, type FetchFunction, type GetEntity, type GetIndex, type IEntityClass, type IEntityInstance, type INormalizeDelegate, type IQueryDelegate, type IndexPath, Invalidate, type KeyofEndpointInstance, Lazy, type Mergeable, type MutateEndpoint, type NI, type NetworkError, type Normalize, type NormalizeNullable, type NormalizeObject, type NormalizedEntity, type NormalizedIndex, type NormalizedNullableObject, Object$1 as Object, type ObjectArgs, type PolymorphicInterface, Query, type Queryable, type ReadEndpoint, type RecordClass, type ResolveType, type Schema, type SchemaArgs, type SchemaClass, type SchemaSimple, type Serializable, type SnapshotInterface, Union, type UnknownError, Values, type Visit, schema_d as schema, unshift, validateRequired }; +export { type AbstractInstanceType, All, Array$1 as Array, type CheckLoop, Collection, type CollectionOptions, type DefaultArgs, type Denormalize, type DenormalizeNullable, type DenormalizeNullableObject, type DenormalizeObject, Endpoint, type EndpointExtendOptions, type EndpointExtraOptions, type EndpointInstance, type EndpointInstanceInterface, type EndpointInterface, type EndpointOptions, type EndpointParam, type EndpointToFunction, type EntitiesInterface, type EntitiesPath, Entity, type EntityFields, type EntityInterface, type EntityMap, EntityMixin, type EntityPath, type EntityTable, type ErrorTypes, type ExpiryStatusInterface, ExtendableEndpoint, type FetchFunction, type GetEntity, type GetIndex, type IDenormalizeDelegate, type IEntityClass, type IEntityInstance, type INormalizeDelegate, type IQueryDelegate, type IndexPath, Invalidate, type KeyofEndpointInstance, Lazy, type Mergeable, type MutateEndpoint, type NI, type NetworkError, type Normalize, type NormalizeNullable, type NormalizeObject, type NormalizedEntity, type NormalizedIndex, type NormalizedNullableObject, Object$1 as Object, type ObjectArgs, type PolymorphicInterface, Query, type Queryable, type ReadEndpoint, type RecordClass, type ResolveType, Scalar, type Schema, type SchemaArgs, type SchemaClass, type SchemaSimple, type Serializable, type SnapshotInterface, Union, type UnknownError, Values, type Visit, schema_d as schema, unshift, validateRequired }; diff --git a/website/src/components/Playground/editor-types/@data-client/graphql.d.ts b/website/src/components/Playground/editor-types/@data-client/graphql.d.ts index e1a70580d2cb..cde55ab8173c 100644 --- a/website/src/components/Playground/editor-types/@data-client/graphql.d.ts +++ b/website/src/components/Playground/editor-types/@data-client/graphql.d.ts @@ -202,11 +202,24 @@ type Serializable = (value: any) => T; interface SchemaSimple { + /** + * Normalize a value into entity table form. + * + * @param input The value being normalized. + * @param parent The parent object/array/dictionary containing `input`. + * @param key The key under which `input` lives on `parent`. + * @param args The endpoint args for this normalize call. + * @param visit Recursive visitor for nested schemas. + * @param delegate Store accessors for reading/writing entities. + * @param parentEntity Nearest enclosing entity-like schema (one with `pk`), + * tracked automatically by the visit walker. `Scalar` + * uses this to discover its entity binding. + */ normalize(input: any, parent: any, key: any, args: any[], visit: (...args: any) => any, delegate: { getEntity: any; setEntity: any; - }): any; - denormalize(input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any): T; + }, parentEntity?: any): any; + denormalize(input: {}, delegate: IDenormalizeDelegate): T; queryKey(args: Args, unvisit: (...args: any) => any, delegate: { getEntity: any; getIndex: any; @@ -248,7 +261,21 @@ interface EntityTable { [pk: string]: unknown; } | undefined; } -/** Visits next data + schema while recurisvely normalizing */ +/** + * Visits next data + schema while recursively normalizing. + * + * @param schema The schema to apply to `value`. + * @param value The value being visited. + * @param parent The parent object/array/dictionary that holds `value`. + * Schemas that recurse via `visit` should pass their own + * `input` (or the surrounding container) here. + * @param key The key under which `value` lives on `parent`. + * @param args The endpoint args for this normalize call. + * + * The walker internally tracks the nearest enclosing entity-like schema and + * forwards it to `schema.normalize` as a trailing `parentEntity` argument — + * see `SchemaSimple.normalize`. Consumers of `visit` don't pass it. + */ interface Visit { (schema: any, value: any, parent: any, key: any, args: readonly any[]): any; creating?: boolean; @@ -289,6 +316,19 @@ interface IQueryDelegate { /** Return to consider results invalid */ INVALID: symbol; } +/** Helpers during schema.denormalize() */ +interface IDenormalizeDelegate { + /** Recursive denormalize of nested schemas */ + unvisit(schema: any, input: any): any; + /** Raw endpoint args. Reading this does NOT contribute to cache + * invalidation — if your output varies with args, register an `argsKey` + * so the cache buckets correctly. */ + readonly args: readonly any[]; + /** Adds a memoization dimension to the surrounding cache frame. + * `fn` must be referentially stable (it doubles as the cache path key). + * Returns `fn(args)` for convenience. */ + argsKey(fn: (args: readonly any[]) => string | undefined): string | undefined; +} /** Helpers during schema.normalize() */ interface INormalizeDelegate { /** Action meta-data for this normalize call */ @@ -545,7 +585,7 @@ interface IEntityClass { * @see https://dataclient.io/rest/api/Entity#queryKey */ queryKey(args: readonly any[], unvisit: any, delegate: IQueryDelegate): any; - denormalize IEntityInstance & InstanceType) & IEntityClass & TBase>(this: T, input: any, args: readonly any[], unvisit: (schema: any, input: any) => any): AbstractInstanceType; + denormalize IEntityInstance & InstanceType) & IEntityClass & TBase>(this: T, input: any, delegate: IDenormalizeDelegate): AbstractInstanceType; /** All instance defaults set */ readonly defaults: any; } @@ -635,7 +675,7 @@ declare class Invalidate any): E extends ProcessableEntity ? AbstractInstanceType : AbstractInstanceType; + }, delegate: IDenormalizeDelegate): E extends ProcessableEntity ? AbstractInstanceType : AbstractInstanceType; _denormalizeNullable(): (E extends ProcessableEntity ? AbstractInstanceType : AbstractInstanceType) | undefined; _normalizeNullable(): string | undefined; } @@ -674,12 +714,12 @@ declare class Lazy implements SchemaSimple { */ constructor(schema: S); normalize(input: any, parent: any, key: any, args: any[], visit: (...args: any) => any, _delegate: any): any; - denormalize(input: {}, _args: readonly any[], _unvisit: any): any; + denormalize(input: {}, _delegate: IDenormalizeDelegate): any; queryKey(_args: readonly any[], _unvisit: (...args: any) => any, _delegate: any): undefined; /** Queryable schema for use with useQuery() to resolve lazy relationships */ get query(): LazyQuery; private _query; - _denormalizeNullable: (input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any) => any; + _denormalizeNullable: (input: {}, delegate: IDenormalizeDelegate) => any; _normalizeNullable: () => NormalizeNullable; } /** @@ -691,12 +731,12 @@ declare class Lazy implements SchemaSimple { declare class LazyQuery> { schema: S; constructor(schema: S); - denormalize(input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any): Denormalize; + denormalize(input: {}, delegate: IDenormalizeDelegate): Denormalize; queryKey(args: Args, unvisit: (...args: any) => any, delegate: { getEntity: any; getIndex: any; }): any; - _denormalizeNullable: (input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any) => DenormalizeNullable; + _denormalizeNullable: (input: {}, delegate: IDenormalizeDelegate) => DenormalizeNullable; } /** @@ -716,15 +756,126 @@ declare class Query; + denormalize(input: {}, delegate: IDenormalizeDelegate): ReturnType

; queryKey(args: ProcessParameters, unvisit: (schema: any, args: any) => any): any; - _denormalizeNullable: (input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any) => ReturnType

| undefined; + _denormalizeNullable: (input: {}, delegate: IDenormalizeDelegate) => ReturnType

| undefined; _normalizeNullable: () => NormalizeNullable; } type ProcessParameters = P extends (entries: any, ...args: infer Par) => any ? Par extends [] ? SchemaArgs : Par & SchemaArgs : SchemaArgs; +interface ScalarOptions { + /** + * Selects the lens value from Endpoint args. + * + * The returned value is part of the stored cell key, so it must be stable + * for a given lens selection. + */ + lens: (args: readonly any[]) => string | undefined; + /** + * Unique namespace for this Scalar's internal entity table. + */ + key: string; + /** + * Entity class this Scalar stores cells for. + * + * Optional when used as a field on `Entity.schema`, where the parent Entity + * is inferred. Required for standalone usage such as `schema.Values`. + */ + entity?: { + key: string; + pk?: (...args: any[]) => string | number | undefined; + }; +} +/** + * Represents lens-dependent scalar fields on entities. + * + * Scalar stores values that belong to an Entity but vary by Endpoint args, + * such as portfolio-, currency-, or locale-specific columns. Use it as an + * `Entity.schema` field, or bind `entity` when using it standalone in + * `schema.Values`. + * + * @see https://dataclient.io/rest/api/Scalar + */ +declare class Scalar implements Mergeable { + readonly key: string; + readonly lensSelector: (args: readonly any[]) => string | undefined; + readonly entity: ScalarOptions['entity']; + readonly entityKey: string | undefined; + /** + * Allow normalize to receive primitive field values. + * + * Scalar stores per-cell values like `0.5`, so the visit walker must not + * apply its primitive short-circuit before dispatching to `normalize()`. + */ + readonly acceptsPrimitives = true; + /** + * Represents lens-dependent scalar fields on entities. + * + * @see https://dataclient.io/rest/api/Scalar + */ + constructor(options: ScalarOptions); + /** + * The bound Entity's pk for a standalone scalar cell. + * + * Prefers the surrounding map key (authoritative for `Values(Scalar)`), + * then falls back to the bound `Entity.pk(...)`. + * + * @see https://dataclient.io/rest/api/Scalar#entityPk + * @param [input] the scalar cell input + * @param [parent] When normalizing, the object which included the cell + * @param [key] When normalizing, the surrounding map key (if any) + * @param [args] ...args sent to Endpoint + */ + entityPk(input: any, parent: any, key: string | undefined, args: readonly any[]): string | number | undefined; + createIfValid(props: any): any; + merge(existing: any, incoming: any): any; + /** + * Determines whether an incoming write is older than the stored cell. + * + * Defaults to comparing `fetchedAt`, matching Entity behavior so older + * responses do not overwrite newer values. + */ + shouldReorder(existingMeta: { + date: number; + fetchedAt: number; + }, incomingMeta: { + date: number; + fetchedAt: number; + }, existing: any, incoming: any): boolean; + mergeWithStore(existingMeta: { + date: number; + fetchedAt: number; + }, incomingMeta: { + date: number; + fetchedAt: number; + }, existing: any, incoming: any): any; + mergeMetaWithStore(existingMeta: { + fetchedAt: number; + date: number; + expiresAt: number; + }, incomingMeta: { + fetchedAt: number; + date: number; + expiresAt: number; + }, existing: any, incoming: any): { + fetchedAt: number; + date: number; + expiresAt: number; + }; + normalize(input: any, parent: any, key: any, args: any[], visit: Visit, delegate: INormalizeDelegate, parentEntity: any): any; + denormalize(input: any, delegate: IDenormalizeDelegate): any; + /** + * Returns the cpks of cells matching the current lens, or undefined. + * + * Only consulted when `Scalar` is an endpoint's top-level schema; field + * usage resolves through the parent entity. Relies on `lens` not + * containing the cpk delimiter `|`. + */ + queryKey(args: readonly any[], unvisit: any, delegate: IQueryDelegate): string[] | undefined; +} + type CollectionOptions = ({ /** Defines lookups for Collections nested in other schemas. * @@ -927,8 +1078,7 @@ declare class Array$1 implements SchemaClass { denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): (S extends EntityMap ? T : Denormalize)[]; queryKey( @@ -982,8 +1132,7 @@ declare class All< denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): (S extends EntityMap ? T : Denormalize)[]; queryKey( @@ -1021,11 +1170,7 @@ declare class Object$1< _denormalizeNullable(): DenormalizeNullableObject; - denormalize( - input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, - ): DenormalizeObject; + denormalize(input: {}, delegate: IDenormalizeDelegate): DenormalizeObject; queryKey( args: ObjectArgs, @@ -1113,8 +1258,7 @@ interface UnionInstance< denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): AbstractInstanceType; queryKey( @@ -1196,8 +1340,7 @@ declare class Values implements SchemaClass { denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): Record< string, Choices extends EntityMap ? T : Denormalize @@ -1252,6 +1395,8 @@ type schema_d_Query, ...args: any) => any> = Query; declare const schema_d_Query: typeof Query; +type schema_d_Scalar = Scalar; +declare const schema_d_Scalar: typeof Scalar; type schema_d_SchemaAttributeFunction = SchemaAttributeFunction; type schema_d_SchemaClass = SchemaClass; type schema_d_SchemaFunction = SchemaFunction; @@ -1269,7 +1414,7 @@ type schema_d_Values = Values; declare const schema_d_Values: typeof Values; declare const schema_d_unshift: typeof unshift; declare namespace schema_d { - export { schema_d_All as All, Array$1 as Array, schema_d_Collection as Collection, type schema_d_CollectionArrayAdder as CollectionArrayAdder, type schema_d_CollectionArrayOrValuesAdder as CollectionArrayOrValuesAdder, type schema_d_CollectionConstructor as CollectionConstructor, type schema_d_CollectionFromSchema as CollectionFromSchema, type schema_d_CollectionInterface as CollectionInterface, schema_d_CollectionRoot as CollectionRoot, type schema_d_CollectionValuesAdder as CollectionValuesAdder, type schema_d_DefaultArgs as DefaultArgs, EntityMixin as Entity, type schema_d_EntityInterface as EntityInterface, type schema_d_EntityMap as EntityMap, schema_d_EntityMixin as EntityMixin, schema_d_Invalidate as Invalidate, schema_d_Lazy as Lazy, type schema_d_MergeFunction as MergeFunction, Object$1 as Object, schema_d_Query as Query, type schema_d_SchemaAttributeFunction as SchemaAttributeFunction, type schema_d_SchemaClass as SchemaClass, type schema_d_SchemaFunction as SchemaFunction, type schema_d_StrategyFunction as StrategyFunction, schema_d_Union as Union, type schema_d_UnionConstructor as UnionConstructor, type schema_d_UnionInstance as UnionInstance, type schema_d_UnionResult as UnionResult, schema_d_UnionRoot as UnionRoot, schema_d_Values as Values, schema_d_unshift as unshift }; + export { schema_d_All as All, Array$1 as Array, schema_d_Collection as Collection, type schema_d_CollectionArrayAdder as CollectionArrayAdder, type schema_d_CollectionArrayOrValuesAdder as CollectionArrayOrValuesAdder, type schema_d_CollectionConstructor as CollectionConstructor, type schema_d_CollectionFromSchema as CollectionFromSchema, type schema_d_CollectionInterface as CollectionInterface, schema_d_CollectionRoot as CollectionRoot, type schema_d_CollectionValuesAdder as CollectionValuesAdder, type schema_d_DefaultArgs as DefaultArgs, EntityMixin as Entity, type schema_d_EntityInterface as EntityInterface, type schema_d_EntityMap as EntityMap, schema_d_EntityMixin as EntityMixin, schema_d_Invalidate as Invalidate, schema_d_Lazy as Lazy, type schema_d_MergeFunction as MergeFunction, Object$1 as Object, schema_d_Query as Query, schema_d_Scalar as Scalar, type schema_d_SchemaAttributeFunction as SchemaAttributeFunction, type schema_d_SchemaClass as SchemaClass, type schema_d_SchemaFunction as SchemaFunction, type schema_d_StrategyFunction as StrategyFunction, schema_d_Union as Union, type schema_d_UnionConstructor as UnionConstructor, type schema_d_UnionInstance as UnionInstance, type schema_d_UnionResult as UnionResult, schema_d_UnionRoot as UnionRoot, schema_d_Values as Values, schema_d_unshift as unshift }; } declare const Entity_base: IEntityClass { @@ -1312,7 +1457,7 @@ declare abstract class Entity extends Entity_base { * @see https://dataclient.io/rest/api/Entity#process */ static process(input: any, parent: any, key: string | undefined, args: any[]): any; - static denormalize: (this: T, input: any, args: readonly any[], unvisit: (schema: any, input: any) => any) => AbstractInstanceType; + static denormalize: (this: T, input: any, delegate: IDenormalizeDelegate) => AbstractInstanceType; } declare function validateRequired(processedEntity: any, requiredDefaults: Record): string | undefined; @@ -1361,4 +1506,4 @@ interface GQLError { path: (string | number)[]; } -export { type AbstractInstanceType, All, Array$1 as Array, type CheckLoop, Collection, type CollectionOptions, type DefaultArgs, type Denormalize, type DenormalizeNullable, type DenormalizeNullableObject, type DenormalizeObject, Endpoint, type EndpointExtendOptions, type EndpointExtraOptions, type EndpointInstance, type EndpointInstanceInterface, type EndpointInterface, type EndpointOptions, type EndpointParam, type EndpointToFunction, type EntitiesInterface, type EntitiesPath, Entity, type EntityFields, type EntityInterface, type EntityMap, EntityMixin, type EntityPath, type EntityTable, type ErrorTypes, type ExpiryStatusInterface, ExtendableEndpoint, type FetchFunction, GQLEndpoint, GQLEntity, type GQLError, GQLNetworkError, type GQLOptions, type GetEntity, type GetIndex, type IEntityClass, type IEntityInstance, type INormalizeDelegate, type IQueryDelegate, type IndexPath, Invalidate, type KeyofEndpointInstance, Lazy, type Mergeable, type MutateEndpoint, type NI, type NetworkError, type Normalize, type NormalizeNullable, type NormalizeObject, type NormalizedEntity, type NormalizedIndex, type NormalizedNullableObject, Object$1 as Object, type ObjectArgs, type PolymorphicInterface, Query, type Queryable, type ReadEndpoint, type RecordClass, type ResolveType, type Schema, type SchemaArgs, type SchemaClass, type SchemaSimple, type Serializable, type SnapshotInterface, Union, type UnknownError, Values, type Visit, schema_d as schema, unshift, validateRequired }; +export { type AbstractInstanceType, All, Array$1 as Array, type CheckLoop, Collection, type CollectionOptions, type DefaultArgs, type Denormalize, type DenormalizeNullable, type DenormalizeNullableObject, type DenormalizeObject, Endpoint, type EndpointExtendOptions, type EndpointExtraOptions, type EndpointInstance, type EndpointInstanceInterface, type EndpointInterface, type EndpointOptions, type EndpointParam, type EndpointToFunction, type EntitiesInterface, type EntitiesPath, Entity, type EntityFields, type EntityInterface, type EntityMap, EntityMixin, type EntityPath, type EntityTable, type ErrorTypes, type ExpiryStatusInterface, ExtendableEndpoint, type FetchFunction, GQLEndpoint, GQLEntity, type GQLError, GQLNetworkError, type GQLOptions, type GetEntity, type GetIndex, type IDenormalizeDelegate, type IEntityClass, type IEntityInstance, type INormalizeDelegate, type IQueryDelegate, type IndexPath, Invalidate, type KeyofEndpointInstance, Lazy, type Mergeable, type MutateEndpoint, type NI, type NetworkError, type Normalize, type NormalizeNullable, type NormalizeObject, type NormalizedEntity, type NormalizedIndex, type NormalizedNullableObject, Object$1 as Object, type ObjectArgs, type PolymorphicInterface, Query, type Queryable, type ReadEndpoint, type RecordClass, type ResolveType, Scalar, type Schema, type SchemaArgs, type SchemaClass, type SchemaSimple, type Serializable, type SnapshotInterface, Union, type UnknownError, Values, type Visit, schema_d as schema, unshift, validateRequired }; diff --git a/website/src/components/Playground/editor-types/@data-client/normalizr.d.ts b/website/src/components/Playground/editor-types/@data-client/normalizr.d.ts index 46117835449a..c78fa87967a1 100644 --- a/website/src/components/Playground/editor-types/@data-client/normalizr.d.ts +++ b/website/src/components/Playground/editor-types/@data-client/normalizr.d.ts @@ -16,8 +16,11 @@ interface SchemaSimple { normalize(input: any, parent: any, key: any, args: any[], visit: (...args: any) => any, delegate: { getEntity: any; setEntity: any; - }): any; - denormalize(input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any): T; + }, + /** The nearest enclosing entity-like schema (one with `pk`), if any. + * Tracked automatically by the visit walker. */ + parentEntity?: any): any; + denormalize(input: {}, delegate: IDenormalizeDelegate): T; queryKey(args: Args, unvisit: (...args: any) => any, delegate: { getEntity: any; getIndex: any; @@ -96,6 +99,19 @@ interface IQueryDelegate { /** Return to consider results invalid */ INVALID: symbol; } +/** Helpers during schema.denormalize() */ +interface IDenormalizeDelegate { + /** Recursive denormalize of nested schemas */ + unvisit(schema: any, input: any): any; + /** Raw endpoint args. Reading this does NOT contribute to cache + * invalidation — if your output varies with args, register an `argsKey` + * so the cache buckets correctly. */ + readonly args: readonly any[]; + /** Adds a memoization dimension to the surrounding cache frame. + * `fn` must be referentially stable (it doubles as the cache path key). + * Returns `fn(args)` for convenience. */ + argsKey(fn: (args: readonly any[]) => string | undefined): string | undefined; +} /** Helpers during schema.normalize() */ interface INormalizeDelegate { /** Action meta-data for this normalize call */ @@ -243,22 +259,36 @@ declare function denormalize(schema: S | undefined, input: any declare function isEntity(schema: Schema): schema is EntityInterface; +/** Function path used by `argsKey` deps. Distinguished from object paths + * via `typeof === 'function'`. */ +type KeyFn = (args: readonly any[]) => string | undefined; /** Maps a (ordered) list of dependencies to a value. * * Useful as a memoization cache for flat/normalized stores. * - * All dependencies are only weakly referenced, allowing automatic garbage collection - * when any dependencies are no longer used. + * Object dependencies are weakly referenced (via `WeakMap`), allowing + * automatic garbage collection when the dependency is no longer used. + * String-keyed dependencies (used by `argsKey`) sit on a `Map` keyed by the + * value returned from `path(args)`, branching on a stable function reference. */ declare class WeakDependencyMap { private readonly next; private nextPath; - get(entity: K, getDependency: GetDependency): readonly [undefined, undefined] | readonly [V, Path[]]; - set(dependencies: Dep[], value: V): void; + /** Sticky: true once any function-typed (`argsKey`) dep has been stored. + * Lets `get` pick the entity-only fast path when no schema in this map + * uses `argsKey` — avoids a polymorphic `typeof` branch per walk step. */ + private hasStr; + get(entity: K, getDependency: GetDependency, args?: readonly any[]): readonly [undefined, undefined] | readonly [V, Path[]]; + /** Slow path: dep chain may interleave entity and `argsKey`-style deps. */ + private _getMixed; + set(dependencies: Dep[], value: V, args?: readonly any[]): void; + /** True once any `argsKey`-style dep has been written. Consumers can use + * this to skip function-stripping work on the hit path when false. */ + get hasStringDeps(): boolean; } type GetDependency = (lookup: Path) => K | undefined; interface Dep { - path: Path; + path: Path | KeyFn; entity: K | undefined; } @@ -468,4 +498,4 @@ type FetchFunction = (...args: A) => Pr declare function validateQueryKey(queryKey: unknown): boolean; -export { type AbstractInstanceType, type ArrayElement, BaseDelegate, type CheckLoop, type DenormGetEntity, type Denormalize, type DenormalizeNullable, type EndpointExtraOptions, type EndpointInterface, type EndpointsCache, type EntitiesInterface, type EntitiesPath, type EntityCache, type EntityInterface, type EntityPath, type EntityTable, type ErrorTypes, ExpiryStatus, type ExpiryStatusInterface, type FetchFunction, type GetEntity, type GetIndex, type IMemoPolicy, INVALID, type INormalizeDelegate, type IQueryDelegate, type IndexInterface, type IndexParams, type IndexPath, type InferReturn, MemoCache, MemoPolicy, type Mergeable, type MutateEndpoint, type NI, type NetworkError, type Normalize, type NormalizeNullable, type NormalizeReturnType, type NormalizedIndex, type NormalizedSchema, type OptimisticUpdateParams, type QueryPath, type Queryable, type ReadEndpoint, type ResolveType, type Schema, type SchemaArgs, type SchemaClass, type SchemaSimple, type Serializable, type SnapshotInterface, type UnknownError, type UpdateFunction, type Visit, WeakDependencyMap, denormalize, isEntity, normalize, validateQueryKey }; +export { type AbstractInstanceType, type ArrayElement, BaseDelegate, type CheckLoop, type DenormGetEntity, type Denormalize, type DenormalizeNullable, type EndpointExtraOptions, type EndpointInterface, type EndpointsCache, type EntitiesInterface, type EntitiesPath, type EntityCache, type EntityInterface, type EntityPath, type EntityTable, type ErrorTypes, ExpiryStatus, type ExpiryStatusInterface, type FetchFunction, type GetEntity, type GetIndex, type IDenormalizeDelegate, type IMemoPolicy, INVALID, type INormalizeDelegate, type IQueryDelegate, type IndexInterface, type IndexParams, type IndexPath, type InferReturn, MemoCache, MemoPolicy, type Mergeable, type MutateEndpoint, type NI, type NetworkError, type Normalize, type NormalizeNullable, type NormalizeReturnType, type NormalizedIndex, type NormalizedSchema, type OptimisticUpdateParams, type QueryPath, type Queryable, type ReadEndpoint, type ResolveType, type Schema, type SchemaArgs, type SchemaClass, type SchemaSimple, type Serializable, type SnapshotInterface, type UnknownError, type UpdateFunction, type Visit, WeakDependencyMap, denormalize, isEntity, normalize, validateQueryKey }; diff --git a/website/src/components/Playground/editor-types/@data-client/rest.d.ts b/website/src/components/Playground/editor-types/@data-client/rest.d.ts index b7b8bc4e8008..641d3925d8a4 100644 --- a/website/src/components/Playground/editor-types/@data-client/rest.d.ts +++ b/website/src/components/Playground/editor-types/@data-client/rest.d.ts @@ -204,11 +204,24 @@ type Serializable = (value: any) => T; interface SchemaSimple { + /** + * Normalize a value into entity table form. + * + * @param input The value being normalized. + * @param parent The parent object/array/dictionary containing `input`. + * @param key The key under which `input` lives on `parent`. + * @param args The endpoint args for this normalize call. + * @param visit Recursive visitor for nested schemas. + * @param delegate Store accessors for reading/writing entities. + * @param parentEntity Nearest enclosing entity-like schema (one with `pk`), + * tracked automatically by the visit walker. `Scalar` + * uses this to discover its entity binding. + */ normalize(input: any, parent: any, key: any, args: any[], visit: (...args: any) => any, delegate: { getEntity: any; setEntity: any; - }): any; - denormalize(input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any): T; + }, parentEntity?: any): any; + denormalize(input: {}, delegate: IDenormalizeDelegate): T; queryKey(args: Args, unvisit: (...args: any) => any, delegate: { getEntity: any; getIndex: any; @@ -250,7 +263,21 @@ interface EntityTable { [pk: string]: unknown; } | undefined; } -/** Visits next data + schema while recurisvely normalizing */ +/** + * Visits next data + schema while recursively normalizing. + * + * @param schema The schema to apply to `value`. + * @param value The value being visited. + * @param parent The parent object/array/dictionary that holds `value`. + * Schemas that recurse via `visit` should pass their own + * `input` (or the surrounding container) here. + * @param key The key under which `value` lives on `parent`. + * @param args The endpoint args for this normalize call. + * + * The walker internally tracks the nearest enclosing entity-like schema and + * forwards it to `schema.normalize` as a trailing `parentEntity` argument — + * see `SchemaSimple.normalize`. Consumers of `visit` don't pass it. + */ interface Visit { (schema: any, value: any, parent: any, key: any, args: readonly any[]): any; creating?: boolean; @@ -291,6 +318,19 @@ interface IQueryDelegate { /** Return to consider results invalid */ INVALID: symbol; } +/** Helpers during schema.denormalize() */ +interface IDenormalizeDelegate { + /** Recursive denormalize of nested schemas */ + unvisit(schema: any, input: any): any; + /** Raw endpoint args. Reading this does NOT contribute to cache + * invalidation — if your output varies with args, register an `argsKey` + * so the cache buckets correctly. */ + readonly args: readonly any[]; + /** Adds a memoization dimension to the surrounding cache frame. + * `fn` must be referentially stable (it doubles as the cache path key). + * Returns `fn(args)` for convenience. */ + argsKey(fn: (args: readonly any[]) => string | undefined): string | undefined; +} /** Helpers during schema.normalize() */ interface INormalizeDelegate { /** Action meta-data for this normalize call */ @@ -543,7 +583,7 @@ interface IEntityClass { * @see https://dataclient.io/rest/api/Entity#queryKey */ queryKey(args: readonly any[], unvisit: any, delegate: IQueryDelegate): any; - denormalize IEntityInstance & InstanceType) & IEntityClass & TBase>(this: T, input: any, args: readonly any[], unvisit: (schema: any, input: any) => any): AbstractInstanceType; + denormalize IEntityInstance & InstanceType) & IEntityClass & TBase>(this: T, input: any, delegate: IDenormalizeDelegate): AbstractInstanceType; /** All instance defaults set */ readonly defaults: any; } @@ -633,7 +673,7 @@ declare class Invalidate any): E extends ProcessableEntity ? AbstractInstanceType : AbstractInstanceType; + }, delegate: IDenormalizeDelegate): E extends ProcessableEntity ? AbstractInstanceType : AbstractInstanceType; _denormalizeNullable(): (E extends ProcessableEntity ? AbstractInstanceType : AbstractInstanceType) | undefined; _normalizeNullable(): string | undefined; } @@ -672,12 +712,12 @@ declare class Lazy implements SchemaSimple { */ constructor(schema: S); normalize(input: any, parent: any, key: any, args: any[], visit: (...args: any) => any, _delegate: any): any; - denormalize(input: {}, _args: readonly any[], _unvisit: any): any; + denormalize(input: {}, _delegate: IDenormalizeDelegate): any; queryKey(_args: readonly any[], _unvisit: (...args: any) => any, _delegate: any): undefined; /** Queryable schema for use with useQuery() to resolve lazy relationships */ get query(): LazyQuery; private _query; - _denormalizeNullable: (input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any) => any; + _denormalizeNullable: (input: {}, delegate: IDenormalizeDelegate) => any; _normalizeNullable: () => NormalizeNullable; } /** @@ -689,12 +729,12 @@ declare class Lazy implements SchemaSimple { declare class LazyQuery> { schema: S; constructor(schema: S); - denormalize(input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any): Denormalize; + denormalize(input: {}, delegate: IDenormalizeDelegate): Denormalize; queryKey(args: Args, unvisit: (...args: any) => any, delegate: { getEntity: any; getIndex: any; }): any; - _denormalizeNullable: (input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any) => DenormalizeNullable; + _denormalizeNullable: (input: {}, delegate: IDenormalizeDelegate) => DenormalizeNullable; } /** @@ -714,15 +754,126 @@ declare class Query; + denormalize(input: {}, delegate: IDenormalizeDelegate): ReturnType

; queryKey(args: ProcessParameters, unvisit: (schema: any, args: any) => any): any; - _denormalizeNullable: (input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any) => ReturnType

| undefined; + _denormalizeNullable: (input: {}, delegate: IDenormalizeDelegate) => ReturnType

| undefined; _normalizeNullable: () => NormalizeNullable; } type ProcessParameters = P extends (entries: any, ...args: infer Par) => any ? Par extends [] ? SchemaArgs : Par & SchemaArgs : SchemaArgs; +interface ScalarOptions { + /** + * Selects the lens value from Endpoint args. + * + * The returned value is part of the stored cell key, so it must be stable + * for a given lens selection. + */ + lens: (args: readonly any[]) => string | undefined; + /** + * Unique namespace for this Scalar's internal entity table. + */ + key: string; + /** + * Entity class this Scalar stores cells for. + * + * Optional when used as a field on `Entity.schema`, where the parent Entity + * is inferred. Required for standalone usage such as `schema.Values`. + */ + entity?: { + key: string; + pk?: (...args: any[]) => string | number | undefined; + }; +} +/** + * Represents lens-dependent scalar fields on entities. + * + * Scalar stores values that belong to an Entity but vary by Endpoint args, + * such as portfolio-, currency-, or locale-specific columns. Use it as an + * `Entity.schema` field, or bind `entity` when using it standalone in + * `schema.Values`. + * + * @see https://dataclient.io/rest/api/Scalar + */ +declare class Scalar implements Mergeable { + readonly key: string; + readonly lensSelector: (args: readonly any[]) => string | undefined; + readonly entity: ScalarOptions['entity']; + readonly entityKey: string | undefined; + /** + * Allow normalize to receive primitive field values. + * + * Scalar stores per-cell values like `0.5`, so the visit walker must not + * apply its primitive short-circuit before dispatching to `normalize()`. + */ + readonly acceptsPrimitives = true; + /** + * Represents lens-dependent scalar fields on entities. + * + * @see https://dataclient.io/rest/api/Scalar + */ + constructor(options: ScalarOptions); + /** + * The bound Entity's pk for a standalone scalar cell. + * + * Prefers the surrounding map key (authoritative for `Values(Scalar)`), + * then falls back to the bound `Entity.pk(...)`. + * + * @see https://dataclient.io/rest/api/Scalar#entityPk + * @param [input] the scalar cell input + * @param [parent] When normalizing, the object which included the cell + * @param [key] When normalizing, the surrounding map key (if any) + * @param [args] ...args sent to Endpoint + */ + entityPk(input: any, parent: any, key: string | undefined, args: readonly any[]): string | number | undefined; + createIfValid(props: any): any; + merge(existing: any, incoming: any): any; + /** + * Determines whether an incoming write is older than the stored cell. + * + * Defaults to comparing `fetchedAt`, matching Entity behavior so older + * responses do not overwrite newer values. + */ + shouldReorder(existingMeta: { + date: number; + fetchedAt: number; + }, incomingMeta: { + date: number; + fetchedAt: number; + }, existing: any, incoming: any): boolean; + mergeWithStore(existingMeta: { + date: number; + fetchedAt: number; + }, incomingMeta: { + date: number; + fetchedAt: number; + }, existing: any, incoming: any): any; + mergeMetaWithStore(existingMeta: { + fetchedAt: number; + date: number; + expiresAt: number; + }, incomingMeta: { + fetchedAt: number; + date: number; + expiresAt: number; + }, existing: any, incoming: any): { + fetchedAt: number; + date: number; + expiresAt: number; + }; + normalize(input: any, parent: any, key: any, args: any[], visit: Visit, delegate: INormalizeDelegate, parentEntity: any): any; + denormalize(input: any, delegate: IDenormalizeDelegate): any; + /** + * Returns the cpks of cells matching the current lens, or undefined. + * + * Only consulted when `Scalar` is an endpoint's top-level schema; field + * usage resolves through the parent entity. Relies on `lens` not + * containing the cpk delimiter `|`. + */ + queryKey(args: readonly any[], unvisit: any, delegate: IQueryDelegate): string[] | undefined; +} + type CollectionOptions = ({ /** Defines lookups for Collections nested in other schemas. * @@ -925,8 +1076,7 @@ declare class Array$1 implements SchemaClass { denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): (S extends EntityMap ? T : Denormalize)[]; queryKey( @@ -980,8 +1130,7 @@ declare class All< denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): (S extends EntityMap ? T : Denormalize)[]; queryKey( @@ -1019,11 +1168,7 @@ declare class Object$1< _denormalizeNullable(): DenormalizeNullableObject; - denormalize( - input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, - ): DenormalizeObject; + denormalize(input: {}, delegate: IDenormalizeDelegate): DenormalizeObject; queryKey( args: ObjectArgs, @@ -1111,8 +1256,7 @@ interface UnionInstance< denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): AbstractInstanceType; queryKey( @@ -1194,8 +1338,7 @@ declare class Values implements SchemaClass { denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): Record< string, Choices extends EntityMap ? T : Denormalize @@ -1250,6 +1393,8 @@ type schema_d_Query, ...args: any) => any> = Query; declare const schema_d_Query: typeof Query; +type schema_d_Scalar = Scalar; +declare const schema_d_Scalar: typeof Scalar; type schema_d_SchemaAttributeFunction = SchemaAttributeFunction; type schema_d_SchemaClass = SchemaClass; type schema_d_SchemaFunction = SchemaFunction; @@ -1267,7 +1412,7 @@ type schema_d_Values = Values; declare const schema_d_Values: typeof Values; declare const schema_d_unshift: typeof unshift; declare namespace schema_d { - export { schema_d_All as All, Array$1 as Array, schema_d_Collection as Collection, type schema_d_CollectionArrayAdder as CollectionArrayAdder, type schema_d_CollectionArrayOrValuesAdder as CollectionArrayOrValuesAdder, type schema_d_CollectionConstructor as CollectionConstructor, type schema_d_CollectionFromSchema as CollectionFromSchema, type schema_d_CollectionInterface as CollectionInterface, schema_d_CollectionRoot as CollectionRoot, type schema_d_CollectionValuesAdder as CollectionValuesAdder, type schema_d_DefaultArgs as DefaultArgs, EntityMixin as Entity, type schema_d_EntityInterface as EntityInterface, type schema_d_EntityMap as EntityMap, schema_d_EntityMixin as EntityMixin, schema_d_Invalidate as Invalidate, schema_d_Lazy as Lazy, type schema_d_MergeFunction as MergeFunction, Object$1 as Object, schema_d_Query as Query, type schema_d_SchemaAttributeFunction as SchemaAttributeFunction, type schema_d_SchemaClass as SchemaClass, type schema_d_SchemaFunction as SchemaFunction, type schema_d_StrategyFunction as StrategyFunction, schema_d_Union as Union, type schema_d_UnionConstructor as UnionConstructor, type schema_d_UnionInstance as UnionInstance, type schema_d_UnionResult as UnionResult, schema_d_UnionRoot as UnionRoot, schema_d_Values as Values, schema_d_unshift as unshift }; + export { schema_d_All as All, Array$1 as Array, schema_d_Collection as Collection, type schema_d_CollectionArrayAdder as CollectionArrayAdder, type schema_d_CollectionArrayOrValuesAdder as CollectionArrayOrValuesAdder, type schema_d_CollectionConstructor as CollectionConstructor, type schema_d_CollectionFromSchema as CollectionFromSchema, type schema_d_CollectionInterface as CollectionInterface, schema_d_CollectionRoot as CollectionRoot, type schema_d_CollectionValuesAdder as CollectionValuesAdder, type schema_d_DefaultArgs as DefaultArgs, EntityMixin as Entity, type schema_d_EntityInterface as EntityInterface, type schema_d_EntityMap as EntityMap, schema_d_EntityMixin as EntityMixin, schema_d_Invalidate as Invalidate, schema_d_Lazy as Lazy, type schema_d_MergeFunction as MergeFunction, Object$1 as Object, schema_d_Query as Query, schema_d_Scalar as Scalar, type schema_d_SchemaAttributeFunction as SchemaAttributeFunction, type schema_d_SchemaClass as SchemaClass, type schema_d_SchemaFunction as SchemaFunction, type schema_d_StrategyFunction as StrategyFunction, schema_d_Union as Union, type schema_d_UnionConstructor as UnionConstructor, type schema_d_UnionInstance as UnionInstance, type schema_d_UnionResult as UnionResult, schema_d_UnionRoot as UnionRoot, schema_d_Values as Values, schema_d_unshift as unshift }; } declare const Entity_base: IEntityClass { @@ -1310,7 +1455,7 @@ declare abstract class Entity extends Entity_base { * @see https://dataclient.io/rest/api/Entity#process */ static process(input: any, parent: any, key: string | undefined, args: any[]): any; - static denormalize: (this: T, input: any, args: readonly any[], unvisit: (schema: any, input: any) => any) => AbstractInstanceType; + static denormalize: (this: T, input: any, delegate: IDenormalizeDelegate) => AbstractInstanceType; } declare function validateRequired(processedEntity: any, requiredDefaults: Record): string | undefined; @@ -2035,4 +2180,4 @@ declare class NetworkError extends Error { }; } -export { type AbstractInstanceType, type AddEndpoint, All, Array$1 as Array, type CheckLoop, Collection, type CollectionOptions, type ContentType, type CustomResource, type DefaultArgs, type Defaults, type Denormalize, type DenormalizeNullable, type DenormalizeNullableObject, type DenormalizeObject, Endpoint, type EndpointExtendOptions, type EndpointExtraOptions, type EndpointInstance, type EndpointInstanceInterface, type EndpointInterface, type EndpointOptions, type EndpointParam, type EndpointToFunction, type EntitiesInterface, type EntitiesPath, Entity, type EntityFields, type EntityInterface, type EntityMap, EntityMixin, type EntityPath, type EntityTable, type ErrorTypes, type ExpiryStatusInterface, ExtendableEndpoint, type ExtendedResource, type FetchFunction, type FetchGet, type FetchMutate, type FromFallBack, type GetEndpoint, type GetEntity, type GetIndex, type HookResource, type HookableEndpointInterface, type IEntityClass, type IEntityInstance, type INormalizeDelegate, type IQueryDelegate, type RestEndpoint$1 as IRestEndpoint, type IndexPath, Invalidate, type KeyofEndpointInstance, type KeyofRestEndpoint, type KeysToArgs, Lazy, type Mergeable, type MethodToSide, type MoveEndpoint, type MutateEndpoint, type NI, NetworkError, type Normalize, type NormalizeNullable, type NormalizeObject, type NormalizedEntity, type NormalizedIndex, type NormalizedNullableObject, Object$1 as Object, type ObjectArgs, type OptionsToFunction, type PaginationEndpoint, type PaginationFieldEndpoint, type ParamFetchNoBody, type ParamFetchWithBody, type ParamToArgs, type PartialRestGenerics, type PathArgs, type PathArgsAndSearch, type PathKeys, type PolymorphicInterface, Query, type Queryable, type ReadEndpoint, type RecordClass, type RemoveEndpoint, type ResolveType, type Resource, type ResourceEndpointExtensions, type ResourceExtension, type ResourceGenerics, type ResourceInterface, type ResourceOptions, RestEndpoint, type RestEndpointConstructor, type RestEndpointConstructorOptions, type RestEndpointExtendOptions, type RestEndpointOptions, type RestExtendedEndpoint, type RestFetch, type RestGenerics, type RestInstance, type RestInstanceBase, type RestType, type RestTypeNoBody, type RestTypeWithBody, type Schema, type SchemaArgs, type SchemaClass, type SchemaSimple, type Serializable, type ShortenPath, type SnapshotInterface, Union, type UnknownError, Values, type Visit, resource as createResource, getUrlBase, getUrlTokens, hookifyResource, resource, schema_d as schema, unshift, validateRequired }; +export { type AbstractInstanceType, type AddEndpoint, All, Array$1 as Array, type CheckLoop, Collection, type CollectionOptions, type ContentType, type CustomResource, type DefaultArgs, type Defaults, type Denormalize, type DenormalizeNullable, type DenormalizeNullableObject, type DenormalizeObject, Endpoint, type EndpointExtendOptions, type EndpointExtraOptions, type EndpointInstance, type EndpointInstanceInterface, type EndpointInterface, type EndpointOptions, type EndpointParam, type EndpointToFunction, type EntitiesInterface, type EntitiesPath, Entity, type EntityFields, type EntityInterface, type EntityMap, EntityMixin, type EntityPath, type EntityTable, type ErrorTypes, type ExpiryStatusInterface, ExtendableEndpoint, type ExtendedResource, type FetchFunction, type FetchGet, type FetchMutate, type FromFallBack, type GetEndpoint, type GetEntity, type GetIndex, type HookResource, type HookableEndpointInterface, type IDenormalizeDelegate, type IEntityClass, type IEntityInstance, type INormalizeDelegate, type IQueryDelegate, type RestEndpoint$1 as IRestEndpoint, type IndexPath, Invalidate, type KeyofEndpointInstance, type KeyofRestEndpoint, type KeysToArgs, Lazy, type Mergeable, type MethodToSide, type MoveEndpoint, type MutateEndpoint, type NI, NetworkError, type Normalize, type NormalizeNullable, type NormalizeObject, type NormalizedEntity, type NormalizedIndex, type NormalizedNullableObject, Object$1 as Object, type ObjectArgs, type OptionsToFunction, type PaginationEndpoint, type PaginationFieldEndpoint, type ParamFetchNoBody, type ParamFetchWithBody, type ParamToArgs, type PartialRestGenerics, type PathArgs, type PathArgsAndSearch, type PathKeys, type PolymorphicInterface, Query, type Queryable, type ReadEndpoint, type RecordClass, type RemoveEndpoint, type ResolveType, type Resource, type ResourceEndpointExtensions, type ResourceExtension, type ResourceGenerics, type ResourceInterface, type ResourceOptions, RestEndpoint, type RestEndpointConstructor, type RestEndpointConstructorOptions, type RestEndpointExtendOptions, type RestEndpointOptions, type RestExtendedEndpoint, type RestFetch, type RestGenerics, type RestInstance, type RestInstanceBase, type RestType, type RestTypeNoBody, type RestTypeWithBody, Scalar, type Schema, type SchemaArgs, type SchemaClass, type SchemaSimple, type Serializable, type ShortenPath, type SnapshotInterface, Union, type UnknownError, Values, type Visit, resource as createResource, getUrlBase, getUrlTokens, hookifyResource, resource, schema_d as schema, unshift, validateRequired }; diff --git a/website/src/components/Playground/editor-types/globals.d.ts b/website/src/components/Playground/editor-types/globals.d.ts index c5fbed85316f..8f04089476a8 100644 --- a/website/src/components/Playground/editor-types/globals.d.ts +++ b/website/src/components/Playground/editor-types/globals.d.ts @@ -208,11 +208,24 @@ type Serializable = (value: any) => T; interface SchemaSimple { + /** + * Normalize a value into entity table form. + * + * @param input The value being normalized. + * @param parent The parent object/array/dictionary containing `input`. + * @param key The key under which `input` lives on `parent`. + * @param args The endpoint args for this normalize call. + * @param visit Recursive visitor for nested schemas. + * @param delegate Store accessors for reading/writing entities. + * @param parentEntity Nearest enclosing entity-like schema (one with `pk`), + * tracked automatically by the visit walker. `Scalar` + * uses this to discover its entity binding. + */ normalize(input: any, parent: any, key: any, args: any[], visit: (...args: any) => any, delegate: { getEntity: any; setEntity: any; - }): any; - denormalize(input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any): T; + }, parentEntity?: any): any; + denormalize(input: {}, delegate: IDenormalizeDelegate): T; queryKey(args: Args, unvisit: (...args: any) => any, delegate: { getEntity: any; getIndex: any; @@ -254,7 +267,21 @@ interface EntityTable { [pk: string]: unknown; } | undefined; } -/** Visits next data + schema while recurisvely normalizing */ +/** + * Visits next data + schema while recursively normalizing. + * + * @param schema The schema to apply to `value`. + * @param value The value being visited. + * @param parent The parent object/array/dictionary that holds `value`. + * Schemas that recurse via `visit` should pass their own + * `input` (or the surrounding container) here. + * @param key The key under which `value` lives on `parent`. + * @param args The endpoint args for this normalize call. + * + * The walker internally tracks the nearest enclosing entity-like schema and + * forwards it to `schema.normalize` as a trailing `parentEntity` argument — + * see `SchemaSimple.normalize`. Consumers of `visit` don't pass it. + */ interface Visit { (schema: any, value: any, parent: any, key: any, args: readonly any[]): any; creating?: boolean; @@ -295,6 +322,19 @@ interface IQueryDelegate { /** Return to consider results invalid */ INVALID: symbol; } +/** Helpers during schema.denormalize() */ +interface IDenormalizeDelegate { + /** Recursive denormalize of nested schemas */ + unvisit(schema: any, input: any): any; + /** Raw endpoint args. Reading this does NOT contribute to cache + * invalidation — if your output varies with args, register an `argsKey` + * so the cache buckets correctly. */ + readonly args: readonly any[]; + /** Adds a memoization dimension to the surrounding cache frame. + * `fn` must be referentially stable (it doubles as the cache path key). + * Returns `fn(args)` for convenience. */ + argsKey(fn: (args: readonly any[]) => string | undefined): string | undefined; +} /** Helpers during schema.normalize() */ interface INormalizeDelegate { /** Action meta-data for this normalize call */ @@ -547,7 +587,7 @@ interface IEntityClass { * @see https://dataclient.io/rest/api/Entity#queryKey */ queryKey(args: readonly any[], unvisit: any, delegate: IQueryDelegate): any; - denormalize IEntityInstance & InstanceType) & IEntityClass & TBase>(this: T, input: any, args: readonly any[], unvisit: (schema: any, input: any) => any): AbstractInstanceType; + denormalize IEntityInstance & InstanceType) & IEntityClass & TBase>(this: T, input: any, delegate: IDenormalizeDelegate): AbstractInstanceType; /** All instance defaults set */ readonly defaults: any; } @@ -637,7 +677,7 @@ declare class Invalidate any): E extends ProcessableEntity ? AbstractInstanceType : AbstractInstanceType; + }, delegate: IDenormalizeDelegate): E extends ProcessableEntity ? AbstractInstanceType : AbstractInstanceType; _denormalizeNullable(): (E extends ProcessableEntity ? AbstractInstanceType : AbstractInstanceType) | undefined; _normalizeNullable(): string | undefined; } @@ -676,12 +716,12 @@ declare class Lazy implements SchemaSimple { */ constructor(schema: S); normalize(input: any, parent: any, key: any, args: any[], visit: (...args: any) => any, _delegate: any): any; - denormalize(input: {}, _args: readonly any[], _unvisit: any): any; + denormalize(input: {}, _delegate: IDenormalizeDelegate): any; queryKey(_args: readonly any[], _unvisit: (...args: any) => any, _delegate: any): undefined; /** Queryable schema for use with useQuery() to resolve lazy relationships */ get query(): LazyQuery; private _query; - _denormalizeNullable: (input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any) => any; + _denormalizeNullable: (input: {}, delegate: IDenormalizeDelegate) => any; _normalizeNullable: () => NormalizeNullable; } /** @@ -693,12 +733,12 @@ declare class Lazy implements SchemaSimple { declare class LazyQuery> { schema: S; constructor(schema: S); - denormalize(input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any): Denormalize; + denormalize(input: {}, delegate: IDenormalizeDelegate): Denormalize; queryKey(args: Args, unvisit: (...args: any) => any, delegate: { getEntity: any; getIndex: any; }): any; - _denormalizeNullable: (input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any) => DenormalizeNullable; + _denormalizeNullable: (input: {}, delegate: IDenormalizeDelegate) => DenormalizeNullable; } /** @@ -718,15 +758,126 @@ declare class Query; + denormalize(input: {}, delegate: IDenormalizeDelegate): ReturnType

; queryKey(args: ProcessParameters, unvisit: (schema: any, args: any) => any): any; - _denormalizeNullable: (input: {}, args: readonly any[], unvisit: (schema: any, input: any) => any) => ReturnType

| undefined; + _denormalizeNullable: (input: {}, delegate: IDenormalizeDelegate) => ReturnType

| undefined; _normalizeNullable: () => NormalizeNullable; } type ProcessParameters = P extends (entries: any, ...args: infer Par) => any ? Par extends [] ? SchemaArgs : Par & SchemaArgs : SchemaArgs; +interface ScalarOptions { + /** + * Selects the lens value from Endpoint args. + * + * The returned value is part of the stored cell key, so it must be stable + * for a given lens selection. + */ + lens: (args: readonly any[]) => string | undefined; + /** + * Unique namespace for this Scalar's internal entity table. + */ + key: string; + /** + * Entity class this Scalar stores cells for. + * + * Optional when used as a field on `Entity.schema`, where the parent Entity + * is inferred. Required for standalone usage such as `schema.Values`. + */ + entity?: { + key: string; + pk?: (...args: any[]) => string | number | undefined; + }; +} +/** + * Represents lens-dependent scalar fields on entities. + * + * Scalar stores values that belong to an Entity but vary by Endpoint args, + * such as portfolio-, currency-, or locale-specific columns. Use it as an + * `Entity.schema` field, or bind `entity` when using it standalone in + * `schema.Values`. + * + * @see https://dataclient.io/rest/api/Scalar + */ +declare class Scalar implements Mergeable { + readonly key: string; + readonly lensSelector: (args: readonly any[]) => string | undefined; + readonly entity: ScalarOptions['entity']; + readonly entityKey: string | undefined; + /** + * Allow normalize to receive primitive field values. + * + * Scalar stores per-cell values like `0.5`, so the visit walker must not + * apply its primitive short-circuit before dispatching to `normalize()`. + */ + readonly acceptsPrimitives = true; + /** + * Represents lens-dependent scalar fields on entities. + * + * @see https://dataclient.io/rest/api/Scalar + */ + constructor(options: ScalarOptions); + /** + * The bound Entity's pk for a standalone scalar cell. + * + * Prefers the surrounding map key (authoritative for `Values(Scalar)`), + * then falls back to the bound `Entity.pk(...)`. + * + * @see https://dataclient.io/rest/api/Scalar#entityPk + * @param [input] the scalar cell input + * @param [parent] When normalizing, the object which included the cell + * @param [key] When normalizing, the surrounding map key (if any) + * @param [args] ...args sent to Endpoint + */ + entityPk(input: any, parent: any, key: string | undefined, args: readonly any[]): string | number | undefined; + createIfValid(props: any): any; + merge(existing: any, incoming: any): any; + /** + * Determines whether an incoming write is older than the stored cell. + * + * Defaults to comparing `fetchedAt`, matching Entity behavior so older + * responses do not overwrite newer values. + */ + shouldReorder(existingMeta: { + date: number; + fetchedAt: number; + }, incomingMeta: { + date: number; + fetchedAt: number; + }, existing: any, incoming: any): boolean; + mergeWithStore(existingMeta: { + date: number; + fetchedAt: number; + }, incomingMeta: { + date: number; + fetchedAt: number; + }, existing: any, incoming: any): any; + mergeMetaWithStore(existingMeta: { + fetchedAt: number; + date: number; + expiresAt: number; + }, incomingMeta: { + fetchedAt: number; + date: number; + expiresAt: number; + }, existing: any, incoming: any): { + fetchedAt: number; + date: number; + expiresAt: number; + }; + normalize(input: any, parent: any, key: any, args: any[], visit: Visit, delegate: INormalizeDelegate, parentEntity: any): any; + denormalize(input: any, delegate: IDenormalizeDelegate): any; + /** + * Returns the cpks of cells matching the current lens, or undefined. + * + * Only consulted when `Scalar` is an endpoint's top-level schema; field + * usage resolves through the parent entity. Relies on `lens` not + * containing the cpk delimiter `|`. + */ + queryKey(args: readonly any[], unvisit: any, delegate: IQueryDelegate): string[] | undefined; +} + type CollectionOptions = ({ /** Defines lookups for Collections nested in other schemas. * @@ -929,8 +1080,7 @@ declare class Array$1 implements SchemaClass { denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): (S extends EntityMap ? T : Denormalize)[]; queryKey( @@ -984,8 +1134,7 @@ declare class All< denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): (S extends EntityMap ? T : Denormalize)[]; queryKey( @@ -1023,11 +1172,7 @@ declare class Object$1< _denormalizeNullable(): DenormalizeNullableObject; - denormalize( - input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, - ): DenormalizeObject; + denormalize(input: {}, delegate: IDenormalizeDelegate): DenormalizeObject; queryKey( args: ObjectArgs, @@ -1115,8 +1260,7 @@ interface UnionInstance< denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): AbstractInstanceType; queryKey( @@ -1198,8 +1342,7 @@ declare class Values implements SchemaClass { denormalize( input: {}, - args: readonly any[], - unvisit: (schema: any, input: any) => any, + delegate: IDenormalizeDelegate, ): Record< string, Choices extends EntityMap ? T : Denormalize @@ -1254,6 +1397,8 @@ type schema_d_Query, ...args: any) => any> = Query; declare const schema_d_Query: typeof Query; +type schema_d_Scalar = Scalar; +declare const schema_d_Scalar: typeof Scalar; type schema_d_SchemaAttributeFunction = SchemaAttributeFunction; type schema_d_SchemaClass = SchemaClass; type schema_d_SchemaFunction = SchemaFunction; @@ -1271,7 +1416,7 @@ type schema_d_Values = Values; declare const schema_d_Values: typeof Values; declare const schema_d_unshift: typeof unshift; declare namespace schema_d { - export { schema_d_All as All, Array$1 as Array, schema_d_Collection as Collection, type schema_d_CollectionArrayAdder as CollectionArrayAdder, type schema_d_CollectionArrayOrValuesAdder as CollectionArrayOrValuesAdder, type schema_d_CollectionConstructor as CollectionConstructor, type schema_d_CollectionFromSchema as CollectionFromSchema, type schema_d_CollectionInterface as CollectionInterface, schema_d_CollectionRoot as CollectionRoot, type schema_d_CollectionValuesAdder as CollectionValuesAdder, type schema_d_DefaultArgs as DefaultArgs, EntityMixin as Entity, type schema_d_EntityInterface as EntityInterface, type schema_d_EntityMap as EntityMap, schema_d_EntityMixin as EntityMixin, schema_d_Invalidate as Invalidate, schema_d_Lazy as Lazy, type schema_d_MergeFunction as MergeFunction, Object$1 as Object, schema_d_Query as Query, type schema_d_SchemaAttributeFunction as SchemaAttributeFunction, type schema_d_SchemaClass as SchemaClass, type schema_d_SchemaFunction as SchemaFunction, type schema_d_StrategyFunction as StrategyFunction, schema_d_Union as Union, type schema_d_UnionConstructor as UnionConstructor, type schema_d_UnionInstance as UnionInstance, type schema_d_UnionResult as UnionResult, schema_d_UnionRoot as UnionRoot, schema_d_Values as Values, schema_d_unshift as unshift }; + export { schema_d_All as All, Array$1 as Array, schema_d_Collection as Collection, type schema_d_CollectionArrayAdder as CollectionArrayAdder, type schema_d_CollectionArrayOrValuesAdder as CollectionArrayOrValuesAdder, type schema_d_CollectionConstructor as CollectionConstructor, type schema_d_CollectionFromSchema as CollectionFromSchema, type schema_d_CollectionInterface as CollectionInterface, schema_d_CollectionRoot as CollectionRoot, type schema_d_CollectionValuesAdder as CollectionValuesAdder, type schema_d_DefaultArgs as DefaultArgs, EntityMixin as Entity, type schema_d_EntityInterface as EntityInterface, type schema_d_EntityMap as EntityMap, schema_d_EntityMixin as EntityMixin, schema_d_Invalidate as Invalidate, schema_d_Lazy as Lazy, type schema_d_MergeFunction as MergeFunction, Object$1 as Object, schema_d_Query as Query, schema_d_Scalar as Scalar, type schema_d_SchemaAttributeFunction as SchemaAttributeFunction, type schema_d_SchemaClass as SchemaClass, type schema_d_SchemaFunction as SchemaFunction, type schema_d_StrategyFunction as StrategyFunction, schema_d_Union as Union, type schema_d_UnionConstructor as UnionConstructor, type schema_d_UnionInstance as UnionInstance, type schema_d_UnionResult as UnionResult, schema_d_UnionRoot as UnionRoot, schema_d_Values as Values, schema_d_unshift as unshift }; } declare const Entity_base: IEntityClass { @@ -1314,7 +1459,7 @@ declare abstract class Entity extends Entity_base { * @see https://dataclient.io/rest/api/Entity#process */ static process(input: any, parent: any, key: string | undefined, args: any[]): any; - static denormalize: (this: T, input: any, args: readonly any[], unvisit: (schema: any, input: any) => any) => AbstractInstanceType; + static denormalize: (this: T, input: any, delegate: IDenormalizeDelegate) => AbstractInstanceType; } declare function validateRequired(processedEntity: any, requiredDefaults: Record): string | undefined; @@ -2222,4 +2367,4 @@ declare function useController(): Controller; declare function useLive>(endpoint: E, ...args: readonly [...Parameters]): E['schema'] extends undefined | null ? ResolveType$1 : Denormalize$1; declare function useLive>(endpoint: E, ...args: readonly [...Parameters] | readonly [null]): E['schema'] extends undefined | null ? ResolveType$1 | undefined : DenormalizeNullable$1; -export { type AbstractInstanceType, type AddEndpoint, All, Array$1 as Array, _default as AsyncBoundary, type CheckLoop, Collection, type CollectionOptions, type ContentType, type CustomResource, DataProvider, type DefaultArgs, type Defaults, type Denormalize, type DenormalizeNullable, type DenormalizeNullableObject, type DenormalizeObject, Endpoint, type EndpointExtendOptions, type EndpointExtraOptions, type EndpointInstance, type EndpointInstanceInterface, type EndpointInterface, type EndpointOptions, type EndpointParam, type EndpointToFunction, type EntitiesInterface, type EntitiesPath, Entity, type EntityFields, type EntityInterface, type EntityMap, EntityMixin, type EntityPath, type EntityTable, type ErrorTypes$1 as ErrorTypes, type ExpiryStatusInterface, ExtendableEndpoint, type ExtendedResource, type FetchFunction, type FetchGet, type FetchMutate, type FromFallBack, type GetEndpoint, type GetEntity, type GetIndex, type HookResource, type HookableEndpointInterface, type IEntityClass, type IEntityInstance, type INormalizeDelegate, type IQueryDelegate, type RestEndpoint$1 as IRestEndpoint, type IndexPath, Invalidate, type KeyofEndpointInstance, type KeyofRestEndpoint, type KeysToArgs, Lazy, type Mergeable, type MethodToSide, type MoveEndpoint, type MutateEndpoint, type NI, NetworkError, ErrorBoundary as NetworkErrorBoundary, type Normalize, type NormalizeNullable, type NormalizeObject, type NormalizedEntity, type NormalizedIndex, type NormalizedNullableObject, Object$1 as Object, type ObjectArgs, type OptionsToFunction, type PaginationEndpoint, type PaginationFieldEndpoint, type ParamFetchNoBody, type ParamFetchWithBody, type ParamToArgs, type PartialRestGenerics, type PathArgs, type PathArgsAndSearch, type PathKeys, type PolymorphicInterface, Query, type Queryable, type ReadEndpoint, type RecordClass, type RemoveEndpoint, type ResolveType, type Resource, type ResourceEndpointExtensions, type ResourceExtension, type ResourceGenerics, type ResourceInterface, type ResourceOptions, RestEndpoint, type RestEndpointConstructor, type RestEndpointConstructorOptions, type RestEndpointExtendOptions, type RestEndpointOptions, type RestExtendedEndpoint, type RestFetch, type RestGenerics, type RestInstance, type RestInstanceBase, type RestType, type RestTypeNoBody, type RestTypeWithBody, type Schema, type SchemaArgs, type SchemaClass, type SchemaSimple, type Serializable, type ShortenPath, type SnapshotInterface, Union, type UnknownError, Values, type Visit, resource as createResource, getUrlBase, getUrlTokens, hookifyResource, resource, schema_d as schema, unshift, useCache, useController, useDLE, useError, useFetch, useLive, useQuery, useSubscription, useSuspense, validateRequired }; +export { type AbstractInstanceType, type AddEndpoint, All, Array$1 as Array, _default as AsyncBoundary, type CheckLoop, Collection, type CollectionOptions, type ContentType, type CustomResource, DataProvider, type DefaultArgs, type Defaults, type Denormalize, type DenormalizeNullable, type DenormalizeNullableObject, type DenormalizeObject, Endpoint, type EndpointExtendOptions, type EndpointExtraOptions, type EndpointInstance, type EndpointInstanceInterface, type EndpointInterface, type EndpointOptions, type EndpointParam, type EndpointToFunction, type EntitiesInterface, type EntitiesPath, Entity, type EntityFields, type EntityInterface, type EntityMap, EntityMixin, type EntityPath, type EntityTable, type ErrorTypes$1 as ErrorTypes, type ExpiryStatusInterface, ExtendableEndpoint, type ExtendedResource, type FetchFunction, type FetchGet, type FetchMutate, type FromFallBack, type GetEndpoint, type GetEntity, type GetIndex, type HookResource, type HookableEndpointInterface, type IDenormalizeDelegate, type IEntityClass, type IEntityInstance, type INormalizeDelegate, type IQueryDelegate, type RestEndpoint$1 as IRestEndpoint, type IndexPath, Invalidate, type KeyofEndpointInstance, type KeyofRestEndpoint, type KeysToArgs, Lazy, type Mergeable, type MethodToSide, type MoveEndpoint, type MutateEndpoint, type NI, NetworkError, ErrorBoundary as NetworkErrorBoundary, type Normalize, type NormalizeNullable, type NormalizeObject, type NormalizedEntity, type NormalizedIndex, type NormalizedNullableObject, Object$1 as Object, type ObjectArgs, type OptionsToFunction, type PaginationEndpoint, type PaginationFieldEndpoint, type ParamFetchNoBody, type ParamFetchWithBody, type ParamToArgs, type PartialRestGenerics, type PathArgs, type PathArgsAndSearch, type PathKeys, type PolymorphicInterface, Query, type Queryable, type ReadEndpoint, type RecordClass, type RemoveEndpoint, type ResolveType, type Resource, type ResourceEndpointExtensions, type ResourceExtension, type ResourceGenerics, type ResourceInterface, type ResourceOptions, RestEndpoint, type RestEndpointConstructor, type RestEndpointConstructorOptions, type RestEndpointExtendOptions, type RestEndpointOptions, type RestExtendedEndpoint, type RestFetch, type RestGenerics, type RestInstance, type RestInstanceBase, type RestType, type RestTypeNoBody, type RestTypeWithBody, Scalar, type Schema, type SchemaArgs, type SchemaClass, type SchemaSimple, type Serializable, type ShortenPath, type SnapshotInterface, Union, type UnknownError, Values, type Visit, resource as createResource, getUrlBase, getUrlTokens, hookifyResource, resource, schema_d as schema, unshift, useCache, useController, useDLE, useError, useFetch, useLive, useQuery, useSubscription, useSuspense, validateRequired }; diff --git a/website/src/fixtures/companies.ts b/website/src/fixtures/companies.ts new file mode 100644 index 000000000000..41d42e9c72ac --- /dev/null +++ b/website/src/fixtures/companies.ts @@ -0,0 +1,57 @@ +import { RestEndpoint } from '@data-client/rest'; + +const getCompanies = new RestEndpoint({ + path: '/companies', + searchParams: {} as { portfolio: string }, +}); +const getPortfolioColumns = new RestEndpoint({ + path: '/companies/columns', + searchParams: {} as { portfolio: string }, +}); + +const companiesByPortfolio: Record< + string, + { + id: string; + name: string; + price: number; + pct_equity: number; + shares: number; + }[] +> = { + A: [ + { + id: '1', + name: 'Acme Corp', + price: 145.2, + pct_equity: 0.5, + shares: 10000, + }, + { id: '2', name: 'Globex', price: 89.5, pct_equity: 0.2, shares: 4000 }, + { id: '3', name: 'Initech', price: 32.1, pct_equity: 0.1, shares: 2500 }, + ], + B: [ + { id: '1', name: 'Acme Corp', price: 145.2, pct_equity: 0.3, shares: 6000 }, + { id: '2', name: 'Globex', price: 89.5, pct_equity: 0.4, shares: 8000 }, + { id: '3', name: 'Initech', price: 32.1, pct_equity: 0.05, shares: 1200 }, + ], +}; + +export const companyFixtures = [ + { + endpoint: getCompanies, + response({ portfolio }: { portfolio: string }) { + return companiesByPortfolio[portfolio] ?? []; + }, + delay: 150, + }, + { + endpoint: getPortfolioColumns, + response({ portfolio }: { portfolio: string }) { + return (companiesByPortfolio[portfolio] ?? []).map( + ({ id, pct_equity, shares }) => ({ id, pct_equity, shares }), + ); + }, + delay: 150, + }, +]; diff --git a/website/static/codemods/__tests__/v0.17.test.js b/website/static/codemods/__tests__/v0.17.test.js new file mode 100644 index 000000000000..856ce55d787d --- /dev/null +++ b/website/static/codemods/__tests__/v0.17.test.js @@ -0,0 +1,253 @@ +'use strict'; + +const { defineInlineTest } = require('jscodeshift/dist/testUtils'); + +const transform = require('../v0.17'); + +describe('v0.17 codemod', () => { + describe('class denormalize methods', () => { + defineInlineTest( + transform, + { path: 'MySchema.ts' }, + ` +import { Entity } from '@data-client/rest'; + +class MySchema { + denormalize(input: {}, args: readonly any[], unvisit: (s: any, v: any) => any) { + return unvisit(this.schema, input); + } +} + `, + ` +import { Entity, type IDenormalizeDelegate } from '@data-client/rest'; + +class MySchema { + denormalize(input: {}, delegate: IDenormalizeDelegate) { + return delegate.unvisit(this.schema, input); + } +} + `, + 'rewrites class denormalize and adds IDenormalizeDelegate import', + ); + + defineInlineTest( + transform, + { path: 'MySchema.js' }, + ` +import { Entity } from '@data-client/rest'; + +class MySchema { + denormalize(input, args, unvisit) { + return unvisit(this.schema, input); + } +} + `, + ` +import { Entity } from '@data-client/rest'; + +class MySchema { + denormalize(input, delegate) { + return delegate.unvisit(this.schema, input); + } +} + `, + 'rewrites class denormalize without adding type import in JS files', + ); + + defineInlineTest( + transform, + { path: 'PassThrough.ts' }, + ` +import { Entity } from '@data-client/rest'; + +class Wrapper { + denormalize(input: {}, args: readonly any[], unvisit: any) { + return this.inner.denormalize(input, args, unvisit); + } +} + `, + ` +import { Entity, type IDenormalizeDelegate } from '@data-client/rest'; + +class Wrapper { + denormalize(input: {}, delegate: IDenormalizeDelegate) { + return this.inner.denormalize(input, delegate); + } +} + `, + 'rewrites pass-through .denormalize(input, args, unvisit) calls', + ); + + defineInlineTest( + transform, + { path: 'WithArgs.ts' }, + ` +import { Entity } from '@data-client/rest'; + +class WithArgs { + denormalize(input: {}, args: readonly any[], unvisit: any) { + const value = unvisit(this.schema, input); + return this.process(value, ...args); + } +} + `, + ` +import { Entity, type IDenormalizeDelegate } from '@data-client/rest'; + +class WithArgs { + denormalize(input: {}, delegate: IDenormalizeDelegate) { + const value = delegate.unvisit(this.schema, input); + return this.process(value, ...delegate.args); + } +} + `, + 'rewrites bare args references inside the body', + ); + }); + + describe('TypeScript signatures', () => { + defineInlineTest( + transform, + { path: 'iface.ts' }, + ` +import { Entity } from '@data-client/rest'; + +interface MySchema { + denormalize(input: {}, args: readonly any[], unvisit: (s: any, v: any) => any): any; +} + `, + ` +import { Entity, type IDenormalizeDelegate } from '@data-client/rest'; + +interface MySchema { + denormalize(input: {}, delegate: IDenormalizeDelegate): any +} + `, + 'rewrites TS interface method signature', + ); + + defineInlineTest( + transform, + { path: 'lazy.ts' }, + ` +import { Entity } from '@data-client/rest'; + +class Lazy { + declare _denormalizeNullable: ( + input: {}, + args: readonly any[], + unvisit: (schema: any, input: any) => any, + ) => any; +} + `, + ` +import { Entity, type IDenormalizeDelegate } from '@data-client/rest'; + +class Lazy { + declare _denormalizeNullable: (input: {}, delegate: IDenormalizeDelegate) => any; +} + `, + 'rewrites class field with TSFunctionType annotation', + ); + }); + + describe('safety', () => { + defineInlineTest( + transform, + { path: 'Other.ts' }, + ` +class Other { + denormalize(input: any, args: any[], unvisit: any) { + return unvisit(this.schema, input); + } +} + `, + ` +class Other { + denormalize(input: any, args: any[], unvisit: any) { + return unvisit(this.schema, input); + } +} + `, + 'skips files without @data-client imports', + ); + + defineInlineTest( + transform, + { path: 'TwoArg.ts' }, + ` +import { Entity } from '@data-client/rest'; + +class TwoArg { + denormalize(input: any, delegate: any) { + return delegate.unvisit(this.schema, input); + } +} + `, + ` +import { Entity } from '@data-client/rest'; + +class TwoArg { + denormalize(input: any, delegate: any) { + return delegate.unvisit(this.schema, input); + } +} + `, + 'skips already-migrated 2-arg signatures', + ); + + defineInlineTest( + transform, + { path: 'Aliased.ts' }, + ` +import { Entity } from '@data-client/rest'; + +class Aliased { + denormalize(input: any, _args: readonly any[], _unvisit: any) { + return input; + } +} + `, + ` +import { Entity, type IDenormalizeDelegate } from '@data-client/rest'; + +class Aliased { + denormalize(input: any, delegate: IDenormalizeDelegate) { + return input; + } +} + `, + 'still rewrites underscore-prefixed param names', + ); + + // Shadow handling is intentionally narrow: only nested function + // parameters that re-bind 'args' / 'unvisit' are detected. + // Block-scoped re-bindings inside the same body are NOT detected + // and will be rewritten. This test locks that contract. + defineInlineTest( + transform, + { path: 'NestedFn.ts' }, + ` +import { Entity } from '@data-client/rest'; + +class Outer { + denormalize(input: any, args: readonly any[], unvisit: any) { + const inner = (unvisit: any) => unvisit(input); + return inner(unvisit); + } +} + `, + ` +import { Entity, type IDenormalizeDelegate } from '@data-client/rest'; + +class Outer { + denormalize(input: any, delegate: IDenormalizeDelegate) { + const inner = (unvisit: any) => unvisit(input); + return inner(delegate.unvisit); + } +} + `, + 'preserves nested-function shadowing of unvisit param', + ); + }); +}); diff --git a/website/static/codemods/v0.17.js b/website/static/codemods/v0.17.js new file mode 100644 index 000000000000..0c9c827846ff --- /dev/null +++ b/website/static/codemods/v0.17.js @@ -0,0 +1,432 @@ +'use strict'; + +/** + * @data-client v0.17 migration codemod + * + * Transforms: + * 1. Schema.denormalize(input, args, unvisit) → denormalize(input, delegate) + * - Class methods, object methods, function declarations + * - TypeScript method signatures and property types + * - Renames usages: unvisit(...) → delegate.unvisit(...), args → delegate.args + * - Pass-through calls foo.denormalize(input, args, unvisit) + * → foo.denormalize(input, delegate) when delegate is in scope + * 2. Adds `IDenormalizeDelegate` import from `@data-client/endpoint` (or + * `@data-client/rest` / `@data-client/normalizr` if those are already used) + * when type annotations are required. + * + * Usage: + * npx jscodeshift -t https://dataclient.io/codemods/v0.17.js --extensions=ts,tsx,js,jsx src/ + */ + +const DATA_CLIENT_PREFIX = '@data-client/'; +const DELEGATE_TYPE_NAME = 'IDenormalizeDelegate'; + +function hasDataClientImport(j, root) { + return ( + root.find(j.ImportDeclaration).filter(p => { + const v = p.node.source.value; + return typeof v === 'string' && v.startsWith(DATA_CLIENT_PREFIX); + }).length > 0 + ); +} + +// --- helpers -------------------------------------------------------------- + +function getParamName(param) { + if (!param) return undefined; + if (param.type === 'Identifier') return param.name; + if (param.type === 'AssignmentPattern' && param.left.type === 'Identifier') + return param.left.name; + if (param.type === 'TSParameterProperty' && param.parameter) + return getParamName(param.parameter); + return undefined; +} + +function isDenormalizeKey(key) { + if (!key) return false; + if (key.type === 'Identifier') return key.name === 'denormalize'; + if (key.type === 'StringLiteral' || key.type === 'Literal') + return key.value === 'denormalize'; + return false; +} + +// Broader for type-only declarations (class fields, property signatures): +// matches `denormalize`, `_denormalize`, `_denormalizeNullable`, etc. +const DENORMALIZE_LIKE = /^_?denormalize(Nullable)?$/; +function isDenormalizeLikeKey(key) { + if (!key) return false; + if (key.type === 'Identifier') return DENORMALIZE_LIKE.test(key.name); + if (key.type === 'StringLiteral' || key.type === 'Literal') + return DENORMALIZE_LIKE.test(String(key.value)); + return false; +} + +function buildDelegateParam(j, withTypes) { + const id = j.identifier('delegate'); + if (withTypes) { + id.typeAnnotation = j.tsTypeAnnotation( + j.tsTypeReference(j.identifier(DELEGATE_TYPE_NAME)), + ); + } + return id; +} + +function paramsLookLikeDenormalize(params) { + if (!params || params.length !== 3) return false; + const argsName = getParamName(params[1]); + const unvisitName = getParamName(params[2]); + if (argsName === 'args' && unvisitName === 'unvisit') return true; + if (argsName === '_args' && unvisitName === '_unvisit') return true; + // Typed signature where 3rd param looks like a function type. + const ta3 = params[2] && params[2].typeAnnotation; + const tn = ta3 && (ta3.typeAnnotation || ta3); + if ( + tn && + (tn.type === 'TSFunctionType' || tn.type === 'FunctionTypeAnnotation') + ) + return true; + return false; +} + +// Rewrite identifier references inside a method body NodePath: +// → delegate.args +// (...) → delegate.unvisit(...) +// foo.denormalize(x, args, unvisit) → foo.denormalize(x, delegate) +function rewriteBody(j, methodPath, argsName, unvisitName) { + const bodyCol = j(methodPath).find(j.BlockStatement).at(0); + if (!bodyCol.length) return; + const bodyPath = bodyCol.paths()[0]; + const body = j(bodyPath); + + function isShadowed(p, name) { + let cur = p; + while (cur && cur.parent) { + cur = cur.parent; + if (cur.node === bodyPath.node) return false; + const t = cur.node.type; + if ( + t === 'FunctionDeclaration' || + t === 'FunctionExpression' || + t === 'ArrowFunctionExpression' || + t === 'ObjectMethod' || + t === 'ClassMethod' || + t === 'MethodDefinition' + ) { + const params = + (cur.node.params && cur.node.params) || + (cur.node.value && cur.node.value.params) || + []; + if (params.some(prm => getParamName(prm) === name)) return true; + } + } + return false; + } + + // 1. Rewrite pass-through calls: x.denormalize(input, args, unvisit) → x.denormalize(input, delegate) + body + .find(j.CallExpression) + .filter(p => { + const callee = p.node.callee; + if (!callee || callee.type !== 'MemberExpression') return false; + if (!isDenormalizeKey(callee.property)) return false; + const args = p.node.arguments; + if (args.length !== 3) return false; + return ( + args[1].type === 'Identifier' && + args[1].name === argsName && + args[2].type === 'Identifier' && + args[2].name === unvisitName + ); + }) + .forEach(p => { + if (isShadowed(p, argsName) || isShadowed(p, unvisitName)) return; + p.node.arguments = [p.node.arguments[0], j.identifier('delegate')]; + }); + + // 2. unvisit(...) → delegate.unvisit(...) + if (unvisitName) { + body + .find(j.CallExpression, { + callee: { type: 'Identifier', name: unvisitName }, + }) + .forEach(p => { + if (isShadowed(p, unvisitName)) return; + p.node.callee = j.memberExpression( + j.identifier('delegate'), + j.identifier('unvisit'), + ); + }); + + // bare reference: passing `unvisit` to another fn → `delegate.unvisit` + body + .find(j.Identifier, { name: unvisitName }) + .filter(p => { + const parent = p.parent.node; + if (!parent) return false; + if ( + parent.type === 'MemberExpression' && + parent.property === p.node && + !parent.computed + ) + return false; + if ( + (parent.type === 'Property' || parent.type === 'ObjectProperty') && + parent.key === p.node && + !parent.computed + ) + return false; + if (parent.type === 'CallExpression' && parent.callee === p.node) + return false; + if (parent.type === 'VariableDeclarator' && parent.id === p.node) + return false; + return true; + }) + .forEach(p => { + if (isShadowed(p, unvisitName)) return; + j(p).replaceWith( + j.memberExpression(j.identifier('delegate'), j.identifier('unvisit')), + ); + }); + } + + // 3. bare `args` → `delegate.args` + if (argsName) { + body + .find(j.Identifier, { name: argsName }) + .filter(p => { + const parent = p.parent.node; + if (!parent) return false; + if ( + parent.type === 'MemberExpression' && + parent.property === p.node && + !parent.computed + ) + return false; + if ( + (parent.type === 'Property' || parent.type === 'ObjectProperty') && + parent.key === p.node && + !parent.computed + ) + return false; + if (parent.type === 'VariableDeclarator' && parent.id === p.node) + return false; + if ( + parent.type === 'TSTypeReference' || + parent.type === 'TSQualifiedName' + ) + return false; + return true; + }) + .forEach(p => { + if (isShadowed(p, argsName)) return; + j(p).replaceWith( + j.memberExpression(j.identifier('delegate'), j.identifier('args')), + ); + }); + } +} + +// --- runtime denormalize methods (class/object/function) ------------------- + +function transformDenormalizeMethods(j, root, state) { + let dirty = false; + + function processFunctionLike(methodPath, paramsHolder) { + const params = paramsHolder.params; + if (!paramsLookLikeDenormalize(params)) return false; + const argsName = getParamName(params[1]); + const unvisitName = getParamName(params[2]); + + paramsHolder.params = [params[0], buildDelegateParam(j, state.useTypes)]; + rewriteBody(j, methodPath, argsName, unvisitName); + + if (state.useTypes) state.needsImport = true; + return true; + } + + // ClassMethod (Babel/tsx parser) + if (j.ClassMethod) { + root.find(j.ClassMethod).forEach(p => { + if (!isDenormalizeKey(p.node.key)) return; + if (processFunctionLike(p, p.node)) dirty = true; + }); + } + + // MethodDefinition (ESTree) + if (j.MethodDefinition) { + root.find(j.MethodDefinition).forEach(p => { + if (!isDenormalizeKey(p.node.key)) return; + const fn = p.node.value; + if (!fn) return; + // For MethodDefinition, search inside the FunctionExpression value. + const fnPath = p.get('value'); + if (processFunctionLike(fnPath, fn)) dirty = true; + }); + } + + // ObjectMethod (Babel) + if (j.ObjectMethod) { + root.find(j.ObjectMethod).forEach(p => { + if (!isDenormalizeKey(p.node.key)) return; + if (processFunctionLike(p, p.node)) dirty = true; + }); + } + + // Property: { denormalize: function(...) {} } and { denormalize: (...) => {} } + root + .find(j.Property) + .filter(p => { + if (!isDenormalizeKey(p.node.key)) return false; + const v = p.node.value; + return ( + v && + (v.type === 'FunctionExpression' || + v.type === 'ArrowFunctionExpression') + ); + }) + .forEach(p => { + const fn = p.node.value; + const fnPath = p.get('value'); + if (processFunctionLike(fnPath, fn)) dirty = true; + }); + + // Top-level: function denormalize(input, args, unvisit) { ... } + root.find(j.FunctionDeclaration).forEach(p => { + if (!p.node.id || p.node.id.name !== 'denormalize') return; + if (processFunctionLike(p, p.node)) dirty = true; + }); + + return dirty; +} + +// --- TS interface / type signatures --------------------------------------- + +function transformTypeSignatures(j, root, state) { + if (!j.TSMethodSignature && !j.TSPropertySignature) return false; + let dirty = false; + + if (j.TSMethodSignature) { + root.find(j.TSMethodSignature).forEach(p => { + if (!isDenormalizeKey(p.node.key)) return; + const params = p.node.parameters || p.node.params; + if (!paramsLookLikeDenormalize(params)) return; + const newParams = [params[0], buildDelegateParam(j, true)]; + if (p.node.parameters) p.node.parameters = newParams; + else p.node.params = newParams; + state.needsImport = true; + dirty = true; + }); + } + + function rewriteFnTypeNode(fn) { + if (!fn) return false; + if (fn.type !== 'TSFunctionType') return false; + const params = fn.parameters; + if (!paramsLookLikeDenormalize(params)) return false; + fn.parameters = [params[0], buildDelegateParam(j, true)]; + state.needsImport = true; + return true; + } + + if (j.TSPropertySignature) { + root.find(j.TSPropertySignature).forEach(p => { + if (!isDenormalizeLikeKey(p.node.key)) return; + const ann = p.node.typeAnnotation; + if (!ann || !ann.typeAnnotation) return; + if (rewriteFnTypeNode(ann.typeAnnotation)) dirty = true; + }); + } + + ['ClassProperty', 'PropertyDefinition'].forEach(t => { + if (!j[t]) return; + root.find(j[t]).forEach(p => { + if (!isDenormalizeLikeKey(p.node.key)) return; + const ann = p.node.typeAnnotation; + if (!ann || !ann.typeAnnotation) return; + if (rewriteFnTypeNode(ann.typeAnnotation)) dirty = true; + }); + }); + + return dirty; +} + +// --- Import injection ----------------------------------------------------- + +function ensureDelegateImport(j, root) { + let alreadyImported = false; + root.find(j.ImportDeclaration).forEach(p => { + if (alreadyImported) return; + p.node.specifiers.forEach(s => { + if ( + s.type === 'ImportSpecifier' && + s.imported && + s.imported.name === DELEGATE_TYPE_NAME + ) { + alreadyImported = true; + } + }); + }); + if (alreadyImported) return false; + + const preferred = [ + '@data-client/endpoint', + '@data-client/rest', + '@data-client/normalizr', + ]; + let target = null; + for (const name of preferred) { + const found = root + .find(j.ImportDeclaration) + .filter(p => p.node.source.value === name); + if (found.length) { + target = found.paths()[0]; + break; + } + } + if (!target) { + const newImport = j.importDeclaration( + [j.importSpecifier(j.identifier(DELEGATE_TYPE_NAME))], + j.stringLiteral('@data-client/endpoint'), + ); + newImport.importKind = 'type'; + const allImports = root.find(j.ImportDeclaration); + if (allImports.length) { + const paths = allImports.paths(); + j(paths[paths.length - 1]).insertAfter(newImport); + } else { + root.get().node.program.body.unshift(newImport); + } + return true; + } + + // Inline `type` keyword so this works under verbatimModuleSyntax / + // consistent-type-imports without importing a value at runtime. + const spec = j.importSpecifier(j.identifier(DELEGATE_TYPE_NAME)); + spec.importKind = 'type'; + target.node.specifiers.push(spec); + return true; +} + +// --- Main ----------------------------------------------------------------- + +module.exports = function transformer(fileInfo, api) { + const j = api.jscodeshift; + const root = j(fileInfo.source); + + if (!hasDataClientImport(j, root)) return undefined; + + const isTs = /\.tsx?$/.test(fileInfo.path || ''); + const state = { useTypes: isTs, needsImport: false }; + + let dirty = false; + dirty = transformDenormalizeMethods(j, root, state) || dirty; + dirty = transformTypeSignatures(j, root, state) || dirty; + + if (state.needsImport) { + ensureDelegateImport(j, root); + } + + return dirty ? root.toSource({ quote: 'single' }) : undefined; +}; + +module.exports.parser = 'tsx';