fix(seed-loader): multi-value lookup natural keys, and count the references they used to lose (#3911, #3932)

.changeset/seed-loader-dropped-reference-counter.md

@@ -0,0 +1,33 @@
+---
+"@objectstack/spec": patch
+"@objectstack/metadata-protocol": patch
+"@objectstack/runtime": patch
+"@objectstack/cli": patch
+"@objectstack/cloud-connection": patch
+---
+
+fix(seed-loader): count reference fields dropped from rows that were still written
+
+The loader had two failure outcomes and only counted one. A record it cannot
+write is counted in `errored`. But an unusable **reference value** (an object
+where a natural key belongs, an array on a single-value field) is removed from
+the record — never written as NULL, which would sever an existing link on
+upsert replay — and the row is written **without it**. Nothing counted that.
+
+So a load that quietly severed N associations reported `totalErrored: 0`, and
+every count-driven surface read clean. The CLI boot banner — the one seed signal
+that survives `os dev`'s boot-quiet window and the default `warn` level — printed
+`showcase 42 rows`, and the warn line said `0 dropped record(s)`: true, and
+useless ([#3932](https://github.com/objectstack-ai/objectstack/issues/3932)).
+
+`SeedLoadResult.referencesDropped` and `SeedLoaderSummary.totalReferencesDropped`
+now count it. It is deliberately **not** folded into `errored` — the row *was*
+written, so that would break the `inserted + updated + skipped` reconciliation
+against `total`. The banner names it separately:
+
+```
+⚠ Seeds:   showcase 42 ok / 3 lost links ⚠
+```
+
+Both counters are additive with a `0` default, so an existing producer or
+consumer of `SeedLoaderResult` is unaffected.

.changeset/seed-loader-multi-value-lookup.md

@@ -0,0 +1,41 @@
+---
+"@objectstack/spec": patch
+"@objectstack/metadata-protocol": patch
+---
+
+fix(seed-loader): resolve natural-key ARRAYS for multi-value lookups
+
+A `multiple: true` lookup / `user` field stores an array of ids, so its seed
+value is an array of natural keys (`authors: ['Alice', 'Bob']`). Reference
+resolution only ever accepted a single string: the array tripped the
+"expected a natural-key string but got an object. Pass the target's `name`
+value as a plain string" guard — impossible advice for a field that holds
+several references — and was then DROPPED from the record. The row landed with
+the whole association missing and only a warn in the log
+([#3911](https://github.com/objectstack-ai/objectstack/issues/3911)).
+
+Every element now resolves independently (in-load records first, then the
+database, then pass 2), and the field lands as an array of target ids. A lone
+string is accepted as one-element shorthand for the array shape the field
+stores. Deferral is all-or-nothing per field — a partially-resolved array is a
+corrupt association, so pass 2 re-resolves the whole authored array — and a key
+that never materializes is a reported load error naming that element, not a
+silent drop.
+
+An array passed to a genuinely **single-value** reference field is still
+rejected, now with advice an author can act on: declare the field
+`multiple: true`, or pass one natural key.
+
+`ReferenceResolution` (`@objectstack/spec/data`) gains an optional `multiple`
+flag carrying the field's array-ness into resolution; it is additive and
+defaulted-absent, so existing dependency graphs are unaffected.
+
+**Authoring types.** `defineSeed`'s per-field value type now widens a
+`multiple: true` lookup to `string | string[] | null` (a lone string stays legal
+— the loader accepts it as one-element shorthand). `master_detail` is inherently
+single and is not widened, and an array on a single-value lookup is still a
+compile error. To make that reachable, `Field.lookup` became generic over its
+config (`<const C extends FieldInput>`) so `multiple: true` survives as a
+literal instead of widening to `boolean`; the return type is intersected with
+`FieldInput` so its optional surface is unchanged. Type-level only — the
+returned object is byte-identical at runtime.

content/docs/data-modeling/seed-data.mdx

@@ -238,6 +238,36 @@ const contactsSeed = defineSeed(Contact, {
 export const SeedData = [accountsSeed, contactsSeed];
 ```
 
+### Multi-Value Lookups (`multiple: true`)
+
+A lookup declared `multiple: true` stores an **array** of related record ids, so its
+seed value is an **array of natural keys**. Each element resolves independently.
+
+```typescript
+const Book = ObjectSchema.create({
+  name: 'book',
+  fields: {
+    name: Field.text({ label: 'Title', required: true }),
+    authors: Field.lookup('author', { label: 'Authors', multiple: true }),
+  },
+});
+
+const booksSeed = defineSeed(Book, {
+  externalId: 'name',
+  records: [
+    { name: 'Refactoring', authors: ['Alice', 'Bob'] },  // one natural key per author
+  ],
+});
+```
+
+A lone string is accepted as one-element shorthand (`authors: 'Alice'` stores
+`[<alice id>]`). Resolution is **all-or-nothing per field**: if any element is still
+missing, the whole array is deferred to pass 2 rather than written half-resolved, and
+a key that never materializes is reported as a load error naming that element.
+
+Passing an array to a **single-value** lookup is rejected with a load error — declare
+the field `multiple: true`, or pass one natural key.
+
 ### Composite `externalId` — Join / Junction Tables
 
 A join (junction) table linking two objects many-to-many has **no single-field

content/docs/references/data/seed-loader.mdx

@@ -61,7 +61,7 @@ Complete object dependency graph for seed data loading
 
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
-| **nodes** | `{ object: string; dependsOn: string[]; references: { field: string; targetObject: string; targetField: string; fieldType: Enum<'lookup' \| 'master_detail' \| 'user'> }[] }[]` | ✅ | All objects in the dependency graph |
+| **nodes** | `{ object: string; dependsOn: string[]; references: { field: string; targetObject: string; targetField: string; fieldType: Enum<'lookup' \| 'master_detail' \| 'user'>; … }[] }[]` | ✅ | All objects in the dependency graph |
 | **insertOrder** | `string[]` | ✅ | Topologically sorted insert order |
 | **circularDependencies** | `string[][]` | ✅ | Circular dependency chains (e.g., [["a", "b", "a"]]) |
 
@@ -78,7 +78,7 @@ Object node in the seed data dependency graph
 | :--- | :--- | :--- | :--- |
 | **object** | `string` | ✅ | Object name (snake_case) |
 | **dependsOn** | `string[]` | ✅ | Objects this object depends on |
-| **references** | `{ field: string; targetObject: string; targetField: string; fieldType: Enum<'lookup' \| 'master_detail' \| 'user'> }[]` | ✅ | Field-level reference details |
+| **references** | `{ field: string; targetObject: string; targetField: string; fieldType: Enum<'lookup' \| 'master_detail' \| 'user'>; … }[]` | ✅ | Field-level reference details |
 
 
 ---
@@ -95,6 +95,7 @@ Describes how a field reference is resolved during seed loading
 | **targetObject** | `string` | ✅ | Target object name (snake_case) |
 | **targetField** | `string` | ✅ | Field on target object used for matching |
 | **fieldType** | `Enum<'lookup' \| 'master_detail' \| 'user'>` | ✅ | Relationship field type |
+| **multiple** | `boolean` | optional | Field stores an array of references (multiple: true) |
 
 
 ---
@@ -149,6 +150,7 @@ Result of loading a single dataset
 | **total** | `integer` | ✅ | Total records in dataset |
 | **referencesResolved** | `integer` | ✅ | References resolved via externalId |
 | **referencesDeferred** | `integer` | ✅ | References deferred to second pass |
+| **referencesDropped** | `integer` | ✅ | Reference fields dropped from records that were still written |
 | **errors** | `{ sourceObject: string; field: string; targetObject: string; targetField: string; … }[]` | ✅ | Reference resolution errors |
 
 
@@ -199,7 +201,7 @@ Complete seed loader result
 | :--- | :--- | :--- | :--- |
 | **success** | `boolean` | ✅ | Overall success status |
 | **dryRun** | `boolean` | ✅ | Whether this was a dry-run |
-| **dependencyGraph** | `{ nodes: { object: string; dependsOn: string[]; references: { field: string; targetObject: string; targetField: string; fieldType: Enum<'lookup' \| 'master_detail' \| 'user'> }[] }[]; insertOrder: string[]; circularDependencies: string[][] }` | ✅ | Object dependency graph |
+| **dependencyGraph** | `{ nodes: { object: string; dependsOn: string[]; references: { field: string; targetObject: string; targetField: string; fieldType: Enum<'lookup' \| 'master_detail' \| 'user'>; … }[] }[]; insertOrder: string[]; circularDependencies: string[][] }` | ✅ | Object dependency graph |
 | **results** | `{ object: string; mode: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; inserted: integer; updated: integer; … }[]` | ✅ | Per-object load results |
 | **errors** | `{ sourceObject: string; field: string; targetObject: string; targetField: string; … }[]` | ✅ | All reference resolution errors |
 | **summary** | `{ objectsProcessed: integer; totalRecords: integer; totalInserted: integer; totalUpdated: integer; … }` | ✅ | Summary statistics |

examples/app-showcase/src/data/objects/field-zoo.object.ts

@@ -99,10 +99,22 @@ export const FieldZoo = ObjectSchema.create({
 
     // ── Relational ───────────────────────────────────────────────────────
     f_lookup: Field.lookup('showcase_account', { label: 'Lookup → Account' }),
+    // Multi-value reference — stores an ARRAY of account ids (JSON column).
+    // Seeded from an array of natural keys, one per element (framework#3911);
+    // this is the seedable half of the `multiple: true` reference surface —
+    // see `f_users` below for the half that a fresh boot cannot seed.
+    f_lookups: Field.lookup('showcase_account', { label: 'Lookup → Accounts (multiple)', multiple: true }),
     f_master_detail: Field.masterDetail('showcase_project', { label: 'Master-Detail → Project' }),
     f_tree: { type: 'tree', label: 'Tree (self/category)', reference: 'showcase_category' },
 
     // ── User (lookup specialized to sys_user) ────────────────────────────
+    // NOT seeded, and deliberately so: `sys_user` rows are created by SIGN-UP,
+    // not by seeds (see security/seed-approval-demo.ts — "users can't be
+    // seeded (they sign up)"), so on a fresh boot there is no human user for a
+    // natural key to resolve against. Authoring one here would make the whole
+    // showcase seed load report `success: false` on every first boot. The
+    // multi-value REFERENCE mechanics these fields share are demonstrated by
+    // `f_lookups` above; assign these two in the UI once you have signed up.
     f_user: Field.user({ label: 'User → sys_user (single)' }),
     f_users: Field.user({ label: 'Users (multiple)', multiple: true }),
     f_owner: Field.user({ label: 'Owner (current_user default)', defaultValue: 'current_user' }),

examples/app-showcase/src/data/seed/index.ts

@@ -256,6 +256,15 @@ const memberships = defineSeed(ProjectMembership, {
 // Relational/computed fields (lookup/master_detail/tree/record-map, formula/
 // summary/autonumber) resolve or generate at runtime. `f_master_detail` is the
 // owning Project; `f_lookup` the Account — referenced by their seed externalId.
+//
+// `f_lookups` is the MULTI-VALUE reference case (framework#3911): a
+// `multiple: true` lookup stores an array of ids, so the seed value is an
+// ARRAY of natural keys and each element resolves independently. It is also
+// the regression guard for that resolution — before the fix the array was
+// rejected as a non-string reference and the field vanished from the row,
+// leaving the specimen silently missing one relational type.
+// The multi-value USER field (`f_users`) stays unseeded on purpose: sys_user
+// rows come from sign-up, not seeds — see the note on the object.
 const fieldZoo = defineSeed(FieldZoo, {
   mode: 'upsert',
   externalId: 'name',
@@ -272,7 +281,7 @@ const fieldZoo = defineSeed(FieldZoo, {
       f_date: '2026-06-17', f_datetime: '2026-06-17T14:30:00Z', f_time: '14:30',
       f_boolean: true, f_toggle: true,
       f_select: 'high', f_multiselect: ['red', 'green'], f_radio: 'yes', f_checkboxes: ['email', 'push'], f_tags: ['alpha', 'beta'],
-      f_lookup: 'Northwind', f_master_detail: 'Website Relaunch',
+      f_lookup: 'Northwind', f_lookups: ['Northwind', 'Contoso'], f_master_detail: 'Website Relaunch',
       f_location: { lat: 47.6062, lng: -122.3321 }, f_address: { street: '1 Main St', city: 'Seattle', state: 'WA', postal_code: '98101', country: 'US' },
       f_code: '{\n  "ok": true\n}', f_json: { nested: { k: 'v' }, list: [1, 2, 3] }, f_color: '#2563EB',
       f_rating: 4, f_slider: 60, f_progress: 80,

examples/app-showcase/src/system/translations/index.ts

@@ -266,6 +266,7 @@ export const ShowcaseTranslationBundle = {
       },
       showcase_field_zoo: {
         label: '字段动物园', pluralLabel: '字段动物园',
+        fields: { f_lookups: { label: '查找 → 客户（多值）' } },
       },
     },
   },

packages/cli/src/utils/format.seed-summary.test.ts

@@ -47,6 +47,28 @@ describe('printServerReady seed summary (#3415/#3430)', () => {
     expect(lines.some((l) => l.includes('OS_LOG_LEVEL=info'))).toBe(true);
   });
 
+  it('screams about lost links even when every row landed (#3932)', () => {
+    // The "wrote the row, lost the link" case: nothing rejected, the row counts
+    // look perfect, an association is silently missing. This line used to read
+    // `showcase 42 rows` — clean, and wrong.
+    printServerReady({ ...base, seeds: [s({ source: 'showcase', inserted: 42, droppedRefs: 3 })] });
+    expect(seedLines()).toHaveLength(1);
+    expect(seedLines()[0]).toContain('showcase 42 ok / 3 lost links ⚠');
+    expect(lines.some((l) => l.includes('OS_LOG_LEVEL=info'))).toBe(true);
+  });
+
+  it('reports rejected records and lost links together, singular-aware', () => {
+    printServerReady({ ...base, seeds: [s({ source: 'showcase', inserted: 20, rejected: 1, droppedRefs: 1 })] });
+    expect(seedLines()[0]).toContain('showcase 20 ok / 1 error / 1 lost link ⚠');
+  });
+
+  it('stays quiet when no reference was dropped', () => {
+    printServerReady({ ...base, seeds: [s({ source: 'showcase', inserted: 42, droppedRefs: 0 })] });
+    expect(seedLines()[0]).toContain('showcase 42 rows');
+    expect(seedLines()[0]).not.toContain('lost link');
+    expect(seedLines()[0]).not.toContain('⚠');
+  });
+
   it('labels a marketplace package and marks a fresh-DB heal', () => {
     printServerReady({
       ...base,

packages/cli/src/utils/format.ts

@@ -331,6 +331,12 @@ export interface SeedSourceSummary {
   skipped: number;
   /** Records dropped by validation/reference errors — the silent-loss case. */
   rejected: number;
+  /**
+   * Reference FIELDS dropped from rows that WERE written (#3932). The row
+   * count stays healthy — the association is what went missing — so unless
+   * this is said out loud, nothing on this line hints at it.
+   */
+  droppedRefs?: number;
   /**
    * Rows were (re)seeded onto a fresh/empty database during rehydrate — the
    * "swap the DB out from under an installed package" self-heal (#3430).
@@ -460,20 +466,26 @@ function printSeedSummary(sources: SeedSourceSummary[]) {
   const shown = sources.filter((s) => {
     // Empty installs and rejections are ALWAYS shown (they're the whole point);
     // a source that touched no rows and had no problem is noise — drop it.
-    if (s.emptyInstall || s.rejected > 0) return true;
+    if (s.emptyInstall || s.rejected > 0 || (s.droppedRefs ?? 0) > 0) return true;
     return s.inserted + s.updated + s.skipped > 0;
   });
   if (shown.length === 0) return;
 
-  const anyProblem = shown.some((s) => s.rejected > 0 || s.emptyInstall);
+  const anyProblem = shown.some((s) => s.rejected > 0 || (s.droppedRefs ?? 0) > 0 || s.emptyInstall);
 
   const fragment = (s: SeedSourceSummary): string => {
     const label = s.marketplace ? `${s.source}(marketplace)` : s.source;
     if (s.emptyInstall) return `${label} installed but 0 rows ⚠`;
     const ok = s.inserted + s.updated + s.skipped;
+    // A dropped reference leaves the row in place, so it never shows up in the
+    // row counts — name it separately or the line reads clean over a severed
+    // association (#3932).
+    const dropped = s.droppedRefs ?? 0;
+    const lostLinks = dropped > 0 ? ` / ${dropped} lost link${dropped === 1 ? '' : 's'}` : '';
     if (s.rejected > 0) {
-      return `${label} ${ok} ok / ${s.rejected} error${s.rejected === 1 ? '' : 's'} ⚠`;
+      return `${label} ${ok} ok / ${s.rejected} error${s.rejected === 1 ? '' : 's'}${lostLinks} ⚠`;
     }
+    if (dropped > 0) return `${label} ${ok} ok${lostLinks} ⚠`;
     return `${label} ${ok} rows${s.healed ? ' (healed on fresh db)' : ''}`;
   };
 

packages/cloud-connection/src/marketplace-install-local-plugin.ts

@@ -263,6 +263,7 @@ export class MarketplaceInstallLocalPlugin implements Plugin {
                     updated: summary.updated ?? 0,
                     skipped: summary.skipped ?? 0,
                     rejected: summary.errors ?? 0,
+                    droppedRefs: summary.droppedRefs ?? 0,
                     healed: true,
                 });
             } else {
@@ -310,6 +311,7 @@ export class MarketplaceInstallLocalPlugin implements Plugin {
             updated: number;
             skipped: number;
             rejected: number;
+            droppedRefs?: number;
             healed?: boolean;
             emptyInstall?: boolean;
         },
@@ -1014,7 +1016,7 @@ export class MarketplaceInstallLocalPlugin implements Plugin {
         ctx: PluginContext,
         datasets: any[],
         organizationId?: string,
-    ): Promise<{ inserted: number; updated: number; skipped: number; errors: number; errorSample?: string }> => {
+    ): Promise<{ inserted: number; updated: number; skipped: number; errors: number; droppedRefs: number; errorSample?: string }> => {
         const ql: any = ctx.getService('objectql');
         let metadata: any;
         try { metadata = ctx.getService('metadata'); } catch { /* none */ }
@@ -1040,6 +1042,9 @@ export class MarketplaceInstallLocalPlugin implements Plugin {
             updated: result.summary.totalUpdated,
             skipped: result.summary.totalSkipped ?? 0,
             errors: result.errors.length,
+            // Reference fields dropped from rows that WERE written (#3932) —
+            // invisible in every row count, so carried explicitly.
+            droppedRefs: result.summary.totalReferencesDropped ?? 0,
             // Surface the first write/resolution failure so the caller can
             // report WHY nothing landed (e.g. a locked DB, a missing table,
             // a failed validation) instead of a bare "0 rows".