Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .changeset/discovery-data-slot-computed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
'@objectstack/metadata-protocol': patch
'@objectstack/runtime': patch
---

Both discovery builders now derive the `data` service entry from the implementation in the slot, closing the hardcoded "kernel-provided" block (#4130).

#4089 computed `metadata`; `data` was the last entry that judged itself, reporting `status: 'available'` and `handlerReady: true` unconditionally. That was true — but by a convention in a different package, not by anything either builder checked: ObjectQL is the slot's only producer, and plugin-dev always loads `ObjectQLPlugin` as a child, so plugin-dev's `data` stub (`find()` returns `[]`, `insert()` mints an id and stores nothing) never reaches the slot. A second producer, or a trimmed dev config, and the hardcode starts lying about the platform's most load-bearing capability.

Both builders now read the registered service's `__serviceInfo`:

- a real engine carries no marker ⇒ `available` + `handlerReady: true`, byte-identical to the hardcode it replaces (verified on a real kernel boot);
- a self-declared stub ⇒ its own `status` and `message`, with `handlerReady: false` (the default for `stub`), so a consumer that gates on `handlerReady` stops treating an empty query engine as a real one.

`handlerReady` is derived here rather than pinned `true` as it is for `metadata`, because the two routes differ: `/meta` answers from the protocol whatever fills the metadata slot, while `/data` needs the `protocol` or an objectql-shaped service and 503s without them — and the only stack where a stub occupies the `data` slot is one where ObjectQL never registered. No routing, gating or dispatch behavior changes: the `data` domain resolves its engine directly and never consulted this slot.
2 changes: 1 addition & 1 deletion docs/adr/0076-objectql-core-tiering.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ Decision: each capability plugin registers its routes as a **normalized handler*
**Inventory of current fakes / mis-reports:**
- `plugin-dev` registers ~8 dev stubs — `storage` / `search` / `automation` / `graphql` / `analytics` / `realtime` / `notification` / `ai` (they already carry a `_dev: true` marker that nothing respects). *(Update #4000: the `analytics` one is **gone** — after the fallback's retirement (#3891/#3989) it was the last thing refilling that slot, and refilling it re-created the retired shape in dev: the dispatcher gated on service presence, so the stub was called like an engine and answered 200 with fabricated rows. The remaining stubs are unchanged, and the class-wide question they raise is tracked separately — see conclusion 3 below.)* *(Update #4058 step 1: the blanket `_dev: true` is **gone** too. Every dev implementation now carries the standard `__serviceInfo`, classified in one reviewable table (`DEV_STUB_SELF_INFO`) by what it actually is — `degraded` for the ones that really work with reduced capability (`file-storage` / `search` / `metadata` / `workflow` / `realtime`, plus the wrapped kernel fallbacks), `stub` for the ones whose answer is fabricated (`ai` / `automation` / `notification` / `data` / `auth` / `security.*`). The single marker made those two indistinguishable and declared the working ones fake, which is why "adopt the dispatcher gate everywhere?" could not be answered. The gates themselves are untouched — that is #4058 step 2.)*
- `ObjectQLPlugin` registers a ~66-line `analytics` **fallback** (the D10 note — deliberate, but it reports as fully available).
- `http-dispatcher.ts` `svcAvailable` — the hardcode above. *(Update #4089: `svcAvailable` respects the marker, but two entries never went through it at all — a "kernel-provided (always available)" block above the loop, hardcoded per builder and **contradicting itself across them**: the dispatcher declared `metadata` permanently `degraded` with "In-memory registry; DB persistence pending", while metadata-protocol declared the same slot permanently `available`. So one host called a `sys_metadata`-backed registry degraded and the other called the in-memory fallback fully real — the marker #4058 had just added to `createMemoryMetadata` went unread on both sides, because neither side read anything. Both now compute the slot from the registered implementation's `__serviceInfo` and agree; `handlerReady: true` stays unconditional on both, since `/api/v1/meta` is served by the protocol whichever implementation occupies the slot. `data` keeps its hardcode: ObjectQL is the only producer that ever fills it, and `ObjectQLPlugin` — which plugin-dev always loads as a child — registers the real engine, so plugin-dev's `data` stub is unreachable in a stack that has a discovery builder at all.)*
- `http-dispatcher.ts` `svcAvailable` — the hardcode above. *(Update #4089: `svcAvailable` respects the marker, but two entries never went through it at all — a "kernel-provided (always available)" block above the loop, hardcoded per builder and **contradicting itself across them**: the dispatcher declared `metadata` permanently `degraded` with "In-memory registry; DB persistence pending", while metadata-protocol declared the same slot permanently `available`. So one host called a `sys_metadata`-backed registry degraded and the other called the in-memory fallback fully real — the marker #4058 had just added to `createMemoryMetadata` went unread on both sides, because neither side read anything. Both now compute the slot from the registered implementation's `__serviceInfo` and agree; `handlerReady: true` stays unconditional on both, since `/api/v1/meta` is served by the protocol whichever implementation occupies the slot. `data` kept its hardcode in that pass: ObjectQL is the only producer that ever fills it, and `ObjectQLPlugin` — which plugin-dev always loads as a child — registers the real engine, so plugin-dev's `data` stub is unreachable in a stack that has a discovery builder at all.)* *(Update #4130: `data` is computed now too, which closes the block. Its hardcoded `available` was true, but by a **load-order convention in another package** rather than by anything either builder verified — the same shape the `metadata` hardcodes had while they were still "roughly right". Both builders derive the entry from the slot's `__serviceInfo`: an unmarked engine reproduces the old `available` / `handlerReady: true` byte for byte, while plugin-dev's `data` stub — `status: 'stub'`, so `handlerReady` defaults to `false` — is reported as the non-handler it declares itself to be. Unlike `metadata`, `handlerReady` here is NOT pinned `true`: `/meta` answers from the protocol (down to a last-resort default type list) whatever fills its slot, whereas `callData` needs the `protocol` service or an objectql-shaped one and throws 503 without them — and the only way a stub reaches the `data` slot is a stack where ObjectQL never registered, i.e. exactly the stack where `/data` cannot serve. Deliberately NOT done: re-deriving serveability from `protocol`/`objectql` inside the builder. Discovery resolves services **unscoped**, so that could report the required data capability as `handlerReady: false` on a multi-kernel host while per-request scoped resolution serves it fine — a worse lie than the one being fixed. No domain gates on this slot either: `isServiceServeable` (#4058 step 2) covers the optional domains, and `/data` resolves its engine directly.)*
- *(Added by #4058: the **kernel's own** fallbacks — `createMemoryCache` / `Queue` / `Job` / `I18n` / `Metadata`, auto-registered by ObjectKernel — were missing from this inventory and were the worst case in it: they carried `_fallback: true`, a marker **no** reader recognized (not even `readServiceSelfInfo`), so both discovery builders reported them as fully `available`. They now self-describe as `degraded` with `handlerReady: false` where no HTTP surface exists (`cache` / `queue` / `job`). The lone duck-typed consumer of `_fallback`/`_dev` — an i18n diagnostic in `app-plugin.ts` — reads `readServiceSelfInfo` instead.)*

**Decision — honest capabilities:**
Expand Down
19 changes: 18 additions & 1 deletion packages/metadata-protocol/src/protocol.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1481,6 +1481,16 @@ export class ObjectStackProtocolImplementation implements
// mounted whichever implementation occupies the slot.
const metadataSelf = readServiceSelfInfo(registeredServices.get('metadata'));

// [#4130] `data` was the last self-judging entry in this block. Same
// reasoning, one degree weaker: its hardcoded `available` is currently
// true, but only because ObjectQL is the slot's sole producer and
// plugin-dev (whose `data` stub declares `stub`) always loads
// ObjectQLPlugin as a child. That is a load-order convention in another
// package, not something this builder verifies — so verify it here.
// Unmarked implementation ⇒ `available` + `handlerReady: true`, i.e.
// exactly what the hardcode said, now derived rather than asserted.
const dataSelf = readServiceSelfInfo(registeredServices.get('data'));

const services: Record<string, ServiceInfo> = {
// --- Kernel-provided (objectql is an example kernel implementation) ---
metadata: {
Expand All @@ -1491,7 +1501,14 @@ export class ObjectStackProtocolImplementation implements
provider: 'objectql',
...(metadataSelf?.message ? { message: metadataSelf.message } : {}),
},
data: { enabled: true, status: 'available' as const, route: '/api/v1/data', provider: 'objectql' },
data: {
enabled: true,
status: dataSelf?.status ?? ('available' as const),
handlerReady: dataSelf?.handlerReady ?? true,
route: '/api/v1/data',
provider: 'objectql',
...(dataSelf?.message ? { message: dataSelf.message } : {}),
},
};

// [#4000, #4058] The dispatcher answers a self-declared non-handler in
Expand Down
7 changes: 7 additions & 0 deletions packages/objectql/src/plugin.integration.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,13 @@ describe('ObjectQLPlugin - Metadata Service Integration', () => {
// Serving is unaffected — `/api/v1/meta` is the protocol's route.
expect(discovery.services.metadata.handlerReady).toBe(true);
expect(discovery.routes.metadata).toBe('/api/v1/meta');

// …and on the same boot, `data` — derived from its slot too now (#4130)
// — is unchanged: ObjectQLPlugin registers the real engine there and it
// carries no marker, so the derivation reproduces the old hardcode.
expect(discovery.services.data.status).toBe('available');
expect(discovery.services.data.handlerReady).toBe(true);
expect(discovery.services.data.message).toBeUndefined();
});

it('should serve in-memory metadata definitions', async () => {
Expand Down
36 changes: 36 additions & 0 deletions packages/objectql/src/protocol-discovery.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,42 @@ describe('ObjectStackProtocolImplementation - Dynamic Service Discovery', () =>
expect(discovery.services.metadata.handlerReady).toBe(true);
});

// #4130 — `data` was the last hardcoded entry in the kernel block. Its
// `available` happened to be true, but only because ObjectQL is the slot's
// sole producer and plugin-dev always loads it as a child, so its `data` stub
// never lands there. A convention in another package is not something this
// builder verifies — now it does.
it('should report a self-declared data stub as a stub, never available', async () => {
const mockServices = new Map<string, any>();
mockServices.set('data', {
__serviceInfo: { status: 'stub', message: 'Dev stub — find() always returns []. Register ObjectQLPlugin for a real engine.' },
find: async () => [],
});

protocol = new ObjectStackProtocolImplementation(engine, () => mockServices);
const discovery = await protocol.getDiscovery();

expect(discovery.services.data.enabled).toBe(true);
expect(discovery.services.data.status).toBe('stub');
expect(discovery.services.data.handlerReady).toBe(false);
expect(discovery.services.data.message).toContain('ObjectQLPlugin');
});

it('should keep reporting a real (unmarked) data engine as available', async () => {
const mockServices = new Map<string, any>();
mockServices.set('data', { find: async () => [], insert: async () => ({ id: '1' }) });

protocol = new ObjectStackProtocolImplementation(engine, () => mockServices);
const discovery = await protocol.getDiscovery();

// Byte-for-byte what the hardcode said, now derived from the slot.
expect(discovery.services.data.status).toBe('available');
expect(discovery.services.data.handlerReady).toBe(true);
expect(discovery.services.data.route).toBe('/api/v1/data');
expect(discovery.services.data.provider).toBe('objectql');
expect(discovery.services.data.message).toBeUndefined();
});

// #3891 — the degraded ObjectQL fallback is retired. Analytics is an
// ordinary optional service now: absent ⇒ unavailable, and the route must
// NOT be advertised (an advertised route with no handler 404s — the exact
Expand Down
65 changes: 65 additions & 0 deletions packages/runtime/src/http-dispatcher.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2415,6 +2415,71 @@ describe('HttpDispatcher', () => {
expect(fromDispatcher.message).toBe(fromProtocol.message);
expect(fromDispatcher.route).toBe(fromProtocol.route);
});

// ── The `data` slot: the same hardcode, one degree weaker (#4130) ─────
//
// `available` / `handlerReady: true` was true here only by a convention
// in another package (ObjectQL is the slot's only producer, and
// plugin-dev always loads it as a child so its `data` stub never lands).
// These pin the computation that replaces the convention.

it('keeps reporting a real (unmarked) data engine as available — the hardcode it replaces', async () => {
(kernel as any).getService = vi.fn().mockImplementation((name: string) =>
name === 'data' ? mockObjectQL : null,
);

const info = await dispatcher.getDiscoveryInfo('/api/v1');
expect(info.services.data.enabled).toBe(true);
expect(info.services.data.status).toBe('available');
expect(info.services.data.handlerReady).toBe(true);
expect(info.services.data.route).toBe('/api/v1/data');
expect(info.services.data.provider).toBe('kernel');
expect(info.services.data.message).toBeUndefined();
});

it('reports plugin-dev\'s data stub as a stub with no ready handler, not as a query engine', async () => {
// The real marker plugin-dev attaches (DEV_STUB_SELF_INFO.data):
// `stub` with no explicit handlerReady, which readServiceSelfInfo
// defaults to false.
const devDataStub = {
__serviceInfo: {
status: 'stub',
message: 'Dev stub — find() always returns [], insert() mints an id and stores nothing. Register ObjectQLPlugin for a real engine.',
},
find: vi.fn().mockResolvedValue([]),
insert: vi.fn().mockResolvedValue({ id: 'x' }),
};
(kernel as any).getService = vi.fn().mockImplementation((name: string) =>
name === 'data' ? devDataStub : null,
);

const info = await dispatcher.getDiscoveryInfo('/api/v1');
expect(info.services.data.status).toBe('stub');
expect(info.services.data.handlerReady).toBe(false);
expect(info.services.data.message).toContain('ObjectQLPlugin');
});

it('answers the data slot identically to the metadata-protocol builder', async () => {
const { ObjectStackProtocolImplementation } = await import('@objectstack/metadata-protocol');
const degradedData = {
__serviceInfo: { status: 'degraded', message: 'read-only engine' },
find: vi.fn().mockResolvedValue([]),
};
(kernel as any).getService = vi.fn().mockImplementation((name: string) =>
name === 'data' ? degradedData : null,
);

const fromDispatcher = (await dispatcher.getDiscoveryInfo('/api/v1')).services.data;
const fromProtocol = (await new ObjectStackProtocolImplementation(
mockObjectQL as any,
() => new Map<string, any>([['data', degradedData]]),
).getDiscovery()).services.data;

expect(fromDispatcher.status).toBe(fromProtocol.status);
expect(fromDispatcher.handlerReady).toBe(fromProtocol.handlerReady);
expect(fromDispatcher.message).toBe(fromProtocol.message);
expect(fromDispatcher.route).toBe(fromProtocol.route);
});
});

// ═══════════════════════════════════════════════════════════════
Expand Down
Loading
Loading