fix(plugin-dev,runtime): retire the dev analytics stub; /analytics gates on handlerReady, not presence (#4000)

.changeset/retire-dev-analytics-stub.md

@@ -0,0 +1,13 @@
+---
+'@objectstack/plugin-dev': patch
+'@objectstack/runtime': patch
+---
+
+Retire the dev-mode `analytics` stub, and make the dispatcher gate `/analytics` on `handlerReady` rather than on service presence (#4000).
+
+Retiring the degraded analytics shim (#3891) made an empty `analytics` slot the honest signal: `/api/v1/analytics/*` 404s and discovery reports `unavailable`. `plugin-dev` refilled that slot with a stub, which re-created the retired shape in dev mode — the dispatcher gated on "is a service registered", so the stub was called like a real engine and its empty result came back as a 200.
+
+- `plugin-dev` no longer registers an `analytics` dev stub; the slot stays empty (`NO_DEV_STUB_SERVICES`). Every other dev stub is unchanged.
+- The `/analytics` domain, its route-mount gate, and discovery's `routes`/`features` now share one predicate (`isAnalyticsServiceServeable`): a service that self-declares `handlerReady: false` (ADR-0076 D12 — `__serviceInfo`, or plugin-dev's legacy `_dev: true`) is treated as an empty slot. A `degraded` implementation that genuinely serves requests keeps serving; `discovery.services.analytics` still reports a registered stub as `status: 'stub'`, which says more than `unavailable` would.
+
+FROM → TO for dev setups that relied on the stub answering `POST /api/v1/analytics/query` with `{ rows: [], fields: [] }`: install the real engine — `@objectstack/service-analytics` runs an InMemory strategy and needs no database of its own. Nothing else changes; hosts that already install it (including `os serve`, where `analytics` is in `ALWAYS_ON_CAPABILITIES`) are unaffected.

content/docs/kernel/services-checklist.mdx

@@ -186,6 +186,13 @@ caller's `ExecutionContext` — aggregates ran without RLS/tenant scoping — an
 silently ignored the contract `where` filter. Without
 `@objectstack/service-analytics`, `/api/v1/analytics/*` now answers **404** and
 discovery reports `analytics: { enabled: false, status: "unavailable" }`.
+
+**This holds in dev mode too** (#4000): `plugin-dev` no longer registers an
+`analytics` dev stub, and the dispatcher treats a slot filled by any
+self-declared stub (`handlerReady: false`, ADR-0076 D12) exactly like an empty
+one — routes unmounted, request 404. To use analytics locally, install the real
+engine; `@objectstack/service-analytics` runs an InMemory strategy, so it needs
+no database of its own.
 </Callout>
 
 ### Features (via `@objectstack/service-analytics`)

docs/adr/0076-objectql-core-tiering.md

@@ -132,14 +132,14 @@ Decision: each capability plugin registers its routes as a **normalized handler*
 **Root cause of agents being misled.** Several plugins register stub / dev / fallback services under canonical names, and the discovery builder reports *any* present service as fully real: `runtime/http-dispatcher.ts`'s `svcAvailable` hardcodes `{ enabled: true, status: 'available', handlerReady: true }` for every registered service — it **ignores stub markers** (its own comment even says "handlerReady:false … may be served by a stub", but the code never computes it). So `discovery.services.*` claims capabilities that are only stubbed, and consumers (AI agents, the console) trust them. A dev AI stub advertised this way has already confused an agent.
 
 **Inventory of current fakes / mis-reports:**
-- `plugin-dev` registers ~8 dev stubs — `storage` / `search` / `automation` / `graphql` / `analytics` / `realtime` / `notification` / `ai` (they already carry a `_dev: true` marker that nothing respects).
+- `plugin-dev` registers ~8 dev stubs — `storage` / `search` / `automation` / `graphql` / `analytics` / `realtime` / `notification` / `ai` (they already carry a `_dev: true` marker that nothing respects). *(Update #4000: the `analytics` one is **gone** — after the fallback's retirement (#3891/#3989) it was the last thing refilling that slot, and refilling it re-created the retired shape in dev: the dispatcher gated on service presence, so the stub was called like an engine and answered 200 with fabricated rows. The remaining stubs are unchanged, and the class-wide question they raise is tracked separately — see conclusion 3 below.)*
 - `ObjectQLPlugin` registers a ~66-line `analytics` **fallback** (the D10 note — deliberate, but it reports as fully available).
 - `http-dispatcher.ts` `svcAvailable` — the hardcode above.
 
 **Decision — honest capabilities:**
 1. A registered service that is a stub / dev / fallback MUST self-identify with a **standard marker** (standardise one; `_dev` is the existing precedent).
 2. Discovery MUST respect it: such services report **`status: 'stub'` (or `'degraded'`) with `handlerReady: false`**, never `status: 'available'`. Add the status value to the discovery schema.
-3. Consumers (agents, console) treat **only `handlerReady: true` / `status: 'available'`** as a real capability; `stub`/`unavailable` ⇒ do not use for real work.
+3. Consumers (agents, console) treat **only `handlerReady: true` / `status: 'available'`** as a real capability; `stub`/`unavailable` ⇒ do not use for real work. *(Update #4000: the **dispatcher is a consumer too**, and was not executing this — every service domain gates on "is the slot occupied", never on `handlerReady`, so a stub in the slot was called like a real implementation. Enforced for `analytics` (`isAnalyticsServiceServeable` in `runtime/src/domains/analytics.ts`, read by the domain, the route-mount gate, and `routes`/`features` in discovery — one predicate, so advertised and served cannot disagree). The other stub-backed domains — `file-storage` / `search` / `automation` / `realtime` / `notification` / `ai` / `i18n` — still gate on presence alone; adopting the same rule across them is a per-domain call, tracked in #4058.)*
 
 This fixes the **whole class at once — without deleting any fallback** (no `/analytics` 404 regression): the analytics fallback and the dev stubs simply stop *lying*; they keep serving but are honestly labelled. It is the runtime enforcement of the D9-refinement principle (capabilities = what is actually installed, computed at runtime).
 

packages/metadata-protocol/src/protocol.ts

@@ -1408,6 +1408,19 @@ export class ObjectStackProtocolImplementation implements
             data:      { enabled: true, status: 'available' as const, route: '/api/v1/data', provider: 'objectql' },
         };
 
+        // [#4000] The dispatcher answers a self-declared stub in the `analytics`
+        // slot with the same 404 an empty slot gets (`isAnalyticsServiceServeable`,
+        // runtime/src/domains/analytics.ts), so this builder must not advertise a
+        // route for it — that would be the `declared ≠ enforced` gap discovery
+        // exists to close. Analytics-only on purpose: every other stub-backed
+        // slot IS still served by its dispatcher domain, so their route
+        // advertisement stays presence-gated and honest. Unifying the rule
+        // across domains is #4058.
+        const analyticsUnserveable =
+            readServiceSelfInfo(registeredServices.get('analytics'))?.handlerReady === false;
+        const advertisedRoute = (serviceName: string, route?: string) =>
+            serviceName === 'analytics' && analyticsUnserveable ? undefined : route;
+
         // Check which services are actually registered
         for (const [serviceName, config] of Object.entries(SERVICE_CONFIG)) {
             if (registeredServices.has(serviceName)) {
@@ -1420,7 +1433,7 @@ export class ObjectStackProtocolImplementation implements
                 services[serviceName] = {
                     enabled: true,
                     status: self?.status ?? (noHttpSurface ? ('degraded' as const) : ('available' as const)),
-                    route: config.route,
+                    route: advertisedRoute(serviceName, config.route),
                     provider: config.plugin,
                     ...(noHttpSurface || self?.handlerReady !== undefined
                         ? { handlerReady: noHttpSurface ? false : self?.handlerReady }
@@ -1460,10 +1473,11 @@ export class ObjectStackProtocolImplementation implements
         // Add routes for available plugin services. Services without an HTTP
         // surface (config.route undefined) advertise no route (D12, #2462).
         for (const [serviceName, config] of Object.entries(SERVICE_CONFIG)) {
-            if (registeredServices.has(serviceName) && config.route) {
+            const route = advertisedRoute(serviceName, config.route);
+            if (registeredServices.has(serviceName) && route) {
                 const routeKey = serviceToRouteKey[serviceName];
                 if (routeKey) {
-                    optionalRoutes[routeKey] = config.route;
+                    optionalRoutes[routeKey] = route;
                 }
             }
         }

packages/objectql/src/protocol-discovery.test.ts

@@ -175,6 +175,25 @@ describe('ObjectStackProtocolImplementation - Dynamic Service Discovery', () =>
     expect(discovery.routes.analytics).toBe('/api/v1/analytics');
   });
 
+  // #4000 — the same rule one step further: a stub occupying the slot is not
+  // the capability either. The dispatcher answers it with the empty-slot 404,
+  // so advertising a route here would re-tell the lie #3891 removed. The
+  // service entry itself still self-reports as a stub — that says more than
+  // `unavailable` would.
+  it('should not advertise the analytics route for a self-declared stub', async () => {
+    const mockServices = new Map<string, any>();
+    mockServices.set('analytics', { _dev: true, query: async () => ({ rows: [], fields: [] }) });
+
+    protocol = new ObjectStackProtocolImplementation(engine, () => mockServices);
+    const discovery = await protocol.getDiscovery();
+
+    expect(discovery.services.analytics.enabled).toBe(true);
+    expect(discovery.services.analytics.status).toBe('stub');
+    expect(discovery.services.analytics.handlerReady).toBe(false);
+    expect(discovery.services.analytics.route).toBeUndefined();
+    expect(discovery.routes.analytics).toBeUndefined();
+  });
+
   it('should map file-storage service to storage route', async () => {
     const mockServices = new Map<string, any>();
     mockServices.set('file-storage', {});

packages/plugins/plugin-dev/src/dev-plugin.test.ts

@@ -67,7 +67,7 @@ describe('DevPlugin', () => {
     await expect(plugin.init(ctx)).resolves.not.toThrow();
   });
 
-  it('should register contract-compliant dev stubs for all core services', async () => {
+  it('should register contract-compliant dev stubs for every core service except analytics', async () => {
     const registeredServices = new Map<string, any>();
     const ctx: any = {
       logger: {
@@ -159,12 +159,14 @@ describe('DevPlugin', () => {
     expect(execResult.success).toBe(true);
     expect(Array.isArray(await automation.listFlows())).toBe(true);
 
-    // ── Verify IAnalyticsService contract ──
-    const analytics = registeredServices.get('analytics');
-    const analyticsResult = await analytics.query({ cube: 'test' });
-    expect(Array.isArray(analyticsResult.rows)).toBe(true);
-    expect(Array.isArray(analyticsResult.fields)).toBe(true);
-    expect(Array.isArray(await analytics.getMeta())).toBe(true);
+    // ── analytics: deliberately NOT stubbed (#4000) ──
+    // #3891/#3989 retired the degraded analytics shim so an empty slot is the
+    // honest signal (route unmounted → 404, discovery `unavailable`). A dev
+    // stub refilled the slot and the dispatcher, gating on presence alone,
+    // served its fabricated rows with a 200 — the retired shape, one layer
+    // down. The slot stays empty; install @objectstack/service-analytics.
+    expect(registeredServices.has('analytics')).toBe(false);
+    expect(stubLog[0]).not.toContain('analytics');
 
     // ── Verify IRealtimeService contract ──
     const realtime = registeredServices.get('realtime');

packages/plugins/plugin-dev/src/dev-plugin.ts

@@ -97,16 +97,6 @@ function createAutomationStub() {
 }
 
 
-/** IAnalyticsService — dev stub returning empty results */
-function createAnalyticsStub() {
-  return {
-    _dev: true, _serviceName: 'analytics',
-    async query() { return { rows: [], fields: [] }; },
-    async getMeta() { return []; },
-    async generateSql() { return { sql: '', params: [] }; },
-  };
-}
-
 /** IRealtimeService — in-memory pub/sub stub */
 function createRealtimeStub() {
   const subs = new Map<string, Function>();
@@ -266,7 +256,6 @@ const DEV_STUB_FACTORIES: Record<string, () => Record<string, any>> = {
   'file-storage': createStorageStub,
   'search':      createSearchStub,
   'automation':  createAutomationStub,
-  'analytics':   createAnalyticsStub,
   'realtime':    createRealtimeStub,
   'notification': createNotificationStub,
   'ai':          createAIStub,
@@ -281,6 +270,24 @@ const DEV_STUB_FACTORIES: Record<string, () => Record<string, any>> = {
   'security.fieldMasker': createSecurityFieldMaskerStub,
 };
 
+/**
+ * Core service slots that deliberately get NO dev stub — not even the
+ * shapeless `{ _dev: true }` fallback the registration loop uses for slots
+ * without a factory.
+ *
+ * `analytics` (#4000): #3891/#3989 retired the degraded analytics shim, making
+ * an unoccupied slot the honest signal — `/analytics/*` is not mounted, the
+ * request 404s, and discovery reports `unavailable`. Registering a dev stub
+ * re-filled that slot and re-created the retired shape one layer down: the
+ * dispatcher gates on service presence, so a stub was called like a real
+ * engine and answered 200 with fabricated rows. Dev mode is explicit opt-in
+ * and the fake was empty, so this was never the security hole the shim was —
+ * but "the capability is present" must mean the same thing in dev as in
+ * production. To use analytics locally, install the real engine:
+ * `@objectstack/service-analytics` runs an InMemory strategy.
+ */
+const NO_DEV_STUB_SERVICES = new Set<string>(['analytics']);
+
 /**
  * Dev Plugin Options
  *
@@ -657,11 +664,13 @@ export class DevPlugin implements Plugin {
     // dev stub (implementing the interface from packages/spec/src/contracts/)
     // so that the full kernel service map is populated and downstream code
     // receives correct return types (arrays, booleans, objects — not undefined).
+    // Exception: NO_DEV_STUB_SERVICES slots stay empty on purpose (#4000).
 
     const stubNames: string[] = [];
 
     for (const svc of CORE_SERVICE_NAMES) {
       if (!enabled(svc)) continue;
+      if (NO_DEV_STUB_SERVICES.has(svc)) continue;
       try {
         ctx.getService(svc);
         // Already registered by a real plugin — skip

packages/runtime/src/dispatcher-plugin.routes.test.ts

@@ -155,6 +155,31 @@ describe('createDispatcherPlugin — HTTP route registration', () => {
       for (const r of ANALYTICS_ROUTES) expect(routes).toContain(r);
     });
 
+    // [#4000] "Registered" is not the test — `handlerReady` is (ADR-0076 D12).
+    // plugin-dev used to fill this slot with a stub, which mounted the three
+    // routes and served its fabricated rows with a 200. The stub is retired,
+    // and the mount gate now reads the same predicate the domain does, so the
+    // wire surface can't advertise routes that could only 404.
+    it('does NOT mount /analytics when the registered analytics service self-declares as a stub', async () => {
+      const { server, routes } = makeFakeServer();
+      const plugin = createDispatcherPlugin({ prefix: '/api/v1', securityHeaders: false });
+      await plugin.start?.(ctxWithServices(server, {
+        analytics: { _dev: true, query: async () => ({ rows: [], fields: [] }) },
+      }));
+
+      for (const r of ANALYTICS_ROUTES) expect(routes).not.toContain(r);
+    });
+
+    it('DOES mount /analytics for a degraded-but-serving analytics service', async () => {
+      const { server, routes } = makeFakeServer();
+      const plugin = createDispatcherPlugin({ prefix: '/api/v1', securityHeaders: false });
+      await plugin.start?.(ctxWithServices(server, {
+        analytics: { __serviceInfo: { status: 'degraded' }, query: async () => ({ rows: [] }) },
+      }));
+
+      for (const r of ANALYTICS_ROUTES) expect(routes).toContain(r);
+    });
+
     it('mounts /analytics unconditionally on a multi-tenant host (kernel-resolver wired)', async () => {
       // Host-global mounts, per-project services: presence is a per-request
       // question the analytics domain answers (`handled:false` → 404), so the

packages/runtime/src/dispatcher-plugin.ts

@@ -4,6 +4,7 @@ import { Plugin, PluginContext, IHttpServer } from '@objectstack/core';
 import { looksLikeInternalErrorLeak, INTERNAL_ERROR_MESSAGE } from '@objectstack/types';
 import { DispatcherErrorCode } from '@objectstack/spec/api';
 import { HttpDispatcher, HttpDispatcherResult } from './http-dispatcher.js';
+import { isAnalyticsServiceServeable } from './domains/analytics.js';
 import { validationFailureDetails, VALIDATION_FAILED_STATUS } from './validation-failure.js';
 import { buildApiError } from './error-envelope.js';
 import {
@@ -648,14 +649,23 @@ export function createDispatcherPlugin(config: DispatcherPluginConfig = {}): Plu
             // Route via dispatch() (not handleAnalytics directly) so the host
             // dispatcher's project-aware kernel swap runs first — the
             // per-project kernel owns the `analytics` service.
+            //
+            // [#4000] "Registered" is not the test — `handlerReady` is. A
+            // service that self-declares as a stub (ADR-0076 D12) occupies the
+            // slot without being the capability, and the domain answers it with
+            // the same `handled:false` 404 an empty slot gets; mounting routes
+            // that can only 404 would re-advertise a capability that isn't
+            // there. Same predicate on both, so the wire surface and the
+            // handler can't disagree.
             const analyticsInstalled = dispatcher.isMultiTenantHost() || await (async () => {
                 const k: any = kernel;
                 try {
+                    let svc: unknown;
                     if (k && typeof k.getServiceAsync === 'function') {
-                        const svc = await k.getServiceAsync('analytics').catch(() => undefined);
-                        if (svc) return true;
+                        svc = await k.getServiceAsync('analytics').catch(() => undefined);
                     }
-                    return !!k?.getService?.('analytics');
+                    if (!svc) svc = k?.getService?.('analytics');
+                    return isAnalyticsServiceServeable(svc);
                 } catch {
                     return false;
                 }
@@ -690,7 +700,8 @@ export function createDispatcherPlugin(config: DispatcherPluginConfig = {}): Plu
                 });
             } else {
                 ctx.logger?.info?.(
-                    '[dispatcher] /analytics not mounted — no `analytics` service is registered. ' +
+                    '[dispatcher] /analytics not mounted — no `analytics` service is registered ' +
+                    '(or the registered one self-declares as a stub). ' +
                     'Install @objectstack/service-analytics to enable the analytics API.',
                 );
             }