Skip to content

Commit 4965bfa

Browse files
authored
feat(automation,formula): warn on undeclared flow-node config keys (#4045) (#4059)
`FlowNodeSchema.config` is `z.record(z.unknown())`, so a misspelled or invented config key was accepted in total silence: `visibleIf` instead of `visibleWhen` registered cleanly, was never read, and the only symptom was a feature that quietly did not happen — the diagnostic vacuum that made #3528 take three passes. `registerFlow` now walks each node's `config` against its descriptor's configSchema and warns on anything undeclared, located to the exact path with the declared set listed. The walk descends where the schema declares structure and STOPS at free-form keyValue maps, whose keys are author data. Descending is load-bearing: `visibleWhen` is a property of screen's `fields[].items`, so a top-level-only check would miss the exact typo class this exists to catch. Warn, never reject. An undeclared key is an author typo, a key the executor reads that its configSchema never declared (`notify.source` was exactly this until #4050), or dead config — and only 4 of 13 schema-carrying builtins have been audited for the middle case. This warning measures the distribution; tightening is a later per-key decision. The published configSchema is unchanged, so no consumer sees a different shape. `@objectstack/formula` exports `nearestName`, the edit-distance helper already behind unknown-field/role suggestions. It is a bonus, not the mechanism: `visibleIf` → `visibleWhen` is distance 4 against a threshold of 3, so the declared set is printed always rather than only as a fallback. Measured over 30 flows in app-crm + app-showcase: exactly one warning, and it was a real finding — `showcase_inquiry_purge`'s get_record carried `mode: 'records'`, which no executor reads (`limit > 1` selects the list read), with a comment crediting `mode` for what `limit` actually does. Fixed here.
1 parent ec796d5 commit 4965bfa

6 files changed

Lines changed: 347 additions & 6 deletions

File tree

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
'@objectstack/service-automation': minor
3+
'@objectstack/formula': minor
4+
---
5+
6+
Warn on flow-node `config` keys the node type does not declare (#4045).
7+
8+
`FlowNodeSchema.config` is `z.record(z.unknown())`, so a misspelled or invented
9+
config key was accepted in total silence: `visibleIf` instead of `visibleWhen`
10+
registered cleanly, was never read, and the only symptom was a feature that quietly
11+
did not happen. That diagnostic vacuum is what made #3528 take three passes and two
12+
wrong diagnoses to resolve.
13+
14+
`registerFlow` now compares each node's `config` against its descriptor's
15+
`configSchema` and warns on anything undeclared, located and with the declared set
16+
listed:
17+
18+
```
19+
[flow 'lead_conversion'] node 'screen_1' (screen): unknown config key `visibleIf`
20+
at config.fields[0].visibleIf — It is not declared by this node type's
21+
configSchema, so nothing reads it. Declared here: name, label, type, required,
22+
visibleWhen.
23+
```
24+
25+
The walk descends where the schema declares structure and **stops at free-form
26+
keyValue maps**, whose keys are author data (`filter: { status: 'stale' }`).
27+
Descending matters: the #3528 typo class lives *inside* the `screen` field
28+
repeater, so a top-level-only comparison would miss the exact mistake this exists
29+
to catch.
30+
31+
**Warn, never reject.** An undeclared key is an author typo, a key the executor
32+
genuinely reads that its hand-written `configSchema` never declared (`notify.source`
33+
was exactly this), or dead config. Only 4 of the 13 schema-carrying builtins have
34+
been audited for the second population, so hard-failing would gamble on the other
35+
nine. Tightening to an error is a later, per-key decision once this warning has
36+
measured the real distribution. Nothing about the published `configSchema` changes,
37+
so no consumer sees a different shape.
38+
39+
`@objectstack/formula` now exports `nearestName`, the edit-distance helper already
40+
used for unknown-field and unknown-role suggestions, so "did you mean?"
41+
diagnostics share one threshold. It is deliberately a bonus rather than the
42+
mechanism — `visibleIf``visibleWhen` is distance 4 against a threshold of 3, so
43+
the declared set is always listed instead of only as a fallback.
44+
45+
Also fixes the first real finding from the new check: `showcase_inquiry_purge`'s
46+
`get_record` node carried `mode: 'records'`, which no executor reads, with a comment
47+
crediting it for behaviour that `limit > 1` actually produces.

examples/app-showcase/src/automation/flows/index.ts

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1357,13 +1357,16 @@ export const InquiryPurgeFlow = defineFlow({
13571357
id: 'purge_check',
13581358
type: 'get_record',
13591359
label: 'Find closed inquiries',
1360-
// records mode + outputVariable: the result lands in the variable
1361-
// context, where the out-edge CEL below reads it (the engine routes on
1362-
// EDGE conditions — same contract as BudgetApprovalFlow's gate).
1360+
// `limit > 1` is what makes this a LIST read (`find`, not `findOne`) — the
1361+
// executor has no `mode` key, so a `mode: 'records'` used to sit here doing
1362+
// nothing while the comment credited it for the behaviour `limit` actually
1363+
// produced (found by the #4045 unknown-config-key check). `outputVariable`
1364+
// then lands the array in the variable context, where the out-edge CEL below
1365+
// reads it — the engine routes on EDGE conditions, same contract as
1366+
// BudgetApprovalFlow's gate.
13631367
config: {
13641368
objectName: 'showcase_inquiry',
13651369
filter: { status: 'closed' },
1366-
mode: 'records',
13671370
limit: 200,
13681371
outputVariable: 'closedInquiries',
13691372
},

packages/formula/src/index.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ export type { CelFilterCompileResult, CelFilterCompileOptions, CelFilterFailReas
2828
export { matchesFilterCondition } from './matches-filter';
2929
// ADR-0032 — shared validator + introspection (one validator for build,
3030
// registration, and the agent-callable validate_expression tool).
31-
export { validateExpression, introspectScope, expectedDialect, inferExpressionType, CEL_STDLIB_FUNCTIONS } from './validate';
31+
export { validateExpression, introspectScope, expectedDialect, inferExpressionType, nearestName, CEL_STDLIB_FUNCTIONS } from './validate';
3232
export type { FieldRole, ExprInput, ExprSchemaHint, ExprValidationError, ExprValidationResult, InferredValueType } from './validate';
3333
export type { SeedValue, SeedPrimitive } from './seed-eval';
3434
export type { DialectEngine, EvalContext, EvalResult, EvalError } from './types';

packages/formula/src/validate.ts

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -216,6 +216,20 @@ function checkFieldExistence(source: string, schema: ExprSchemaHint | undefined,
216216
}
217217

218218
/** Cheap edit-distance suggestion for typo'd field names. */
219+
/**
220+
* The closest candidate to `name`, or `undefined` when nothing is close enough
221+
* to be worth suggesting.
222+
*
223+
* The public face of the same edit-distance heuristic this module already uses
224+
* for unknown field refs and unknown roles, exported so other "did you mean?"
225+
* diagnostics reuse one threshold instead of each inventing their own — an
226+
* author (increasingly an agent) should not get a suggestion here and silence
227+
* there for the same class of typo.
228+
*/
229+
export function nearestName(name: string, candidates: readonly string[]): string | undefined {
230+
return nearest(name, candidates);
231+
}
232+
219233
function nearest(name: string, candidates: readonly string[]): string | undefined {
220234
let best: string | undefined;
221235
let bestD = Infinity;
Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* Unknown flow-node `config` keys are reported at registration (#4045).
5+
*
6+
* `FlowNodeSchema.config` is `z.record(z.unknown())`, so before this a misspelled
7+
* or invented config key was accepted in **total silence**: `visibleIf` instead of
8+
* `visibleWhen` registered cleanly, was never read, and the only symptom was a
9+
* feature that quietly did not happen. That is the diagnostic vacuum that made
10+
* #3528 take three passes and two wrong diagnoses.
11+
*
12+
* Warn, never reject — see `validateNodeConfigKeys` for why. These tests pin both
13+
* halves of that: the warning fires with an actionable suggestion, AND
14+
* registration still succeeds.
15+
*/
16+
17+
import { describe, it, expect } from 'vitest';
18+
import { AutomationEngine } from '../engine.js';
19+
import { installBuiltinNodes } from './index.js';
20+
21+
function recordingLogger() {
22+
const warnings: string[] = [];
23+
const logger: any = {
24+
info() {}, error() {}, debug() {},
25+
warn(msg: string) { warnings.push(msg); },
26+
child() { return logger; },
27+
};
28+
return { logger, warnings };
29+
}
30+
31+
function engineWith(logger: any) {
32+
const engine = new AutomationEngine(logger);
33+
installBuiltinNodes(engine, { logger, getService() { throw new Error('none'); } } as any);
34+
return engine;
35+
}
36+
37+
/** A one-node flow carrying `config` verbatim on a node of `type`. */
38+
function flowWith(type: string, config: Record<string, unknown>) {
39+
return {
40+
name: 'f', label: 'F', type: 'screen', status: 'active', version: 1,
41+
nodes: [
42+
{ id: 'start', type: 'start', label: 'S', config: {} },
43+
{ id: 'n1', type, label: 'N', config },
44+
{ id: 'end', type: 'end', label: 'E' },
45+
],
46+
edges: [
47+
{ id: 'e1', source: 'start', target: 'n1', type: 'default' },
48+
{ id: 'e2', source: 'n1', target: 'end', type: 'default' },
49+
],
50+
};
51+
}
52+
53+
describe('unknown node config keys (#4045)', () => {
54+
it('flags a misspelled key with a did-you-mean suggestion', () => {
55+
const { logger, warnings } = recordingLogger();
56+
const engine = engineWith(logger);
57+
58+
// The typo lives INSIDE the field repeater — where the real #3528 one did.
59+
engine.registerFlow('f', flowWith('screen', {
60+
fields: [{ name: 'opportunityName', required: true, visibleIf: 'createOpportunity == true' }],
61+
}));
62+
63+
const hit = warnings.find((w) => w.includes('visibleIf'));
64+
expect(hit, 'the typo should be reported').toBeDefined();
65+
expect(hit).toContain("node 'n1'");
66+
expect(hit).toContain('(screen)');
67+
// Located to the exact element, so the author knows WHICH field.
68+
expect(hit).toContain('config.fields[0].visibleIf');
69+
// The load-bearing half for an agent author: the correct key is named. Note
70+
// it comes from the DECLARED SET, not the suggestion — `visibleIf` →
71+
// `visibleWhen` is edit-distance 4 against `nearestName`'s threshold of 3, so
72+
// this exact typo gets no did-you-mean. Printing the declared set is what
73+
// makes the diagnostic actionable regardless.
74+
expect(hit).toContain('Declared here: name, label, type, required, visibleWhen.');
75+
});
76+
77+
it('adds a did-you-mean when the typo IS within edit distance', () => {
78+
const { logger, warnings } = recordingLogger();
79+
const engine = engineWith(logger);
80+
81+
engine.registerFlow('f', flowWith('screen', { titl: 'Details' }));
82+
83+
const hit = warnings.find((w) => w.includes('titl'));
84+
expect(hit).toContain('did you mean `title`?');
85+
// …and still lists the declared set alongside it.
86+
expect(hit).toContain('Declared here:');
87+
});
88+
89+
it('still registers the flow — warn, never reject', () => {
90+
const { logger } = recordingLogger();
91+
const engine = engineWith(logger);
92+
expect(() => engine.registerFlow('f', flowWith('screen', { visibleIf: 'a == true' }))).not.toThrow();
93+
// And it is really registered, not swallowed.
94+
expect(engine.getFlow('f')).toBeDefined();
95+
});
96+
97+
it('lists the declared keys when nothing is close enough to suggest', () => {
98+
const { logger, warnings } = recordingLogger();
99+
const engine = engineWith(logger);
100+
101+
engine.registerFlow('f', flowWith('screen', { totallyMadeUp: 42 }));
102+
103+
const hit = warnings.find((w) => w.includes('totallyMadeUp'));
104+
expect(hit).toBeDefined();
105+
expect(hit).not.toContain('did you mean');
106+
// The declared set is always present, so there is always somewhere to go.
107+
expect(hit).toContain('Declared here:');
108+
expect(hit).toContain('fields');
109+
});
110+
111+
it('reports every undeclared key, not just the first', () => {
112+
const { logger, warnings } = recordingLogger();
113+
const engine = engineWith(logger);
114+
115+
engine.registerFlow('f', flowWith('screen', { hideWhen: 'x', submitLabel: 'Go' }));
116+
117+
expect(warnings.some((w) => w.includes('hideWhen'))).toBe(true);
118+
expect(warnings.some((w) => w.includes('submitLabel'))).toBe(true);
119+
});
120+
121+
it('stays quiet on a fully declared config', () => {
122+
const { logger, warnings } = recordingLogger();
123+
const engine = engineWith(logger);
124+
125+
engine.registerFlow('f', flowWith('screen', {
126+
title: 'Details',
127+
fields: [{ name: 'a', type: 'boolean', required: true, visibleWhen: 'b == true' }],
128+
waitForInput: true,
129+
defaults: { x: 1 },
130+
}));
131+
132+
expect(warnings.filter((w) => w.includes('unknown config key'))).toEqual([]);
133+
});
134+
135+
it('stays quiet for a node type that publishes no configSchema', () => {
136+
// `decision` / `script` are deliberately schemaless (config-schemas.test.ts):
137+
// nothing is declared, so nothing can be undeclared.
138+
const { logger, warnings } = recordingLogger();
139+
const engine = engineWith(logger);
140+
141+
engine.registerFlow('f', flowWith('decision', { condition: 'a == b', whateverElse: 1 }));
142+
143+
expect(warnings.filter((w) => w.includes('unknown config key'))).toEqual([]);
144+
});
145+
146+
it('does not flag the keys of a keyValue map — those are author data', () => {
147+
// The walk descends where the schema declares structure and STOPS at a
148+
// free-form map: `filter: { status: 'stale' }` keys are data, not config keys.
149+
const { logger, warnings } = recordingLogger();
150+
const engine = engineWith(logger);
151+
152+
engine.registerFlow('f', flowWith('get_record', {
153+
objectName: 'crm_lead',
154+
filter: { status: 'stale', anythingAtAll: true },
155+
}));
156+
157+
expect(warnings.filter((w) => w.includes('unknown config key'))).toEqual([]);
158+
});
159+
});

packages/services/service-automation/src/engine.ts

Lines changed: 119 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,18 @@ import { ConnectorSchema } from '@objectstack/spec/integration';
1717
// silently returned `false`, so EVERY start-node / edge condition (record-change
1818
// `previous.*`, `budget > 100000`, …) skipped its flow. A static import binds the
1919
// engine at module load in both ESM and CJS builds.
20-
import { ExpressionEngine, validateExpression } from '@objectstack/formula';
20+
import { ExpressionEngine, validateExpression, nearestName } from '@objectstack/formula';
21+
22+
/**
23+
* The slice of a descriptor's JSON-Schema `configSchema` that the undeclared-key
24+
* walk reads (#4045). Structural only — no validation semantics.
25+
*/
26+
interface ConfigSchemaNode {
27+
type?: string;
28+
properties?: Record<string, ConfigSchemaNode>;
29+
items?: ConfigSchemaNode;
30+
additionalProperties?: unknown;
31+
}
2132
import { runIsUnscopedUserMode, flowTouchesData } from './runtime-identity.js';
2233
import { isGuardRefusal } from './guard-refusal.js';
2334

@@ -1473,6 +1484,12 @@ export class AutomationEngine implements IAutomationService {
14731484
// executeNode() already throws NO_EXECUTOR at run time for unknown types.
14741485
this.validateNodeTypes(name, parsed);
14751486

1487+
// #4045 — warn on config keys the node's descriptor does not declare
1488+
// (a `visibleIf` typo is silently accepted today). Warn-only: see
1489+
// validateNodeConfigKeys for why hard-failing would gamble on the nine
1490+
// builtins whose read-vs-declared drift has not been audited.
1491+
this.validateNodeConfigKeys(name, parsed);
1492+
14761493
// ADR-0032 §Decision 1a — parse-validate every predicate at registration,
14771494
// so a malformed condition (e.g. the #1491 `{record.x}` template-brace-in-
14781495
// CEL mistake) is a LOUD registration error with the offending source,
@@ -2717,6 +2734,107 @@ export class AutomationEngine implements IAutomationService {
27172734
}
27182735
}
27192736

2737+
/**
2738+
* Warn about node `config` keys the node type's descriptor does not declare
2739+
* (#4045 — the warn half of the unknown-key ladder).
2740+
*
2741+
* `FlowNodeSchema.config` is `z.record(z.unknown())`, so a misspelled or
2742+
* invented config key is accepted in total silence today: `visibleIf` instead
2743+
* of `visibleWhen` registers cleanly and then does nothing, which is exactly
2744+
* the failure shape that made #3528 take three passes to diagnose. The key is
2745+
* never read, so there is no runtime error to trace back — the only symptom
2746+
* is a feature that quietly does not happen.
2747+
*
2748+
* **Warn, never reject.** An undeclared key falls into three populations and
2749+
* this seam cannot yet tell them apart: an author typo (which we want to
2750+
* reject), a key the executor genuinely reads that its hand-written
2751+
* `configSchema` never declared (`notify.source` was exactly this until
2752+
* #4045 — rejecting those breaks working apps), and dead config nobody reads
2753+
* (harmless). Only 4 of the 13 schema-carrying builtins have been audited for
2754+
* the second population, so hard-failing here would gamble on the 9 that have
2755+
* not. The warning is what measures that distribution; tightening to an error
2756+
* is a later, per-key decision once the data exists, and belongs with a
2757+
* tombstone that carries the prescription (the `UNKNOWN_KEY_GUIDANCE` pattern
2758+
* in `object.zod.ts`).
2759+
*
2760+
* Soft-fail matches {@link validateNodeTypes} directly above — same function,
2761+
* same `logger.warn` channel, same reasoning about not breaking flows over a
2762+
* diagnostic. Nothing about the published `configSchema` changes, so no
2763+
* consumer (designer form generation included) sees a different shape.
2764+
*/
2765+
private validateNodeConfigKeys(flowName: string, flow: FlowParsed): void {
2766+
for (const node of flow.nodes) {
2767+
const schema = this.actionDescriptors.get(node.type)?.configSchema as ConfigSchemaNode | undefined;
2768+
// No descriptor, or a deliberately schemaless type (`decision`,
2769+
// `script`, `wait`, `subflow` publish none — see config-schemas.test)
2770+
// ⇒ nothing is declared, so nothing can be undeclared.
2771+
if (!schema) continue;
2772+
this.warnUndeclaredConfigKeys(flowName, node, schema, node.config, 'config');
2773+
}
2774+
}
2775+
2776+
/**
2777+
* Walk `value` against `schema` in lockstep, warning on keys the schema does
2778+
* not declare.
2779+
*
2780+
* Descends **only where the schema declares structure** — an object with fixed
2781+
* `properties`, or an array whose `items` do. It deliberately stops at a
2782+
* keyValue map (`additionalProperties: true` with no `properties`): those keys
2783+
* are author *data* (`filter: { status: 'stale' }`), not config keys, and
2784+
* flagging them would make the check useless noise.
2785+
*
2786+
* Descending matters rather than being thoroughness for its own sake: the
2787+
* #3528 typo class lives *inside* the `screen` field repeater, not at the top
2788+
* level. `visibleWhen` is a property of `fields[].items`, so a top-level-only
2789+
* comparison would miss `visibleIf` — the exact mistake this check exists to
2790+
* catch — while still reporting the rarer top-level ones. A test pins it.
2791+
*/
2792+
private warnUndeclaredConfigKeys(
2793+
flowName: string,
2794+
node: FlowNodeParsed,
2795+
schema: ConfigSchemaNode,
2796+
value: unknown,
2797+
path: string,
2798+
): void {
2799+
if (schema.type === 'array' && schema.items) {
2800+
if (!Array.isArray(value)) return;
2801+
value.forEach((element, index) => {
2802+
this.warnUndeclaredConfigKeys(flowName, node, schema.items!, element, `${path}[${index}]`);
2803+
});
2804+
return;
2805+
}
2806+
2807+
// A free-form map declares no properties — its keys are author data.
2808+
if (!schema.properties || schema.additionalProperties === true) return;
2809+
if (value == null || typeof value !== 'object' || Array.isArray(value)) return;
2810+
2811+
const declared = Object.keys(schema.properties);
2812+
for (const key of Object.keys(value as Record<string, unknown>)) {
2813+
const child = schema.properties[key];
2814+
if (child) {
2815+
this.warnUndeclaredConfigKeys(flowName, node, child, (value as Record<string, unknown>)[key], `${path}.${key}`);
2816+
continue;
2817+
}
2818+
// The declared set is printed ALWAYS, not only as a fallback when the
2819+
// edit-distance heuristic misses. It misses more than you would
2820+
// expect: `visibleIf` → `visibleWhen` is distance 4 against a
2821+
// threshold of 3, so the very typo this check exists to catch gets no
2822+
// suggestion. Loosening `nearestName` is the wrong fix — it is shared
2823+
// with the unknown-field and unknown-role diagnostics, where a looser
2824+
// threshold means confidently wrong suggestions over hundreds of
2825+
// candidates. A config object declares at most a dozen keys, so simply
2826+
// listing them is both cheap and complete, and the suggestion becomes
2827+
// a bonus for the cases it does catch.
2828+
const suggestion = nearestName(key, declared);
2829+
this.logger.warn(
2830+
`[flow '${flowName}'] node '${node.id}' (${node.type}): unknown config key \`${key}\` at ${path}.${key}` +
2831+
(suggestion ? ` — did you mean \`${suggestion}\`?` : '') +
2832+
` It is not declared by this node type's configSchema, so nothing reads it.` +
2833+
` Declared here: ${declared.join(', ')}.`,
2834+
);
2835+
}
2836+
}
2837+
27202838
/**
27212839
* ADR-0032 §Decision 1a — parse-validate every predicate in the flow at
27222840
* registration. Predicates are bare CEL; this catches the #1491 class

0 commit comments

Comments
 (0)