fix(workflow-executor): tolerate LLM field name variations in findField#1585
Conversation
|
Coverage Impact ⬇️ Merging this pull request will decrease total coverage on Modified Files with Diff Coverage (1)
🛟 Help
|
|
[claude-opus-4-7] [Code conventions / defence-in-depth — not blocking] The normalized fallback runs after both exact passes ( Cheap mitigation if we ever want it: swap the two normalized const exact =
schema.fields.find(f => f.displayName === name) ??
schema.fields.find(f => f.fieldName === name);
if (exact) return exact;
const fuzzy = schema.fields.filter(
f => normalizeFieldName(f.displayName) === normalized
|| normalizeFieldName(f.fieldName) === normalized,
);
return fuzzy.length === 1 ? fuzzy[0] : undefined;Leaving it as-is is also defensible — happy to defer. |
| // Schema does constrain fieldName to exact values. However, invokeWithTool returns | ||
| // toolCall.args as-is without re-running Zod validation on the response, so the LLM can | ||
| // silently ignore the constraint and return a formatting variation (e.g. "first_name" | ||
| // instead of "firstname"). The normalized fallback catches these cosmetic mismatches. |
There was a problem hiding this comment.
I'd tighten this, for example :
// LLMs occasionally return formatting variants of field names (e.g. "first_name" for
// "firstname", "full-name" for "Full Name") even though the tool schema declares them
// as literals. Fall back to a normalized comparison so a cosmetic variation doesn't
// fail an otherwise correct step.
But it's preferential
…indField Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
c61b015 to
c3496f0
Compare
…d match in findField Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…field test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Summary
findFieldpreviously did exact string matching (===) ondisplayNameandfieldNamefirst_nameinstead offirstname) even when the tool schema constrains the value viaz.literal_,-, spaces) and lowercases before matching, so a hallucinated variation no longer kills an otherwise correct stepRoot cause
invokeWithToolreturnstoolCall.argsdirectly without re-validating against the Zod schema. Thez.literalconstraint is only used to generate the tool definition sent to the LLM — it is not enforced on the response.Test plan
update-record/read-record/load-related-recordtests passaccount.firstnamefield no longer fails when the LLM outputsfirst_namefixes PRD-214
Note
Fix
findFieldinRecordStepExecutorto tolerate LLM field name variantsLLMs may return field names in snake_case, camelCase, lowercase, or hyphenated forms. The
findFieldmethod in record-step-executor.ts already uses normalized comparison to handle these cosmetic variants; this PR clarifies the inline comment and adds a parameterized test suite in update-record-step-executor.test.ts to verify each variant resolves to the correct normalized field name.Changes since #1585 opened
RecordStepExecutor.findFieldmethod inworkflow-executorto implement two-phase field resolution: first attempting exact match ondisplayNameorfieldName, then computing fuzzy matches using normalized names (lowercased with spaces, underscores, and hyphens removed), returning a field only if exactly one fuzzy match exists, otherwise returning undefined instead of the first fuzzy match [065d4b0]update-record-step-executor.test.tsinworkflow-executorunder the 'automaticExecution: update direct (Branch B)' suite that constructs a schema with two fields normalizing to the same string and asserts the executor returns an error status [065d4b0]update-record-step-executor.test.ts[e3f8e58]Macroscope summarized c3496f0.