Skip to content

Commit 93a61be

Browse files
os-zhuangclaude
andauthored
docs(spec): fix the L3 Connector example in SYNC_ARCHITECTURE.md and gate it (#5515) (#5603)
The `sapConnector` example taught four spellings `integration/connector.zod.ts` turns down. Measured with the compiler API before the fix, verbatim: TS2353 'sourceField' does not exist in '{ source: string; target: string; ... }' TS2322 '"custom"' is not assignable to '"map" | "lookup" | "constant" | "cast" | "javascript"' TS2353 'retryPolicy' does not exist in the webhook shape TS2322 'string' is not assignable to '{ dialect: "cel"|"cron"|"template"; ... }' Fixed in the document: - fieldMappings[].sourceField/targetField -> source/target, the canonical spelling of the base protocol in shared/mapping.zod.ts. - transform { type: 'custom', function } -> { type: 'javascript', expression }, the nearest real member of the five-way discriminated union. The bare string is ExpressionInput shorthand and parses to { dialect, source }. - webhooks[].retryPolicy removed. WebhookSchema is a strictObject and already carries a curated tombstone for it (#3494): delivery retries are owned by the messaging outbox on a fixed schedule. There is no equivalent key, so the block becomes a comment saying why -- and saying that the sibling `retryConfig` is a different thing (the connector's own calls). - the annotation is now ConnectorInput (z.input), which is what an author writes. The bare `Connector` is z.infer on this file, so it is the shape a parse RETURNS. Flipping this file's 20 aliases to the #4963 X / XParsed house convention is a real but separate appetite -- filed as #5551 -- and is deliberately NOT done here. New gate: packages/spec/src/integration/connector-author-shape.test.ts, a sibling of automation/etl-author-shape.test.ts (#4963 / PR #5514) owned by the schema that owns the example. It compiles the L3 block verbatim, import line included, with a harness self-test against vacuity; classifies the document's three `Connector` blocks (two Migration-Guide sketches elide with a bare `...` and are not TypeScript); restores each of the four defects as a probe that must go red with a named diagnostic; and pins what the schema SAYS at runtime for each -- a curated tombstone for `retryPolicy`, a silent strip plus missing `source`/`target` for `sourceField` (the curated alias for that word lives on ./data's ImportFieldMappingSchema, not this one), a value verdict naming the five members for 'custom'. The document's total ```typescript block count is unchanged at 6; the sibling gate's pin holds. Fixes #5515 Claude-Session: https://claude.ai/code/session_018fxLGQdatPbBUvCgiVxg6D Co-authored-by: Claude <noreply@anthropic.com>
1 parent cbb6a5c commit 93a61be

3 files changed

Lines changed: 448 additions & 26 deletions

File tree

packages/spec/docs/SYNC_ARCHITECTURE.md

Lines changed: 42 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -210,10 +210,29 @@ Complete, production-grade integration with external systems. Includes authentic
210210

211211
### Example
212212

213+
> **`ConnectorInput` is the AUTHOR shape.** It is `z.input` of
214+
> `ConnectorSchema`, so every key carrying a `.default()``enabled`,
215+
> `status`, `connectionTimeoutMs`, `requestTimeoutMs`, all of `syncConfig`'s
216+
> `strategy` / `direction` / `realtimeSync` / `conflictResolution` /
217+
> `batchSize` / `deleteMode`, a mapping's `required` / `syncMode`, a webhook's
218+
> `method` / `timeoutMs` / `isActive` / `signatureAlgorithm` — is optional when
219+
> you write a connector, and `syncConfig.schedule` takes the bare cron string
220+
> the schema wraps for you. Annotate the **result** of
221+
> `ConnectorSchema.parse(…)` with the bare **`Connector`**, which is `z.infer`:
222+
> there those keys are all present and `schedule` is already the
223+
> `{ dialect: 'cron', source }` envelope. Note the asymmetry with L2 above,
224+
> where the bare `ETLPipeline` *is* the author shape and the parse result is
225+
> `ETLPipelineParsed``integration/connector.zod.ts` has not been moved onto
226+
> that house convention yet (#5551). The example below states the defaulted
227+
> keys anyway, because it is a tour of the surface; the Migration Guide's
228+
> sketches omit them, because that is what ordinary authoring looks like.
229+
> To have the literal validated as you write it, prefer `defineConnector(…)`,
230+
> which takes this same input shape and returns the parsed one.
231+
213232
```typescript
214-
import { Connector } from '@objectstack/spec/integration';
233+
import type { ConnectorInput } from '@objectstack/spec/integration';
215234

216-
const sapConnector: Connector = {
235+
const sapConnector: ConnectorInput = {
217236
name: 'sap_erp_connector',
218237
label: 'SAP ERP Integration',
219238
type: 'saas',
@@ -241,22 +260,28 @@ const sapConnector: Connector = {
241260
deleteMode: 'soft_delete'
242261
},
243262

244-
// Field Mappings with Transformations
263+
// Field Mappings with Transformations.
264+
// The keys are `source` / `target` — the canonical spelling of the base
265+
// protocol in `shared/mapping.zod.ts`, which every mapping surface extends.
245266
fieldMappings: [
246267
{
247-
sourceField: 'customer_number',
248-
targetField: 'customer_id',
268+
source: 'customer_number',
269+
target: 'customer_id',
249270
dataType: 'string',
250271
required: true,
251272
syncMode: 'bidirectional'
252273
},
253274
{
254-
sourceField: 'order_value',
255-
targetField: 'order_total',
275+
source: 'order_value',
276+
target: 'order_total',
256277
dataType: 'number',
278+
// `transform.type` is a discriminated union with exactly five members:
279+
// `constant` / `cast` / `lookup` / `javascript` / `map`. The bare string
280+
// below is `ExpressionInput` shorthand — the schema wraps it into an
281+
// `{ dialect, source }` envelope on parse.
257282
transform: {
258-
type: 'custom',
259-
function: 'value => parseFloat(value) / 100' // Convert cents to dollars
283+
type: 'javascript',
284+
expression: 'value / 100' // Convert cents to dollars
260285
},
261286
syncMode: 'bidirectional'
262287
}
@@ -270,11 +295,11 @@ const sapConnector: Connector = {
270295
events: ['record.created', 'record.updated'],
271296
secret: process.env.WEBHOOK_SECRET!,
272297
signatureAlgorithm: 'hmac_sha256',
273-
retryPolicy: {
274-
maxRetries: 3,
275-
backoffStrategy: 'exponential',
276-
initialDelayMs: 1000
277-
},
298+
// (`retryPolicy` sat here until #3494 retired it — webhook delivery
299+
// retries are owned by the messaging outbox on a fixed schedule, and the
300+
// authored policy was never read. There is no replacement, and it is a
301+
// different thing from `retryConfig` below, which governs the calls this
302+
// connector MAKES.)
278303
timeoutMs: 30000,
279304
isActive: true
280305
}
@@ -283,7 +308,7 @@ const sapConnector: Connector = {
283308
// (`rateLimitConfig` sat here until #4911 retired it — no outbound
284309
// rate-limiting engine ever existed. Throttle at the provider/gateway.)
285310

286-
// Retry Configuration
311+
// Retry Configuration — for the connector's own outbound requests
287312
retryConfig: {
288313
strategy: 'exponential_backoff',
289314
maxAttempts: 5,
@@ -378,7 +403,7 @@ When a connector's declarative sync needs complex transformations:
378403

379404
**Before (L3 `syncConfig`):**
380405
```typescript
381-
const connector: Connector = {
406+
const connector: ConnectorInput = {
382407
name: 'orders',
383408
type: 'saas',
384409
authentication: { type: 'api-key', ... },
@@ -423,7 +448,7 @@ const pipeline: ETLPipeline = {
423448

424449
**After (L3):**
425450
```typescript
426-
const connector: Connector = {
451+
const connector: ConnectorInput = {
427452
authentication: { type: 'oauth2', ... },
428453
webhooks: [...],
429454
retryConfig: { ... }

packages/spec/src/automation/etl-author-shape.test.ts

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -113,15 +113,17 @@ describe('[#4963] SYNC_ARCHITECTURE.md pipeline examples compile', () => {
113113
// assertion below pass over an empty program — the way a gate goes dormant.
114114
expect(pipelineBlocks.length, 'ETLPipeline examples in SYNC_ARCHITECTURE.md').toBe(3);
115115
// The other three are the L3 `Connector` examples, out of this gate's scope
116-
// because they belong to `integration/connector.zod.ts`. Two of them are
117-
// Migration-Guide sketches that elide with a bare `...`, which is not
118-
// TypeScript. The third — the full `sapConnector` example — is NOT exempt on
119-
// its merits: run through this same harness it reports four diagnostics, and
120-
// three of them are keys or values the schema REJECTS (`sourceField` /
121-
// `targetField` for `source` / `target`, `transform.type: 'custom'`,
122-
// `webhooks[].retryPolicy`). That is filed as #5515, not fixed here, because
123-
// the fourth diagnostic is `Connector` being `z.infer` — this issue's twin
124-
// on a file whose migration surface is NOT empty, so it needs its own ruling.
116+
// because they belong to `integration/connector.zod.ts` — and covered, since
117+
// #5515, by that file's own gate: `integration/connector-author-shape.test.ts`
118+
// classifies the same three (two Migration-Guide sketches that elide with a
119+
// bare `...`, which is not TypeScript; one full `sapConnector` example) and
120+
// compiles the third. When this pin was written that example reported four
121+
// diagnostics, three of them keys or values the schema REJECTS (`sourceField`
122+
// / `targetField` for `source` / `target`, `transform.type: 'custom'`,
123+
// `webhooks[].retryPolicy`); those are fixed in the document. The fourth was
124+
// `Connector` being `z.infer` — this issue's twin on a file whose migration
125+
// surface is NOT empty — and it is solved there by annotating the example
126+
// with `ConnectorInput`; the alias flip itself is still open as #5551.
125127
// The total is pinned rather than left open so that ADDING a block to this
126128
// document is a decision someone has to make on purpose: a new ETL example
127129
// is picked up automatically by the selector above, and anything else

0 commit comments

Comments
 (0)