Skip to content

Commit 92f9f09

Browse files
committed
fix(connectors): rehydrate forces a full listing so containers aren't omitted
A rehydrate request set forceRehydrate but left listing incremental. For a connector that is both incremental and rehydrateOnFullSync, an unchanged container page that transcludes a changed page would be omitted from the incremental listing and never re-hydrated. Force a full (non-incremental) listing on rehydrate so every document is seen; deletion-safety guards stay armed (unlike fullSync). No-op for Confluence, which is already non-incremental.
1 parent 6e0595f commit 92f9f09

4 files changed

Lines changed: 33 additions & 10 deletions

File tree

apps/sim/lib/api/contracts/knowledge/connectors.ts

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -154,10 +154,10 @@ export const deleteKnowledgeConnectorContract = defineRouteContract({
154154
export const triggerKnowledgeConnectorSyncQuerySchema = z.object({
155155
/**
156156
* Force re-hydration: for connectors whose rendered content can drift without a
157-
* hash change (e.g. Confluence transclusions), re-fetch and re-index every
158-
* already-synced document rather than only hash-changed ones. Listing and
159-
* deletion reconciliation are unchanged from a normal sync. Defaults to the
160-
* normal hash-gated sync.
157+
* hash change (e.g. Confluence transclusions), do a full listing and re-fetch +
158+
* re-index every already-synced document rather than only hash-changed ones. The
159+
* deletion-reconciliation safety guards stay armed. Defaults to the normal
160+
* hash-gated sync.
161161
*/
162162
rehydrate: booleanQueryFlagSchema.optional().default(false),
163163
})

apps/sim/lib/knowledge/connectors/queue.test.ts

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,7 @@ describe('connector sync queue', () => {
9898
{
9999
connectorId: 'connector-1',
100100
fullSync: true,
101+
rehydrate: undefined,
101102
requestId: 'request-1',
102103
billingAttribution: BILLING_ATTRIBUTION,
103104
},
@@ -113,6 +114,20 @@ describe('connector sync queue', () => {
113114
)
114115
})
115116

117+
it('carries the rehydrate flag into the queued payload', async () => {
118+
await dispatchSync('connector-1', {
119+
billingAttribution: BILLING_ATTRIBUTION,
120+
rehydrate: true,
121+
requestId: 'request-1',
122+
})
123+
124+
expect(mockTrigger).toHaveBeenCalledWith(
125+
'knowledge-connector-sync',
126+
expect.objectContaining({ connectorId: 'connector-1', rehydrate: true }),
127+
expect.anything()
128+
)
129+
})
130+
116131
it('rejects legacy payloads without billing attribution', () => {
117132
expect(() =>
118133
assertConnectorSyncPayload({

apps/sim/lib/knowledge/connectors/queue.ts

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,9 @@ export interface ConnectorSyncPayload {
2222
/**
2323
* Force re-hydration + re-indexing of already-synced documents for connectors
2424
* whose rendered content can drift without a hash change (see
25-
* `ConnectorMeta.rehydrateOnFullSync`). Unlike `fullSync`, this does NOT alter
26-
* listing or bypass any deletion-reconciliation safety guard.
25+
* `ConnectorMeta.rehydrateOnFullSync`). Forces a full (non-incremental) listing
26+
* so every document is re-hydrated, but — unlike `fullSync` — keeps every
27+
* deletion-reconciliation safety guard armed.
2728
*/
2829
rehydrate?: boolean
2930
requestId: string

apps/sim/lib/knowledge/connectors/sync-engine.ts

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -432,11 +432,17 @@ export async function executeSync(
432432
let hasMore = true
433433
const syncContext: Record<string, unknown> = { syncRunId: generateId() }
434434

435-
// Determine if this sync should be incremental
435+
/**
436+
* Determine if this sync should be incremental. A `rehydrate` request forces a
437+
* full listing too: re-hydration must see *every* document (a container page can
438+
* be unchanged itself yet transclude a page that changed), and an incremental
439+
* listing would omit those unchanged containers, so they'd never be re-fetched.
440+
*/
436441
const isIncremental =
437442
connectorConfig.supportsIncrementalSync &&
438443
connector.syncMode !== 'full' &&
439444
!options?.fullSync &&
445+
!options?.rehydrate &&
440446
connector.lastSyncAt != null
441447
const lastSyncAt =
442448
isIncremental && connector.lastSyncAt ? new Date(connector.lastSyncAt) : undefined
@@ -445,9 +451,10 @@ export async function executeSync(
445451
* Re-hydrate and re-index connectors whose rendered content can drift without a
446452
* hash change (transclusions) — see `ConnectorMeta.rehydrateOnFullSync`. Driven
447453
* by the dedicated `rehydrate` request (the "Full resync" action) or implied by a
448-
* true `fullSync`. This ONLY affects hydration/indexing — it does not change
449-
* listing or bypass any deletion-reconciliation guard. Incremental syncs of other
450-
* connectors stay hash-gated.
454+
* true `fullSync`. It forces a full listing (above) and re-indexes unchanged
455+
* deferred docs, but — unlike `fullSync` — it does NOT bypass any
456+
* deletion-reconciliation safety guard. Incremental syncs of other connectors
457+
* stay hash-gated.
451458
*/
452459
const forceRehydrate = Boolean(
453460
(options?.rehydrate || options?.fullSync) && connectorConfig.rehydrateOnFullSync

0 commit comments

Comments
 (0)