fix(confluence): index mirrored/included page content via rendered view format by waleedlatif1 · Pull Request #5746 · simstudioai/sim

waleedlatif1 · 2026-07-17T20:55:05Z

Summary

KB Confluence connector fetched page bodies with body-format=storage, which only contains unexpanded macro references (Include Page / Excerpt Include). "Mirrored" articles that pull in another page's body were stripped to empty content by htmlToPlainText and never synced.
Switch getDocument to body-format=view — the v2 single-item page/blogpost GET supports it, and it returns server-rendered HTML with built-in include/excerpt macros expanded inline, so the included text is now indexed.
Scope is intentionally limited to the connector's per-document body fetch. contentHash is keyed on version number (not body), so this does not trigger re-sync churn.

Not covered (by Confluence's design)

Smart Link embeds (the /embed/{id} "embed" content type) are not pages/blogposts and have no body in the API — GET /embeds/{id} returns only title + embedUrl. That content is rendered client-side from the source system and cannot be extracted via the Confluence API. This PR does not attempt to index embed bodies.

Type of Change

Bug fix

Testing

Tested manually. Verified body-format=view is accepted on the v2 single-item /pages/{id} and /blogposts/{id} GET endpoints and returns rendered HTML with built-in macros expanded; htmlToPlainText handles the rendered HTML identically to storage. Typecheck/lint clean.

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…ew format The KB connector fetched page bodies as body-format=storage, which only carries unexpanded macro references (Include Page / Excerpt Include). Those 'mirrored' articles were stripped to empty content by htmlToPlainText and never synced. Switch getDocument to body-format=view (supported on the v2 single-item page/blogpost GET) so built-in include/excerpt macros render inline and the included text is indexed.

vercel · 2026-07-17T20:55:11Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		Jul 17, 2026 10:15pm

cursor · 2026-07-17T20:55:16Z

PR Summary

Medium Risk
Changes core connector sync classification and re-indexing behavior plus Confluence content extraction; well-covered by tests but affects knowledge base document correctness and sync cost on full resync.

Overview
Fixes Confluence KB indexing for pages that include other pages via macros, and adds a way to refresh already-synced content when hashes do not change.

Confluence extraction: Per-document fetches now use body-format=view instead of storage, so Include Page / Excerpt macros are expanded before htmlToPlainText. Listing stubs embed a view marker in contentHash so existing indexed pages are treated as changed once and re-hydrated with the new extraction.

Full resync (rehydrate): Connectors can set rehydrateOnFullSync (Confluence does). Manual sync accepts ?rehydrate=true, which flows through the queue/worker into executeSync. That forces a full listing, re-fetches deferred docs even when the hash is unchanged, and re-indexes without relaxing deletion-reconciliation guards (unlike fullSync). The connectors UI shows a Sync now / Full resync menu only for those connector types.

^{Reviewed by Cursor Bugbot for commit 92f9f09. Configure here.}

greptile-apps · 2026-07-17T20:56:43Z

Greptile Summary

This PR indexes rendered Confluence content and adds a manual recovery path for stale transclusions. The main changes are:

Fetches the rendered view body so included page and excerpt content is expanded.
Invalidates previously indexed storage-format content with a representation marker.
Adds a manual full rehydrate action for Confluence connectors.
Forces a complete listing and re-indexes unchanged deferred documents during rehydration.
Propagates the rehydrate flag through the API, queue, worker, and sync engine.

Confidence Score: 5/5

This looks safe to merge.

The full rehydrate action now reaches the sync engine through both execution paths.
Rehydration forces a complete listing, so unchanged container pages are not omitted.
Unchanged deferred documents are fetched and re-indexed despite version-based hashes.
Deletion safeguards remain active during manual rehydration.
No blocking issues were found in the updated code.

Important Files Changed

Filename	Overview
apps/sim/connectors/confluence/confluence.ts	Uses rendered Confluence bodies and changes the hash representation marker to refresh previously indexed documents.
apps/sim/lib/knowledge/connectors/sync-engine.ts	Adds complete-listing rehydration while retaining deletion safeguards and bypassing stale version-hash checks.
apps/sim/lib/knowledge/connectors/queue.ts	Validates and propagates the rehydrate option through queued and local sync execution.
apps/sim/background/knowledge-connector-sync.ts	Forwards the rehydrate option from worker payloads to the sync engine.
apps/sim/lib/api/contracts/knowledge/connectors.ts	Adds the optional rehydrate query flag with a false default.
apps/sim/app/workspace/[workspaceId]/knowledge/[id]/components/connectors-section/connectors-section.tsx	Adds normal sync and full rehydrate actions for eligible connectors.

_{Reviews (7): Last reviewed commit: "fix(connectors): rehydrate forces a full..." | Re-trigger Greptile}

The version-based contentHash meant already-synced mirrored documents (with stale empty content) classified as 'unchanged' and never re-hydrated with the new rendered view content. Embed a body-representation marker in the hash so a representation change invalidates every previously-synced Confluence document, forcing a one-time re-hydration that picks up the expanded include/excerpt text.

waleedlatif1 · 2026-07-17T21:03:06Z

@greptile

waleedlatif1 · 2026-07-17T21:03:07Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 0980add. Configure here.}

…ions) Version-based change detection can't see when a Confluence page's rendered view changes because an *included* page was edited (the container's version doesn't bump). Add a 'Full resync' path so that drift can be recovered: - ConnectorMeta.rehydrateOnFullSync flag (set for Confluence) - on fullSync, classifyExternalDoc promotes unchanged deferred docs to update and the hydration guard re-indexes unconditionally, so rendered content is refreshed - fullSync threaded through the manual sync contract (query param), route, and hook - 'Sync now' / 'Full resync' dropdown on the connector card Incremental syncs stay hash-gated and cheap; only the deliberate full resync pays the re-index cost.

waleedlatif1 · 2026-07-17T21:34:36Z

@greptile

waleedlatif1 · 2026-07-17T21:34:37Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 5789a6a. Configure here.}

…tics The transclusion refresh only needs re-hydration, but reusing the fullSync flag also activated its deletion-cleanup semantics — which bypass three previously unreachable safety guards (empty-listing wipe, listingCapped, and the >50% mass-deletion threshold). Since fullSync had no caller before this PR, the new 'Full resync' button would have exposed all three to any KB editor. Introduce a dedicated 'rehydrate' request that ONLY forces re-hydration + re-index of already-synced docs. Listing and deletion reconciliation are identical to a normal sync (all safety guards stay armed). fullSync's cleanup semantics remain dormant and untouched.

waleedlatif1 · 2026-07-17T21:50:07Z

@greptile

waleedlatif1 · 2026-07-17T21:50:09Z

@cursor review

…rted connectors Cleanup/simplify pass over the connector changes: - use shared booleanQueryFlagSchema for the rehydrate query param (typed boolean at the boundary instead of a hand-rolled 'true'/'false' string enum) - move rehydrateOnFullSync onto the client-safe ConnectorMeta so the UI can gate on it - only Confluence (rehydrateOnFullSync) shows the Sync now / Full resync dropdown; every other connector keeps its original one-click sync button (Full resync is a no-op for them, and this restores the pre-change one-click UX) - wrap the sync trigger in a span so its tooltip still shows while disabled (cooldown)

waleedlatif1 · 2026-07-17T22:05:08Z

@greptile

waleedlatif1 · 2026-07-17T22:05:09Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

1 issue from previous review remains unresolved.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 60d2b80. Configure here.}

executeConnectorSyncJob (the production async sync path) destructured only fullSync from the payload and forwarded only fullSync to executeSync, silently dropping rehydrate. A manual Full resync would therefore never re-hydrate on the default Trigger.dev path. Forward rehydrate too.

waleedlatif1 · 2026-07-17T22:07:52Z

@greptile

waleedlatif1 · 2026-07-17T22:07:53Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 6e0595f. Configure here.}

… omitted A rehydrate request set forceRehydrate but left listing incremental. For a connector that is both incremental and rehydrateOnFullSync, an unchanged container page that transcludes a changed page would be omitted from the incremental listing and never re-hydrated. Force a full (non-incremental) listing on rehydrate so every document is seen; deletion-safety guards stay armed (unlike fullSync). No-op for Confluence, which is already non-incremental.

waleedlatif1 · 2026-07-17T22:16:04Z

@greptile

waleedlatif1 · 2026-07-17T22:16:05Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 92f9f09. Configure here.}

greptile-apps Bot reviewed Jul 17, 2026

View reviewed changes

Comment thread apps/sim/connectors/confluence/confluence.ts

vercel Bot temporarily deployed to Preview July 17, 2026 21:02 Inactive