fix(knowledge): bound tombstone-forced full syncs, resurrect kept docs on connector delete

Two more real findings from this review round:

- A document whose refresh keeps failing every sync (e.g. permanently
  oversized) never resurrects and never hard-deletes (it's present in
  the listing, just unreadable) — correct on its own, but it also never
  stops being counted by hasTombstonedDocs, so it would force a full
  listing for this connector forever, permanently disabling incremental
  sync on account of one stuck document. Bounded the check to the same
  RETRY_WINDOW_DAYS already used for the stuck-document retry sweep
  below: past the window, this connector stops forcing full syncs on
  the stuck document's account. The document itself is unaffected —
  it stays tombstoned either way, matching the existing 'last-known-good
  forever' tolerance this codebase already accepts for any document
  whose hydration keeps failing.

- The connector-DELETE route's deleteDocuments=false path (kept docs)
  counted tombstoned documents but never resolved them one way or the
  other. With the connector gone, there's no future sync left to ever
  confirm or resurrect them, so they'd become permanent invisible
  orphans holding storage forever. Since 'kept' documents become normal
  standalone KB entries once detached from their connector, resurrect
  any pending-removal ones as part of that transition — consistent
  with what happens to their non-tombstoned sibling documents.

Author: waleedlatif1

apps/sim/app/api/knowledge/[id]/connectors/[connectorId]/route.ts

@@ -335,12 +335,18 @@ export const DELETE = withRouteHandler(async (request: NextRequest, { params }:
         .from(document)
         .where(and(eq(document.connectorId, connectorId), isNull(document.archivedAt)))
 
+      const documentIds = docs.map((doc) => doc.id)
       if (deleteDocuments) {
-        const documentIds = docs.map((doc) => doc.id)
         if (documentIds.length > 0) {
           await tx.delete(embedding).where(inArray(embedding.documentId, documentIds))
           await tx.delete(document).where(inArray(document.id, documentIds))
         }
+      } else if (documentIds.length > 0) {
+        // Kept documents become normal standalone KB entries once their connector
+        // is gone — resurrect any pending-removal ones rather than leaving them
+        // invisible tombstones with no future sync left to ever confirm or
+        // resurrect them.
+        await tx.update(document).set({ deletedAt: null }).where(inArray(document.id, documentIds))
       }
 
       const deletedConnectors = await tx

apps/sim/lib/knowledge/connectors/sync-engine.ts

@@ -516,14 +516,23 @@ export async function executeSync(
     let hasMore = true
     const syncContext: Record<string, unknown> = { syncRunId: generateId() }
 
+    /**
+     * Bounded to the same retry window as the stuck-document retry below: a
+     * document whose refresh keeps failing every sync (e.g. permanently
+     * oversized) would otherwise be a tombstone that never resolves, forcing a
+     * full listing — and its listing-time overhead — for this connector
+     * forever. Past the window, this connector stops forcing full syncs on its
+     * account; the document itself is unaffected and stays tombstoned either way.
+     */
     const hasTombstonedDocs = await db
       .select({ id: document.id })
       .from(document)
       .where(
         and(
           eq(document.connectorId, connectorId),
           isNull(document.archivedAt),
-          isNotNull(document.deletedAt)
+          isNotNull(document.deletedAt),
+          gt(document.deletedAt, new Date(Date.now() - RETRY_WINDOW_DAYS * 24 * 60 * 60 * 1000))
         )
       )
       .limit(1)