Skip to content

Commit 023e3be

Browse files
committed
fix(confluence): fix nested-block duplication in callout text extraction
Greptile P1: filtering the found blocks to only top-level ones still wasn't enough — a nested block (an outer <li> containing its own nested <ul><li>, a <td> containing a <blockquote>) matched the selector once, but its .text() call recurses into and flattens its own matched descendants with no separator, reproducing the exact word-fusion bug one level deeper (and any duplicate-selection would have double-counted the same text). Replaces the block-selector approach with a recursive text-node walk: extractBlockJoinedText now visits every text node individually and joins them all with a single space, so word boundaries are preserved at every nesting depth with no double-counting, matching the pattern html-parser.ts already uses elsewhere in this codebase for the same class of problem.
1 parent f379ebc commit 023e3be

2 files changed

Lines changed: 56 additions & 19 deletions

File tree

apps/sim/connectors/confluence/confluence.test.ts

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -163,4 +163,35 @@ describe('preserveConfluenceCallouts', () => {
163163
expect(result).not.toContain('sentence.Second')
164164
}
165165
)
166+
167+
it.concurrent(
168+
'does not duplicate or fuse text from a nested list inside a callout body (nesting regression)',
169+
() => {
170+
const html =
171+
'<div class="confluence-information-macro confluence-information-macro-note">' +
172+
'<div class="confluence-information-macro-body">' +
173+
'<ul><li>Outer item' +
174+
'<ul><li>Nested item A</li><li>Nested item B</li></ul>' +
175+
'</li><li>Outer item two</li></ul>' +
176+
'</div></div>'
177+
const result = preserveConfluenceCallouts(html)
178+
// Each nested <li>'s text must appear exactly once, not duplicated by the
179+
// outer <li> also being matched and its .text() recursing into it.
180+
const occurrences = (result.match(/Nested item A/g) ?? []).length
181+
expect(occurrences).toBe(1)
182+
expect(result).not.toContain('Nested item ANested item B')
183+
expect(result).toContain('Outer item Nested item A Nested item B Outer item two')
184+
}
185+
)
186+
187+
it.concurrent('does not fuse text from a blockquote nested inside a table cell', () => {
188+
const html =
189+
'<div class="panel"><div class="panelContent">' +
190+
'<table><tr><td>Cell text<blockquote><p>quoted text</p></blockquote>after quote</td></tr></table>' +
191+
'</div></div>'
192+
const result = preserveConfluenceCallouts(html)
193+
expect(result).not.toContain('quotedtext')
194+
expect(result).not.toContain('textafter')
195+
expect(result).toContain('Cell text quoted text after quote')
196+
})
166197
})

apps/sim/connectors/confluence/confluence.ts

Lines changed: 25 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -19,28 +19,34 @@ const CALLOUT_LABELS: Record<string, string> = {
1919
}
2020

2121
/**
22-
* Selectors for elements whose own text is a natural word-boundary unit —
23-
* paragraphs, list items, headings, table cells, quotes. Cheerio's `.text()`
24-
* concatenates every descendant text node with no separator at all, so pulling
25-
* a macro body's text in one call fuses adjacent blocks together (e.g. a
26-
* `<p>...for:</p>` immediately followed by `<li>GitLab</li>` becomes
27-
* `for:GitLab`, corrupting the very word boundaries RAG chunking depends on).
28-
* Extracting block-by-block and joining with a single space keeps each block's
29-
* text intact and separated, matching how `htmlToPlainText` already treats the
30-
* rest of the page.
22+
* Cheerio's `.text()` concatenates every descendant text node with no
23+
* separator at all, so pulling a macro body's text in one call fuses adjacent
24+
* blocks together (e.g. a `<p>...for:</p>` immediately followed by
25+
* `<li>GitLab</li>` becomes `for:GitLab`, corrupting the very word boundaries
26+
* RAG chunking depends on). Selecting "block" elements and taking `.text()` on
27+
* each independently isn't enough either — nested blocks (a `<li>` containing
28+
* its own nested `<ul><li>`, a `<td>` containing a `<blockquote>`) still fuse
29+
* together the same way, one level deeper, since a matched outer block's
30+
* `.text()` recurses into and flattens its matched descendants too. Walking
31+
* every text node individually and joining them all with a single space
32+
* fixes both cases at once, with no double-counting, regardless of nesting
33+
* depth — matching how `html-parser.ts` already walks HTML for the same
34+
* reason elsewhere in this codebase.
3135
*/
32-
const BLOCK_TEXT_SELECTOR = 'p, li, h1, h2, h3, h4, h5, h6, td, th, blockquote, pre'
33-
3436
function extractBlockJoinedText($: cheerio.CheerioAPI, $el: cheerio.Cheerio<any>): string {
35-
const blocks = $el.find(BLOCK_TEXT_SELECTOR)
36-
if (blocks.length === 0) {
37-
return $el.text().trim()
37+
const parts: string[] = []
38+
const visit = ($node: cheerio.Cheerio<any>) => {
39+
$node.contents().each((_, child) => {
40+
if (child.type === 'text') {
41+
const text = $(child).text().trim()
42+
if (text) parts.push(text)
43+
} else if (child.type === 'tag') {
44+
visit($(child))
45+
}
46+
})
3847
}
39-
return blocks
40-
.map((_, block) => $(block).text().trim())
41-
.get()
42-
.filter(Boolean)
43-
.join(' ')
48+
visit($el)
49+
return parts.join(' ').trim()
4450
}
4551

4652
/**

0 commit comments

Comments
 (0)