Skip to content

Commit 6aa3e0e

Browse files
authored
fix(logs): match copilot log-grep patterns with RE2 (linear time) (#5987)
* fix(logs): match copilot log-grep patterns literally, never as a regex `grepSpans` compiled a caller-supplied pattern with a bare `new RegExp`, whose only guard was a `catch` for syntax errors. Trace text is attacker-influenced — a workflow can emit arbitrarily long uniform runs into its own block outputs — and matching runs synchronously on the shared event loop, so an authenticated caller could choose both the pattern and the input and stall every request on the instance. Patterns are now matched as an escaped, case-insensitive literal. Screening was implemented first and abandoned, because each rule only excludes the shapes someone thought to enumerate: - `safe-regex2` (already used by the guardrails validator) documents itself as having false negatives. It screens star height only, so it passes `(a|a)*b`, measured >61s on V8 at 30 characters. - Rejecting quantified groups on top of that catches `(a|a)*b`, and every attack `safe-regex2` catches, but still passes `a*a*b` — adjacent quantifiers over overlapping character sets — measured 213s on JSC and 132s on V8 over a 10k-character run, well inside what a block output can hold. An escaped literal is linear on every engine, needs no dependency, and does not depend on which runtime serves the request. `safe-regex2` stays a dependency for its existing guardrails/PII callers; it is simply not relied on here. `query_logs` loses regex matching. Its catalog entry describes `pattern` as "greps" rather than promising regex, but it is generated from a contract in another repository and cannot be updated here, so a `patternNotice` is returned whenever a pattern contains regex syntax — the caller is told the pattern was taken literally instead of reading zero matches as "not in the trace". Two bounds are kept as backstops: a cumulative match-time budget charging only time inside `test`/`exec` (never the blob-store reads the scan awaits between matches, which would truncate slow-but-legitimate greps under load), and a total scanned-character cap. * improvement(logs): only flag genuine regex intent, document the truncated invariant Review follow-ups on the literal log-grep. The `patternNotice` fired on any regex metacharacter, which includes `.` — so ordinary literal searches (`example.com`, `file.pdf`, `v1.2.3`, `block_1.output`) were told their pattern "was not interpreted as a regex", inviting the agent to retry a search that had in fact worked. Measured 10 false notices across 19 realistic literal searches. `REGEX_INTENT` now keys on actual regex intent — escape classes, character class, group, alternation, a leading/trailing anchor, or a repetition quantifier — which drops that to 0/19 while still flagging all 10 regex attempts tested. `truncated` gains a TSDoc block stating its invariant: it reports that trace was left unread, not that a budget was reached. Every point that skips work goes through `done()`, which sets it, so a budget exhausted by the final match correctly reports `false`. Raised by Cursor Bugbot; behavior is right, the contract was just implicit. Also drops `snippetAround`'s `maxChars` parameter, which every caller passed from the state object it already receives. * fix(logs): match log-grep patterns with RE2 instead of dropping regex support Restores full regex support on copilot log-grep, which the previous commits in this PR had removed to close the ReDoS. Deleting the capability was the wrong trade: RE2JS is a port of RE2 that matches in time linear in the input, so no pattern can blow up regardless of the text, and callers keep their regexes. Measured through the real compile path, against a 100k-character adversarial input — 10x the run that took 213s on JSC / 132s on V8: (a+)+$ 13.0ms (\w+\s?)*$ 10.7ms (a|a)*b 6.7ms (x+x+)+y 4.0ms a*a*b 8.1ms ^(\d+)*$ 0.0ms Two escape hatches keep this honest: - A pattern with no regex metacharacter takes the built-in engine. Semantics are identical when there is nothing to interpret, and RE2JS costs ~100x more per byte (~25ms/MB), so the common plain-text search stays native. - Lookaround and backreferences are not in RE2. RE2JS rejects them at compile time, so those fall back to a literal and return a `patternNotice` naming the constructs — a narrow, reported gap rather than a silent behavior change. The match-time and scanned-character budgets stop being formalities: RE2JS's throughput is what they now bound, not backtracking. Also collapses the old test-then-exec double scan — `compilePattern` returns a single `find` returning a match index, which `recordIfMatch` passes straight to `snippetAround`, so each field is scanned once instead of twice. re2js@2.8.6 is MIT, pure JavaScript (no native addon, so nothing changes for the oven/bun image), server-only, and published 2026-07-05 — clearing the 7-day bunfig minimumReleaseAge gate without an exclusion. * fix(security): route every caller-supplied regex through the linear-time engine Extends the RE2 fix to the three other places that compiled a caller-supplied pattern and ran it on the shared event loop, behind one shared helper — `lib/core/security/linear-regex.ts` — rather than four local copies. - `lib/copilot/vfs/operations.ts` — copilot VFS grep took a caller pattern over caller-supplied file content with no screen at all. - `lib/guardrails/validate_regex.ts` (`validateRegex`) — a guardrail rule's pattern matched against LLM output, screened only by `safe-regex2`. - `lib/chunkers/regex-chunker.ts` — the KB chunker's split pattern. This one screened for catastrophic backtracking by running the pattern against six probes including `'a'.repeat(10000)` and rejecting anything over 50ms — but it measured elapsed time *after* the match returned, so the screen was the denial of service it existed to prevent (`a*a*b` on that probe: 213s). The probe is deleted rather than repaired. Deliberately unchanged: `validateRegexPattern`. Those patterns are persisted for Presidio, a separate service where a slow pattern times out instead of stalling this loop, and Presidio's Python engine accepts constructs RE2 does not — so narrowing it to the RE2 subset would reject patterns that work. Its TSDoc now says plainly that its `safe-regex2` screen is a courtesy, not a ReDoS defense, and points in-process callers at `compileLinearRegex`. `safe-regex2` therefore stays a dependency, used only there. Lookaround is the standard way to split while keeping the delimiter (`(?=#\s)`) and RE2 cannot represent it, so the chunker keeps the built-in engine for those patterns — the only remaining path that can backtrack, bounded by the existing 500-character cap. RE2JS `split` was verified identical to `String.split` across six representative chunking patterns. Adds 27 tests for the shared helper (every catastrophic pattern asserted linear over a 50k-character input) plus regression tests at each site, plus the first tests for `validateRegex`. Also removes the now-duplicated `escapeRegExp` and literal-finder locals from `log-views.ts`. * fix(security): drop safe-regex2, keep lookaround splits on the linear engine Audit follow-ups, plus the Cursor round-4 finding. Removes `safe-regex2` entirely. Its last caller was `validateRegexPattern`, the PII custom-pattern boundary, where it was pure cost: it screens star height alone and passes `(a|a)*b` and `a*a*b`, while rejecting patterns that work — lookbehind and optional groups could not be saved as custom PII rules at all. Nor could any screen be sound there: those patterns execute in Presidio's Python engine, which backtracks on shapes RE2 accepts, so RE2-representability says nothing about their runtime. Validation is now syntax-only and the dependency is gone. Cursor flagged the chunker's lookaround fallback as reintroducing backtracking, which was correct. The fallback is removed. Verified first that no bounded probe can replace it: at a probe size small enough to survive an exponential pattern (24 chars), `(?=a*a*b)` completes in 0.0ms and passes, because polynomial blowup only shows on long input — small probes miss it and large probes hang, the same trap as the original guard. Instead `compileLookaroundSplit` puts the two idioms that actually need lookaround onto RE2: splitting on `(?=X)` is slicing at every match start of X, and on `(?<=X)` at every match end, neither of which needs the assertion. Both `(?=#\s)` and the pre-existing `(?<=</section>)` case keep working, now in linear time; anything else RE2 cannot represent is rejected with an actionable message rather than run on the backtracking engine. Also from the audit: - `literalRegex` recompiled its `RegExp` on every call to dodge `lastIndex` state from the `g` flag. VFS grep calls it per line, so that was a compile per line. It now keeps one non-global instance for `test`/`find` and a global one for `split`. - `validateRegex` and VFS grep log a warning when a pattern degrades. A guardrail rule that used lookaround now fails closed, which reads as the guardrail tripping on every input, and VFS grep's return shape has nowhere to report that a pattern was taken literally — both need to be findable in logs. Measured: RE2 costs 6-13x native on per-line matching (20k-line, 1.66MB file greps in 7ms), against ~100x on single multi-megabyte strings. * test(pii): drop the custom-pattern editor's backtracking-screen assertion CI caught this; my local runs only covered `lib/**` and missed the `.tsx` counterpart of removing `safe-regex2`. The editor asserted that `(a+)+$` renders "potentially unsafe". That screen is gone, so the assertion is inverted: the test now pins that `(a+)+$`, lookbehind and optional groups all render cleanly, which is the point of the removal — they are valid Presidio patterns that could not be saved before. Syntactically invalid patterns still surface "Invalid regex". * fix(chunkers): support lookaround with an affix, and keep JS whitespace semantics Three independent reviewers with no shared context audited the RE2 migration. Two found the same high-severity regression, and one found the divergence that silently rewrites stored data. Both are fixed here. **Lookaround with an affix threw.** `compileLookaroundSplit` only accepted a pattern that was *entirely* one assertion, so `(?<=\.)\s+` — split after the period — and `(?<=[.!?])\s+(?=[A-Z])` — the textbook sentence splitter — fell through to the chunker's `throw`. Those are persisted KB configs, and the compile runs in the constructor, so an existing knowledge base would stop re-ingesting with a hard error. The TSDoc claiming the bare assertions "cover the reason a split pattern reaches for lookaround" was simply wrong. Fixed generally rather than by adding another special case: splitting never needs the assertion, only the span the delimiter consumes, so `(?<=X)Y(?=Z)` compiles to `(?:X)(Y)(?:Z)` and group 1's span is exactly what `String.split` removes. One rule now covers every combination, and the four shapes above match `String.prototype.split` output exactly. **`\s` silently lost 14 codepoints.** RE2's `\s` is ASCII-only; ECMAScript's also covers `\v`, NBSP, U+2028/9, U+202F, U+3000, U+FEFF and the rest of `\p{Zs}`. A reviewer measured 9 of 35 realistic pattern/document pairs chunking differently through the real `RegexChunker` — NBSP from PDF/DOCX/HTML extraction, U+3000 in CJK, U+202F in French text. That rewrites stored chunk boundaries and embeddings for a document that never changed, with nothing thrown and nothing logged. `translateToRe2` now rewrites `\s`/`\S` to the verified-equivalent RE2 class, and `\uXXXX` to `\x{XXXX}` (RE2 rejects the former outright, turning a non-ASCII delimiter into a hard failure). Also from the audit: - `truncated`'s new TSDoc asserted an invariant the code violates — reaching `maxMatches` sets it even when nothing was left unread. Reworded to what it actually means. - The chunker's rejection message blamed "backreferences and lookaround" for repeat-count and `\uXXXX` rejections too. - `escapeRegExp` was a byte-identical copy of `executor/constants.ts:472` and exported for nobody; now module-private. - The guardrails wand prompt told the LLM to emit "valid JavaScript regex", which reliably produces `(?=.*\d)`-style patterns that now fail every check. - The PII editor's TSDoc still described the removed backtracking screen. - VFS grep's TSDoc omitted that invalid patterns now match literally. Test gaps the audit exposed: the match-time budget's only mechanism was unguarded (deleting the accumulation left every test green) — now covered by a mutation-verified test; the escape test survived dropping `.` from the class; the split-parity test dodged the one real divergence; and two test names described the built-in engine that no longer runs. * fix(security): correct three defects in the lookaround split decomposition A fourth reviewer, given no context beyond "these scanners are new and unreviewed", differential-tested them against the built-in engine over ~62k comparisons and found three real defects in code added an hour earlier. All were reachable through RegexChunker with plausible patterns, and all silently produced different chunks rather than failing. 1. **Top-level alternation was reshaped into a different pattern.** `(?<=\.)\s+|\n\n` means `((?<=\.)\s+)|(\n\n)`, but decomposing it produced `(?:\.)(\s+|\n\n)` — demanding the period before *both* branches. It could lose boundaries (`"Heading\n\nAlpha beta.\nGamma delta."` split in 2, not 3) and invent them. No grouping recovers the original meaning, so patterns whose middle alternates at the top level are now declined outright, which also removes an asymmetry: `\n\n|(?<=\.)\s` already threw. 2. **A capturing group inside the lookbehind stole group 1.** `(?<=(a))b` cut at the assertion instead of the delimiter, and where the group did not participate `start(1)` was -1, so `find` and `test` contradicted each other. The middle is now captured by name, immune to numbering. 3. **Consuming the assertion text swallowed every other boundary.** The compiled form matched `behind + middle + ahead`, so any boundary whose lookahead doubles as the next one's lookbehind was skipped: `(?<=\w)\s+(?=[A-Z])` over `"A B C D"` split 2 of 3 gaps. Each iteration now resumes at the end of the delimiter rather than the end of the match. This was documented as affecting only self-overlapping delimiters; that was wrong and far too narrow. The reviewer confirmed `translateToRe2` and `closingParen` clean across exhaustive escape-state, character-class, and paren-matching probes, and that nothing added is superlinear. One divergence remains and is now documented rather than overstated: when delimiters themselves overlap (a single-character middle whose matches abut, as in `(?<=\w).(?=\w)`), a match starting behind the cursor is dropped instead of emitting the empty segment the built-in engine produces. Splitters that consume whitespace or punctuation between tokens do not overlap and match exactly. * test(security): pin engine parity with a differential suite Every defect in this module has been a silent semantic divergence rather than a crash, and each was found by comparing engines over a corpus in a throwaway script. That comparison now lives in the suite: ~700 pattern/document pairs plus match/index/ignore-case parity, checked against the built-in engine on every run, with the accepted divergences enumerated and individually pinned. It earned its place immediately by falsifying my own documentation. The "self-overlapping delimiter" caveat was stale — fixing the boundary-advance in the previous commit also made `(?=#{1,6}\s)`, `(?=aa)` and `(?=##)` exact, so the entry asserting they still diverge failed. One real divergence remains and is now stated precisely: a lookbehind whose body self-overlaps (`(?<=aa)` over `aaaaa`). Making that exact means restarting the scan one character past each match start, which is quadratic on a multi-megabyte document and forfeits the linear guarantee — so it is a deliberate trade, not an oversight. Mutation-verified against the three bugs that actually escaped review: disabling the `\s` translation fails 12 tests, changing the boundary-advance fails 2, and removing the top-level-alternation guard fails 3. That last one initially passed, because the corpus had no alternation pattern in it — the gap is closed and the mutation now fails as it should.
1 parent a042f0b commit 6aa3e0e

18 files changed

Lines changed: 1322 additions & 97 deletions

apps/sim/blocks/blocks/guardrails.ts

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,11 @@ export const GuardrailsBlock: BlockConfig<GuardrailsResponse> = {
7676
enabled: true,
7777
prompt: `Generate a regular expression pattern based on the user's description.
7878
The regex should be:
79-
- Valid JavaScript regex syntax
79+
- Valid RE2 syntax: no lookahead ((?=...), (?!...)), no lookbehind ((?<=...), (?<!...)),
80+
no backreferences (\\1), and no \\uXXXX escapes (use \\x41 or the literal character).
81+
Patterns are matched by a linear-time engine that does not implement these, and one
82+
that uses them fails every check. To express "must NOT contain X", match X and invert
83+
the result downstream with a Condition block instead of using negative lookahead.
8084
- Properly escaped for special characters
8185
- Optimized for the use case
8286

apps/sim/components/pii/custom-patterns-editor.test.tsx

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -49,10 +49,18 @@ describe('CustomPatternsEditor', () => {
4949
expect(container.textContent).toMatch(/Invalid regex/)
5050
})
5151

52-
it('shows an inline error for a catastrophic-backtracking pattern', () => {
53-
renderEditor([row('(a+)+$')], vi.fn())
54-
expect(container.textContent).toMatch(/potentially unsafe/)
55-
})
52+
it.each(['(a+)+$', '(?<=id: )\\w+', '(?:https?://)?example\\.com'])(
53+
'accepts %s, which the removed safe-regex2 screen rejected',
54+
(pattern) => {
55+
// The editor no longer runs a catastrophic-backtracking screen. It caught
56+
// `(a+)+$` but passed `a*a*b`, so it deterred only the obvious spelling
57+
// while blocking valid Presidio patterns like lookbehind. These patterns
58+
// run in Presidio, not in this process.
59+
renderEditor([row(pattern)], vi.fn())
60+
expect(container.textContent).not.toMatch(/potentially unsafe/)
61+
expect(container.textContent).not.toMatch(/Invalid regex/)
62+
}
63+
)
5664

5765
it('appends an empty row when "Add pattern" is clicked', () => {
5866
const onChange = vi.fn()

apps/sim/components/pii/custom-patterns-editor.tsx

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,13 @@ interface CustomPatternsEditorProps {
1515

1616
/**
1717
* Editor for user-supplied custom regex patterns. Each row is a name label, the
18-
* regex (validated inline for syntax + catastrophic-backtracking safety), and the
19-
* verbatim replacement token that matches are redacted to. Shared by the Data
20-
* Retention settings and any other PII-policy surface.
18+
* regex (validated inline for syntax only), and the verbatim replacement token
19+
* that matches are redacted to. Shared by the Data Retention settings and any
20+
* other PII-policy surface.
21+
*
22+
* Syntax is the only check because these patterns execute in Presidio, not in
23+
* this process — see `validateRegexPattern` for why a backtracking screen here
24+
* was removed rather than kept.
2125
*/
2226
export function CustomPatternsEditor({ patterns, onChange }: CustomPatternsEditorProps) {
2327
function updateRow(index: number, patch: Partial<CustomPiiPattern>) {

apps/sim/lib/api/contracts/primitives.test.ts

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -35,10 +35,22 @@ describe('customPatternSchema', () => {
3535
}
3636
})
3737

38-
it('rejects a catastrophic-backtracking regex at the boundary', () => {
39-
expect(
40-
customPatternSchema.safeParse({ name: 'evil', regex: '(a+)+$', replacement: '' }).success
41-
).toBe(false)
38+
it('no longer screens for catastrophic backtracking, which never worked here', () => {
39+
// This used to reject `(a+)+$` via `safe-regex2`. The screen was removed:
40+
// it caught that shape but passed `a*a*b`, which is just as catastrophic,
41+
// so it only ever deterred the obvious spelling of a misconfiguration a
42+
// user can inflict on their own workspace. It also rejected valid patterns
43+
// (lookbehind, optional groups) that Presidio accepts.
44+
//
45+
// These patterns run in Presidio, not in this process, so they cannot
46+
// stall this event loop; Presidio's own timeout is the bound. In-process
47+
// matching uses `compileLinearRegex`, which cannot backtrack at all.
48+
for (const regex of ['(a+)+$', 'a*a*b', '(?<=id: )\\w+']) {
49+
expect(
50+
customPatternSchema.safeParse({ name: 'p', regex, replacement: '' }).success,
51+
`pattern ${regex} should be accepted`
52+
).toBe(true)
53+
}
4254
})
4355
})
4456

apps/sim/lib/chunkers/regex-chunker.test.ts

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -387,5 +387,39 @@ describe('RegexChunker', () => {
387387
)
388388
}
389389
})
390+
it.concurrent(
391+
'chunks with a catastrophic pattern without hanging',
392+
async () => {
393+
// `a*a*b` defeats every syntactic backtracking screen. The previous
394+
// guard probed it against 'a'.repeat(10000) and measured the elapsed
395+
// time only after the match returned, so the guard itself hung for
396+
// ~213s. RE2 has no backtracking, so this simply completes.
397+
const chunker = new RegexChunker({ pattern: 'a*a*b', chunkSize: 100, chunkOverlap: 0 })
398+
399+
const start = Date.now()
400+
await chunker.chunk(`${'a'.repeat(10000)}!`)
401+
402+
expect(Date.now() - start).toBeLessThan(2000)
403+
},
404+
10000
405+
)
406+
407+
it.concurrent('still supports lookahead split patterns', async () => {
408+
// RE2 has no lookaround; compileLookaroundSplit expresses this as slicing
409+
// at each match start, so the idiom keeps working on the linear engine.
410+
// (No `m` flag is applied, so anchor the lookahead on the delimiter
411+
// itself rather than on `^`.)
412+
const chunker = new RegexChunker({
413+
pattern: '(?=#\\s)',
414+
chunkSize: 1024,
415+
chunkOverlap: 0,
416+
strictBoundaries: true,
417+
})
418+
const chunks = await chunker.chunk('# One\nalpha\n# Two\nbeta')
419+
420+
expect(chunks.length).toBe(2)
421+
expect(chunks[0].text).toContain('# One')
422+
expect(chunks[1].text).toContain('# Two')
423+
})
390424
})
391425
})

apps/sim/lib/chunkers/regex-chunker.ts

Lines changed: 34 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,11 @@ import {
1010
splitAtWordBoundaries,
1111
tokensToChars,
1212
} from '@/lib/chunkers/utils'
13+
import {
14+
compileLinearRegex,
15+
compileLookaroundSplit,
16+
type LinearRegex,
17+
} from '@/lib/core/security/linear-regex'
1318

1419
const logger = createLogger('RegexChunker')
1520

@@ -56,7 +61,7 @@ function toNonCapturing(pattern: string): string {
5661
export class RegexChunker {
5762
private readonly chunkSize: number
5863
private readonly chunkOverlap: number
59-
private readonly regex: RegExp
64+
private readonly regex: LinearRegex
6065
private readonly strictBoundaries: boolean
6166

6267
constructor(options: RegexChunkerOptions) {
@@ -67,7 +72,24 @@ export class RegexChunker {
6772
this.strictBoundaries = options.strictBoundaries ?? false
6873
}
6974

70-
private compilePattern(pattern: string): RegExp {
75+
/**
76+
* Compile the caller's split pattern on an engine that cannot backtrack.
77+
*
78+
* This previously screened for catastrophic backtracking by running the
79+
* pattern against six probe strings — including `'a'.repeat(10000)` — and
80+
* rejecting anything slower than 50ms. It measured the elapsed time *after*
81+
* the match returned, so the screen was the denial of service it existed to
82+
* prevent: `a*a*b` against that probe measured 213s on JSC. RE2 removes the
83+
* failure mode outright, so the probe is gone rather than repaired.
84+
*
85+
* Keeping the delimiter — `(?=X)` before a chunk, `(?<=X)` after one — is the
86+
* reason a split pattern reaches for lookaround, and `compileLookaroundSplit`
87+
* runs both on RE2 without it. Anything else RE2 cannot represent is rejected
88+
* rather than run on the built-in engine: no probe can tell a safe pattern
89+
* from an unsafe one without running it, which is what made the old guard
90+
* hang, so there is nothing to fall back *to*.
91+
*/
92+
private compilePattern(pattern: string): LinearRegex {
7193
if (!pattern) {
7294
throw new Error('Regex pattern is required')
7395
}
@@ -77,34 +99,18 @@ export class RegexChunker {
7799
}
78100

79101
try {
80-
const regex = new RegExp(toNonCapturing(pattern), 'g')
81-
82-
const testStrings = [
83-
'a'.repeat(10000),
84-
' '.repeat(10000),
85-
'a '.repeat(5000),
86-
'aB1 xY2\n'.repeat(1250),
87-
`${'a'.repeat(30)}!`,
88-
`${'a b '.repeat(25)}!`,
89-
]
90-
for (const testStr of testStrings) {
91-
regex.lastIndex = 0
92-
const start = Date.now()
93-
regex.test(testStr)
94-
const elapsed = Date.now() - start
95-
if (elapsed > 50) {
96-
throw new Error('Regex pattern appears to have catastrophic backtracking')
97-
}
98-
}
99-
100-
regex.lastIndex = 0
101-
return regex
102+
new RegExp(pattern)
102103
} catch (error) {
103-
if (error instanceof Error && error.message.includes('catastrophic')) {
104-
throw error
105-
}
106104
throw new Error(`Invalid regex pattern "${pattern}": ${toError(error).message}`)
107105
}
106+
107+
const source = toNonCapturing(pattern)
108+
const compiled = compileLinearRegex(source) ?? compileLookaroundSplit(source)
109+
if (compiled) return compiled
110+
111+
throw new Error(
112+
`Regex pattern "${pattern}" uses syntax that cannot be evaluated safely. Unsupported: negative lookaround ("(?!...)", "(?<!...)"), backreferences, and repeat counts above 1000. Positive lookaround is supported — "(?=X)" splits before a delimiter, "(?<=X)" after one, and "(?<=X)Y(?=Z)" consumes Y between them.`
113+
)
108114
}
109115

110116
async chunk(content: string): Promise<Chunk[]> {
@@ -119,8 +125,7 @@ export class RegexChunker {
119125
return buildChunks([cleaned], 0)
120126
}
121127

122-
this.regex.lastIndex = 0
123-
const segments = cleaned.split(this.regex).filter((s) => s.trim().length > 0)
128+
const segments = this.regex.split(cleaned).filter((s) => s.trim().length > 0)
124129

125130
if (segments.length <= 1) {
126131
if (this.strictBoundaries) {

apps/sim/lib/copilot/tools/server/workflow/query-logs.ts

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,12 +145,17 @@ export const queryLogsServerTool: BaseServerTool<QueryLogsArgs, unknown> = {
145145

146146
if (args.pattern) {
147147
logger.info('query_logs grep', { workspaceId, executionId: args.executionId })
148-
const { matches, truncated } = await grepSpans(traceSpans, args.pattern, viewCtx)
148+
const { matches, truncated, patternNotice } = await grepSpans(
149+
traceSpans,
150+
args.pattern,
151+
viewCtx
152+
)
149153
return {
150154
executionId: detail.executionId,
151155
workflowId: detail.workflowId,
152156
status: detail.status,
153157
pattern: args.pattern,
158+
...(patternNotice ? { patternNotice } : {}),
154159
matches,
155160
truncated,
156161
}

apps/sim/lib/copilot/vfs/operations.test.ts

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -138,3 +138,35 @@ describe('grep', () => {
138138
expect(hits).toHaveLength(1)
139139
})
140140
})
141+
142+
describe('grep regex safety', () => {
143+
it('runs a catastrophic pattern in linear time', () => {
144+
// `a*a*b` takes minutes on a backtracking engine against this content;
145+
// both the pattern and the file content are caller-supplied.
146+
const files = vfsFromEntries([['notes.md', `${'a'.repeat(10000)}!`]])
147+
148+
const start = Date.now()
149+
grep(files, 'a*a*b')
150+
151+
expect(Date.now() - start).toBeLessThan(2000)
152+
})
153+
154+
it('still interprets regex syntax', () => {
155+
const files = vfsFromEntries([['log.txt', 'req finished status=503']])
156+
expect(grep(files, 'status=\\d+')).toHaveLength(1)
157+
expect(grep(files, 'status=\\d+', undefined, { outputMode: 'count' })).toEqual([
158+
{ path: 'log.txt', count: 1 },
159+
])
160+
})
161+
162+
it('matches syntax RE2 cannot represent literally instead of not at all', () => {
163+
const files = vfsFromEntries([['log.txt', 'contains (?=x) verbatim']])
164+
expect(grep(files, '(?=x)')).toHaveLength(1)
165+
})
166+
167+
it('honours ignoreCase across repeated line tests', () => {
168+
const files = vfsFromEntries([['log.txt', 'Alpha\nALPHA\nalpha']])
169+
expect(grep(files, 'alpha', undefined, { ignoreCase: true })).toHaveLength(3)
170+
expect(grep(files, 'alpha')).toHaveLength(1)
171+
})
172+
})

apps/sim/lib/copilot/vfs/operations.ts

Lines changed: 34 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,13 @@
1+
import { createLogger } from '@sim/logger'
12
import micromatch from 'micromatch'
3+
import {
4+
compileLinearRegex,
5+
isPlainText,
6+
type LinearRegex,
7+
literalRegex,
8+
} from '@/lib/core/security/linear-regex'
9+
10+
const logger = createLogger('VfsOperations')
211

312
export interface GrepMatch {
413
path: string
@@ -125,7 +134,16 @@ export function pathWithinGrepScope(filePath: string, scope: string): boolean {
125134
}
126135

127136
/**
128-
* Regex search over VFS file contents using ECMAScript `RegExp` syntax.
137+
* Regex search over VFS file contents using RE2 syntax — a subset of
138+
* ECMAScript `RegExp` (see `@/lib/core/security/linear-regex`).
139+
*
140+
* A pattern RE2 cannot represent — negative lookaround, backreferences — is
141+
* matched as a literal instead of on the backtracking engine, as is a pattern
142+
* that does not compile at all (which previously returned no results). Both
143+
* cases log a warning: the return shape carries results only, so there is
144+
* nowhere to tell the caller inline, and a literal fallback can match the
145+
* pattern's own text when grepping source that contains regexes.
146+
*
129147
* `content` and `count` are line-oriented (split on newline, CR stripped per line).
130148
* `files_with_matches` tests the entire file string once, so multiline patterns can match there
131149
* but not in line modes.
@@ -142,19 +160,27 @@ export function grep(
142160
const showLineNumbers = opts?.lineNumbers ?? true
143161
const contextLines = opts?.context ?? 0
144162

145-
const flags = ignoreCase ? 'gi' : 'g'
146-
let regex: RegExp
147-
try {
148-
regex = new RegExp(pattern, flags)
149-
} catch {
150-
return []
163+
// Caller-supplied pattern over caller-supplied file content on the shared
164+
// event loop — matched by RE2 so it cannot backtrack. Syntax RE2 cannot
165+
// represent degrades to a literal rather than to the backtracking engine.
166+
let regex: LinearRegex
167+
if (isPlainText(pattern)) {
168+
regex = literalRegex(pattern, { ignoreCase })
169+
} else {
170+
const linear = compileLinearRegex(pattern, { ignoreCase })
171+
if (!linear) {
172+
// The return shape carries results only, so the caller cannot be told
173+
// inline that its regex was taken literally — log it, since silently
174+
// returning "no matches" reads as "not in the file".
175+
logger.warn('Grep pattern is not RE2-representable; matching it literally', { pattern })
176+
}
177+
regex = linear ?? literalRegex(pattern, { ignoreCase })
151178
}
152179

153180
if (outputMode === 'files_with_matches') {
154181
const matchingFiles: string[] = []
155182
for (const [filePath, content] of files) {
156183
if (path && !pathWithinGrepScope(filePath, path)) continue
157-
regex.lastIndex = 0
158184
if (regex.test(content)) {
159185
matchingFiles.push(filePath)
160186
if (matchingFiles.length >= maxResults) break
@@ -170,7 +196,6 @@ export function grep(
170196
const lines = splitLinesForGrep(content)
171197
let count = 0
172198
for (const line of lines) {
173-
regex.lastIndex = 0
174199
if (regex.test(line)) count++
175200
}
176201
if (count > 0) {
@@ -188,7 +213,6 @@ export function grep(
188213

189214
const lines = splitLinesForGrep(content)
190215
for (let i = 0; i < lines.length; i++) {
191-
regex.lastIndex = 0
192216
if (regex.test(lines[i])) {
193217
if (contextLines > 0) {
194218
const start = Math.max(0, i - contextLines)

0 commit comments

Comments
 (0)