You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ScriptTrigger / CommandsTrigger compile the user's exitCondition as a regex and match it against the script's output on every poll. Most modules put no time limit on this match. Ruby has one, but it only stops waiting for the match. Either way, a catastrophic-backtracking exitCondition keeps a CPU core busy forever:
Guarded (Ruby): one more runaway ForkJoinPool.commonPool thread per poll, until the JVM's common pool is exhausted.
Unguarded (all the others): the evaluating thread itself is blocked inside Matcher.find().
Actual Behaviour
Guarded implementations (Ruby on main, Perl in PR #446):
future.get times out and the trigger falls back to a substring match, so evaluation returns and the trigger correctly doesn't fire. But the Matcher keeps running on its commonPool thread: CompletableFuture.cancel can't interrupt it, and Matcher.find() doesn't check for interrupts anyway. Each poll leaves one more thread at 100% CPU.
Once commonPool is saturated (parallelism = cores − 1), later matches just queue up behind the runaway threads. Everything else in the same JVM that relies on commonPool stalls too: CompletableFuture.*Async without an executor, parallel streams, and so on.
Observed with interval: PT10S on a 16-core host. The flow was enabled at 14:17:59:
Time (UTC)
Container CPU
Threads at ~100% CPU
14:18:25
108%
1
14:18:36
210%
2
14:18:48
328%
3
14:19:10
421%
4
14:19:21
522%
5
after disabling the flow
~657%
6 (still spinning until the container restarted)
In top -H, the hot threads were all ForkJoinPool.commonPool-worker-*.
The catch (Exception) fallback never fires, because catastrophic backtracking doesn't throw. The thread evaluating the trigger stays stuck in find(). This case wasn't exercised in QA; it follows from the code and from the timing check below.
The #388 security audit flagged this as MEDIUM-001. The timeout was only added to Ruby, and it has the leak described above.
Test gap: the natural test pattern (a+)+$ fails in about 0ms on JDK 25 because the regex engine memoises that loop, so a test using it passes without ever reaching the timeout path. Measured against "{k=" + "a".repeat(40) + "!}" on JDK 25:
Pattern
Result
(a+)+$, (a|aa)+$, (a*)*b, (x+x+)+y
false in 0ms
(.*a){20}$, (\w+)*\1!\d, (.*){1,32000}[bc]
still running after 4s
Expected Behaviour
A slow or pathological exitCondition is abandoned within a bounded time and falls back to a substring match. That fallback already exists.
Abandoning a match leaves no work running: no leaked threads, no CPU burned.
commonPool is never occupied by user-controlled work.
The behaviour is the same in every trigger module.
Reproducer
Start Kestra OSS (kestra/kestra:v1.3.39) with plugin-scripts from main.
docker stats --no-stream <kestra-container>
docker exec<kestra-container> sh -c "top -H -b -n1 | head -20"
Actual: the trigger never fires, which is correct. But CPU goes up by about one core per poll, and new ForkJoinPool.commonPool-worker-* threads stay at 100%. Disabling the flow doesn't free them; only restarting the JVM does.
For an unguarded module, run the same flow with io.kestra.plugin.scripts.shell.ScriptTrigger and script: echo '::{"outputs":{"k":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!"}}::'.
The evidence above was gathered with the Perl triggers from PR #446, which carry the same guard as Ruby line for line.
Logs / Stack Trace
No log entries or exceptions. The only symptom is CPU use and the hot commonPool threads.
Environment
Kestra version: 1.3.39 (OSS, Docker image kestra/kestra:v1.3.39, standalone with H2, JDK 25.0.4 in the container)
Unguarded:plugin-script-bun, -deno, -dotnet, -go, -node, -powershell, -python (AbstractPythonTrigger), -shell. Both triggers in each, where applicable.
Suggested fix: reuse the fix already in Kestra core. The same problem in Pebble regex filters (kestra-io/kestra#15488) was fixed by kestra-io/kestra#15490. That PR added io.kestra.core.utils.RegexUtils, which wraps the input CharSequence so that every charAt() checks a deadline. Once the deadline passes it throws RegexTimeoutException, so the match stops on the caller's thread and no extra thread or executor is involved. Its API includes matches(Pattern, CharSequence[, Duration]), matcher(...), isSafeUserRegex(...) and syntaxError(...), and the timeout is configured globally by RegexConfiguration.
RegexUtils ships in Kestra 2.0.0 and isn't in 1.3.x. plugin-scripts currently compiles against kestraVersion=1.3.39.
When the plugin targets Kestra ≥ 2.0: replace every matchesCondition regex call with RegexUtils.matches(pattern, haystack). Catch RegexTimeoutException (plus PatternSyntaxException) and fall back to haystack.contains(cond). Remove Ruby's CompletableFuture guard. Optionally reject a pathological exitCondition up front with isSafeUserRegex.
While the plugin still targets 1.3.x: copy the same small deadline-CharSequence wrapper into plugin-script as a temporary shim that mirrors RegexUtils' behaviour, and swap it for RegexUtils when the plugin is bumped.
Either way, put the logic in a shared helper in plugin-script (see the AbstractScriptTrigger TODO) so all trigger modules behave the same.
Tests:
Use a pattern that is still catastrophic on JDK 21+/25, such as (.*a){20}$, not(a+)+$.
Assert both the fallback result and that it returns within about 1–2× the deadline.
Where possible, also assert that no extra commonPool threads are left busy afterwards.
Summary
ScriptTrigger/CommandsTriggercompile the user'sexitConditionas a regex and match it against the script's output on every poll. Most modules put no time limit on this match. Ruby has one, but it only stops waiting for the match. Either way, a catastrophic-backtrackingexitConditionkeeps a CPU core busy forever:ForkJoinPool.commonPoolthread per poll, until the JVM's common pool is exhausted.Matcher.find().Actual Behaviour
Guarded implementations (Ruby on
main, Perl in PR #446):future.gettimes out and the trigger falls back to a substring match, so evaluation returns and the trigger correctly doesn't fire. But theMatcherkeeps running on itscommonPoolthread:CompletableFuture.cancelcan't interrupt it, andMatcher.find()doesn't check for interrupts anyway. Each poll leaves one more thread at 100% CPU.Once
commonPoolis saturated (parallelism = cores − 1), later matches just queue up behind the runaway threads. Everything else in the same JVM that relies oncommonPoolstalls too:CompletableFuture.*Asyncwithout an executor, parallel streams, and so on.Observed with
interval: PT10Son a 16-core host. The flow was enabled at 14:17:59:In
top -H, the hot threads were allForkJoinPool.commonPool-worker-*.Unguarded implementations (Bun, Deno, .NET, Go, Node, PowerShell, Python, Shell):
The
catch (Exception)fallback never fires, because catastrophic backtracking doesn't throw. The thread evaluating the trigger stays stuck infind(). This case wasn't exercised in QA; it follows from the code and from the timing check below.The #388 security audit flagged this as MEDIUM-001. The timeout was only added to Ruby, and it has the leak described above.
Test gap: the natural test pattern
(a+)+$fails in about 0ms on JDK 25 because the regex engine memoises that loop, so a test using it passes without ever reaching the timeout path. Measured against"{k=" + "a".repeat(40) + "!}"on JDK 25:(a+)+$,(a|aa)+$,(a*)*b,(x+x+)+y(.*a){20}$,(\w+)*\1!\d,(.*){1,32000}[bc]Expected Behaviour
exitConditionis abandoned within a bounded time and falls back to a substring match. That fallback already exists.commonPoolis never occupied by user-controlled work.Reproducer
kestra/kestra:v1.3.39) withplugin-scriptsfrommain.ForkJoinPool.commonPool-worker-*threads stay at 100%. Disabling the flow doesn't free them; only restarting the JVM does.io.kestra.plugin.scripts.shell.ScriptTriggerandscript: echo '::{"outputs":{"k":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!"}}::'.The evidence above was gathered with the Perl triggers from PR #446, which carry the same guard as Ruby line for line.
Logs / Stack Trace
No log entries or exceptions. The only symptom is CPU use and the hot
commonPoolthreads.Environment
kestra/kestra:v1.3.39, standalone with H2, JDK 25.0.4 in the container)plugin-scripts1.10.1-SNAPSHOT (mainat a03cd85, plus PR feat(perl): add script commands trigger #446 for Perl)Additional Context
Affected on
main:plugin-script-ruby(ScriptTrigger,CommandsTrigger). Also Perl, open in feat(perl): add script commands trigger #446.plugin-script-bun,-deno,-dotnet,-go,-node,-powershell,-python(AbstractPythonTrigger),-shell. Both triggers in each, where applicable.Suggested fix: reuse the fix already in Kestra core. The same problem in Pebble regex filters (kestra-io/kestra#15488) was fixed by kestra-io/kestra#15490. That PR added
io.kestra.core.utils.RegexUtils, which wraps the inputCharSequenceso that everycharAt()checks a deadline. Once the deadline passes it throwsRegexTimeoutException, so the match stops on the caller's thread and no extra thread or executor is involved. Its API includesmatches(Pattern, CharSequence[, Duration]),matcher(...),isSafeUserRegex(...)andsyntaxError(...), and the timeout is configured globally byRegexConfiguration.RegexUtilsships in Kestra 2.0.0 and isn't in 1.3.x.plugin-scriptscurrently compiles againstkestraVersion=1.3.39.matchesConditionregex call withRegexUtils.matches(pattern, haystack). CatchRegexTimeoutException(plusPatternSyntaxException) and fall back tohaystack.contains(cond). Remove Ruby'sCompletableFutureguard. Optionally reject a pathologicalexitConditionup front withisSafeUserRegex.CharSequencewrapper intoplugin-scriptas a temporary shim that mirrorsRegexUtils' behaviour, and swap it forRegexUtilswhen the plugin is bumped.Either way, put the logic in a shared helper in
plugin-script(see theAbstractScriptTriggerTODO) so all trigger modules behave the same.Tests:
(.*a){20}$, not(a+)+$.commonPoolthreads are left busy afterwards.See also: #388 (security audit, MEDIUM-001) and kestra-io/kestra#15488 (the same issue in core, fixed by kestra-io/kestra#15490).
View as Artifact