Skip to content

Commit 3e78636

Browse files
committed
feat(regex): close every D-447 gap — named groups, \A \z \Z, class algebra
The regex-equivalence audit's remaining gaps, all landed or settled: - Named groups `(?<name>e)`: a capturing group sharing Java's numbering, duplicate names rejected, name→index bindings on the Program, and `(.group m "name")` on Matcher. - `\A \z \Z` anchors; `\Z` honours a final `\n` / `\r` / `\r\n` (Java parity), all three MULTILINE-independent. - Character-class ALGEBRA: nested unions `[a[b]]` and `&&` intersection (`[a-z&&[^m-p]]`), via a recursive class parser over the existing bitmap+RangeSet representation. This is also what `[[:alpha:]]` IS in Java — the D-447 row called it "POSIX classes", but Java has no POSIX bracket classes; it parses them as a nested union (oracle-verified: clj matches "a" in "ab1"). The row's framing was a mis-diagnosis. - Empty alternatives and groups: `()` and `a|` are legal Java (match empty) and were misrouted to the unsupported signal. - Lookbehind had already landed; the row was stale on it. Backreferences are PERMANENTLY DECLINED as AD-060, not deferred: ADR-0031's Pike-NFA linear-time guarantee is the property that lets untrusted patterns run without a ReDoS budget, a backreference forces a backtracking engine, and RE2 rejects them for exactly this reason. The signal split deferred from the data-vs-feature sweep lands with it: a MALFORMED pattern raises the CATCHABLE `regex_pattern_invalid` (Java PatternSyntaxException parity), routed through one shared `regex_value.raiseCompileError` used by every compile surface — re-pattern, regex literals, Pattern/compile+matches, String .matches/.split/replace, clojure.string/split — while a valid-in-Java feature cljw declines stays the uncatchable unsupported signal. Corpus-backed per the row's own mandate: 15 clj-byte-matched goldens in regex_equivalence.txt covering every landed feature plus the catchable invalid cases; pin e2e regex_parity_gaps.sh asserts the catchable/ uncatchable split the corpus cannot express. compile.zig's header now states the true surface (the row's depth-1 doc-fix item). D-447 discharged. Smell-audited: 2: the row said "POSIX classes" and the first instinct was to implement POSIX classes — the oracle said Java doesn't have them, so what landed is what Java actually does. Also caught my own stale unit test asserting the pre-split error for "(" and updated it to the new contract rather than weakening the contract to fit the test.
1 parent d33e687 commit 3e78636

15 files changed

Lines changed: 446 additions & 99 deletions

File tree

.dev/accepted_divergences.yaml

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -945,3 +945,25 @@ accepted:
945945
be wrong.
946946
derives_from: "ADR-0181 (doc coverage is a ledger of decisions) + .claude/rules/clj_attribution.md (the two header variants) + EPL-1.0 §7 redistribution."
947947
pin: "scripts/check_clj_attribution.sh --gate (the variant-1 file set must equal legal/NOTICE's list, both directions) + test/diff/clj_corpus/set_arity.txt (17 golden pairs incl. every arity and the :doc/:arglists reads)."
948+
- id: "AD-060"
949+
area: "regex"
950+
summary: |-
951+
Backreferences (`\1` … `\9`, `\k<name>`) raise the UNCATCHABLE
952+
unsupported-feature error at pattern-compile time; Java compiles and
953+
matches them. This is the RE2 posture, permanent by design, not a
954+
pending gap: cljw's matcher is a Pike-NFA (ADR-0031) whose worst case is
955+
linear in the input, which is the property that lets the playground run
956+
untrusted patterns without a ReDoS budget. A backreference makes
957+
matching NP-hard in general and forces a backtracking engine — RE2
958+
rejects backreferences for exactly this reason. The raise is the
959+
unsupported-feature signal (not the catchable invalid-pattern error),
960+
because the pattern is VALID Java that cljw deliberately declines,
961+
distinct from a malformed pattern the user should catch and handle
962+
(regex_pattern_invalid, which IS catchable, matching Java's
963+
PatternSyntaxException). Every other D-447 gap closed 2026-08-05:
964+
lookbehind, named groups `(?<name>…)` (+ `.group m "name"`), `\A \z \Z`,
965+
nested class unions `[a[b]]` / `&&` intersection, empty
966+
alternatives/groups.
967+
example: '(re-pattern "(a)\\1") => cljw UNCATCHABLE "this regex feature … is not supported" / clj #"(a)\1" compiles and (re-matches #"(a)\1" "aa") => ["aa" "a"]'
968+
derives_from: "ADR-0031 (Pike-NFA, non-backtracking invariant — linear-time matching is the security property, F-002 finished form)."
969+
pin: "test/e2e/regex_parity_gaps.sh — backref compile raises the uncatchable unsupported signal and a (catch Throwable …) does NOT swallow it."

.dev/debt.yaml

Lines changed: 27 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1354,11 +1354,6 @@ active:
13541354
category: "perf"
13551355
barrier: "none hard — O-023 already ships the effective form. The 正しい姿 unification is a deliberate refactor that touches the gate-green O-023 path; a transducer stack (map's fn) ≠ a pred vector, so the unified representation needs design. Take up alongside a reduce-heavy bench that the current `[xform coll]` descriptor under-serves, or when the lazy-walk collapse is revisited with a real workload. Not blocking; the existing fused-reduce is correct + fast."
13561356
last_reviewed: "2026-06-15"
1357-
- id: "D-447"
1358-
status: "open (2026-06-15, regex equivalence audit / ADR-0147). Regex parity gaps cljw's Pike-NFA matcher does NOT support — all currently RAISE NotImplemented (never silently wrong; the 48-golden `test/diff/clj_corpus/regex_equivalence.txt` deliberately excludes them; reluctant quantifiers `*? +? ?? {n,m}?` LANDED 2026-06-16 — split-priority swap pairing with the cut-on-match leftmost-first fix + a dedicated matchFull loop for the re-matches full-span case, corpus regex_equivalence): lookbehind `(?<=)`/`(?<!)`, named groups `(?<name>)`, backreferences `\\1`, `\\A \\z \\Z`, POSIX `[[:class:]]`. (Lookahead `(?=)`/`(?!)` and all four flags `(?imsx)` DO work.) NOT campaign-blocking — addressed only when a real workload needs them OR opportunistically alongside the ADR-0147 regex-perf engine work (reluctant quantifiers are a cheap split-priority swap that pairs naturally with the matcher changes; backrefs would need a backtracking fallback cljw rejects per ADR-0031, so likely stay a documented gap / RE2 posture). When a gap closes, MOVE its goldens into the equivalence corpus in the same cycle (corpus-backed discharge, anti-D-177). Also a depth-1 doc fix: `compile.zig:22-23` claims 'lookaround' unsupported but lookahead works."
1359-
category: "clj-parity"
1360-
barrier: "none — each gap is a bounded matcher/compiler feature. Reluctant quantifiers DONE 2026-06-16 (the cheap split-priority swap). Remaining: take up opportunistically when a lib/workload needs one. Backrefs conflict with the non-backtracking invariant (ADR-0031) — likely an accepted gap, not a fix. lookbehind/named-groups/`\\A\\z\\Z`/POSIX-classes are bounded compiler features."
1361-
last_reviewed: "2026-06-15"
13621357
- id: "D-462"
13631358
status: "PARTIAL (2026-06-18 — java.time LOCAL FAMILY WIRED; only ZonedDateTime + arithmetic methods residual). Instant, Duration, LocalDateTime, LocalDate, LocalTime are all wired as `.typed_instance` values (per-Runtime descriptors; timestamp.zig/date.zig model — NO new NaN-box tag): statics (of/now/parse/ofEpoch*) + readers + (str) ISO-grounded vs clj + value `=` + cross-refs (LocalDateTime.toLocalDate/toLocalTime). Shared civil + ISO date/time format/parse helpers live in runtime/time/instant.zig; the typed_instance print form is a `temporal_print` enum on TypeDescriptor; value-wrap files in the compat_tiers `wrap:` slot (G3-clean). e2e: phase15_java_time_{instant,duration,local_date_time,local_date}.sh (clj-grounded, incl. negative/pre-1970/1900-non-leap/ns-precision edges). pr-str divergence (bare toString vs clj `#object[…]`) = AD-042. RESIDUAL: (1) ZonedDateTime DEFERRED — needs a bundled IANA tz database for named zones (scope decision, user/ADR-owned; see compat_tiers status); (3) LocalDateTime.now is UTC-based (no zone DB). RESIDUAL (2) RESOLVED (2026-06-20): a verify-sweep (anti-D-177, 24 methods × clj oracle) found the 2026-06-18 'arithmetic NOT implemented' claim was STALE — plus*/minus*/plusWeeks/plusMonths/plusYears/isBefore/isAfter/isEqual (LocalDate), plus*/minus*/isBefore/isEqual (LocalDateTime/LocalTime), plusSeconds/plusMillis/plusNanos/minus*/isBefore (Instant), multipliedBy/negated/toMinutes/between (Duration) ALL already worked + matched clj. The ONLY genuinely-missing methods were LocalDate.atStartOfDay (→ LocalDateTime at midnight) + LocalDate.atTime (h m / h m s / h m s ns → LocalDateTime), now IMPLEMENTED in local_date_value.zig (atStartOfDayFn/atTimeFn → local_date_time_value.make(rt, epoch_day, nano_of_day); out-of-range time fields raise like JVM DateTimeException, message differs per AD-007). clj-verified incl. ns-precision + toLocalDate round-trip; e2e phase15_java_time_local_date_arith.sh (at-time case). The original UNWIRED finding (static-method class resolution + empty method_tables + compat_tiers over-claim) is fully resolved for the local family. ROW NOW OPEN ONLY FOR ZonedDateTime (residual 1, user/ADR-owned tz-DB) + the LDT.now-UTC note (residual 3)."
13641359
category: "clj-parity"
@@ -2696,6 +2691,33 @@ standing:
26962691
barrier: "open after the D-440 arc completes (R4 + R5 must first rewrite the bulk of 'Phase N' citations to gap-area terms). Premature adoption = ~70 dangling citations with no 1:1 redirect. Trigger: D-440 R4/R5 done + a user nod to re-architect the planning axis (it is depth-4, the project's organizing metaphor)."
26972692
last_reviewed: "2026-06-15"
26982693
discharged:
2694+
- id: "D-447"
2695+
status: |-
2696+
DISCHARGED 2026-08-05 — every named gap closed or settled, corpus-backed.
2697+
IMPLEMENTED: named groups `(?<name>e)` (shared Java numbering, duplicate
2698+
names rejected, `.group m "name"` on Matcher); `\A \z \Z` anchors (\Z
2699+
honours a final \n / \r / \r\n); nested class unions `[a[b]]` and `&&`
2700+
intersection (which is what `[[:alpha:]]` actually IS in Java — the row's
2701+
"POSIX classes" framing was a mis-diagnosis: Java has no POSIX bracket
2702+
classes, it parses them as a nested union, verified on the oracle);
2703+
empty alternatives/groups `()` `a|` (were misrouted to NotImplemented).
2704+
Lookbehind had already landed (the row was stale). SETTLED: backreferences
2705+
are PERMANENTLY DECLINED as AD-060 — the RE2 posture; ADR-0031's Pike-NFA
2706+
linear-time guarantee is the security property, and a backref forces
2707+
backtracking. SIGNAL SPLIT landed with it (the task-3 deferral): a
2708+
MALFORMED pattern now raises the CATCHABLE `regex_pattern_invalid`
2709+
(Java PatternSyntaxException parity) via the shared
2710+
`regex_value.raiseCompileError` used by every compile surface (re-pattern,
2711+
literals, Pattern/compile, String .matches/.split/replace,
2712+
clojure.string/split); a valid-in-Java feature cljw declines stays the
2713+
uncatchable unsupported signal. Corpus: 15 clj-byte-matched goldens
2714+
appended to regex_equivalence.txt; pin e2e regex_parity_gaps.sh (3 cases
2715+
incl. the catchable/uncatchable split). The compile.zig header doc now
2716+
states the true surface (the row's depth-1 doc-fix item).
2717+
category: clj-parity
2718+
barrier: none — discharged
2719+
last_reviewed: "2026-08-05"
2720+
26992721
- id: "D-446"
27002722
status: "PARTIAL (2026-06-18 — MID/UNDER/over-arity sweep COMPLETE; only multidim residual open). An empirical both-sides arity probe (193 common clojure.core fns × arities 0-7, BOTH runtimes catch clojure.lang.ArityException; harness + data in private/arity_sweep/) found 22 over-strict fns (cljw REJECTS an arity clj accepts; ZERO over-lenient — the 0-arg class from the 2026-06-16 pass below stays clean). 14 fixed bug→fix (F-011): n-ary map/mapv/mapcat/list* (declare-d top-level -map-n-step + -spread, D-147 no-named-local-fn), bit-and-not variadic fold, resolve 2-arg env form (a local named in env → nil) [commit 982890aa]; the 8 typed array ctors gained the 2-arg (X-array size init-or-seq) form (uniform -array-from2 — value init fills, sequential init copies; the byte/short/char NUMBER-init divergence vs clj IS recorded as AD-036 deriving from AD-019, Devil-advocate-forked) [array commit]. Mechanised: test/diff/clj_corpus/arity_envelopes.txt (22 cases, all clj-byte-matched) + e2e phase14_arity_parity.sh (35/0). REMAINING OPEN SCOPE OF THIS ROW = multidim array indexing: (aget a i j …) and (aset a i j … v) + the typed aset-* multidim forms diverge at arity 3+/4+ (8 aset-* fns; aget is a builtin the ns-publics fn? sweep did not enumerate) — see barrier. EARLIER 0-ARG PASS (2026-06-16, kept for history): a fresh-context subagent probed the 0-ARG boundary (the not= precedent) + 22 over-arity + 20 extra-0-arg; found 10 divergences all aligned to clj — cljw-LENIENT = < > <= >= distinct? every-pred some-fn now throw; cljw-STRICT into + conj! now return clj values. ORIGINAL (2026-06-15, user-directed): arity-divergence audit — enumerate cljw fn arities, diff vs clj, classify each bug→fix or accepted→AD-NNN; big-bang-then-closed (clj_diff_sweep Discipline 2)."
27012723
category: "clj-parity"

src/eval/analyzer/analyzer.zig

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -563,9 +563,7 @@ pub fn parseRatioLiteral(rt: *Runtime, digits: []const u8, loc: error_mod.Source
563563
pub fn parseRegexLiteral(rt: *Runtime, body: []const u8, loc: error_mod.SourceLocation) AnalyzeError!Value {
564564
return regex_value.alloc(rt, body, .{}) catch |err| switch (err) {
565565
error.OutOfMemory => return error.OutOfMemory,
566-
else => return error_catalog.raise(.feature_not_supported, loc, .{
567-
.name = "regex literal (unsupported syntax in cycle 1)",
568-
}),
566+
else => return regex_value.raiseCompileError(err, loc),
569567
};
570568
}
571569

src/lang/primitive/regex.zig

Lines changed: 4 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -56,15 +56,8 @@ pub fn rePattern(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocati
5656
});
5757
}
5858
const src = string_collection.asString(args[0]);
59-
return regex_value.alloc(rt, src, .{}) catch |err| switch (err) {
60-
error.OutOfMemory => err,
61-
// INV-1: a nested-counted-repetition compile-bomb is a catchable error,
62-
// not an OOM / process kill.
63-
error.PatternTooLarge => error_catalog.raise(.regex_pattern_too_large, loc, .{}),
64-
else => error_catalog.raise(.feature_not_supported, loc, .{
65-
.name = "re-pattern (unsupported syntax in cycle 1)",
66-
}),
67-
};
59+
return regex_value.alloc(rt, src, .{}) catch |err|
60+
regex_value.raiseCompileError(err, loc);
6861
}
6962

7063
/// `(re-find re s)` — search for the first match of `re` anywhere
@@ -214,14 +207,8 @@ fn coerceRegex(rt: *Runtime, v: Value, loc: SourceLocation, fn_name: []const u8)
214207
if (v.tag() == .regex) return regex_value.asRegex(v);
215208
if (v.tag() == .string) {
216209
const src = string_collection.asString(v);
217-
const compiled = regex_value.alloc(rt, src, .{}) catch |err| switch (err) {
218-
error.OutOfMemory => return err,
219-
// INV-1: compile-bomb pattern is a catchable error, not an OOM.
220-
error.PatternTooLarge => return error_catalog.raise(.regex_pattern_too_large, loc, .{}),
221-
else => return error_catalog.raise(.feature_not_supported, loc, .{
222-
.name = "re-find / re-matches with invalid pattern (cycle 1)",
223-
}),
224-
};
210+
const compiled = regex_value.alloc(rt, src, .{}) catch |err|
211+
return regex_value.raiseCompileError(err, loc);
225212
return regex_value.asRegex(compiled);
226213
}
227214
return error_catalog.raise(.type_arg_not_string, loc, .{

src/lang/primitive/string.zig

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -491,7 +491,7 @@ fn coerceRegex(rt: *Runtime, v: Value, loc: SourceLocation, fn_name: []const u8)
491491
if (v.tag() == .string) {
492492
const compiled = regex_value.alloc(rt, string_collection.asString(v), .{}) catch |err| switch (err) {
493493
error.OutOfMemory => return err,
494-
else => return error_catalog.raise(.feature_not_supported, loc, .{ .name = "clojure.string/split with invalid regex source" }),
494+
else => |e| return regex_value.raiseCompileError(e, loc),
495495
};
496496
return regex_value.asRegex(compiled);
497497
}

src/runtime/error/catalog.zig

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,13 @@ pub const Code = enum {
5757
inst_string_invalid,
5858
json_string_invalid,
5959
edn_string_invalid,
60+
/// A malformed regex PATTERN (unclosed group/class, dangling quantifier,
61+
/// bad escape) — Java throws the catchable PatternSyntaxException
62+
/// (⊂ IllegalArgumentException), so this is bad DATA, kind value_error.
63+
/// Distinct from a VALID-in-Java pattern using a feature cljw does not
64+
/// implement (backreferences), which stays the uncatchable
65+
/// `feature_not_supported`.
66+
regex_pattern_invalid,
6067
/// A user-written FORM SHAPE the runtime rejects — a malformed libspec,
6168
/// destructuring directive, import spec, `.` member form, reader
6269
/// conditional. Bad DATA (the user's code is the data), so CATCHABLE:
@@ -578,6 +585,11 @@ pub fn entry(comptime code: Code) Entry {
578585
.phase = .parse,
579586
.template = "EDN error ({[reason]s})",
580587
},
588+
.regex_pattern_invalid => .{
589+
.kind = .value_error,
590+
.phase = .eval,
591+
.template = "invalid regex pattern ({[reason]s})",
592+
},
581593
.form_malformed => .{
582594
.kind = .syntax_error,
583595
.phase = .analysis,

src/runtime/java/lang/String.zig

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ const SourceLocation = @import("../../error/info.zig").SourceLocation;
2323
const error_catalog = @import("../../error/catalog.zig");
2424
const charset = @import("../../charset.zig");
2525
const string_collection = @import("../../collection/string.zig");
26+
const regex_value = @import("../../regex/value.zig");
2627
const regex_compile = @import("../../regex/compile.zig");
2728
const regex_match = @import("../../regex/match.zig");
2829
const regex_replace = @import("../../regex/replace.zig");
@@ -446,8 +447,8 @@ fn matches(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) an
446447
try error_catalog.checkArity(".matches", args, 2, loc);
447448
if (args[1].tag() != .string)
448449
return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = ".matches", .actual = @tagName(args[1].tag()) });
449-
var program = regex_compile.compile(rt.gpa, string_collection.asString(args[1]), .{}) catch
450-
return error_catalog.raise(.feature_not_supported, loc, .{ .name = ".matches (invalid regex pattern)" });
450+
var program = regex_compile.compile(rt.gpa, string_collection.asString(args[1]), .{}) catch |err|
451+
return regex_value.raiseCompileError(err, loc);
451452
defer program.deinit(rt.gpa);
452453
const m = regex_match.matchFull(rt.gpa, &program, string_collection.asString(args[0])) catch
453454
return error_catalog.raise(.feature_not_supported, loc, .{ .name = ".matches" });
@@ -465,8 +466,8 @@ fn replaceRegex(rt: *Runtime, fn_name: []const u8, kind: regex_replace.ReplaceKi
465466
return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = fn_name, .actual = @tagName(args[1].tag()) });
466467
if (args[2].tag() != .string)
467468
return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = fn_name, .actual = @tagName(args[2].tag()) });
468-
var program = regex_compile.compile(rt.gpa, string_collection.asString(args[1]), .{}) catch
469-
return error_catalog.raise(.feature_not_supported, loc, .{ .name = "regex-replace (invalid regex pattern)" });
469+
var program = regex_compile.compile(rt.gpa, string_collection.asString(args[1]), .{}) catch |err|
470+
return regex_value.raiseCompileError(err, loc);
470471
defer program.deinit(rt.gpa);
471472
return regex_replace.replaceString(rt, &program, string_collection.asString(args[0]), string_collection.asString(args[2]), kind);
472473
}
@@ -494,8 +495,8 @@ fn split(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anye
494495
if (args[1].tag() != .string)
495496
return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = ".split", .actual = @tagName(args[1].tag()) });
496497
const limit: i64 = if (args.len == 3) try error_catalog.expectInteger(args[2], ".split", loc) else 0;
497-
var program = regex_compile.compile(rt.gpa, string_collection.asString(args[1]), .{}) catch
498-
return error_catalog.raise(.feature_not_supported, loc, .{ .name = ".split (invalid regex pattern)" });
498+
var program = regex_compile.compile(rt.gpa, string_collection.asString(args[1]), .{}) catch |err|
499+
return regex_value.raiseCompileError(err, loc);
499500
defer program.deinit(rt.gpa);
500501
return regex_replace.splitToVector(rt, &program, string_collection.asString(args[0]), limit);
501502
}

src/runtime/java/util/regex/Matcher.zig

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -159,6 +159,20 @@ fn group(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anye
159159
const input = inputOf(args[0]);
160160
if (args.len == 1)
161161
return string_collection.alloc(rt, input[ms.start..ms.end]);
162+
// `(.group m "name")` — a `(?<name>…)` group by name (Java group(String)).
163+
if (args[1].tag() == .string) {
164+
const want = string_collection.asString(args[1]);
165+
for (programOf(args[0]).group_names) |g| {
166+
if (std.mem.eql(u8, g.name, want)) {
167+
const gs = ms.slots[@intCast(2 * @as(u32, g.index))];
168+
const ge = ms.slots[@intCast(2 * @as(u32, g.index) + 1)];
169+
if (gs < 0 or ge < 0) return Value.nil_val;
170+
return string_collection.alloc(rt, input[@intCast(gs)..@intCast(ge)]);
171+
}
172+
}
173+
// Java: IllegalArgumentException "No group with name <x>" — catchable.
174+
return error_catalog.raise(.arg_value_invalid, loc, .{ .fn_name = ".group", .expected = "a declared (?<name>…) group", .actual = "an unknown group name" });
175+
}
162176
if (args[1].tag() != .integer)
163177
return error_catalog.raise(.type_arg_not_integer, loc, .{ .fn_name = ".group", .actual = @tagName(args[1].tag()) });
164178
const n = args[1].asInteger();

0 commit comments

Comments
 (0)