diff --git a/docs/language_status/README.md b/docs/language_status/README.md index cb17d295c..32d280eff 100644 --- a/docs/language_status/README.md +++ b/docs/language_status/README.md @@ -104,6 +104,7 @@ epic #813), not that no cases exist. | **[pli](pli.md)** | production | standard_block | 46/54 | 54 | 158 | **written** (#2502) | | powershell | production | embedded_syntax | 50/52 | 68 | 85 | not written | | **[python](python.md)** | production | line_exclusive | 61/64 | 60 | 92 | **written** | +| **[rexx](rexx.md)** | production | recursive_block_rexx | 35/53 | | 97 | **written** (#2504) | | ruby | production | line_exclusive | 51/52 | 4* | 66 | not written | | rust | production | recursive_block | 52/52 | 51 | 68 | not written | | scala | production | recursive_block | 51/52 | 63 | 90 | not written | diff --git a/docs/language_status/rexx.md b/docs/language_status/rexx.md new file mode 100644 index 000000000..c3613a560 --- /dev/null +++ b/docs/language_status/rexx.md @@ -0,0 +1,157 @@ +# REXX — Structural Signature Coverage + +Snapshot written 2026-09-16 with the language's addition (#2504, under the legacy-mainframe +epic #2516). Source: `LANGUAGE_DEFINITIONS["rexx"]` in +`gitgalaxy/standards/language_standards/languages/rexx.py` and +`tests/extraction/languages/test_rexx_strict.py`. Re-run the `language-status` skill's +data-gathering commands before trusting these numbers if this doc looks old relative to +`last_updated` below. + +**Scope note:** REXX is tree-sitter-blind to this repo's comparison tooling (the +jcl/cobol/bms/hlasm position), and no ground-truth parser diff was run, so there is no §9. +The pinned language-crucible corpus carries no REXX sources — its `.cmd` files are Windows +batch wrappers (`@echo off` + `%~dp0`) and must STAY classified `batch` through this change; +that regression is pinned by the strict suite's routing tests in both directions and by the +golden-master run (whose only diffs were `.bat` identity upgrades from batch's new +discriminator). The rules were validated by the 97-case strict suite and the keyword-rosetta +`data/rexx/` control shell (97 gate assertions). + +## 1. At a glance + +| Field | Value | +|---|---| +| `_meta.status` | `production` | +| `_meta.target_version` | z/OS TSO/E REXX (SAA) + Open Object Rexx 5.0 directives | +| `_meta.blueprint_version` | v6.3 | +| `_meta.last_updated` | 2026-09-16 | +| `lexical_family` | `recursive_block_rexx` — a new prism dialect family (#2504, the haskell/lisp #621/#770 precedent): the same nested-peel algorithm as `recursive_block` (REXX block comments genuinely nest), but **without** the shared family's `//` line token — `//` is REXX's integer-remainder operator — and with `--` (ooRexx/Regina/NetRexx) as the line comment. prism's combined literal pass also swaps to REXX quote morphology for this family: quotes double to escape (`'don''t'`), never backslash, and strings cannot span lines (both quote branches line-bounded, no backtick branch). The classic-REXX `5--3` double-negation is truncated by the `--` token — a documented, strict-suite-pinned trade (vanishingly rare vs. ubiquitous ooRexx comments). | +| `invocation_model` | default (`by_name`): `CALL name`, the function form `name()`, and `SIGNAL name` reach labels by writing their names, so the `unreferenced_by_name` census applies (#2866). | +| Structural signature keys wired | 35 / 53 (18 explicit `None`, see §4) | +| Extraction-gauntlet tests | — (strict suite drives the real extractor directly) | +| Strict-signature tests (`test_rexx_strict.py`) | 97 | + +## 2. Identification surface — the `.cmd` collision (#2504's contested extension) + +`rexx` claims `.rexx` (uncontested), `.exec` (z/OS SYSEXEC convention, uncontested) and +`.cmd` (contested — Windows/OS2 `batch` claims it too). `.cmd` is registered in +`_lens_config.py`'s `COLLISION_FREQUENCIES`, so Tier 1 never locks it on extension alone. +Routing then resolves: + +1. **Tier 2 — `internal_discriminator`s, both claimants.** rexx's: a file whose first token + is `/*` (the OS/2 and z/OS loaders' own dispatch rule for .cmd), or a line-anchored + `PARSE ARG/PULL/SOURCE/VAR`, `ADDRESS `, `SIGNAL/CALL ON|OFF`, `EXECIO`, or an + ooRexx `::requires/::routine/::class/::method` directive. batch — whose `rules` dict is + empty, so it can never win a Tier 3 lexical scan — gained its own discriminator in the + same change (`@echo off/on`, `SETLOCAL/ENDLOCAL`, `goto :label`, `set NAME=` with no + spaces, `if exist/defined`, `%~dp0`-style modifiers, `%ERRORLEVEL%`); registry order + checks batch first, and its shapes are chosen to be impossible in REXX (bare `rem` and + spaced `set = 1` are deliberately excluded — both are legal REXX assignments). +2. **Tier 1.5 — ecosystem gravity**: discriminators `.rexx`, `.exec`, `.jcl`; disqualifiers + `.bat`, `.btm` (a `.cmd` beside `.bat` files is a Windows tree). +3. **Tier 3 — lexical scan** as the last resort. + +Shebangs `rexx`, `regina`, `rexx64`, `oorexx` (Regina documents skipping a `#!` first +line). `case_insensitive_imports: True` (PDS members / case-blind host filesystems). + +## 3. What GitGalaxy detects + +The x/y/z coverage #2504 asked for, plus the rest of the baseline schema. The identifier +guards are explicit classes (`(?` and ooRexx + `USE [STRICT] ARG`: the constructs that stand in for a declared parameter list (#2773's + fallback family — labels carry no formal list). `ARG(1)` is the built-in and never + matches. +- Extraction runs through **Mode A** ("greedy to the next func_start match", COBOL's + paragraph slot): routines never nest, and `RETURN`/`EXIT` are already in the shared + `assembly_returns` terminator vocabulary, so each label's body ends at its return or the + next label. + +### I/O & bridging (y) +- **`io`** — `EXECIO` (z/OS dataset I/O, issued as a quoted host command — string + literals stay in the code stream, #2535, so the quoted form is exactly what fires); the + SAA stream functions `LINEIN( LINEOUT( CHARIN( CHAROUT( STREAM(` and TSO's `OUTTRAP(`; + `PULL` / `PARSE PULL` (external data queue / terminal reads) and statement-position + `PUSH` / `QUEUE` (stack writes — the stack is how execs feed EXECIO and host commands). +- **`ipc_rpc_bridges`** — the `ADDRESS` statement (`ADDRESS TSO`, `ADDRESS ISPEXEC`, + `ADDRESS VALUE expr`): REXX's host-command bridge. **Documented deviation from the issue + text:** #2504 grouped ADDRESS under `io`, but the io contract's unit is a data mover and + the bridge statement is ipc's "site that crosses a process or host boundary" (the + hlasm-DSECT contract-over-issue-text shape, pinned in the strict suite). `ADDRESS()` is + the built-in and never matches. + +### Control flow (z) +- **`branch`** — `IF`/`ELSE` and `SELECT`'s `WHEN`/`OTHERWISE` arms (SELECT anchored to + its `;`/EOL/`LABEL` shape), loop openers `DO WHILE/UNTIL/FOREVER` and the iterative + `DO i = ...`. `THEN` is #2822's excluded continuation word, `END` a closer, plain `DO;` + a group, `LEAVE`/`ITERATE` transfers. + +### Safety & risk +- **`safety`** — `SIGNAL ON ` / `CALL ON ` handler installs, and the + `IF RC` / `WHEN RC` return-code test every host-command exec writes (pli's `IF SQLCODE` + precedent; `when rc = 8` is a deliberate branch+safety dual, pinned). +- **`safety_bypasses`** — `SIGNAL OFF` / `CALL OFF` (handler removal) and the bare + `SIGNAL label` / `SIGNAL VALUE expr` unstructured jump (the GO TO ruling). +- **`high_risk_execution`** — `INTERPRET` (running text as code) and the `EXIT` statement + (termination; the #2878 dual with `panics_and_aborts`, which also carries ooRexx + `RAISE `). `SIGNAL EXIT` / `CALL EXIT` / a label line `EXIT:` fire neither. +- **`state_mutation`** — statement-anchored assignment (REXX has no declaration syntax, + so the assignment is the write — the shell/php/tcl ruling) with compound/stem lvalues, + plus `PARSE VAR` / `PARSE VALUE` (one owner per PARSE form: ARG is args', PULL io's). + +### The rest, by owner +`globals` (`PROCEDURE EXPOSE`, `SYSVAR(`/`MVSVAR(`, ooRexx `.environment`/`.local`); +`telemetry` (the `TRACE` statement; `TRACE(` is the built-in); `debug_prints` (`SAY`); +`ui_framework` (ISPF panel services: `ISPEXEC ... DISPLAY/ADDPOP/REMPOP/SETMSG/PQUERY/ +LMDDISP`); `cleanup` (`DROP`, EXECIO's `FINIS`, quoted TSO `FREE F|FI|DD|DDNAME|DA| +DATASET|DSNAME(`); `pointers` (`STORAGE(` — TSO/E absolute-address access); +`explicit_casts` (the radix built-ins `C2D( C2X( D2C( D2X( X2C( X2D( B2X( X2B(`); +`bitwise_ops` (`BITAND( BITOR( BITXOR(`); `scientific` (`RANDOM(`); +`reflection_metaprogramming` (`VALUE( SYMBOL( SOURCELINE(`); `time_date_logic` +(`DATE( TIME(`); `thread_sleeps` (`SysSleep`); `import`/`_dependency_capture` (ooRexx +`::REQUIRES`, quoted or bare member); `api` (`::ROUTINE/CLASS/METHOD/ATTRIBUTE ... +PUBLIC`); `encapsulation` (`... PRIVATE`; `PROCEDURE` alone is lexical scope, which #2766 +excludes — the perl/shell precedent); `immutability_locks` (`::CONSTANT`); `doc` (`/**` +block open past its own line + `PURPOSE:/DESCRIPTION:/ABSTRACT:/REMARKS:/FUNCTION:` +header tags); `ownership` (author tags in `/* */`, `*` and `--` comments); `dead_code` +(commented-out CALL/IF-THEN/DO WHILE/EXECIO/PARSE/assignment under BOTH comment styles); +`structural_boundaries` (`RETURN PROCEDURE END NOP` + `CALL`, excluding `CALL ON/OFF`); +`planned_debt`/`fragile_debt` (the shared GLOBAL rules); `spec_exposure` (`[SPEC-n]`). + +## 4. What it deliberately doesn't detect (18 explicit `None`s) + +`test` (no framework executes classic REXX cases; ooTest has no per-case keyword), +`concurrency` (strictly serial; ooRexx early-REPLY has no anchorable shape), +`sync_locks` (serialization is the host's — a quoted `"ENQ ..."` is the host-command +surface), `closures` (every routine is a label), `decorators` (OPTIONS is a runtime +instruction, not an attribute), `generics`, `comprehensions`, `memory_alloc` (storage is +implicit), `inline_asm`, `macros` (no preprocessor), `dependency_injection`, +`ssr_boundaries`, `events`, `listeners`, `test_skip`, `serialization_parsing` (PARSE is +template parsing of strings, not an interchange format), `regex_execution` +(POS/INDEX/VERIFY take no pattern), `hardcoded_secrets` (the security lens's detector +covers rexx). The keyword-rosetta side ledgers the gated three as +`rexx-stated-absences` (concurrency|sync_locks|test). + +## 5. Issues & evidence + +- #2504 — the language-addition issue (this change; engine PR + keyword-rosetta corpus PR + landed together). +- `tests/extraction/languages/test_rexx_strict.py` — 97 cases: per-signature + positive/negative coverage, the nested `/* /* */ */` shielding proof the issue + demanded, the `.cmd` collision in both directions, the SIGNAL/EXIT/PARSE ownership + pins, the ADDRESS-is-ipc deviation pin, `re.M` and schema-completeness audits, and the + scaled ReDoS detonation — which caught a real Rule-14 adjacent-quantifier defect in the + first draft's compound-lvalue tails (`(?:\.[class-with-dot]{0,64}){0,6}` on a long dot + run), fixed by excluding the dot from the segment class. +- keyword-rosetta `data/rexx/` — the 12-probe control shell (97 gate assertions), with + the string decoy planted as a bare-string host-command statement (REXX's own idiom) so + it adds no assignment. diff --git a/gitgalaxy/core/detector.py b/gitgalaxy/core/detector.py index 1cc093a5d..9f89a8673 100644 --- a/gitgalaxy/core/detector.py +++ b/gitgalaxy/core/detector.py @@ -603,6 +603,11 @@ def get_mode(cls, lang_id: str) -> Optional[str]: "pli", "powershell", "python", + # #2504: rexx's class_start is ooRexx's `::CLASS name` with the name in + # group 1; the generic fallback (`class|struct|...`, lowercase-only, no + # `::` anchor) can never match the directive form. Tree-sitter-blind; + # verified against planted corpus programs. + "rexx", "ruby", "rust", "scala", @@ -3410,6 +3415,12 @@ def _function_slice( # body early; that is the same approximation ada's nested # subprograms already take. "pli", + # #2504: rexx is COBOL's own shape -- a subroutine runs from + # its `label:` to its RETURN/EXIT (both already in the shared + # assembly_returns terminator vocabulary) or the next label, + # never a brace. Routines don't nest, so Mode A's greedy + # label-to-label body is the real boundary. + "rexx", ) or family in ("column_sensitive"): mode_name = "Mode_A_Labels" sats, impact = self._slice_by_labels(code, rules, offset, spatial_map) diff --git a/gitgalaxy/core/prism.py b/gitgalaxy/core/prism.py index 9b5279cca..9194f406b 100644 --- a/gitgalaxy/core/prism.py +++ b/gitgalaxy/core/prism.py @@ -423,10 +423,11 @@ def _positional_comment_segment(self, text: str, lang_id: str, family: str) -> s ) return positional_comments - if family in ("recursive_block", "recursive_block_haskell", "recursive_block_lisp"): + if family in ("recursive_block", "recursive_block_haskell", "recursive_block_lisp", "recursive_block_rexx"): # #2908 Phase 2 follow-up: rust/scala/swift (recursive_block), - # haskell (recursive_block_haskell) and scheme - # (recursive_block_lisp) -- see _positional_nested_comments. + # haskell (recursive_block_haskell), scheme + # (recursive_block_lisp) and rexx (recursive_block_rexx, #2504) + # -- see _positional_nested_comments. return self._positional_nested_comments(text, family) if lang_id == "perl": @@ -724,17 +725,24 @@ def _positional_nested_comments(self, text: str, family: str) -> str: s_line, b_start, b_end = delims[0], delims[1], delims[2] # Same construction as _strip_nested_comments -- see that method's - # own comment for why each alternative exists and why lisp's char - # literal must be tried first. + # own comment for why each alternative exists, why lisp's char + # literal must be tried first, and why rexx swaps the quote branches. lisp_char_literal = r"#\\(?:[a-zA-Z0-9][a-zA-Z0-9-]{0,31}|[^\s])|" if family == "recursive_block_lisp" else "" - combined_pattern = re.compile( - lisp_char_literal - + r'(?(),&|\]\s])(?:\\.|[^'\\]){0,10}'" - + r"|(?(),&|\]\s])(?:\\.|[^'\\]){0,10}'" + + r"|(? str: return "".join(ch if ch == "\n" else "\x00" for ch in s) @@ -808,7 +816,7 @@ def _strip_segment_comments(self, text: str, lang_id: str, family: str) -> tuple # line_exclusive/recursive_block/positional_anchored/block_exclusive/ # non_lexical), so none of these branches, nor the generic REGEX_MATRIX # stripper below, ever actually ran for any language. - if family in ("recursive_block", "recursive_block_haskell", "recursive_block_lisp"): + if family in ("recursive_block", "recursive_block_haskell", "recursive_block_lisp", "recursive_block_rexx"): # #621: recursive_block_haskell added because Haskell's {- -} # blocks genuinely nest (unlike the standard_block family's flat # delimiters) but use -- for line comments and {- -} rather than @@ -1494,14 +1502,30 @@ def _strip_nested_comments(self, text: str, family: str = "recursive_block") -> # it goes through the same mask/unmask path as a string so the code stream # keeps it verbatim. Bounded exactly like detector.py's _LISP_SCOPE_TOKEN. lisp_char_literal = r"#\\(?:[a-zA-Z0-9][a-zA-Z0-9-]{0,31}|[^\s])|" if family == "recursive_block_lisp" else "" - combined_pattern = re.compile( - lisp_char_literal - + r'(?(),&|\]\s])(?:\\.|[^'\\]){0,10}'" - + r"|(?(),&|\]\s])(?:\\.|[^'\\]){0,10}'" + + r"|(? str: diff --git a/gitgalaxy/standards/analysis_lens.py b/gitgalaxy/standards/analysis_lens.py index 3c92fc719..61f82f418 100644 --- a/gitgalaxy/standards/analysis_lens.py +++ b/gitgalaxy/standards/analysis_lens.py @@ -198,6 +198,11 @@ def get_policy(mode="baseline"): "powershell": (False, False, True, True), # non-terminating errors continue by default "proto": None, "python": (False, False, True, True), + # #2504: dynamically typed (everything is a string); conditions (SYNTAX/ + # ERROR/NOVALUE) are ignored unless SIGNAL ON installs a handler; no raw + # memory model of its own; a routine without PROCEDURE shares the caller's + # whole variable pool -- implicit globals by default. + "rexx": (False, False, True, False), "ruby": (False, False, True, True), "rust": (True, True, True, True), "scala": (True, False, True, True), @@ -991,6 +996,9 @@ def strictness_constants(lang_id: str) -> tuple[int, float]: "infra": { "shell", "powershell", + # #2504: z/OS automation scripting -- ADDRESS TSO/ISPEXEC host + # commands are the language's purpose, shell's own profile. + "rexx", "dockerfile", "yaml", "makefile", diff --git a/gitgalaxy/standards/gitgalaxy_config.py b/gitgalaxy/standards/gitgalaxy_config.py index 76c13f5fb..e442f0381 100644 --- a/gitgalaxy/standards/gitgalaxy_config.py +++ b/gitgalaxy/standards/gitgalaxy_config.py @@ -538,6 +538,18 @@ # a stateless per-line stripper is exactly what caused the bug. # Examples: scheme. "recursive_block_lisp": {"delimiters": [";", "#|", "|#"]}, + # 2d. Recursive Block, REXX dialect (#2504) + # Same nested-block-peeling algorithm as recursive_block ("comments + # may be nested within other comments", TSO/E REXX Reference), but + # WITHOUT recursive_block's `//` line token: `//` is REXX's + # integer-remainder operator (`a // b`), so the shared family would + # truncate real arithmetic lines. The line token is ooRexx/Regina/ + # NetRexx's `--` (classic z/OS REXX has no line comment at all; + # adjacent `--` double-negation is legal but vanishingly rare in real + # source, the accepted trade). prism.py also swaps the quote-masking + # branches for this family: REXX strings double their quote to escape, + # never backslash, and cannot span lines. + "recursive_block_rexx": {"delimiters": ["--", "/*", "*/"]}, # 3. Line Exclusive # The language possesses no native multi-line block syntax. The engine ignores closing tags. # Examples: Python, Shell, Makefile, Ruby, Perl, Assembly. diff --git a/gitgalaxy/standards/language_addition_costs.jsonl b/gitgalaxy/standards/language_addition_costs.jsonl index 994d90c15..69b8e53dd 100644 --- a/gitgalaxy/standards/language_addition_costs.jsonl +++ b/gitgalaxy/standards/language_addition_costs.jsonl @@ -3,3 +3,4 @@ {"recorded_at":"2026-09-16T22:11:10.047283+00:00","lang":"db2_sql","issue":"2511","phase":"combined","runner":"claude-code","primary_model":"claude-fable-5","notes":"SQL-dialect family; Mode E one-statement-one-hit; caught a census contract violation","started_at":"2026-09-16T12:22:30.638000+00:00","elapsed_min":196.8,"active_min":107.2,"output_tokens":724758,"fresh_tokens":1560430,"cache_read_tokens":154423681,"total_tokens":156708869,"est_cost_usd":210.16,"model_output_split":{"claude-fable-5":724758,"":0},"unpriced_models":[""],"estimate":false,"sessions":["6418a243-0f16-4a37-8289-76955f19b0a9.jsonl"]} {"recorded_at":"2026-09-16T22:11:10.078475+00:00","lang":"hlasm","issue":"2503","phase":"combined","runner":"claude-code","primary_model":"claude-fable-5","notes":"built from bms (bms IS HLASM macro source); widened positional comment gate; user's '~1h/150k' recollection = this run (150k was the live context-gauge peak, not cumulative)","started_at":"2026-09-16T15:42:01.685000+00:00","elapsed_min":79.5,"active_min":75.0,"output_tokens":600318,"fresh_tokens":1123295,"cache_read_tokens":130950075,"total_tokens":132673688,"est_cost_usd":175.0,"model_output_split":{"claude-fable-5":600318},"unpriced_models":[],"estimate":false,"sessions":["c2b5e5ef-0509-421e-8c45-ef502a26dd56.jsonl"]} {"recorded_at":"2026-09-16T22:18:12.689521+00:00","lang":"pli","issue":"3057","phase":"engine","runner":"agy-gemini","primary_model":"gemini-3.1-pro","notes":"engine half; gitgalaxy PR #3057 (issues #2502/#1142): +1661/-5 across 12 files, CICS/SQL/DLI parity with cobol; committed 03:25-03:41Z 2026-09-15; agy/Gemini build, no Claude transcript so tokens/cost unmeasured","started_at":"2026-09-15T03:25Z","elapsed_min":null,"active_min":null,"output_tokens":null,"fresh_tokens":null,"cache_read_tokens":null,"total_tokens":null,"est_cost_usd":null,"model_output_split":{},"unpriced_models":[],"estimate":true} +{"recorded_at":"2026-09-17T01:28:44.233040+00:00","lang":"rexx","issue":"2504","phase":"combined","runner":"claude-code","primary_model":"","notes":"engine+corpus in one session; recursive_block_rexx prism dialect (// is REXX remainder); .cmd batch/rexx collision required giving batch an internal_discriminator (empty rules dict loses every Tier-3 scan); rules draft + strict skeleton delegated to agy/gemini-3.1-pro, adjudication+wiring Claude; bias report caught the groupless-args avg_func_args artifact; gemini draft output tokens not in this transcript","started_at":"2026-09-17T00:34:05.540000+00:00","elapsed_min":54.6,"active_min":54.5,"output_tokens":498906,"fresh_tokens":890101,"cache_read_tokens":102656642,"total_tokens":104045649,"est_cost_usd":138.73,"model_output_split":{"claude-fable-5":498906},"unpriced_models":[],"estimate":false,"sessions":["6e0399f5-2949-4c64-9358-53e767286614.jsonl"]} diff --git a/gitgalaxy/standards/language_standards/__init__.py b/gitgalaxy/standards/language_standards/__init__.py index 4b0153044..c73f91802 100644 --- a/gitgalaxy/standards/language_standards/__init__.py +++ b/gitgalaxy/standards/language_standards/__init__.py @@ -118,6 +118,7 @@ from .languages import powershell as _powershell from .languages import proto as _proto from .languages import python as _python +from .languages import rexx as _rexx from .languages import ruby as _ruby from .languages import rust as _rust from .languages import scala as _scala @@ -209,4 +210,5 @@ "bms": _bms.DEFINITION, "db2_sql": _db2_sql.DEFINITION, "hlasm": _hlasm.DEFINITION, + "rexx": _rexx.DEFINITION, } diff --git a/gitgalaxy/standards/language_standards/_lens_config.py b/gitgalaxy/standards/language_standards/_lens_config.py index 2e2723d42..c84aa5dc5 100644 --- a/gitgalaxy/standards/language_standards/_lens_config.py +++ b/gitgalaxy/standards/language_standards/_lens_config.py @@ -58,7 +58,25 @@ class LensConfig(TypedDict): # internal_discriminator (Tier 2: CSECT/DSECT/USING/... in operation-field # position), mainframe-sibling ecosystem gravity (Tier 1.5) or the lexical # scan (Tier 3). ".mac" and ".hlasm" are uncontested and stay Tier 1. - "COLLISION_FREQUENCIES": {".inc", ".h", ".py", ".cshtml", ".c", ".y", ".m", ".map", ".sql", ".ddl", ".dml", ".asm"}, + # #2504: ".cmd" is claimed by BOTH batch (Windows/OS2) and rexx (z/OS, + # OS/2), same mechanism -- rexx's internal_discriminator (a .cmd opening + # with `/*` is REXX, the platform loaders' own dispatch rule) resolves it; + # ".rexx" and ".exec" are uncontested and stay Tier 1. + "COLLISION_FREQUENCIES": { + ".inc", + ".h", + ".py", + ".cshtml", + ".c", + ".y", + ".m", + ".map", + ".sql", + ".ddl", + ".dml", + ".asm", + ".cmd", + }, "PROSE_ANCHORS": { "README", "LICENSE", diff --git a/gitgalaxy/standards/language_standards/languages/batch.py b/gitgalaxy/standards/language_standards/languages/batch.py index a72f0ec73..47df43736 100644 --- a/gitgalaxy/standards/language_standards/languages/batch.py +++ b/gitgalaxy/standards/language_standards/languages/batch.py @@ -8,6 +8,7 @@ # of this project, or at https://polyformproject.org/licenses/noncommercial/1.0.0/ # ============================================================================== +import re from typing import Any DEFINITION: dict[str, Any] = { @@ -17,5 +18,26 @@ "discriminators": [], "shebangs": [], "lexical_family": "line_exclusive", + # Collision resolution for `.cmd` (#2504: rexx claims it too, and batch's + # empty rules dict scores 0 in the Tier 3 lexical scan, so without a + # Tier 2 anchor every real batch file would lose the scan to any language + # with rules). These are batch-only line shapes REXX cannot carry: + # `@echo off/on`, SETLOCAL/ENDLOCAL, `goto :label`, the `%~dp0`/`%%~x` + # argument modifiers and `%ERRORLEVEL%`/`errorlevel N` tests. Bare `rem` + # and `set X=1` are deliberately left out -- `rem = n // 7` and + # `set = 1` are legal REXX assignments (registry order checks batch's + # discriminator FIRST, so a false batch hit on real REXX would lock the + # wrong language, the costlier direction). + "internal_discriminator": re.compile( + r"^[ \t]*@?ECHO[ \t]+(?:OFF|ON)\b" + r"|^[ \t]*(?:SETLOCAL|ENDLOCAL)\b" + r"|^[ \t]*GOTO[ \t]+:?[A-Za-z_]" + r"|^[ \t]*@?SET[ \t]+\"?[A-Za-z_][\w]{0,63}=" + r"|^[ \t]*IF[ \t]+(?:NOT[ \t]+)?(?:EXIST|DEFINED)\b" + r"|%%?~[A-Za-z]{0,10}[0-9]" + r"|%ERRORLEVEL%" + r"|\bERRORLEVEL[ \t]+[0-9]", + re.M | re.I, + ), "rules": {}, } diff --git a/gitgalaxy/standards/language_standards/languages/rexx.py b/gitgalaxy/standards/language_standards/languages/rexx.py new file mode 100644 index 000000000..487dbff45 --- /dev/null +++ b/gitgalaxy/standards/language_standards/languages/rexx.py @@ -0,0 +1,519 @@ +# ============================================================================== +# GitGalaxy +# Copyright (c) 2026 Joe Esquibel +# +# This source code is licensed under the PolyForm Noncommercial License 1.0.0. +# You may not use this file except in compliance with the License. +# A copy of the license can be found in the LICENSE file in the root directory +# of this project, or at https://polyformproject.org/licenses/noncommercial/1.0.0/ +# ============================================================================== + +import re +from typing import Any + +from .._shared_patterns import GLOBAL_FRAGILE_DEBT, GLOBAL_PLANNED_DEBT + +# #2504: REXX -- z/OS TSO/E REXX and classic (SAA) REXX, plus the ooRexx +# directive surface (`::requires` / `::routine` / `::class` / `::method`), +# since `.rexx` files in the wild host ooRexx code too. +# +# A REXX symbol is letters, digits and `. ! ? _` plus the national characters +# `@ # $` (TSO/E REXX Reference, "Tokens"). `\w` covers letters/digits/`_`; +# `! ? @ # $` are regex NON-word characters, so a plain `\b` cannot guard a +# keyword against them (`?EXIT` would satisfy `\bEXIT`): bare keywords use +# these explicit guards, PL/I's discipline. `.` joins both guards -- a +# compound variable's tail (`rc.exit`) must not read as the keyword -- and +# stays OUT of the label-name class below (a label is a simple symbol). +_ID = r"[\w.!?@#$]" +_NAME = r"[\w!?@#$]" +_L = r"(?`, statement-position `ARG