feat(extract): add Perl extractor — packages and subs (.pl/.pm, tree-sitter-perl) - #3013
feat(extract): add Perl extractor — packages and subs (.pl/.pm, tree-sitter-perl)#3013ad-astra-bot wants to merge 3 commits into
Conversation
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Formal verification. 2 change(s) tested, no difference found (not proven).
Graphify review — findings
Adds a Perl extractor (graphify/extractors/perl.py) for .pl/.pm files covering package declarations (statement, block, and mid-file-switch forms plus implicit main) and sub definitions including qualified sub Pkg::sub, with budget-bounded iterative tree walks. Wires it into detect.CODE_EXTENSIONS, extract._DISPATCH, _LANG_FAMILY_BY_EXT, the perl shebang route, and the LANGUAGE_EXTRACTORS/extractors.__init__ registry. Bumps the README grammar count to 38 and adds a CHANGELOG entry for 0.9.49.
No blocking issues surfaced. 4 lower-confidence candidates did not survive cross-model review.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 2787 functions depend on the 1454 functions this change touches.
Health — this change adds coupling hotspots:
- new:
extract()— 474 callers, 42 callees - new:
_rebuild_code()— 98 callers, 50 callees - new:
detect()— 108 callers, 15 callees - new:
save_manifest()— 39 callers, 11 callees - new:
extract_files_direct()— 17 callers, 20 callees - new:
extract_xaml()— 19 callers, 17 callees - new:
extract_corpus_parallel()— 26 callers, 11 callees - new:
extract_js()— 80 callers, 3 callees - …and 39 more — each is listed as a finding
Verification — 2787 functions in the blast radius were not formally verified this run (proofs are advisory here).
Health delta baseline: last indexed commit b2cd362, 31 commit(s) behind this PR's base.
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 2592 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify load\_cached.
The verifier did not have enough to check load\_cached, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify save\_semantic\_cache.
The verifier did not have enough to check save\_semantic\_cache, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify dispatch\_command.
The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)
Could not verify: Could not verify \_stamped\_manifest\_files.
The verifier did not have enough to check \_stamped\_manifest\_files, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
No difference found (not proven): No behavior difference found in to\_obsidian (not a proof).
The verifier ran both versions of to\_obsidian on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify extract.
The verifier did not have enough to check extract, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `cache_root` is annotated `Path | None` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_rewire\_unique\_stub\_nodes.
The verifier did not have enough to check \_rewire\_unique\_stub\_nodes, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: the input domain has 81 values but only 9 distinct were tested — a small finite domain must be EXHAUSTED, not sampled (an untested input could invert the result)
Could not verify: Could not verify extract\_bash.
The verifier did not have enough to check extract\_bash, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify extract\_commonlisp.
The verifier did not have enough to check extract\_commonlisp, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_extract\_generic.
The verifier did not have enough to check \_extract\_generic, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify extract\_julia.
The verifier did not have enough to check extract\_julia, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify extract\_powershell.
The verifier did not have enough to check extract\_powershell, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_load\_tsconfig\_aliases.
The verifier did not have enough to check \_load\_tsconfig\_aliases, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `start_dir` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_load\_tsconfig\_base\_url.
The verifier did not have enough to check \_load\_tsconfig\_base\_url, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `start_dir` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_merge\_decl\_def\_classes.
The verifier did not have enough to check \_merge\_decl\_def\_classes, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: the input domain has 81 values but only 9 distinct were tested — a small finite domain must be EXHAUSTED, not sampled (an untested input could invert the result)
Could not verify: Could not verify extract\_sql.
The verifier did not have enough to check extract\_sql, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify extract\_zig.
The verifier did not have enough to check extract\_zig, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify bisect\_slice.
The verifier did not have enough to check bisect\_slice, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `fs` is annotated `FileSlice` — outside the synthesizable primitive/collection set
No difference found (not proven): No behavior difference found in expand\_oversized\_files (not a proof).
The verifier ran both versions of expand\_oversized\_files on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify is\_splittable\_text.
The verifier did not have enough to check is\_splittable\_text, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify read\_slice\_text.
The verifier did not have enough to check read\_slice\_text, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `fs` is annotated `FileSlice` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_estimate\_file\_tokens.
The verifier did not have enough to check \_estimate\_file\_tokens, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `unit` is annotated `'Path | FileSlice'` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_build\_server.
The verifier did not have enough to check \_build\_server, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous
· 2 grounded finding(s) anchored inline below; 45 more finding(s) on lines outside this diff (see the check run).
| ) | ||
|
|
||
|
|
||
| def extract_perl(path: Path) -> dict: |
There was a problem hiding this comment.
extract_perl()
10 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return False | ||
| return True | ||
|
|
||
| def walk_statements(root_node) -> None: |
There was a problem hiding this comment.
walk_statements()
fans out to 7 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.
Formal verification. 2 change(s) tested, no difference found (not proven).
Graphify review — findings
Adds a Perl extractor (graphify/extractors/perl.py) covering package declarations (statement, block, and mid-file forms plus implicit main) and sub definitions, wired into _DISPATCH, _SHEBANG_DISPATCH, _LANG_FAMILY_BY_EXT, and the LANGUAGE_EXTRACTORS registry. Registers .pl/.pm in CODE_EXTENSIONS, bumps the README grammar count to 38, and adds a 0.9.49 changelog entry. Call resolution, imports, and inheritance are explicitly deferred to later slices.
Worth a look
- Extractor package import can fail because Perl module is not added —
graphify/extractors/__init__.py:28· Escalate · high- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 2795 functions depend on the 1462 functions this change touches.
Health — this change adds coupling hotspots:
- new:
extract()— 474 callers, 42 callees - new:
_rebuild_code()— 98 callers, 50 callees - new:
detect()— 108 callers, 15 callees - new:
save_manifest()— 39 callers, 11 callees - new:
extract_files_direct()— 17 callers, 20 callees - new:
extract_xaml()— 19 callers, 17 callees - new:
extract_corpus_parallel()— 26 callers, 11 callees - new:
extract_js()— 80 callers, 3 callees - …and 39 more — each is listed as a finding
Verification — 2795 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 2600 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify load\_cached.
The verifier did not have enough to check load\_cached, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify save\_semantic\_cache.
The verifier did not have enough to check save\_semantic\_cache, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify dispatch\_command.
The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)
Could not verify: Could not verify \_stamped\_manifest\_files.
The verifier did not have enough to check \_stamped\_manifest\_files, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
No difference found (not proven): No behavior difference found in to\_obsidian (not a proof).
The verifier ran both versions of to\_obsidian on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify extract.
The verifier did not have enough to check extract, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `cache_root` is annotated `Path | None` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_rewire\_unique\_stub\_nodes.
The verifier did not have enough to check \_rewire\_unique\_stub\_nodes, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: the input domain has 81 values but only 9 distinct were tested — a small finite domain must be EXHAUSTED, not sampled (an untested input could invert the result)
Could not verify: Could not verify extract\_bash.
The verifier did not have enough to check extract\_bash, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify extract\_commonlisp.
The verifier did not have enough to check extract\_commonlisp, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_extract\_generic.
The verifier did not have enough to check \_extract\_generic, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify extract\_julia.
The verifier did not have enough to check extract\_julia, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify extract\_powershell.
The verifier did not have enough to check extract\_powershell, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_load\_tsconfig\_aliases.
The verifier did not have enough to check \_load\_tsconfig\_aliases, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `start_dir` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_load\_tsconfig\_base\_url.
The verifier did not have enough to check \_load\_tsconfig\_base\_url, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `start_dir` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_merge\_decl\_def\_classes.
The verifier did not have enough to check \_merge\_decl\_def\_classes, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: the input domain has 81 values but only 9 distinct were tested — a small finite domain must be EXHAUSTED, not sampled (an untested input could invert the result)
Could not verify: Could not verify extract\_sql.
The verifier did not have enough to check extract\_sql, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify extract\_zig.
The verifier did not have enough to check extract\_zig, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify bisect\_slice.
The verifier did not have enough to check bisect\_slice, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `fs` is annotated `FileSlice` — outside the synthesizable primitive/collection set
No difference found (not proven): No behavior difference found in expand\_oversized\_files (not a proof).
The verifier ran both versions of expand\_oversized\_files on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify is\_splittable\_text.
The verifier did not have enough to check is\_splittable\_text, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify read\_slice\_text.
The verifier did not have enough to check read\_slice\_text, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `fs` is annotated `FileSlice` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_estimate\_file\_tokens.
The verifier did not have enough to check \_estimate\_file\_tokens, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `unit` is annotated `'Path | FileSlice'` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_build\_server.
The verifier did not have enough to check \_build\_server, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous
· 2 grounded finding(s) anchored inline below; 45 more finding(s) on lines outside this diff (see the check run).
| ) | ||
|
|
||
|
|
||
| def extract_perl(path: Path) -> dict: |
There was a problem hiding this comment.
extract_perl()
14 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return False | ||
| return True | ||
|
|
||
| def walk_statements(root_node) -> None: |
There was a problem hiding this comment.
walk_statements()
fans out to 7 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
a103ed1 to
7bf0c19
Compare
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 2 advisory finding(s) below merit a look before merge.
Graphify review — findings
Adds a Perl extractor (extract_perl) built on tree-sitter-perl that recognizes package declarations in statement, block (package Foo { ... } with correct scope restore), and mid-file switch forms, materializes Perl's implicit main package lazily for package-less code, and emits sub definitions including qualified sub Pkg::sub names; imports, inheritance, and call resolution are deferred to later slices. Registers .pl/.pm across the dispatch, language-family, extractor, and code-extension tables, and routes extensionless #!/usr/bin/perl scripts through the shebang map. Guards against runaway files with iterative, node-budget-bounded tree walks that keep partial output on exhaustion, and discards malformed or non-ASCII package names rather than labeling them.
Worth a look
- block-form package name restore uses stale scope after descent —
graphify/extractors/perl.py:160· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Perl walker skips declarations inside control-flow blocks —
graphify/extractors/perl.py:184· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 2995 functions depend on the 1505 functions this change touches.
Health — this change adds coupling hotspots:
- new:
extract()— 498 callers, 42 callees - new:
_rebuild_code()— 98 callers, 50 callees - new:
detect()— 108 callers, 15 callees - new:
save_manifest()— 40 callers, 11 callees - new:
extract_files_direct()— 17 callers, 20 callees - new:
extract_xaml()— 19 callers, 17 callees - new:
extract_corpus_parallel()— 26 callers, 11 callees - new:
extract_js()— 83 callers, 3 callees - …and 42 more — each is listed as a finding
Verification — 2995 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 2768 function(s) in the blast radius were not formally verified this run
· 2 grounded finding(s) anchored inline below; 48 more finding(s) on lines outside this diff (see the check run).
| ) | ||
|
|
||
|
|
||
| def extract_perl(path: Path) -> dict: |
There was a problem hiding this comment.
extract_perl()
14 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return False | ||
| return True | ||
|
|
||
| def walk_statements(root_node) -> None: |
There was a problem hiding this comment.
walk_statements()
fans out to 7 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
Perl .pl/.pm files (and extensionless #!/usr/bin/perl shebang scripts, now
mapped in _SHEBANG_DISPATCH) extract package declarations in all three forms —
statement, block `package Foo { ... }` with scope restore for statements that
follow the block, and mid-file switches — plus sub definitions including
qualified `sub Pkg::sub` declarations and Perl's implicit `main` package,
materialized lazily so a file with no package-less subs emits no empty node.
Package names are validated ASCII identifiers before they become labels: the
tree-sitter name child is arbitrary source text, and a crafted or malformed
name would otherwise flow raw into node labels and on into exports. The tree
walks are iterative with a per-sibling traversal budget, so pathologically
deep or wide files keep a partial graph instead of RecursionError-dropping.
Imports (`use`/`require`), inheritance (\@ISA/use parent/use base), and call
resolution are deliberately out of scope here; they follow as separate slices
so each lands reviewable. Closes the extensionless-perl gap in the shebang
dispatch (Graphify-Labs#419 mentions the demand).
37 -> 38: .pl/.pm join the code grammar table; entry under 0.9.49 (unreleased).
…onicalize root-qualified names
Two gaps found by review:
A bare `{ ... }` block or a phaser (`BEGIN`/`CHECK`/`INIT`/`END`/`UNITCHECK`)
containing `package`/`sub` declarations was never traversed — the walker only
descended into blocks attached directly to a `package_statement`, so valid
declarations inside ordinary scopes were silently missing. Both forms now
push a walker frame at the enclosing scope; the existing restore-on-exit keeps
statements after the block attributed correctly.
Root-qualified names (::Name == main::Name) produced an empty-label package
node via rpartition on `sub ::foo {}`, and `package ::Outer;` failed label
validation outright. A leading :: is now stripped before validation and
container selection, so both spellings key to the same canonical package node.
Real-corpus effect (Foswiki core/lib): contains edges 2385 -> 2402 — the
missing declarations were predominantly inside BEGIN blocks.
7bf0c19 to
1dd51a5
Compare
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 3 advisory finding(s) below merit a look before merge.
Graphify review — findings
Adds Perl .pl/.pm extraction backed by tree-sitter-perl in a new extract_perl, wired into the dispatch tables, _LANG_FAMILY_BY_EXT, CODE_EXTENSIONS, and the shebang map so extensionless #!/usr/bin/perl scripts also route through. It materializes package declarations in statement, block-scoped, and mid-file-switch forms plus a lazily-created implicit main, and sub definitions including qualified sub Pkg::sub, while validating package names against an ASCII-only pattern so malformed labels are dropped instead of emitted. The tree walks are iterative and capped at _MAX_PERL_TRAVERSAL_NODES, leaving pathologically large files partial rather than crashing; imports, inheritance, and call resolution are deferred to later slices.
Worth a look
- Restored scope after descent uses stale captured values, mis-attributing subs after nested package block —
graphify/extractors/perl.py:191· Escalate · high- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Qualified sub name validation missing before use as label —
graphify/extractors/perl.py· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Qualified sub name validation missing; malformed package qualifier becomes a node label —
graphify/extractors/perl.py:200· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 3013 functions depend on the 1511 functions this change touches.
Health — this change adds coupling hotspots:
- new:
extract()— 500 callers, 42 callees - new:
_rebuild_code()— 98 callers, 50 callees - new:
detect()— 108 callers, 15 callees - new:
save_manifest()— 40 callers, 11 callees - new:
extract_files_direct()— 17 callers, 20 callees - new:
extract_xaml()— 19 callers, 17 callees - new:
extract_corpus_parallel()— 26 callers, 11 callees - new:
extract_js()— 85 callers, 3 callees - …and 42 more — each is listed as a finding
Verification — 3013 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 2780 function(s) in the blast radius were not formally verified this run
· 2 grounded finding(s) anchored inline below; 48 more finding(s) on lines outside this diff (see the check run).
| ) | ||
|
|
||
|
|
||
| def extract_perl(path: Path) -> dict: |
There was a problem hiding this comment.
extract_perl()
14 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return False | ||
| return True | ||
|
|
||
| def walk_statements(root_node) -> None: |
There was a problem hiding this comment.
walk_statements()
fans out to 7 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
1 participant
References #419 ("Add Perl support via tree-sitter", open since 2026-04-17). First slice of the decomposition sketched in #1788 — that PR stays open as the umbrella until all slices land.
This slice: package and sub extraction only — deliberately narrow so it is reviewable without CI (the workflow runs on outside PRs stand at
action_required, so a reviewer sees this diff raw)..pl/.pminCODE_EXTENSIONS; extensionless#!/usr/bin/perlscripts now dispatch through_SHEBANG_DISPATCH(previously detected-but-dropped)package Foo;), block (package Foo { ... }with scope restore — statements after the block are not mis-attributed), and mid-file switchessub Pkg::subdeclarations; Perl's implicitmainpackage is materialized lazily so package-less code gets an honest scope without emitting empty nodes for files that don't need oneDeliberately out of scope here, following as separate slices:
use/requireimports with in-corpus re-pointing,@ISA/use parent/use baseinheritance, and cross-file call resolution (#1788 has those parts; each will come as its own reviewable change).Verification:
v8worktree (test_graphifyignore_hermetic_without_vcs,test_no_git_repo_raises; both assume a directory without VCS)containsedge set is identical to the full extractor's from feat(extract): add Perl support (.pl/.pm, tree-sitter-perl) #1788 (2385 edges), i.e. the package/sub structure is unchanged by the slicingtest_extensionless_without_usable_shebang_stays_unsupportedfixed the old unmapped-perl behavior; its replacement asserts the new dispatch plus keeps the fish case as the still-unmapped guardtree-sitter-perl >=1.2.0,<2.0added to core dependencies (not an extra) since.pl/.pmjoin the default code extensions.