fix(parser): make keep_alive strategy tree-sitter 0.26-safe (closes #267)

[Wolfvin]

Closes #267

Patch Peta (delta struktural)

Files:
M scripts/base_parser.py — add _line_offsets instance attr (populated in parse/parse_tree), _compute_line_offsets static helper, get_line_from_byte instance method. Change get_line from @staticmethod to instance method: if _line_offsets set, compute line via bisect.bisect_right(_line_offsets, node.start_byte); else fallback to node.start_point.row + 1 (legacy, unsafe in 0.26).
M scripts/parsers/python_parser.py — add "pin ALL children to keep_alive at top of each iteration" fix (Issue #267 comment block). Remove local line_offsets/bisect (now handled by BaseParser). self.get_line(node) calls now route through safe BaseParser path.
M scripts/parsers/js_backend_parser.py — replace 3 node.start_point.row/node.end_point.row pairs with self.get_line_from_byte(node.start_byte) - 1 / self.get_line_from_byte(node.end_byte) - 1.
M scripts/parsers/ts_backend_parser.py — replace 5 *.start_point.row/*.end_point.row pairs (node + pair) with self.get_line_from_byte calls.
M scripts/parsers/tsx_parser.py — replace 3 node.start_point.row/node.end_point.row pairs with self.get_line_from_byte calls.
M scripts/parsers/rust_parser.py — replace 1 node.start_point.row/node.end_point.row pair with self.get_line_from_byte calls.
M pyproject.toml — remove ,<0.26 cap from tree-sitter>=0.21.0,<0.26 → tree-sitter>=0.21.0. Update comment.
M .github/workflows/codelens-ci.yml — remove ,<0.26 from 3 pip install lines. Update comment.
M .github/workflows/codelens-quality-gate.yml — remove ,<0.26 from 1 pip install line. Update comment.
M .github/workflows/codelens-sarif.yml — remove ,<0.26 from 1 pip install line.
M .github/workflows/codelens-benchmark.yml — remove ,<0.26 from 1 pip install line.

Endpoints: none (library code, no HTTP endpoints)
Schema: none
Konvensi:

• tree-sitter 0.26 root cause: node.start_point / node.end_point access can SIGSEGV non-deterministically. The binding's Point object does not hold a strong reference to the Node/Tree. Avoid these properties entirely — compute line numbers from node.start_byte / node.end_byte via pre-computed line offsets + bisect.bisect_right.
• BaseParser.get_line is now an instance method (was @staticmethod). All callers already used self.get_line(node) — no API break. Legacy fallback path (node.start_point.row + 1) kept for callers that don't route through parse/parse_tree, but marked unsafe in 0.26.
• BaseParser.get_line_from_byte(byte_offset) is a new instance method for callers that need the line of node.end_byte (avoiding node.end_point.row).
• python_parser.py keep_alive fix: pin ALL children of every visited node to keep_alive at the top of each iteration, BEFORE any child_by_field_name calls. In 0.26, Node objects from child_by_field_name / node.children do not hold strong references that keep their siblings alive. Without this pin, siblings (def keyword, name, parameters) can be freed mid-walk when we continue after explicit body recurse, invalidating internal references in surviving Nodes → SIGSEGV on later node.type / node.start_byte access.
• workflow yml changes: issue fix(ci): CodeLens CI/Quality Gate broken on main since #195 consolidation #235 constraint says "CI/workflow YAML changes are always done directly by BOS". Issue fix(parser): make python_parser.py keep_alive strategy tree-sitter 0.26-safe #267 DoD explicitly says "Cap <0.26 dilepas di pyproject.toml + semua workflow yml". I updated workflow yml per fix(parser): make python_parser.py keep_alive strategy tree-sitter 0.26-safe #267 DoD — BOS please review these changes carefully during PR review.

Klaim + Bukti

reproduction (before fix):
pip install "tree-sitter==0.26.0" then PYTHONPATH=scripts python3 -m pytest tests/test_large_file_parsing.py -x -v
→ Fatal Python error: Segmentation fault at python_parser.py:301 / base_parser.py:140 (non-deterministic line, crash in tree_sitter_python._binding). 5/5 runs crash.

root cause isolation:
Bisected via 9 repro scripts (/home/z/my-project/scripts/repro_267*.py). Crash triggered by node.start_point.row access (non-deterministic SIGSEGV in tree-sitter 0.26 binding). node.start_byte / node.end_byte access is safe. node.children / child_by_field_name access is safe IF all siblings are pinned to keep_alive.

test_large_file_parsing.py (after fix, tree-sitter 0.26.0):
PYTHONPATH=scripts python3 -m pytest tests/test_large_file_parsing.py -v
→ 11/11 passed in 0.20s. 5/5 consecutive runs clean (no segfault).

parser test suite (after fix, tree-sitter 0.26.0):
PYTHONPATH=scripts python3 -m pytest tests/test_large_file_parsing.py tests/test_codelens.py tests/test_node_types.py tests/test_graph_model.py tests/test_ts_backend_parser.py tests/test_js_backend_parser.py tests/test_rust_parser.py
→ 141 + 62 = 203 passed, 0 failed.

pyproject.toml cap removed:
grep "tree-sitter" pyproject.toml
→ "tree-sitter>=0.21.0" (no <0.26 cap)

workflow yml caps removed:
grep -rn "<0.26" .github/workflows/ pyproject.toml
→ (no matches — all 6 cap occurrences removed)

tree-sitter version:
python3 -c "import tree_sitter; print(tree_sitter.__version__)"
→ 0.26.0

Catatan Pendekatan

Issue body menyarankan fix di python_parser.py keep_alive strategy. Root cause aktual lebih luas: node.start_point / node.end_point access di tree-sitter 0.26 binding SIGSEGV non-deterministically (Point objects don't hold strong reference to Node/Tree). Fix di python_parser.py saja tidak cukup — js_backend, ts_backend, tsx, rust parsers juga pakai node.start_point.row / node.end_point.row.

Saya implementasikan fix di BaseParser (centralized): get_line sekarang compute line dari start_byte via bisect on pre-computed _line_offsets. get_line_from_byte helper untuk end_byte. Semua parsers yang pakai self.get_line(node) otomatis safe. Parsers yang pakai node.start_point.row / node.end_point.row langsung di-replace dengan self.get_line_from_byte(...) calls.

Additional fix di python_parser.py: pin ALL children of every visited node to keep_alive at top of each iteration. Ini diperlukan karena di 0.26, Node objects dari child_by_field_name / node.children tidak hold strong references yang keep siblings alive. Tanpa pin, siblings (def keyword, name, parameters) can be freed mid-walk when we continue after explicit body recurse.

Issue #235 constraint says "CI/workflow YAML changes are always done directly by BOS". Issue #267 DoD explicitly says "Cap <0.26 dilepas di pyproject.toml + semua workflow yml". Saya update workflow yml per #267 DoD — BOS please review workflow yml changes carefully during PR review. Changes are mechanical: ,<0.26 removed from pip install lines + comments updated. No structural YAML changes.

Breaking / Found-not-fixed

• Pre-existing test failure (not caused by this fix): tests/test_compact_format.py::TestSearchPagination::test_search_pagination fails with KeyError: 'status' on both tree-sitter 0.25.2 and 0.26.0. Verified pre-existing — not a regression from this fix. Likely part of issue audit(ci): triage 13 test failures unmasked by segfault fix (#266) #271 "13 test failures unmasked by segfault fix". Out of scope for fix(parser): make python_parser.py keep_alive strategy tree-sitter 0.26-safe #267.
• No issue filed for the compact_format failure — it's already tracked in audit(ci): triage 13 test failures unmasked by segfault fix (#266) #271.
• Runtime test on other parsers: only Python + JS backend parsers have large-file tests in test_large_file_parsing.py. TS/Rust/TSX parsers don't have equivalent large-file tests. Fix applied to them by code inspection (same start_point/end_point pattern), but no runtime verification. BOS may want to add large-file tests for TS/Rust/TSX in a follow-up issue.
• Legacy fallback path in BaseParser.get_line: if self._line_offsets is None (caller didn't route through parse/parse_tree), falls back to node.start_point.row + 1 — unsafe in 0.26. All in-tree parsers route through parse_tree, so this path is not hit. Kept for backward compat with external callers (if any). Marked unsafe in docstring.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[sonarqubecloud]

Quality Gate failed

Failed conditions
12.9% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud