Summary
On codegraph init . --force (v1.5.0), a re-index of an existing, previously-clean project walked into two subdirectories whose node_modules trees are correctly excluded for git itself by nested (non-root) .gitignore files — producing a ~63x file-count blowup (509 files -> 32,260 files, 8,906 nodes -> 514,456 nodes, 724,013 edges, 1.8 GB DB) before the run was manually aborted (host-resource discipline; the run was still growing).
This is a real-world regression report from an active project (boba, a torrent-search proxy stack), not a synthetic construction. I (an independent contributor, not affiliated with the project that found it) verified the root-cause evidence first-hand and spent significant effort trying to reproduce it in isolation — full details below, including a negative result that may itself help triage.
Root cause evidence (git side — confirmed first-hand)
$ git check-ignore -v frontend/node_modules
frontend/.gitignore:10:/node_modules frontend/node_modules
$ git check-ignore -v extension/node_modules
extension/.gitignore:2:node_modules/ extension/node_modules
$ git ls-files frontend/node_modules extension/node_modules | wc -l
0
Both directories are correctly and unambiguously excluded for git by their own nested .gitignore (not the project's root .gitignore, which does not mention extension/node_modules at all — only frontend/node_modules/ is separately listed there as a belt-and-suspenders entry). Yet codegraph init . --force on v1.5.0 walked into both anyway (365 MB + 236 MB, tens of thousands of files).
The project's own codegraph version had moved from 0.9.9 (last validated 2026-06-06, 509 files / 8,906 nodes, 7 PASS / 0 FAIL) to 1.5.0 with no re-verification in between — frontend/ and extension/ (each with their own node_modules and nested .gitignore) were both added to the project after the 0.9.9 baseline, so this exclusion path had never been exercised against the newer CodeGraph version before this incident.
What I could NOT reproduce (important negative result)
I read src/extraction/index.ts on main (current HEAD, still 1.5.0 per package.json) and traced both scanning implementations:
getGitVisibleFiles() — the primary path, delegates to real git ls-files -z -s --recurse-submodules + git ls-files -z -o --exclude-standard, which should inherit git's own correct nested-.gitignore handling.scanDirectoryWalk() — the fallback for non-git projects, which explicitly layers a ScopedIgnore matcher per directory level (comment: "Patterns in a nested .gitignore are relative to that directory... mirroring how git applies .gitignore files at every level") — this also looks correct on inspection.node_modules is additionally a hardcoded DEFAULT_IGNORE_DIRS entry (src/extraction/index.ts ~line 165), applied as a bare gitignore-style pattern (node_modules/) that should match at any depth regardless of which path above is taken.
Given that, I built two synthetic fixtures and ran the actual installed v1.5.0 binary against them:
- A minimal 2-level fixture: root + 1 subdirectory with its own nested
.gitignore excluding node_modules/ (400 files under node_modules, no root-level mention). codegraph init --force correctly indexed only the 2 real source files — did not walk into node_modules.
- A larger fixture mimicking the real shape more closely: root + 2 subprojects (each with its own
package.json + nested .gitignore excluding node_modules/), 150 fake packages per subproject (some shipping their own .gitignore, matching how real npm packages sometimes do), .bin symlinks (also common in real npm installs) — 63 nested .gitignore files total, 960 files correctly excluded by git. codegraph init --force again correctly indexed only the 3 real source files.
So the bug is real (confirmed first-hand against the real project) but I could not trigger it in either synthetic fixture at hundreds-of-files scale with the same binary version. This suggests the trigger is scale-dependent and/or specific to something about a genuine npm install-produced tree (24,102 real files under frontend/node_modules, 13,449 under extension/node_modules in the reporting project — deeply nested transitive deps, .map/binary files, very long paths, etc.) that I judged unsafe/disproportionate to reproduce here given the same host-resource concern that caused the original report to abort its own run (this is exactly the failure mode being reported, so reproducing it at full scale would risk the same runaway walk).
Diagnostic gap found (independent of root cause — draft PR attached)
While tracing getGitVisibleFiles(), I found it has an outer catch { return null; } that silently swallows any exception (git missing, a rev-parse/ls-files timeout, a buffer overrun under load, an unreadable repo, etc.) and falls through to scanDirectoryWalk() with zero log signal, even under CODEGRAPH_DEBUG. That means there is currently no way to tell, after the fact, which of the two independent scanning implementations actually ran for a given index/init/sync — which is exactly the ambiguity that makes a report like this one hard to triage (both implementations look correct on inspection, but I can't tell you which one actually executed on the reporting host).
I've opened a small, non-behavioral draft PR adding a logDebug() call there: #TBD (linked below). It doesn't claim to fix the reported regression — it's offered as defensive hardening so a future report at this scale can confirm/rule out the fallback path in one step.
Version / environment
- CodeGraph:
1.5.0 (confirmed via codegraph --version)
- Reproduced git evidence on: Linux, git 2.x, real project checkout
- Reporting project's own investigation (independent of this report):
docs/QA_DISCOVERY_LEDGER.md entry CODEGRAPH-1.5.0-GITIGNORE-2026-08-18 and docs/codegraph/Status.md (2026-08-18 entry) in the boba project — I'm filing this on their behalf as an upstream contribution after independently re-verifying their git check-ignore -v evidence.
What would help
- Any pointer to whether
getGitVisibleFiles() or scanDirectoryWalk() is actually the path taken on a repo of this shape (the attached diagnostic PR would surface this going forward).
- If you have (or can share) a way to safely reproduce against a real multi-ten-thousand-file
node_modules tree, I'm happy to help narrow it down further — my synthetic attempts top out at 960 files / 63 nested .gitignore files without triggering it.
Summary
On
codegraph init . --force(v1.5.0), a re-index of an existing, previously-clean project walked into two subdirectories whosenode_modulestrees are correctly excluded forgititself by nested (non-root).gitignorefiles — producing a ~63x file-count blowup (509 files -> 32,260 files, 8,906 nodes -> 514,456 nodes, 724,013 edges, 1.8 GB DB) before the run was manually aborted (host-resource discipline; the run was still growing).This is a real-world regression report from an active project (
boba, a torrent-search proxy stack), not a synthetic construction. I (an independent contributor, not affiliated with the project that found it) verified the root-cause evidence first-hand and spent significant effort trying to reproduce it in isolation — full details below, including a negative result that may itself help triage.Root cause evidence (git side — confirmed first-hand)
Both directories are correctly and unambiguously excluded for
gitby their own nested.gitignore(not the project's root.gitignore, which does not mentionextension/node_modulesat all — onlyfrontend/node_modules/is separately listed there as a belt-and-suspenders entry). Yetcodegraph init . --forceon v1.5.0 walked into both anyway (365 MB + 236 MB, tens of thousands of files).The project's own
codegraphversion had moved from0.9.9(last validated 2026-06-06, 509 files / 8,906 nodes,7 PASS / 0 FAIL) to1.5.0with no re-verification in between —frontend/andextension/(each with their ownnode_modulesand nested.gitignore) were both added to the project after the 0.9.9 baseline, so this exclusion path had never been exercised against the newer CodeGraph version before this incident.What I could NOT reproduce (important negative result)
I read
src/extraction/index.tsonmain(current HEAD, still1.5.0perpackage.json) and traced both scanning implementations:getGitVisibleFiles()— the primary path, delegates to realgit ls-files -z -s --recurse-submodules+git ls-files -z -o --exclude-standard, which should inherit git's own correct nested-.gitignorehandling.scanDirectoryWalk()— the fallback for non-git projects, which explicitly layers aScopedIgnorematcher per directory level (comment: "Patterns in a nested .gitignore are relative to that directory... mirroring how git applies .gitignore files at every level") — this also looks correct on inspection.node_modulesis additionally a hardcodedDEFAULT_IGNORE_DIRSentry (src/extraction/index.ts~line 165), applied as a bare gitignore-style pattern (node_modules/) that should match at any depth regardless of which path above is taken.Given that, I built two synthetic fixtures and ran the actual installed v1.5.0 binary against them:
.gitignoreexcludingnode_modules/(400 files undernode_modules, no root-level mention).codegraph init --forcecorrectly indexed only the 2 real source files — did not walk intonode_modules.package.json+ nested.gitignoreexcludingnode_modules/), 150 fake packages per subproject (some shipping their own.gitignore, matching how real npm packages sometimes do),.binsymlinks (also common in real npm installs) — 63 nested.gitignorefiles total, 960 files correctly excluded by git.codegraph init --forceagain correctly indexed only the 3 real source files.So the bug is real (confirmed first-hand against the real project) but I could not trigger it in either synthetic fixture at hundreds-of-files scale with the same binary version. This suggests the trigger is scale-dependent and/or specific to something about a genuine
npm install-produced tree (24,102 real files underfrontend/node_modules, 13,449 underextension/node_modulesin the reporting project — deeply nested transitive deps,.map/binary files, very long paths, etc.) that I judged unsafe/disproportionate to reproduce here given the same host-resource concern that caused the original report to abort its own run (this is exactly the failure mode being reported, so reproducing it at full scale would risk the same runaway walk).Diagnostic gap found (independent of root cause — draft PR attached)
While tracing
getGitVisibleFiles(), I found it has an outercatch { return null; }that silently swallows any exception (git missing, arev-parse/ls-filestimeout, a buffer overrun under load, an unreadable repo, etc.) and falls through toscanDirectoryWalk()with zero log signal, even underCODEGRAPH_DEBUG. That means there is currently no way to tell, after the fact, which of the two independent scanning implementations actually ran for a givenindex/init/sync— which is exactly the ambiguity that makes a report like this one hard to triage (both implementations look correct on inspection, but I can't tell you which one actually executed on the reporting host).I've opened a small, non-behavioral draft PR adding a
logDebug()call there: #TBD (linked below). It doesn't claim to fix the reported regression — it's offered as defensive hardening so a future report at this scale can confirm/rule out the fallback path in one step.Version / environment
1.5.0(confirmed viacodegraph --version)docs/QA_DISCOVERY_LEDGER.mdentryCODEGRAPH-1.5.0-GITIGNORE-2026-08-18anddocs/codegraph/Status.md(2026-08-18 entry) in thebobaproject — I'm filing this on their behalf as an upstream contribution after independently re-verifying theirgit check-ignore -vevidence.What would help
getGitVisibleFiles()orscanDirectoryWalk()is actually the path taken on a repo of this shape (the attached diagnostic PR would surface this going forward).node_modulestree, I'm happy to help narrow it down further — my synthetic attempts top out at 960 files / 63 nested.gitignorefiles without triggering it.