Skip to content

Large multi-module repo: extraction+resolution still takes 65 minutes in v1.2.0 #1180

Description

@bayernjava

Summary

Running CodeGraph v1.2.0 (with #965 O(K²) import-resolution fix and #999 "Resolving refs" wedge fix) on an 8,256-file monorepo with 2 git submodules still takes ~65 minutes for the extraction+resolution phase on a full index run.

Repo Structure

<repo-root>/
├── backend/     # submodule: Gradle project
│                # Spring Boot 2.5.12 + Kotlin 1.9.10
│                # ~3,881 Java files + ~1,378 Kotlin files
│                # multiple domain modules
├── frontend/    # submodule: package.json
│                # React 17 + Redux + TypeScript
│                # single-spa micro-frontend
├── docs/
└── references/
  • Total source files (CG's perspective): 8,256
  • No circular cross-submodule references

Observed Behavior

18:25:58 [INFO] codegraph_index_started
19:30:45 [INFO] codegraph_index_completed     # extraction + resolution: ~65 min
19:39:42 [INFO] ingestion_done                # end-to-end: 74 min
Metric Value
Total source files 8,256
Extracted nodes 47,344
Extracted edges 16,940
SQLite DB size ~385 MB
Extraction+resolution 65 min
End-to-end 74 min

Invocation

codegraph index <repo-root>    # full reindex (not sync)

No sub-module parallelism, no --verbose (clean baseline to compare against v1.1.x).

Questions for the maintainer

  1. Is 65 minutes expected for an 8K-file repo after perf(resolution): fix O(K²) import-node blowup in "Resolving refs" (#915) #965 / codegraph index wedges indefinitely at "Resolving refs" 94% on a repo with a large *tracked* JS/TS asset dir and there's no way to exclude it #999?
  2. If still a known issue, is there a tracking issue / ETA?
  3. Otherwise, what diagnostics do you need (verbose log? DB stats? profile trace?)
  4. Are there repo patterns that bypass the perf fixes — e.g. mixed Java+Kotlin sources, cross-submodule imports, or certain Gradle/Spring conventions?

Why this matters

We run a full index once per repo (cold start), then incremental sync runs (seconds on unchanged code). 65 minutes is the blocking cost for first-time ingestion of large repos. A meaningful speedup here would unlock multi-project workspace use cases that currently take half an hour just to ingest.

What I haven't tried

  • Running submodule indexing in parallel (backend/ and frontend/)
  • --verbose to see phase-level progress
  • --jobs parallelism flag (if CG exposes one for resolution)
  • Excluding test/build artifacts from extraction

Happy to follow whichever diagnostic path the maintainer recommends.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions