Skip to content

fix: use correct storage path in run_benchmark.py#134

Closed
Aneesh-382005 wants to merge 2 commits into
elara-labs:mainfrom
Aneesh-382005:fix/benchmark-storage-path-mismatch
Closed

fix: use correct storage path in run_benchmark.py#134
Aneesh-382005 wants to merge 2 commits into
elara-labs:mainfrom
Aneesh-382005:fix/benchmark-storage-path-mismatch

Conversation

@Aneesh-382005

Copy link
Copy Markdown
Contributor

Summary

benchmarks/run_benchmark.py was missed in #106. It still uses Path(config.storage_path) / project_dir.name instead of project_storage_dir(), so it looks in a different directory than where the index is written (<name>-<hash>). Because of that, every benchmark query returns 0 results.

Fix

Use the same project_storage_dir() helper the indexer already uses.

Test plan

  • Ran python benchmarks/run_benchmark.py --repo https://github.com/expressjs/express.git --source-dir lib before the fix: all 20 queries MISS, 0% everywhere
  • Same command after the fix: 20/20 HIT, 93.6% retrieval savings, 97.1% combined
  • Note for maintainers: benchmarks/results/fastapi.md, fiber.md, chi.md were generated before fix: hashed storage paths to prevent basename collisions #106 landed and are stale relative to current main, though still historically valid. Happy to rerun them if useful.

@rajkumarsakthivel

Copy link
Copy Markdown
Member

Closing as superseded by #136, which includes this fix along with the Django and Express benchmark results. The storage path fix is now on main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants