Skip to content

Add tier1 SQLite DB reuse#105

Merged
SonAIengine merged 1 commit into
mainfrom
codex/tier1-sqlite-reuse
Jul 2, 2026
Merged

Add tier1 SQLite DB reuse#105
SonAIengine merged 1 commit into
mainfrom
codex/tier1-sqlite-reuse

Conversation

@SonAIengine

Copy link
Copy Markdown
Contributor

Summary

  • add persistent SQLite DB options to the tier1 benchmark runner
  • write and validate .tier1.json sidecar metadata before reusing a benchmark DB
  • add overwrite cleanup for SQLite/WAL/HNSW/metadata artifacts
  • document persistent MS MARCO 1M repeat-run commands

Why

The MS MARCO 1M run proved the path works, but build/materialization took 1913.3s (31.9m). Reusing a validated persistent DB lets follow-up quality/search runs skip that cost.

Tests

  • uv run --extra dev ruff check examples/ablation/run_tier1_benchmarks.py tests/test_tier1_benchmarks.py
  • uv run --extra dev ruff format --check examples/ablation/run_tier1_benchmarks.py tests/test_tier1_benchmarks.py
  • uv run --extra dev pytest tests/test_tier1_benchmarks.py -q
  • PYTHONUNBUFFERED=1 uv run --extra sqlite python examples/ablation/run_tier1_benchmarks.py --only msmarco --subset 1 --corpus-limit 2 --use-sqlite-graph --sqlite-db-path /tmp/tier1_reuse_smoke.db --overwrite-sqlite-db --ingest-batch 1 --progress-every 1
  • PYTHONUNBUFFERED=1 uv run --extra sqlite python examples/ablation/run_tier1_benchmarks.py --only msmarco --subset 1 --corpus-limit 2 --use-sqlite-graph --sqlite-db-path /tmp/tier1_reuse_smoke.db --reuse-sqlite-db --ingest-batch 1 --progress-every 1

@SonAIengine SonAIengine merged commit d8fdf03 into main Jul 2, 2026
2 checks passed
@SonAIengine SonAIengine deleted the codex/tier1-sqlite-reuse branch July 2, 2026 03:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant