feat: persist download progress in SQLite with queryable API

[roziscoding]

Summary

Persists each peer file download's lifecycle in SQLite so active, completed, failed, and import-queued downloads are queryable without scanning logs or folders. Captures real byte progress through PeerConnector.downloadFile() progress events, splits the blackhole watcher from the download workflow, and exposes a read-only downloads API.

Implements the plan at ai_docs/plans/2026-06-04-download-progress-sqlite/.

What's included

Phase 1 — Queryable downloads store

• Drizzle ORM schema for a downloads table (bun:sqlite), with status/updated_at indexes and CHECK constraints
• Connection module opening /config/database.sqlite and applying committed migrations at startup via migrate() (folder resolved from import.meta.dir)
• DownloadsRepository (create-after-metadata, newest-first list, get, status transitions, stale-row reconciliation)
• GET /downloads and GET /downloads/:id behind the existing requireApiKey middleware
• getApp gains an optional services param (backward compatible)

Phase 2 — Progress capture + runtime wiring

• PeerConnector.downloadFile() emits headers/progress/completed events; a missing/invalid Content-Length stores expectedBytes: null, and a mismatch against releaseSize is flagged
• Completion check switched from contentLength > 0 to expectedBytes != null
• Split blackhole.ts into blackhole.watcher.ts (watch/scan/stability) and DownloadsService (download lifecycle + row persistence). Any post-create failure marks the row failed
• index.ts opens the DB, reconciles stale downloading rows on restart (using .part size when present), passes the repository into the app, and closes the DB on SIGINT/SIGTERM

Security fix (raised in review)

• Sanitize peer-controlled release.filename to a plain basename before building destPath/partPath, rejecting traversal/absolute paths so a malicious peer cannot escape completedPath. Unsafe names mark the row failed. (This hardens logic that pre-existed on main.)

Out of scope

No /api/v3/queue shim, no download resume (stale rows are failed, not resumed — issue #13), no concurrency semaphore, no frontend, no runtime schema editing.

Testing

• bun test apps packages — 84 pass, 0 fail
• bun run eslint . — clean
• bunx --bun tsc --noEmit — clean

e2e suite (mise run test:e2e) is a separate CI job requiring the docker-compose stack and was not run locally.

[roziscoding]

@greptileai review

[greptile-apps]

Greptile Summary

This PR introduces a full SQLite-backed download tracking layer using Drizzle ORM (bun:sqlite), splits the monolithic blackhole.ts into a watcher and a service, wires byte-level progress events through PeerConnector.downloadFile(), and exposes a read-only /downloads API behind the existing requireApiKey middleware. The security fix sanitising peer-controlled release.filename to a plain basename — rejecting traversals before any row is created — is correctly placed.

• Phase 1: Drizzle schema, migration, DownloadsRepository (create, list, get, status transitions, stale-row reconciliation on restart), and GET /downloads / GET /downloads/:id endpoints.
• Phase 2: PeerConnector.downloadFile() now emits headers/progress/completed events; the post-rename completed callback is isolated in its own try/catch (addressing the previous review finding); DownloadsService maps those events to repository transitions.
• Shutdown caveat: database.close() is called in SIGINT/SIGTERM before in-flight downloads finish, leaving stale downloading rows and .part files until the next-start reconciliation — this is an acknowledged design limitation (issue feat: harden peer download handling #13).

Confidence Score: 5/5

Safe to merge; the path-traversal fix is correctly placed, the post-rename progress-callback issue from the previous review has been addressed, and all 84 tests pass.

The download tracking lifecycle is correctly implemented end-to-end: rows are created after metadata resolves and the safety check passes, progress is persisted synchronously, and the completed/failed transitions are well-guarded. The shutdown behaviour leaves stale rows under forced termination, but that is reconciled on restart and explicitly documented in the PR.

apps/backend/src/index.ts — the SIGINT/SIGTERM shutdown sequence closes the database before stopping the HTTP server, which could leave .part files and stale DB rows when downloads are in flight at shutdown time.

Important Files Changed

Filename	Overview
apps/backend/src/modules/downloads/downloads.service.ts	New DownloadsService split from blackhole.ts; isSafeName guard correctly precedes row creation; status lifecycle (downloading → completed → import_queued) is well-handled; onProgress callbacks properly check the download variable guard.
apps/backend/src/lib/servers/peer.ts	Adds PeerDownloadProgressEvent emission; post-rename completed callback is wrapped in its own try/catch (previous review concern addressed); headers, progress, and completed events are correctly sequenced.
apps/backend/src/modules/downloads/downloads.repository.ts	Clean synchronous Drizzle repository; reconcileStaleDownloads correctly materialises stale rows before iterating; markFailed preserves completedAt timestamp set by markCompleted.
apps/backend/src/database/connection.ts	Opens WAL-mode SQLite with foreign keys enabled, runs Drizzle migrations at startup; MIGRATIONS_FOLDER resolved relative to import.meta.dir so it's correct under bun:test and direct invocation.
apps/backend/src/index.ts	Correctly wires DB, repository, reconcileStaleDownloads, and shutdown handlers; SIGINT/SIGTERM close the DB without draining in-progress downloads, leaving .part files on disk (reconciled on next start — accepted limitation).
apps/backend/src/database/schema.ts	Well-defined downloads table with status CHECK constraint, expected_bytes_source CHECK, and indexes on status/updated_at; boolean integer mode and nullable fields correctly modelled.
apps/backend/src/modules/downloads/blackhole.watcher.ts	Clean extraction of watch/scan/stability logic from old blackhole.ts; processing Set deduplication is correct; watcher registered before scan to avoid missed events during startup.
apps/backend/src/app.ts	getApp gains optional services param (backward compatible); downloads router only mounted when repository is present; all existing routes unaffected.
apps/backend/src/tests/downloads-service.test.ts	Good coverage: happy path, metadata failure, path traversal rejection (no row created), and download failure with row marked failed; all assertions are meaningful.

_{Reviews (3): Last reviewed commit: "fix: handle download tracking edge cases" | Re-trigger Greptile}