dbtool-gui: managed profile backups (backup all / restore with target picker)

[Yaraslaut]

Summary

Adds a managed backup workflow to dbtool-gui: configure a backup folder once (Settings), press Backup all to back up every profile's database into it — one safely-overwritten <profile>.zip each — and restore any archive into its own profile, a different profile, or a custom connection string. Backup/restore is promoted out of the experimental --enable-backup-restore flag.

Design doc: docs/superpowers/specs/2026-07-11-dbtool-gui-managed-backups-design.md (visual mockups maintained in the Claude Design project dbtool-gui).

What's new

• ManagedBackupCore — pure, Qt-free decision logic (archive naming/sanitization + collision detection, folder scan, atomic tmp→rename overwrite, ProfileStore+SecretResolver connection-string resolution).
• ManagedBackupController + BackupStatusListModel — sequential per-profile orchestration on a worker thread; per-row status (queued/running/ok/failed + archive size/date); guards against concurrent migration/backup runs; secrets resolved off the GUI thread.
• BackupsPage.qml — replaces the bare BackupRestoreDialog; profile list with status pills, per-row Backup/Restore, destructive-restore confirmation with a Restore to target picker (cross-target warning), and a Custom-archive section for ad-hoc paths.
• Settings: new "Backups folder" card (QSettings backup/folder).
• Library fixes (benefit dbtool CLI too): SqlBackup/Restore helpers no longer connect via the unset global default connection string; metadata.json now redacts PWD=/Password= values instead of embedding resolved plaintext credentials.
• Gate removal: --enable-backup-restore, AppController::backupRestoreEnabled, BackupRunner::setEnabled deleted.

Performance

Backup/restore run with jobs = min(8, hardware_concurrency) and zstd (fallback deflate). Benchmarked on an 87 MB / 800k-row database: restore ~2.4–3 s (MSSQL 2022 jobs=8 / SQLite) vs ~6 s at dbtool's CLI default --jobs 1 on MSSQL.

Risk assessment

• Destructive-op safety: restores confirm in a dialog naming archive, timestamp, and target; archive-name collisions between profiles are refused (never silently merged) on backup-all, per-profile backup, and restore.
• Threading: single worker thread; all model/phase mutations marshalled to the GUI thread via queued invokes; a latent use-after-scope in the progress bridge (pre-existing in BackupRunner) was fixed in both copies.
• Library change blast radius: the SqlConnection{std::nullopt} fix is behavior-preserving for all existing callers (every site immediately calls Connect(...)); metadata redaction is write-only data no restore path consumes.
• Windows: narrow-string fs::path conversions (pre-existing pattern) may mishandle non-ASCII folder names — follow-up candidate.

Code coverage (clang-coverage, dbtool-gui label)

ManagedBackupCore.cpp 100% · BackupStatusListModel.cpp 98.2% · ManagedBackupController.cpp 95.8% (residual: gcov brace artifacts + library-boundary error paths, documented).

Databases tested

• sqlite3: full LightweightTest (1242 passed) + all dbtool-gui suites (204 assertions / 56 cases, qmltest, smoke).
• mssql2022 (Docker): LightweightTest [SqlBackup] 2558 assertions / 170 cases + end-to-end dbtool backup verifying PWD=*** in archive metadata.
• postgres: skipped — no psqlodbc driver installable in this environment (documented; the touched library code is DBMS-agnostic connection plumbing).

---

Update — live per-table backup "Details" panel

Adds a live master-detail view to the Backups page so a running backup/restore shows real-time per-table progress, and fixes a Windows launch blocker.

Design doc: docs/superpowers/specs/2026-07-11-backup-live-details-design.md · plan: docs/superpowers/plans/2026-07-11-backup-live-details.md.

What's new

• BackupTableListModel — GUI-thread-confined per-table model (one per profile), keyed by table name: rows create-on-first-sight and update-in-place, tolerating the up-to-8 concurrent table workers reporting in any order. Cleared at each run start; rows persist after a run so a finished profile still shows its last per-table results.
• Structured progress plumbing — ManagedBackupController gains a queued tableProgress(profile, table, current, total, state, message) signal; EmittingProgressManager now carries the profile name and emits it alongside the existing logLine. A GUI-thread slot routes each update into the right profile's table model. Restore progress routes under the archive profile too.
• BackupDetailPanel.qml — the live pane: per-table progress bar (indeterminate until the row total is known), a state badge (queued/running/done/error/warning), and the latest message, plus an "N workers" caption surfacing the existing 8-thread table parallelism (see the later follow-up correcting the MSSQL worker claim).
• BackupsPage.qml master-detail split — the Profiles card becomes a profile list + detail pane (side-by-side when wide, stacked when narrow). Selection auto-follows the running profile and a click pins a specific one.
• Windows launch fix — dbtool-gui.exe (and the test exes) failed to start with an SxS "requestedExecutionLevel level missing" error: lld-link's /MANIFEST:EMBED merge rewrote the manifest into the wrong XML namespace. Now a hand-written manifest is embedded verbatim via an .rc RT_MANIFEST resource. Also fixed a ::getpid portability break in the GUI test target on clang-cl.

Parallelism

No engine change — backups already run tables with jobs = min(8, hardware_concurrency). This change makes that concurrency visible in the UI.

Risk assessment

• Threading: all table-model mutation is GUI-thread-confined; the worker only posts queued invokeMethod calls (args copied into the event, no this/manager capture). A whole-branch review traced the worker->GUI hand-off, clear-before-progress FIFO ordering, and mid-run profile-reload lifetime — no data race or use-after-free.
• Lifetime: per-profile table models are parented to the status model and explicitly deleteLater()'d on profile reload (a leak found in review and fixed with a QPointer regression test).
• Windows-only manifest/.rc change (no-op elsewhere); DBMS-agnostic (Progress contract is per-DBMS-neutral).

Tests

• C++ dbtool-gui-tests: 240 assertions / 66 cases pass (was 204/56) — new per-table-model tests, the reload-leak regression test, and a controller test that runs a real SQLite backup and asserts tableProgress fires and the profile's model is populated.
• QML dbtool-gui-qmltest: 18/18 (adds a BackupDetailPanel smoke test).
• Windows clangcl-release; the real GUI binary launches clean (no SxS error, no QML warnings).
• Deferred follow-up: a page-level QML test for the selection logic (pin toggle / detail-profile derivation).

Layout fix — details as a dedicated master-detail region

The first cut nested the live details inside the Profiles card's Column, where Layout.* sizing is ignored, so the list and the details overlapped. Reworked into a page-level GridLayout: a fixed-width left rail (folder + profile master-list + custom archive) and a flexible right region that gives the details their own column (stacks on a narrow window). Also fixed a latent bug where the panel keyed visibility on tablesModel.rowCount — an invokable method, so the table list would never have appeared during a run — now keyed on the ListView's reactive count. Verified in the running GUI (no overlap; correct empty state).

🤖 Generated with Claude Code

Follow-up — independent scrolling + summary/active-only details

Two more fixes from real-fleet feedback (~300 profiles, hundreds of tables):

• Independent scroll regions. The page was one big ScrollView, so scrolling the 300-profile master-list scrolled the details away — you had to scroll back to the top to watch progress. It's now a fixed header + a fill-height master-detail body, where the profile list and the detail panel's active-table list each own their own scrollbar and scroll independently. (The Profiles master-list moved out of a content-sized Card into a fill-height bordered panel so its inner ListView can take the slack.)
• Summary + active-only detail. The detail panel no longer renders a wall of finished tables. It shows a compact summary — "N / M tables done", an aggregate progress bar, and per-state chips (running / queued / error / warning) — then a live list of only the in-flight tables (running/error/warning). Done and queued rows are filtered out via a new BackupActiveTablesModel (QSortFilterProxyModel) that re-filters automatically as tables transition state — done in C++, not via fragile zero-height QML delegates. New StatChip.qml renders the summary chips; BackupTableListModel gains NOTIFY-backed per-state count properties feeding the summary.
• Tests: summary tallies + summaryChanged firing (BackupTableListModelTests), and active-only filtering incl. live re-filter on state transition, roleNames pass-through, and source reset (BackupActiveTablesModelTests). Suite now 283 assertions / 76 cases; qmltest 18/18. Running-state layout verified by screenshot (summary + 6 active rows, 140 done rows hidden; both regions scroll independently).

Follow-up — profile search + corrected MSSQL worker claim

Two more fixes from a run with 325 profiles:

• Profile search. The master-list had no way to reach a profile short of scrolling hundreds of rows. Added a search field above the list that filters by name (case-insensitive substring), with a clear (✕) affordance and a "no profiles match" placeholder. Filtering is done in C++ by a new BackupProfileFilterModel (QSortFilterProxyModel over the status model), mirroring BackupActiveTablesModel: the ListView binds to the proxy while selection / auto-follow still resolve against the unfiltered status model, so filtering never changes which profile's details are shown. Search text is whitespace-trimmed (a stray space won't hide every row) and the list caption switches to "X of Y match" while filtering.
• Corrected a false worker-count tooltip. It claimed "MS SQL Server is limited to 1 worker by the driver" — which is wrong: SqlBackup::Backup spawns concurrency worker threads for every DBMS (iota over 0..concurrency) with no MSSQL special-casing; concurrency is only floored at max(1U, …), never capped to 1 for SQL Server, and the GUI passes BackupConcurrency() (min(hw, 8)) for all backends alike. Tooltip now reads "Tables are backed up in parallel by up to N worker thread(s)." Confirmed against a live MSSQL run — all workers active.
• Tests: BackupProfileFilterModelTests — empty source, empty-text pass-through, case-insensitive match, no-match, clear, whitespace trim, filterTextChanged fires only on a real change, and re-filter on source reset. Suite now 301 assertions / 84 cases; qmltest 18/18. Search + parallel MSSQL execution both manually confirmed by the maintainer.

---

Follow-up — review fixes: two data-loss defects + four smaller ones

A review of this branch turned up two ways the feature could destroy data while
reporting success, plus four smaller defects. All six are fixed here (7 commits,
c2219e4d..669de920).

🔴 BLOCKER — a backup that lost tables was committed over the last good archive and reported "ok"

SqlBackup::Backup() does not throw when individual tables fail:
ChunkWorker catches the per-chunk exception, marks the chunk failed, reports
Progress::State::Error, and returns normally. The established way to notice is
ProgressManager::ErrorCount() — exactly what the CLI gates its exit code on
(return pm->ErrorCount() > 0 ? EXIT_FAILURE : EXIT_SUCCESS, dbtool/main.cpp).

EmittingProgressManager derived from ProgressManager, whose ErrorCount()
is a hard-coded 0, and nothing called it. So a run that silently dropped
tables still reached CommitArchive(tmp, final) — the atomic rename replaced
the last good archive with the partial one — and the profile was marked "ok".
Since there is exactly one archive per profile, the only copy of the data the
user had was destroyed by a backup that reported success.

Fix: EmittingProgressManager now derives from
ErrorTrackingProgressManager and chains its Update(). A non-zero
ErrorCount() aborts before the commit: the temporary archive is deleted, the
previous archive is left untouched, the profile is marked failed with the
reason, an error line is logged, and the run summary counts it as a failure.

🔴 BLOCKER — restore reported success after dropping tables it then failed to recreate

Same shape on the restore side: CreateTablesInOrder drops each archived table
and, when the CREATE TABLE fails, reports State::Error and omits the table
from createdTables without throwing. RunRestore caught only
std::exception, so ok stayed true and it emitted finished(true, …) — a
green success banner over a database that had just lost the tables it failed to
recreate.

Fix: same ErrorCount() mechanism. A failed restore additionally states
that a restore is not transactional — tables are dropped and recreated one
at a time and data is loaded by concurrent workers — so the target may be left
incomplete and must not be used before a successful re-run. The thrown-exception
path carries the same warning.

Because no live SQLite database will report table errors without throwing on
demand, the two SqlBackup calls became injectable operations
(setBackupOperation / setRestoreOperation, defaulting to the real API, an
empty callable restoring the default). The operation is copied into the worker
task on the GUI thread, so the worker never races a replacement.

🟠 HIGH — the busy guard was one-way

AppController installed a busy probe on _managedBackups only.
MigrationRunner::applyUpTo and BackupRunner::runBackup/runRestore checked
nothing but their own _phase, and no QML binding disabled them on
managedBackups.phase (SimpleView's "Run migrations" had no enabled
condition at all). A migration — or an ad-hoc restore — could therefore run
during a managed backup, producing a torn archive that was then committed and
marked "ok".

Fix: mutual, in both layers. MigrationRunner and BackupRunner each gained
a setBusyProbe(), consulted by a shared entry guard that logs a warning on
refusal; AppController wires all three probes to each other; and the QML
enabled bindings on the Backups page, ActionsPanel and SimpleView mirror the
same condition. Dry-runs are read-only and stay ungated, matching the C++ side.

🟠 HIGH — the destructive ad-hoc restore had no confirmation

The custom-archive "Restore…" button called BackupRunner.runRestore
directly: one click, no confirmation, on an action that drops and recreates every
table of the connected database. The ellipsis promised a dialog, the adjacent
per-profile restore has one, and this surface used to sit behind
--enable-backup-restore — removed in this PR, so it is now on by default,
~6px from "Backup".

Fix: routed through a confirmation dialog showing the archive path and the
target it will overwrite. The red warning banner moved into a shared
DestructiveWarningBanner.qml used by both restore dialogs so their wording and
prominence cannot drift apart.

🟡 MEDIUM — inverted destruction-order comment (and a violated invariant)

ManagedBackupController.hpp claimed _pool "MUST be declared first so it is
destroyed last". Non-static members are destroyed in reverse declaration
order, so declaring _pool first destroyed it last — _status, _profiles
and _busyProbe were gone before ~QThreadPool joined the worker, which is
precisely the use-after-free the comment said it prevented. Benign today only
because the worker publishes everything through queued invokeMethod. _pool
now really is declared last, and the comment describes what the language does.

🟡 MEDIUM — RedactConnectionStringSecrets leaked brace-quoted passwords

The scan looked for the next ; and ignored ODBC {…} quoting, so
PWD={pa;ss} was written into an archive's metadata.json as PWD=***;ss} —
half the password in cleartext, inside a file that gets copied around. The
connectionString[pos-1] != ';' boundary check also let …; PWD=secret (a
separator followed by whitespace, which driver managers accept) escape redaction
entirely.

Fix: the string is parsed attribute-wise. A value starting with { runs to
its matching } (}} = escaped brace) and is masked whole, so an embedded ;
neither ends the value nor starts a new attribute; whitespace is tolerated around
the key; matching stays case-insensitive and whole-attribute only, so MyPWD=
and Database=PasswordVault are still untouched. MetadataRedactionTests
covered neither case before and now covers both, plus malformed segments.

🟡 MEDIUM — Windows non-ASCII backup folder was mangled

std::filesystem::path(qstring.toStdString()) fed UTF-8 into MSVC's narrow
fs::path constructor, which decodes in the active code page — so
C:\Users\Müller\Sicherungen became mojibake and archives landed somewhere the
user never picked. path::string() has the same defect in reverse, including in
the error messages that name the folder.

Fix: ToPath()/FromPath() convert via UTF-16 (toStdU16String() /
fromStdU16String()), and ManagedBackup::PathToUtf8() replaces four
.string() calls in the core layer's messages.

---

Known limitations (confirmed in review, deliberately not addressed here)

• No archive retention / generations. Exactly one archive exists per profile
  and a successful run replaces it. The failure gate above means a bad run can
  no longer destroy the only copy, but there is still no history to roll back to.
• No cancellation for a running backup-all, and ~QThreadPool blocks process
  exit until the current run finishes.
• Live-details UI inaccuracies: a restore does not call clearTables() or
  setRunState("running"); a cross-target restore animates the source
  profile's row; the details header's worker count is hard-coded to 8 in QML
  while C++ uses min(hardware_concurrency, 8); restoreDialog.openFor
  defaults to currentIndex = 0 when the archive's own profile is not in the
  target list.

These are recorded in src/tools/dbtool-gui/README.md under "Known limitations".

Tests added

Test	What it pins
a backup whose tables failed is reported as failed and never committed	no archive, no .tmp, run reported as failure, profile failed
a failed backup leaves the previous good archive in place	the last good archive is byte-identical after a partial run
a restore whose tables failed is reported as failed and warns about the target	finished(false, …) + "INCOMPLETE" in the summary and the log
a clean injected run still commits and reports success	the gate does not reject warning-only runs; empty callable restores the real API
RunnerBusyGuardTests (4 cases)	both runners refuse + log while busy; an unwired probe changes nothing
the busy guard between the three runners is mutual	a live managed run blocks ad-hoc backup, ad-hoc restore and migration
tst_backups_page (2 new)	the custom "Restore…" click opens the dialog and leaves the runner idle; shared banner wording
PathToUtf8 + CheckWritableFolder non-ASCII (2 cases)	"Müller" survives the path round-trip and the error message
MetadataRedactionTests (4 new cases)	brace-quoted passwords, ; inside another quoted value, whitespace after the separator, malformed segments

The three failure-semantics tests were verified to fail against the
pre-fix code (the partial archive overwrote the good one).

dbtool-gui-tests: 370 assertions / 98 cases (was 301/84). dbtool-gui-qmltest: 20/20.

Verification

Build	sqlite3	mssql2022	postgres
clang-debug (ASan + UBSan + clang-tidy, -Werror)	1248 ✅ / 1 skip	1246 ✅ / 3 skip	1247 ✅ / 2 skip
gcc-release	1248 ✅ / 1 skip	1246 ✅ / 3 skip	1247 ✅ / 2 skip
test_dbtool.py (clang-debug)	✅	✅	✅

dbtool-gui targets built and ctest -L dbtool-gui green (3/3: Catch2,
qmltestrunner, process smoke). clang-tidy clean — no NOLINT added.
clang-format applied to every changed file.

Risk: the failure gate makes previously-"successful" partial backups report
as failures — that is the intended behaviour change and the only user-visible
regression risk. The mutual busy guard can disable a button that used to be
clickable while another run is in flight; the "back up first, then migrate" flow
in SimpleView is unaffected because the backup runner is already Idle when its
finished signal fires. No library ABI change beyond
RedactConnectionStringSecrets's (unchanged) signature; no per-DBMS behaviour
change. Performance impact: none — the added work is one integer comparison per
profile per run.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!