Skip to content

ADFA-5220: Write the database version, and refuse a format we cannot read - #29

Open
davidschachterADFA wants to merge 2 commits into
mainfrom
task/ADFA-5220-database-version
Open

ADFA-5220: Write the database version, and refuse a format we cannot read#29
davidschachterADFA wants to merge 2 commits into
mainfrom
task/ADFA-5220-database-version

Conversation

@davidschachterADFA

Copy link
Copy Markdown
Collaborator

The version table existed and the app already gated on it, but nothing created it. Grepping this repo for DocumentationDatabaseVersion returned nothing — not populate_db.py, not the migration or re-mint scripts, not docdb-studio, not the workflows. The one row in the shipped database was inserted by hand.

Why that was about to break the app

CoGo reads this table now (DatabaseVersionResolver, merged in appdevforall/CodeOnTheGo#1677). A missing table reads as unversioned, and the app then declines to attach the compression dictionary — so a freshly built database, with dictionary-compressed content and no version row, would have the app skip the dictionary and fail to decode every brotli row. A well-formed file that serves nothing.

It isn't broken today only because the currently shipped asset has no dictionary either, so the gate is correctly a no-op. It breaks the moment this pipeline's dictionary work produces an asset — which is what #26 just merged.

populate_db declares it

In the same transaction that mints the dictionary, because the dictionary is what the version means: a reader must never see one without the other.

A row is appended only when the declared version changes. The table logs what the format became, not who ran what — a row per invocation would bury the two or three entries that matter under hundreds of identical ones, and the app reads only the last row anyway.

A last row differing in either direction gets appended, including a lower version than the file already declares. That is deliberate: rebuilding from an older pipeline genuinely is a downgrade, and "the row inserted last wins" is the convention the app's reader implements, so recording it is what keeps the file honest about what it now contains.

docdb-studio refuses what it cannot read

The version is read before anything else touches the database, and a MAJOR above the one this build understands shows what it found and what it expected instead of the browser.

Refusal rather than a warning, because this tool writes: a format it would read incorrectly is a format it would save damage back into. Lower or absent versions open normally — those are strictly simpler (no shared dictionary) and the plain-Brotli fallback already reads them.

A duplication worth naming

Both sides read the row inserted last by rowid, and the rule is duplicated rather than shared, because this repo and the app cannot import from each other. Each implementation carries a comment naming the other, so the next person to change one knows there is a second.

Tests

Five on the declaration: creates the table, a second run adds nothing, an older declaration is superseded, a downgrade is recorded rather than hidden, and the declared MAJOR is at least what the app gates on — that last one fails loudly if anyone lowers the constant without noticing the app depends on it.

Four on docdb-studio: unversioned reads as none, the last row wins over a higher earlier one, supported and older versions are not refused, and the refusal message names both versions and says what to do rather than only what is wrong.

34 pipeline tests and 178 docdb-studio tests pass. SCHEMA.md gains the table; docdb-studio's README explains which versions it opens and why it declines the rest.

davidschachterADFA and others added 2 commits August 24, 2026 18:47
…read

The version table existed and the app already gated on it, but nothing created
it. Grepping this repo for DocumentationDatabaseVersion returned nothing: not
populate_db.py, not the migration or re-mint scripts, not docdb-studio, not the
workflows. The one row in the shipped database was inserted by hand.

That matters because CoGo now reads it. DatabaseVersionResolver treats a missing
table as "unversioned" and declines to attach the compression dictionary, so a
freshly built database -- dictionary-compressed content, no version row -- would
have the app skip the dictionary and fail every brotli row: a well-formed file
that serves nothing. It is not broken today only because the currently shipped
asset has no dictionary either; it breaks the moment this pipeline's dictionary
work produces one.

populate_db declares the version in the same transaction that mints the
dictionary, because the dictionary is what the version means. A row is appended
only when the declared version *changes*: the table logs what the format became,
not who ran what, and a row per invocation would bury the two or three entries
that matter. A last row differing in either direction is appended, including a
lower version than the file already declares -- rebuilding from an older
pipeline genuinely is a downgrade, and "the row inserted last wins" is the
convention the app's reader implements, so recording it is what keeps the file
honest.

docdb-studio now reads that version before anything else touches the database
and refuses a MAJOR above the one it understands, showing what it found and what
it expected instead of the browser. Refusal rather than a warning because this
tool writes: a format it would read incorrectly is a format it would save
damage back into. Lower or absent versions open normally -- those are strictly
simpler, and the plain-Brotli fallback already reads them.

The version-reading rule is duplicated deliberately rather than shared: this
repo and the app cannot import from each other, so both read the row inserted
last by rowid, and both say so in a comment naming the other.

Nine tests. Five on the declaration -- creates the table, a second run adds
nothing, an older declaration is superseded, a downgrade is recorded rather than
hidden, and the declared MAJOR is at least what the app gates on. Four on
docdb-studio -- unversioned reads as none, the last row wins over a higher
earlier one, supported and older versions are not refused, and the refusal
message names both versions and says what to do. 34 pipeline tests and 178
docdb-studio tests pass.

SCHEMA.md gains the table; docdb-studio's README explains which versions it will
open and why it declines the rest.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit dbae587)
The table records the format version the file *is*, not a history of what it has
been, so declare_database_version replaces its row instead of appending, and
collapses a database that somehow accumulated several back to one.

The replacement stays unconditional in either direction, including a version
lower than the file already declares: rebuilding from an older pipeline really
does produce an older format, and the row has to say what the file contains now.

Both readers keep their ORDER BY rowid DESC, now as a defence rather than a
model -- if a file ever breaks the contract, the version read stays
deterministic instead of depending on what SQLite returns first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 1eca425)
@hal-eisen-adfa
hal-eisen-adfa force-pushed the task/ADFA-5220-database-version branch from 1eca425 to a332e08 Compare August 25, 2026 01:47
@hal-eisen-adfa
hal-eisen-adfa changed the base branch from fix/ADFA-4737 to main August 25, 2026 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant