feat: add --precise-types flag for exact CDB type fidelity - #55
Open
mpicciolli wants to merge 2 commits into
Open
feat: add --precise-types flag for exact CDB type fidelity#55mpicciolli wants to merge 2 commits into
mpicciolli wants to merge 2 commits into
Conversation
By default, DB_STRUCTURE and narrow integer types (BOOLEAN,
INTEGER_BYTE, INTEGER_SHORT) now match what the official PCM
SQLiteExporter tool expects, so cdb-converter output stays
importable there. Pass --precise-types / { preciseTypes: true }
to keep the exact CDB type and each table's real Flags in the
.sqlite file for full round-trip fidelity instead.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the CDB→SQLite conversion defaults to match the official PCM SQLiteExporter schema for interoperability, and introduces an opt-in preciseTypes mode (--precise-types / { preciseTypes: true }) to preserve exact CDB type and table-flag fidelity for full round-trips.
Changes:
- Changed default
cdbToSqlmetadata/schema emission to beSQLiteExporter-compatible (noDB_STRUCTURE.Flags, narrow int types collapsed toINTEGERmetadata). - Added
--precise-typesCLI flag andCdbToSqlOptions.preciseTypesto enable exact type + flags fidelity. - Expanded/adjusted tests and documentation to cover both default and
preciseTypesbehavior.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test/roundtrip.test.ts | Splits round-trip assertions into preciseTypes (full fidelity) vs default (row-data fidelity). |
| test/cli.test.ts | Adds preciseTypes defaults and --precise-types parsing coverage. |
| test/cdbToSql.test.ts | Adds unit tests for default type-collapsing vs preciseTypes type preservation. |
| src/types.ts | Documents the new preciseTypes option and its compatibility implications. |
| src/cli.ts | Implements --precise-types flag parsing and wires it into conversion. |
| src/cdbToSql.ts | Implements the default-compatible schema changes and preciseTypes-dependent encoding/Flags behavior. |
| README.md | Documents --precise-types, default compatibility behavior, and DB_STRUCTURE differences. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| ``` | ||
|
|
||
| Each table's flags (their exact meaning is unknown but must be preserved) are stored in the `Flags` column, so they are written into the `.sqlite` file itself and survive an `export()`/reopen cycle. Column indices and data types are encoded into each column's declared type annotation. Together this makes `cdb → sqlite → cdb` lossless even when the SQLite database is saved to disk and reopened in a separate process. | ||
| Column indices and data types are encoded into each column's declared type annotation, which makes `cdb → sqlite → cdb` lossless even when the SQLite database is saved to disk and reopened in a separate process. By default, CDB's narrower integer types (`BOOLEAN`, `INTEGER_BYTE`, `INTEGER_SHORT`) are encoded as plain `INTEGER`, and each table's flags (their exact meaning is unknown but must be preserved) are **not** written to the `.sqlite` file — `sqlToCdb` falls back to a static table of flags extracted from official PCM saves (`TABLE_FLAGS_BY_ID`) instead. Pass `{ preciseTypes: true }` (`--precise-types` on the CLI) to encode the exact CDB type and store each table's real flags in the `Flags` column instead of relying on that fallback. |
- Clarify README round-trip wording: default mode preserves row data, preciseTypes preserves types and flags too. - Guard empty exec() result in roundtrip snapshot helper. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cdbToSqloutput now matches the official PCMSQLiteExportertool's schema (noFlagscolumn, narrow int types collapsed toINTEGER) so it stays importable there.--precise-typesCLI flag /{ preciseTypes: true }option opts back into the exact CDB type + real per-tableFlagsfor full round-trip fidelity.