Skip to content

Merge upstream SQLite 3.46.1#2238

Merged
penberg merged 481 commits into
mainfrom
update-sqlite-3.46.1
May 27, 2026
Merged

Merge upstream SQLite 3.46.1#2238
penberg merged 481 commits into
mainfrom
update-sqlite-3.46.1

Conversation

@penberg
Copy link
Copy Markdown
Collaborator

@penberg penberg commented May 21, 2026

Updates libSQL's vendored SQLite from 3.45.1 → 3.46.1 (upstream tag version-3.46.1, f3d536d).

Despite libSQL claiming to be 3.46.1 in the version strings, it was based on upstream version 3.45.1 (9ed72eb).

What's in here:

  • libsql-sqlite3/ — the vendored SQLite source tree merged up to 3.46.1, with libSQL's modifications preserved (vector indexes, extra pragmas, WASM tweaks, etc.).
  • libsql-ffi/bundled/ — regenerated amalgamation (sqlite3.c/sqlite3.h now report SQLITE_VERSION "3.46.1" with the …alt1 libSQL source-id marker) and session_bindgen.rs, via LIBSQL_DEV=1 cargo build --features multiple-ciphers,session. The SQLite3MultipleCiphers fileio.c/series.c were out-of-band synced to match 3.46.1.
  • Tooling — scripts/sqlite-diff (diffs only upstream's src/ artifacts between two trees) and scripts/update-sqlite-bundle.sh, so future bumps are reproducible.
  • ext/intck/ is new from upstream and was relocated under libsql-sqlite3/ where Makefile.in already expects it.

Conflicts resolved:

  • src/pragma.c (integrity_check) — upstream replaced the per-table count loop with a single pre-pass using b-tree counts from OP_IntegrityCk. Took upstream's refactored form and re-added the IsVectorIndex(pIdx) skip in the relocated loop, keeping cnt++ unconditional so aRoot register indexing stays aligned. The existing per-row vector-index skip was untouched.
  • src/where.c — kept libSQL's if (pProbe->idxIsVector) opMask = 0; alongside upstream's refined bLowQual && pSrc->fg.isIndexedBy==0 (cherry-pick 626d619); took upstream's WHERE_KEEP_ALL_JOINS (cherry-pick 2fdb323).
  • src/shell.c.in — libSQL's .init_wasm_func_table and upstream's new .intck command coexist.
  • ext/wasm/GNUmakefile — preserved libSQL's -DSQLITE_OMIT_SHARED_MEM and -lm next to upstream's new SEE filter hook (c-pp preprocessor only; no effect on WASM output).
  • tool/mksqlite3c.tcl — kept wasmedge_bindings.c in the amalgamation source list.
  • FTS3/FTS5 + mechanical files — taken from upstream; these were merge-base artifacts (base 3.45.0 vs HEAD already at 3.45.1). libSQL's local trigram-tokenizer crash fix (3e56d28) is now superseded by upstream's a751fdd.

How this was verified:

The libsql-sqlite3/src tree was diffed against both clean upstream bases using scripts/sqlite-diff and compared the two patches — they should differ only in context lines and hunk offsets (upstream code that changed between releases), never in libSQL's actual +/- content.

scripts/sqlite-diff <clean-3.45.1> libsql-sqlite3 > libsql-sqlite3-3.45.1.patch
scripts/sqlite-diff <clean-3.46.1> libsql-sqlite3 > libsql-sqlite3-3.46.1.patch

The diffs are the same except for a single divergence is the PRAGMA integrity_check vector-index guard, which is the same libSQL change in two shapes because upstream refactored that loop between releases:

  • vs 3.45.1: if( IsVectorIndex(pIdx) ) continue;
  • vs 3.46.1: folded into upstream's new check → if( pIdx->pPartIdxWhere==0 && !IsVectorIndex(pIdx) ){

Both skip vector indexes during integrity check — identical semantics. Ground truth confirms the tree is correctly ported (src/pragma.c:1804).

Test plan:

  • SQLite TCL test suite

drh and others added 30 commits March 13, 2024 11:29
FossilOrigin-Name: 3128b98d2656c7f23f6071c173f96329c7bdc49cd641050fbb7b105b810ba7c7
FossilOrigin-Name: 233155c174575fa99b34030c1936bc2b04489417f4d5223f18cdb3c8fc274a56
FossilOrigin-Name: f0c9eb0326f189fe5ebbee83532ac2c413e91c6a8e9d5b93aa6170cba66fdd22
FossilOrigin-Name: 5a7b5fc99a9d5c7693a5707c2050a8c59a8f97732e4763a56982579f8d5959ec
testrunner.tcl to run that many jobs.

FossilOrigin-Name: 924281b94d8e6ba674d6fe2f7f01da890351355a854e9e6fe623fad1180f7392
…s affinities other than NONE.

FossilOrigin-Name: 4229b12b327b05561dcf49b8585a66467d17d4e998b14eff65eb886f2434b53c
… the SF_MultiValue flag is correctly set.

FossilOrigin-Name: 94791824a6f04782aac6bc16c182685e2d6ebf5c688dc9f8d59e479c6fd5a40e
FossilOrigin-Name: 38635651f08d42110c01d6b24f6e362e340511846294f8696af4afc795dae80d
…ake it

easier to add configurations in branches.

FossilOrigin-Name: 9349d94e05dcf266b02afcd89f1e433cdc45f23e8536b2f3f9aa242d5d89c307
FossilOrigin-Name: 2085c7f12a3916ec883c31795e29f2e2b6641c30ecf748cce9bff7b13b061d1f
FossilOrigin-Name: 657c7b4f9df53e041a33cfad00ee2f507ac09c97a2ae61164bd189d660d167da
… cycles.

FossilOrigin-Name: 88d5bc91a52675d2a83748b79ad50de0e9732b2afd23107627bdc604f744d275
FossilOrigin-Name: 973d197d9676c066a95cea77a27e36d2d7f5778283919d8cac4424d2ec952f47
…s. No functional changes.

FossilOrigin-Name: f15709430d4ba2fcf5729bc30a45dacb77102927b82cda12de0b67935b526cc0
…s that insert more than one row in cases where the VALUES clause contains a value for all columns of the table.

FossilOrigin-Name: 988f0ea70cd21b3194011e0cd1ddd4990a07669e0fd9c37afa48f69698ab7212
… (...)" and similar statements.

FossilOrigin-Name: e8a2a8198a97046ff376bc5d38e4bc0a24fcac79f5a0dadb9d29d953a862a012
FossilOrigin-Name: d543c829ef74dbd64105bd757ca660e4f02e9ce562be4f1688a701fa535351c4
FossilOrigin-Name: 823e579362c05bb8accf6c3b158c5162a16eb23cf81d6021c9e3246e32583d1c
…ype.c

extension) so that it works on i586 compiled using gcc-13 with -O0.  Details
at [forum:/forumpost/0a7553b0f734c033|forum post 0a7553b0f734c033].

FossilOrigin-Name: d0fbe779bc2460e120da4f39063a9f4121c700f0b86d85f4311715b1366c3515
… ASAN

builds.

FossilOrigin-Name: ce009205a8edc02b7d45ac01bd0e692c3d2c3ffeadb68e4f1bad20c39075e692
FossilOrigin-Name: 82035b9cfd28ef6b0ecc6f469f03d2b001189aa4925147cdb784b6b1964eb3b2
constant functions.  So far the enhancement only applies to the multi-row
VALUES clause, but it could possibly be applied in many other places.

FossilOrigin-Name: c9e0488c6c0135932b6e76b0f3f3acd69ef65327e0a54daa59777f35da1aef26
… WRC_Abort, if the function really is constant.

FossilOrigin-Name: d85dd4de2d0989127e0ae6a2eec9d83a577777f8d5ba40700084a0b498016634
optimization.

FossilOrigin-Name: dae669245c86021bcd28716aff6e316257cc8075d02081745d4b7de17f8ad553
[constant propagation optimization](/info/f4229707ac08d66c).

FossilOrigin-Name: 92afea38881a1a4f266c79ddc6e1027cf17c288dcc8f7e7d50466899ac0054c4
…nstant().

FossilOrigin-Name: f71b28f6713126745bca53e098dfe3444d4235d6a2b87ced5b333a7f0ee79be8
…ant.

FossilOrigin-Name: d7eadcf7dd089f0f4aa963c58f0df32edd951368c9906375fd62b0a61cd393b8
ExprIsConstant enhancements.  So deal with that.

FossilOrigin-Name: 9ba3cff9d48fe8fc8a0a51291169599209d464465e44bdfd60058fd1e314a2b1
optimized VALUES clause.

FossilOrigin-Name: c0e5ceb87d3cf28f22d381f863b8f91043804fd7901f23f3afb02fe698e6a12a
FossilOrigin-Name: 0dce6211690210e802545cbca9b3375e07075e117fad3f8c921879da7671d981
drh and others added 19 commits May 27, 2024 11:38
FossilOrigin-Name: 55eee9f920e5dfdb88be5bb294707e743fa7ffe679fb0ff1e8f04b3a67ee271e
…o erroneously report a corrupt index.

FossilOrigin-Name: 0a7fa0a434bb84023bf9075463613857966ca579e18de903c0d9b1561acf42c6
…ms in

the ORDER BY clause.

FossilOrigin-Name: a7fe90c1581e7e4ffbcb6fb9bcf1db7631cde612ad1d4a33b6b9a22640db5817
…ReservedLock() implementation.

FossilOrigin-Name: a7cd07d96ef420bb1512e6d575bf756809561a536c627900d0eeac879f72e63b
…he case of a file disappearing while the OPFS VFS is waiting to acquire a lock on it.

FossilOrigin-Name: d34e788044eec3bf90d6d2db5a2ce20b0b8aea926f776d21c2cb58124773e544
…in the OPFS VFS which appears in rare instances in some browsers when running high I/O loads. This resolves [sqlite/sqlite-wasm#12 | issue #12 of the npm distribution].

FossilOrigin-Name: c7519d98ff09ed96c3c6f2cccf16f8efa19133e12cbb077fc86526f0e81f0470
…n feedback. No code changes.

FossilOrigin-Name: a61997c315ce70d60f3722a2b3b3d06ba592ce1cafed1639a9d5f162f712ae03
FossilOrigin-Name: 7a65ac42c2723b785786cf15f4b267ebfbd4f848f9fc6b37dcf9fac9abd0398c
…702c79e9da].

FossilOrigin-Name: 569824c7ff79ed1aadd2a6bce34bfcd8bf4f68db9565f56b803182adad526bc7
SQLITE_LIMIT_COMPOUND_SELECT, even if the VALUES clause contains elements
that appear to be variables due to the use of double-quoted string literals.

FossilOrigin-Name: 05dbfad70c3eb39f84b91503fc9b4c1cb96843e5e6599584077b1a630936ddc7
…s with the database handle before returning.

FossilOrigin-Name: 270831444812d77e2b3db9534753600f9aa5c88cbf29094feeba549e860a714b
…solve a compiler warning reported in the fossil forum. This is a build fix, not a functional change.

FossilOrigin-Name: be3de2e600037aa30a8c2cd7d6cfe8f2ba5a9c3e89ac99e87634e2c8c67c1846
Add early detection of over-prediction of covering-indexes so that
sqlite3_prepare() will return an error rather than just generate bad
bytecode.

FossilOrigin-Name: 4afe3201f005bfd1aa66bbeebac659323d8a35dbee04d50fbf77ba96a94db91e
FossilOrigin-Name: 6047b18ee3c4d8f2661386ef68c717f5892651c3d6f7e29a98a6b2795b3382d3
returns SQLITE_INTERNAL instead of SQLITE_ERROR after malformed input.

FossilOrigin-Name: 6decf00f7c56746fac31469ccc098a5be1344b2cafa8011bfcfbc7923e3cd97b
check-in f0b671183f44d0ae.

FossilOrigin-Name: d6fc6e5f0a68c5b34c0f93e14a38a7e1bc1b72a66f2bcc400c3d7e0f0bab5a92
… failures

are not overridden by a syntax error in the tokenizer spec.

FossilOrigin-Name: fc956353d3762d0e655b88f9d0c1a3840b40453a22e97160ccdf60485be56a92
FossilOrigin-Name: c9c2ab54ba1f5f46360f1b4f35d849cd3f080e6fc2b6c60e91b16c63f69a1e33
This merges the version-3.46.1 tag (f3d536d "Version 3.46.1") of
upstream SQLite into libSQL. Conflicts were resolved with rename-following
(libsql vendors SQLite under libsql-sqlite3/, while upstream paths are at
the root).

== Path moves ==

* ext/intck/ — new upstream tree. After the merge tool placed the 9 files
  at top-level ext/intck/, they were moved under libsql-sqlite3/ext/intck/
  where libsql-sqlite3/Makefile.in already expects them (lines 472-473,
  1269-1270).

== Resolved conflicts ==

src/pragma.c — integrity check
  Upstream restructured how "wrong # of entries in index" is verified: the
  per-table count loop at the end of integrity_check was replaced with a
  single pre-pass using b-tree counts from OP_IntegrityCk. Took upstream's
  refactored form and added an IsVectorIndex(pIdx) skip in the new
  relocated loop, while keeping cnt++ unconditional so register indexing
  into aRoot stays aligned. The existing per-row IsVectorIndex skip was
  unaffected (both sides agreed there).

src/where.c
  (1) Vector-index planner clash: kept libsql's
      `if (pProbe->idxIsVector) opMask = 0;`
      alongside upstream's refined
      `if (pProbe->bLowQual && pSrc->fg.isIndexedBy==0)` (cherry-pick
      626d619 — low-quality indexes still considered when selected via
      INDEXED BY).
  (2) Took upstream's `wctrlFlags |= WHERE_KEEP_ALL_JOINS;` (cherry-pick
      2fdb323 — disables omit-noop-join when ORDER BY has 64+ terms).
      The flag definition and consumer were already merged in
      sqliteInt.h:3437 and where.c:6587.

src/shell.c.in
  Pure adjacency. Kept libsql's `.init_wasm_func_table` command (under
  LIBSQL_ENABLE_WASM_RUNTIME) alongside upstream's new `.intck` command,
  in help text and in the command dispatch.

ext/wasm/GNUmakefile
  Three independent edits coexisting:
  * libsql's `-DSQLITE_OMIT_SHARED_MEM` (replacing upstream's OMIT_WAL,
    from dfa4ce4 "ext/wasm: allow WAL mode")
  * libsql's `-lm` (from 9280b8e "link math lib" — needed for vector
    search)
  * upstream's new `C-PP.FILTER.global` / `SQLITE_C_IS_SEE` SEE filter
    hook
  These flags only affect the c-pp.c preprocessor binary, not the WASM
  output (see the comment block above the rule), but preserving libsql's
  intentional swap keeps the build self-consistent with sqliteInt.h:55-56
  where SQLITE_OMIT_SHARED_MEM is defined.

tool/mksqlite3c.tcl
  Kept libsql's `wasmedge_bindings.c` in the amalgamation source list.

ext/fts3/fts3.c, ext/fts3/fts3_write.c, ext/fts5/fts5_main.c,
ext/fts5/fts5_tokenize.c, ext/fts5/test/fts5integrity.test
  All taken from upstream verbatim. These conflicts were an artifact of
  the merge base being 3.45.0 while libsql HEAD already contained 3.45.1
  content (from prior merges 9ed72eb / f996bf9). For four of the
  five files libsql had zero local changes. The fifth — fts5_tokenize.c —
  had a libsql-local trigram-tokenizer crash fix (3e56d28, Aug 2024)
  that is now superseded by upstream's a751fdd (cherry-picked into
  3.46.1), which handles OOM-vs-syntax-error ordering more carefully.

autoconf/tea/configure.ac, configure, doc/testrunner.md,
test/mmapcorrupt.test, test/permutations.test
  Mechanical — took upstream (version string bumps, doc updates,
  ifcapable !mmap guard, test path globs for ext/rbu/ and ext/intck/).

== libsql-ffi bundle sync ==

* libsql-ffi/bundled/SQLite3MultipleCiphers/src/{fileio.c,series.c}
  were updated to match upstream 3.46.1 byte-for-byte. The SQLite3
  MultipleCiphers tree is independently vendored from sqlite3mc 1.8.1
  and these two files are out-of-band synced to keep the cipher bundle
  aligned with libsql's vendored SQLite. Note: the bundle's own
  sqlite3.h still reports 3.45.1 — only sqlite3.c is overlaid at build
  time from libsql-sqlite3 (build.rs:466-471). sqlar.c has a small
  3.45.1→3.46.1 type-safety drift that was not mirrored.
* libsql-ffi/bundled/src/{sqlite3.c,sqlite3.h} and
  libsql-ffi/bundled/bindings/session_bindgen.rs were regenerated via
  `LIBSQL_DEV=1 cargo build --features multiple-ciphers,session`.
  The new amalgamation reports SQLITE_VERSION "3.46.1" with
  SQLITE_SOURCE_ID ending in "alt1" (libsql modification marker).
@penberg penberg changed the title Update sqlite 3.46.1 Merge upstream SQLite 3.46.1 May 21, 2026
@penberg penberg force-pushed the update-sqlite-3.46.1 branch from 86fa63d to 8cedfe2 Compare May 25, 2026 12:51
penberg added 3 commits May 26, 2026 09:18
The 3.46.1 merge updated shell8.test (adding the symlink extraction
tests shell8-3.x) and the bundled MultipleCiphers fileio.c, but left
libsql-sqlite3/ext/misc/fileio.c at 3.44.0. That file is embedded into
the CLI shell via 'INCLUDE ../ext/misc/fileio.c', so '.ar -x' ran the
stale writeFile() and the new tests failed with
'failed to create symlink: link1'.

Bring writeFile() in line with upstream 3.46.1:

  - Skip utimes() on symbolic links. utimes() follows the link to its
    target; the archive only contains link1 -> file1 (no file1), so the
    dangling link made utimes() fail and broke the first extraction
    (shell8-3.2).
  - unlink(zFile) before symlink() so re-extraction does not fail with
    EEXIST (shell8-3.3).
The steps to re-generate all the different variants of SQLite bundles is
a bit complicated. Add a script to automate that.
Add a script to make a diff between two versions of SQLite. Useful when
comparing libsql-sqlite3 after a upstream merge with the plain upstream
version to detect merge errors.
@penberg penberg merged commit 7251ec0 into main May 27, 2026
20 checks passed
@penberg penberg deleted the update-sqlite-3.46.1 branch May 27, 2026 06:12
penberg added a commit that referenced this pull request May 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant