Skip to content

Merge upstream SQLite 3.47.0#2240

Open
penberg wants to merge 1139 commits into
mainfrom
update-sqlite-3.47.0
Open

Merge upstream SQLite 3.47.0#2240
penberg wants to merge 1139 commits into
mainfrom
update-sqlite-3.47.0

Conversation

@penberg
Copy link
Copy Markdown
Collaborator

@penberg penberg commented May 26, 2026

Depends on #2238

drh and others added 30 commits September 5, 2024 11:46
testrunner.tcl.

FossilOrigin-Name: a01d869520329fb9e786cdc65f359785a95d19d289e4c6b844c758d6e5385aaf
as reported by individual test cases and to report that information in the
summary at the end of each test run.

FossilOrigin-Name: 80ebb7c7e686cd936ac834f2258f585a7004762593e0bc859ecd75d6fb0badfd
…nsion

if it is not already available.  (2) testrunner target "devtest" is added as
an alias for "mdevtest".  (3) Try to keep summary information at the end of
a test below 80-characters per line.  (4) Update the Makefile.in so that
the "clean" target removes the tcl extension built by item 1 above.

FossilOrigin-Name: aa5f10f21dbfb24ee54ca96bfb7b013ae29e26fec05b80681f19cc63d9face49
FossilOrigin-Name: 1d12744fe7cc294994e53e1ffe72799d0321dd5c1ef031213f083e200918fee5
…removed

from the FROM clause by the Omit-Noop-Join optimization of
[0cd82ee9a8413cf1].  Fix for the issue described by
[forum:/forum/8a1e467e905b8d27|format post 8a1e467e905b8d27].

FossilOrigin-Name: 22ca5a2ffb89ccb5f337993b5a95e27c449c39014284156eabc33da012a8759c
…b24e].

FossilOrigin-Name: d20c65c3b4256a662ebf7ed024b7b7adaceca90358f58111dc645da322000794
…tructions

for building on Windows.

FossilOrigin-Name: f69ef1a37b2778bdf73ee7e3b3edd74f7344ab8e5eedbedc22203c782e521f5b
…-join

optimization from yesterday does not regress.

FossilOrigin-Name: 224628b1039b996499e0d806fc0215f175da6f1059eb1b9ac491ac11126971ab
FossilOrigin-Name: 60fb8ee153ec293b6b3a4170dafa305e4c16af575aced72daef46116d8dc2bb6
system under test:  hostname, OS, pointer size, byte-order.

FossilOrigin-Name: ec75cfc5d4b69e4aed64d17748cac15cd62a759a1cbe7feaa4580ad8346b0b95
FossilOrigin-Name: 84a6c5f975de36ba93b888e01cc5188717b5644ab8384beb8fd8906bdfc8e227
…isable-tcl.

FossilOrigin-Name: d94541ae76b5d8b69f5524f10dcccc0814283f438a03f553848ed631a1983633
FossilOrigin-Name: e319d43bfd5ee4ed92b93531b239af4d1be0a8215b2a06c3532122ff2c7b6a7c
…TT and

$int_NNN parameters for fuzz testing.  Fix to testing logic only - no changes
to the SQLite core.

FossilOrigin-Name: 6206b90a4ec3f05e3bbb4844e71569bbde7df237550569e6419ff7c3146505dc
…y affect more than one row.

FossilOrigin-Name: 70e42f941c0778a04b82655409c7caf4c1039589f7e43a8ec1e736ea8f931b26
…belongs.

Make it part of the standard build for the CLI.  Bring some of the vfstrace
output up-to-date.

FossilOrigin-Name: 055b97de8d2397987d72dbab1cde78ece2d1c066e95042b4ed6b7b36b2cf9006
FossilOrigin-Name: aa75e701de61fe63ec15c35d70e53e950ff73b0dcb0d871dd8721412f3af297a
FossilOrigin-Name: f23954e604bf4da45f07194b54a4fe1c83002ab65d6c6f0ac095e88baba18547
…sary.

Improved xFileControl() output from vfstrace.c.

FossilOrigin-Name: e8f2d6313075c92fdeebcdfd8b50f43e9d45225890b2ef6b77148a766a42e940
standard part of the CLI.  Improve its output.  Also fix some unrelated
comment typos.

FossilOrigin-Name: 123cb1f579daec3ed092fe9dd1bc0d3250f2b56d4cda1efa92af139029e112e2
new pages onto the end of the database file using INSERT.

FossilOrigin-Name: fe0d67e72d4228661c021f227bfc0d5ddb1b726db0f36c7221ead8dd8bd1dc73
FossilOrigin-Name: d8103684f660ff9b3186d0f89afb113ca580bd16f0bf413ed8a9434236b54426
…iting

a NULL page.  Experimental.

FossilOrigin-Name: eb3c89ee2e4c5425be75deaf46a06a9cd8b210c695b918dd63a78f930c6e6b63
… to be stored in indexed columns of fts5 locale=1 tables.

FossilOrigin-Name: 55c5c119a0a77fac2c9f46d718ef78c0f33ed3520e10c240cf5bf1801e0586ee
… was not defined.

FossilOrigin-Name: 437849c80851da842b5c4fd37d5c147f821abc541e9b4d6f9000c12983548844
Does not work.

FossilOrigin-Name: 397b2d37b7a6619b0c1eee201065585d03496f94786b21540f613e4716d56612
FossilOrigin-Name: 7d87a27a01311153ddee122cedecedc3bcc331618dc2ab1da397a3b257dc21cf
… to be stored in indexed columns of fts5 locale=1 tables.

FossilOrigin-Name: 198305de92ebba7045d8ec7d2de98511f3b00924f808a3811f061dca47b01ec7
…working

on the actual sync protocol.  Still experimental.  Still a work in progress.

FossilOrigin-Name: 9a1a95f523a96303aad57e2422c2b51ea7e125f5490f32f7a2929d49b6c69ef8
sqlite3_mutex_notheld() when compiling with TSAN in as much as those
routines cause TSAN to complain.  Response to
[https://issues.chromium.org/issues/41427446].

FossilOrigin-Name: db702dd78500a0839b0b2810a580d3634df49275470787b170973a86b73826d3
drh and others added 29 commits October 17, 2024 18:12
(1) Extra paranoid defense against integer overflow.
(2) Remove a stray blank line in an FTS5 comment.

FossilOrigin-Name: 8563728deeb1d3ead9ff99a3ad2c7819b2223ead44e7374ecc7bbeda2d955ade
…ithin

the autoconf tarball.

FossilOrigin-Name: 360613a2246ae193d9a1d4bc7a91a0f63f159146709dac8a2de3e0ca8eb62752
FossilOrigin-Name: 6d55c3e238e06c6727f3e816a9268b6222ebfc90ec745bbbd48430bf7b16664c
FossilOrigin-Name: 62d24a69fe9cf6c07aa7e554f54aa7874b90e64c7c5597a78a824c6c9dc8847e
FossilOrigin-Name: 5f0cc7f26453faaea20a7bb0a1bcbab381a9bae7a81e099bb27f4b05fac1cd6c
…NUmakefile, as it is not compatible with too-old versions of gmake. Resolves the problem reported in [forum:e57932e3ccdc2c9c | forum post e57932e3ccdc2c9c].

FossilOrigin-Name: a31a94644113c226a06316a3f95fb38b605821f1c123e2cda06ba90bfcacf59f
…ypo in a makefile.

FossilOrigin-Name: 07843ac245661e8b8e086ad9d36c60bacf11784e9c56482593691fb97732f04f
FossilOrigin-Name: 31179fa78e47cafae49df35f28e5c4b01654665d54ee60d059545ce2d97a39a2
…aping of SQL values to incorrectly fail, and the --column-names setting to not be reset between tests.

FossilOrigin-Name: 779368dd9b8bb20890b06229c4c3c3b5d153ee88d1af472799515cdc599c870a
…ith both OPFS VFSes.

FossilOrigin-Name: ac747d57c2a26b47644bca2a9b191b09f5180f6872dce9c3261c370a18c848a2
…abase

using a UTF16 text encoding.
[forum:/forumpost/e055d50821|Forum post e055d50821].

FossilOrigin-Name: 981347009c4baa9f16d60091d37e11b05a7fc1c62ae262c4de596b584d6a491a
FossilOrigin-Name: ad8d4812420e6a5f1cd5c26d2ed07b76e0e3e94596bae4ca59b667a4099bd1cc
… new words to tool/custom.txt. No code changes.

FossilOrigin-Name: 59b76a99e4a28f4cc8c4f9f39ff6e039c4d29cb7b44183f1902b5792638656d2
vdbe.c so that it will be available when needed, even for non-amalgamation
builds.

FossilOrigin-Name: 5f23036a87f443c82fa7fd48fdac16a31128bb9288b71f97228c29bcfeb632fb
FossilOrigin-Name: 5a594dbcd533aa1e37acea1702db993672c3c0e621add9ea26a497c52037617f
…anges.

FossilOrigin-Name: 10f5c4a2fce8a9ea9b1533cabbaf4ca8549e72950d5c460c0f4501a9fc50e3e8
…ost/49d6a19ec|Forum post 49d6a19ec].

FossilOrigin-Name: 7a7162293c8fdb0078fe56948d697703539dd23273b2072990d4391c761e6ae2
…ike" has been used or the "like" UDF replaced by something unexpected.

FossilOrigin-Name: 9f642b3dbc8febfacad97076030f44e9b40067e27222f2bcb84813c5765d3d2a
FossilOrigin-Name: e904b37fb2621e6bd5e761f3ecb75adb34350f2d1d7b229e655e74bc6a2f5321
FossilOrigin-Name: 03a9703e27c44437c39363d0baf82db4ebc94538a0f28411c85dda156f82636e
This merges the version-3.46.1 tag (f3d536d "Version 3.46.1") of
upstream SQLite into libSQL. Conflicts were resolved with rename-following
(libsql vendors SQLite under libsql-sqlite3/, while upstream paths are at
the root).

== Path moves ==

* ext/intck/ — new upstream tree. After the merge tool placed the 9 files
  at top-level ext/intck/, they were moved under libsql-sqlite3/ext/intck/
  where libsql-sqlite3/Makefile.in already expects them (lines 472-473,
  1269-1270).

== Resolved conflicts ==

src/pragma.c — integrity check
  Upstream restructured how "wrong # of entries in index" is verified: the
  per-table count loop at the end of integrity_check was replaced with a
  single pre-pass using b-tree counts from OP_IntegrityCk. Took upstream's
  refactored form and added an IsVectorIndex(pIdx) skip in the new
  relocated loop, while keeping cnt++ unconditional so register indexing
  into aRoot stays aligned. The existing per-row IsVectorIndex skip was
  unaffected (both sides agreed there).

src/where.c
  (1) Vector-index planner clash: kept libsql's
      `if (pProbe->idxIsVector) opMask = 0;`
      alongside upstream's refined
      `if (pProbe->bLowQual && pSrc->fg.isIndexedBy==0)` (cherry-pick
      626d619 — low-quality indexes still considered when selected via
      INDEXED BY).
  (2) Took upstream's `wctrlFlags |= WHERE_KEEP_ALL_JOINS;` (cherry-pick
      2fdb323 — disables omit-noop-join when ORDER BY has 64+ terms).
      The flag definition and consumer were already merged in
      sqliteInt.h:3437 and where.c:6587.

src/shell.c.in
  Pure adjacency. Kept libsql's `.init_wasm_func_table` command (under
  LIBSQL_ENABLE_WASM_RUNTIME) alongside upstream's new `.intck` command,
  in help text and in the command dispatch.

ext/wasm/GNUmakefile
  Three independent edits coexisting:
  * libsql's `-DSQLITE_OMIT_SHARED_MEM` (replacing upstream's OMIT_WAL,
    from dfa4ce4 "ext/wasm: allow WAL mode")
  * libsql's `-lm` (from 9280b8e "link math lib" — needed for vector
    search)
  * upstream's new `C-PP.FILTER.global` / `SQLITE_C_IS_SEE` SEE filter
    hook
  These flags only affect the c-pp.c preprocessor binary, not the WASM
  output (see the comment block above the rule), but preserving libsql's
  intentional swap keeps the build self-consistent with sqliteInt.h:55-56
  where SQLITE_OMIT_SHARED_MEM is defined.

tool/mksqlite3c.tcl
  Kept libsql's `wasmedge_bindings.c` in the amalgamation source list.

ext/fts3/fts3.c, ext/fts3/fts3_write.c, ext/fts5/fts5_main.c,
ext/fts5/fts5_tokenize.c, ext/fts5/test/fts5integrity.test
  All taken from upstream verbatim. These conflicts were an artifact of
  the merge base being 3.45.0 while libsql HEAD already contained 3.45.1
  content (from prior merges 9ed72eb / f996bf9). For four of the
  five files libsql had zero local changes. The fifth — fts5_tokenize.c —
  had a libsql-local trigram-tokenizer crash fix (3e56d28, Aug 2024)
  that is now superseded by upstream's a751fdd (cherry-picked into
  3.46.1), which handles OOM-vs-syntax-error ordering more carefully.

autoconf/tea/configure.ac, configure, doc/testrunner.md,
test/mmapcorrupt.test, test/permutations.test
  Mechanical — took upstream (version string bumps, doc updates,
  ifcapable !mmap guard, test path globs for ext/rbu/ and ext/intck/).

== libsql-ffi bundle sync ==

* libsql-ffi/bundled/SQLite3MultipleCiphers/src/{fileio.c,series.c}
  were updated to match upstream 3.46.1 byte-for-byte. The SQLite3
  MultipleCiphers tree is independently vendored from sqlite3mc 1.8.1
  and these two files are out-of-band synced to keep the cipher bundle
  aligned with libsql's vendored SQLite. Note: the bundle's own
  sqlite3.h still reports 3.45.1 — only sqlite3.c is overlaid at build
  time from libsql-sqlite3 (build.rs:466-471). sqlar.c has a small
  3.45.1→3.46.1 type-safety drift that was not mirrored.
* libsql-ffi/bundled/src/{sqlite3.c,sqlite3.h} and
  libsql-ffi/bundled/bindings/session_bindgen.rs were regenerated via
  `LIBSQL_DEV=1 cargo build --features multiple-ciphers,session`.
  The new amalgamation reports SQLITE_VERSION "3.46.1" with
  SQLITE_SOURCE_ID ending in "alt1" (libsql modification marker).
The 3.46.1 merge updated shell8.test (adding the symlink extraction
tests shell8-3.x) and the bundled MultipleCiphers fileio.c, but left
libsql-sqlite3/ext/misc/fileio.c at 3.44.0. That file is embedded into
the CLI shell via 'INCLUDE ../ext/misc/fileio.c', so '.ar -x' ran the
stale writeFile() and the new tests failed with
'failed to create symlink: link1'.

Bring writeFile() in line with upstream 3.46.1:

  - Skip utimes() on symbolic links. utimes() follows the link to its
    target; the archive only contains link1 -> file1 (no file1), so the
    dangling link made utimes() fail and broke the first extraction
    (shell8-3.2).
  - unlink(zFile) before symlink() so re-extraction does not fail with
    EEXIST (shell8-3.3).
The steps to re-generate all the different variants of SQLite bundles is
a bit complicated. Add a script to automate that.
Add a script to make a diff between two versions of SQLite. Useful when
comparing libsql-sqlite3 after a upstream merge with the plain upstream
version to detect merge errors.
This merges the version-3.47.0 tag (f5fb820 "Version 3.47.0") of
upstream SQLite into libSQL. Conflicts were resolved with rename-following
(libsql vendors SQLite under libsql-sqlite3/, while upstream paths are at
the root), using a large merge.renameLimit so the ~2000 top-level ->
libsql-sqlite3/ moves are all detected:

    git -c merge.renameLimit=999999 -c diff.renameLimit=999999 \
        merge --no-commit version-3.47.0

Only files under libsql-sqlite3/ are changed by this merge. The
libsql-ffi/ bundled amalgamation and the SQLite3MultipleCiphers tree are
intentionally left untouched here; regenerating the bundle is a separate
step (cargo xtask build-bundled + scripts/update-sqlite-bundle.sh).

== Renames / path moves ==

* ext/wasm reorganization. Upstream renamed several wasm API sources to
  the ".c-pp.js" preprocessing convention and split the exported-symbol
  list. We followed upstream's layout (the new GNUmakefile / mkwasmbuilds.c
  expect it), and the rename/rename content merge preserved libsql's local
  additions inside the renamed files:
    sqlite3-api-glue.js     -> sqlite3-api-glue.c-pp.js
    sqlite3-api-oo1.js      -> sqlite3-api-oo1.c-pp.js   (keeps libsql's
                               DB.getAutocommit()/lastInsertRowid())
    sqlite3-api-worker1.js  -> sqlite3-api-worker1.c-pp.js
    EXPORTED_FUNCTIONS.sqlite3-api -> EXPORTED_FUNCTIONS.sqlite3-core
  plus new files accepted: EXPORTED_FUNCTIONS.sqlite3-extras,
  mkwasmbuilds.c.
* src/test_vfstrace.c -> ext/misc/vfstrace.c (upstream relocated it; no
  libsql changes, followed upstream).
* autoconf/tea/README -> autoconf/tea/README.txt (upstream replacement;
  libsql had no local changes).
* ext/wasm/fiddle/emscripten.css deleted by upstream, but kept here
  because libsql's fiddle/index.html still references it (libsql enables
  the jquery.terminal view and "libSQL" branding).

== Resolved conflicts ==

src/parse.y
  Combined: kept libsql's FUNCTION/LANGUAGE tokens (CREATE/DROP FUNCTION
  ... LANGUAGE, the WASM-UDF grammar) and added upstream's new ISNOT token
  (used by the auto-merged token-class list).

src/where.c
  Kept libsql's vector-index planner guard
  `|| pIndex->idxIsVector` in the ORDER-BY-satisfiability check, and took
  upstream's removal of the now-redundant standalone `nColumn = 1;`. The
  latter is required, not cosmetic: upstream relocated that assignment into
  the new SQLITE_OrderBySubq inner if/else, so keeping libsql's old line
  would have unconditionally clobbered the nColumn=0 that the new
  subquery-ORDER-BY optimization sets.

src/os_unix.c (3 hunks)
  Kept libsql's `#ifndef SQLITE_OMIT_SHARED_MEM` guard around the unix shm
  machinery (forward decl, SQLITE_FCNTL_EXTERNAL_READER, the shm
  implementation block). Upstream had changed those same lines to
  `#if !defined(SQLITE_WASI) && !defined(SQLITE_OMIT_WAL)`. libsql gates
  the whole shm subsystem on SQLITE_OMIT_SHARED_MEM (its "WAL without
  shared memory" design); the surrounding ~10 guards and the matching
  #endif already use that symbol, so all three must too. libsql's WASI/wasm
  builds define SQLITE_OMIT_SHARED_MEM, which subsumes upstream's WASI
  condition.

src/wal.c, src/wal.h
  Took libsql's side for both wal.c hunks: the `struct Wal` definition
  stays out of wal.c (libsql exposes it from wal.h as `struct sqlite3_wal`
  for the virtual-WAL interface), and the SQLITE_USE_SEH path keeps libsql's
  `#error "SEH is not supported in libSQL ..."` (SEH is incompatible with
  the virtual WAL). Upstream 3.47.0 added one new field to struct Wal,
  `int bGetSnapshot;` (SQLITE_ENABLE_SNAPSHOT); since the in-file struct is
  dropped on libsql's side, that field was added to wal.h's
  struct sqlite3_wal so SQLITE_ENABLE_SNAPSHOT builds still compile (the
  auto-merged wal.c references pWal->bGetSnapshot in 3 places).

src/sqlite.h.in
  Took upstream's deletion of the `# define SQLITE_OMIT_WAL 1` lines in the
  `#if defined(__wasi__)` block. libsql had previously commented those
  lines out with a NOTICE; upstream removed them outright. Same end state
  (WAL not forced off for WASI), so the libsql NOTICE/commented-out lines
  are no longer needed.

src/shell.c.in (4 hunks)
  Upstream refactored the shell's output API from oputf()/sputf() to
  sqlite3_fprintf() and removed the old helpers entirely. Re-expressed
  libsql's customizations in the new API rather than keeping the now-
  undefined oputf/sputf calls: the "libSQL %s (based on SQLite %s)" version
  banners (libsql_libversion()) in the .version meta-command, the -version
  flag and the interactive startup line, plus the LIBSQL_STMTSTATUS_ROWS_READ
  / ROWS_WRITTEN prepared-statement counters.

src/test3.c
  Took upstream's consolidated `#include "tclsqlite.h"` (the new header),
  and kept libsql's `#include "wal.h"` (needed for libsql_wal_methods,
  referenced in this file).

src/vacuum.c
  Semantic merge fix. Upstream 3.47.0 changed the VACUUM temp database from
  the fixed name "vacuum_db" to a random "vacuum_%016llx" (zDbVacuum). That
  rename landed in the ATTACH and the auto-merged data-copy, but libsql's
  local "#ifndef SQLITE_OMIT_VECTOR" block (which special-cases vector
  shadow tables) still emitted INSERT SQL against the literal "vacuum_db" -
  a database that no longer exists, so every VACUUM failed with
  SQLITE_ERROR. Updated libsql's vector block to use %s/zDbVacuum, matching
  upstream's #else branch. (Caught by running the Tcl suite; a pure diff
  review missed it because both sides changed adjacent code.)

ext/fts5/{fts5_main.c,fts5_tokenize.c} and fts5 tests
  Aligned to upstream verbatim. libsql's local fts5 changes here were
  superseded by upstream: the pTokApi field (libsql's way of storing the
  tokenizer API) is now provided natively by upstream's Fts5TokenizerConfig
  (t.pApi1/pApi2) - and fts5Int.h/fts5_config.c had already auto-merged to
  that, so taking upstream in fts5_main.c was required for the build, not
  just preferred. The trigram tokenizer odd-argument fix was likewise
  superseded by upstream's up-front `if( nArg%2 )` guard. The corresponding
  test expectations ("no such tokenizer" -> "SQL logic error", new
  fts5secure8 case) follow.

libsql-sqlite3/Makefile.in (8 hunks)
  Adopted upstream's restructuring - the $(libtclsqlite3.la_$(HAVE_TCL)) /
  $(tcl_install_$(HAVE_TCL)) idioms, the new tclextension-install/uninstall/
  list targets, the tidy:/clean:/distclean: split, sqldiff depending on
  ext/misc/sqlite3_stdio.h, src-verify$(BEXE), and dropping dbfuzz2-asan/msan
  - while preserving every libsql target and flag: liblibsql.la,
  liblibsql_wasm, libsqlapi, testlibsql, rusttest, rusttestwasm, libsql.pc,
  $(OPT_STATIC_LIBLIBSQL_WASM), and libsql-specific cleanup entries.

ext/wasm/{GNUmakefile,fiddle.make,fiddle/index.html,api/sqlite3-opfs-async-proxy.js,api/sqlite3-vfs-opfs.c-pp.js}
  Took upstream's reorganized structure (build-rule generation moved into
  mkwasmbuilds.c) while keeping libsql's c-pp flags (-DSQLITE_OMIT_SHARED_MEM,
  -lm) and the wasm-opt/maybe-wasm-shrink size-optimization definition. The
  opfs JS conflicts were upstream cleaning up its own code (operand-order,
  dead else) - no libsql logic. NOTE: libsql's wasm-opt/gzip weaving into
  the old build recipes is dropped, since upstream replaced those recipes
  with generated ones. None of ext/wasm is built in libsql CI.

manifest, manifest.uuid, VERSION, configure, autoconf/tea/configure.ac,
doc/compile-for-windows.md, README.md
  Mechanical. manifest / manifest.uuid taken verbatim from upstream 3.47.0
  (libsql tracks them as-is); VERSION -> 3.47.0; configure and tea
  configure.ac version-bumped to 3.47.0 (configure's only conflicts were
  version strings); compile-for-windows.md took upstream's TCL 9.0 docs;
  top-level README.md kept libsql's.

== Rename-misrouting fix ==

Four files exist twice in the libsql tree - once under libsql-sqlite3/ and
once under libsql-ffi/bundled/SQLite3MultipleCiphers/src/ - and both copies
were byte-identical to the merge base. git's rename detection paired the
base files with the (alphabetically first) libsql-ffi copies, so upstream's
3.47.0 changes were applied to the cipher-bundle copies while the real
libsql-sqlite3/ sources were left at 3.46.1. Corrected by reverting the
libsql-ffi copies to pre-merge HEAD (the bundle is updated out-of-band) and
syncing the libsql-sqlite3/ sources to upstream 3.47.0 (none carry libsql
changes - tclsqlite.c/shathree.c were verbatim upstream, fileio.c differed
only in whitespace, series.c is version-tracked):
    libsql-sqlite3/src/tclsqlite.c        (Tcl 9.0 support: Tcl_Size, etc.)
    libsql-sqlite3/ext/misc/shathree.c
    libsql-sqlite3/ext/misc/series.c
    libsql-sqlite3/ext/misc/fileio.c

== Validation ==

testfixture builds against Tcl 9.0 (pre-merge HEAD does not - exactly the
upstream tclsqlite.c fix restored above) and `make test` passes the full
Tcl suite with a single exception, vector-index-v2-query-4. That test is a
pre-existing over-specified assertion against an exact cosine-distance tie
(the query vector is equidistant from rows 'b' and 'c'); it runs against a
frozen, md5-locked fixture using vector code that this merge does not touch,
so the result is independent of this merge. Left as-is for a separate fix.
…tically

The DiskANN top-k buffer ordered candidates by distance using a strict
less-than comparison (distanceBufferInsertIdx), so when two vectors are
exactly equidistant from the query their relative order in the result fell
out of the search/visit order. That visit order depends on float rounding in
the distance computation, which varies with compiler/build flags -- e.g. the
same sources built as testfixture vs the sqlite3 shell resolved such ties
differently, and the Makefile.in changes in the 3.47.0 merge were enough to
flip the testfixture build's result for vector-index-v2-query-4 (query vector
[-1,1,1,1] is exactly equidistant from rows 'b' and 'c').

Add topCandidateInsertIdx(), which breaks exact-distance ties in the top
candidates buffer by rowid (ascending). Equidistant results now come back in
a stable, reproducible order independent of build flags or how the search
happened to reach them. Non-tied results are unaffected (the distance
comparison still decides first).

Update vector-index-v2-query-4 to assert the deterministic ordering ({d a b},
'b' being the smaller rowid of the tied pair) instead of the build-dependent
value it previously hard-coded.
@penberg penberg force-pushed the update-sqlite-3.47.0 branch from 65104fc to 604cc47 Compare May 26, 2026 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant