Merge Upstream#10
Draft
louislam wants to merge 60 commits into
Draft
Conversation
refs https://www.sqlite.org/releaselog/3_42_0.html - sha3-256sum: `643898e9fcc8f6069bcd47b0e6057221c1ed17bbee57da20d2752c79d91274e8`
…dability - Implemented RAII to ensure resource safety and exception safety - Leveraged range-based for loops for cleaner and more efficient iteration - Addressed and resolved all project-related warnings for enhanced code quality
This reverts commit 7552aa0.
refs https://www.sqlite.org/releaselog/3_44_2.html - sha3-256sum: `6c427f0547e2f7babe636b748dd5d5a1f2f31601adadef7e2805e7d1f7171861`
- this doesn't do anything because we just rethrow the error
- Python 3.12 no longer ships with this and GHA CI has updated to 3.12, which breaks our build scripts - this should fix that until we can update node-gyp
- merged pretest and test - removed `pack` as it is not needed
- right now our way to check semver compatibility is to run the tests on every version - this is really inefficient as it results in a lot of CI jobs that we don't need - this should run a CI job that ensures the minimum version we accept is Node 10.12.0
- bumped building on Node 16 to Node 18 - removed all matrix executions apart from Node 18
…stall` fixes #1641 fixes #1721 fixes #1714 fixes #1713 fixes #1700 fixes #1704 - `@mapbox/node-pre-gyp` is effectively unmaintained [1] as has a few bugs which our users keep running into - it seems the prebuilt binary world has moved in favor of prebuild + it's various other forms - one option would be to use prebuildify to bundle all binaries into the package, but that's a step too far removed from the current situation for now - instead, we can use prebuild-install to download the binaries, and `prebuild` to build + upload the binaries - this means we can remove node-pre-gyp and fix a bunch of issues! - eventually, we could start providing electron prebuilt binaries too [1]: mapbox/node-pre-gyp#657
- switched to just providing a list of platform + arch instead of the full formatted string
- this fixes building in debug mode because the compiler (at least, on macOS) doesn't like the `assert(status == 0)` - this also DRYs up some common code by moving it into a macro
- the path here was incorrect
- this cleans up similar definitions and makes further refactors easier
- the whole benchmark suite needs reworking but this should be a good poor mans method to make the benchmarking suite heavier
…tiation refs https://github.com/nodejs/node-addon-api/blob/main/doc/object.md#set - according to the node-addon-api docs, you can set various things as the key for a Napi::Object, including a std::string ref - instantiating a `Napi::String::New` is quite heavy, especially when we're doing it for every row we return, so we can avoid doing that and speed up the function - locally, this speeds up the benchmark by 5-15% (a lot of variance) but YMMV
- these extra functions aren't really necessary and means there is more redirection occurring - as a bonus, this fixes a variable shadowing issue in the Statement implementation
- this was never passed to the user because the value below is `1` - our documentation says the close event doesn't emit any parameters, so we can just remove it from the array
refs https://www.sqlite.org/releaselog/3_45_0.html - sha3-256sum: `9fc2a78088875ae7c112957d58ccc52b1a0a4afa34ac669290be42f352b1aa76`
Added unmaintained notice and status.
Fixes #1824 — prebuild-install failing on Node 22.14.0 due to N-API version string comparison bug in napi-build-utils < 2.0.0. Dependencies: - prebuild-install: ^7.1.1 -> ^7.1.3 (fixes N-API version detection) - prebuild: 12.1.0 -> 13.0.1 - tar: ^6.1.11 -> ^7.5.10 (fixes 6 HIGH CVEs) - node-addon-api: ^7.0.0 -> ^8.0.0 - node-gyp: 8.x -> 12.x (peer + optional) - Added engines field: node >=20.17.0 - Added yarn.lock CI: - Test matrix: Node 20, 22, 24 (all current LTS lines) - ubuntu-20.04 -> ubuntu-24.04, alpine3.15 -> alpine3.20 - Replaced macos-m1 self-hosted with macos-latest (now ARM64) - Dropped win32-ia32 (Node 24 no longer ships 32-bit Windows) - Excluded macos x64 + Node 20 (Rosetta 2 async hooks bug) - Removed pip install setuptools workaround - Updated actions (checkout v6, setup-node v6, upload-artifact v7, setup-qemu-action v4, setup-buildx-action v4) - Sanitised artifact names to avoid slashes from platform targets - Dockerfile: Node 18 -> 24, bullseye -> bookworm
Two years of upstream improvements including bug fixes, performance enhancements, and security hardening. The extract script now removes the VERSION file that SQLite >= 3.49 ships, which conflicts with the C++20 <version> header on case-insensitive filesystems (macOS/Windows).
- gcc-preinclude.h: include <unistd.h> with _GNU_SOURCE active to provide syscall() on musl (fixes GCC 13 atomic_wait.h bug); use __GLIBC__ instead of __USE_GNU to guard glibc symbol versioning - CI: linux-arm64 glibc builds now use native ubuntu-24.04-arm runners instead of QEMU emulation (much faster) - CI: QEMU/Docker retained only for Alpine/musl builds (renamed to build-musl)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR merges upstream changes that modernize the project’s build/release pipeline (moving from node-pre-gyp to prebuild/prebuild-install), bump supported Node versions, update the bundled SQLite version, and refactor parts of the C++ bindings toward safer memory management.
Changes:
- Update CI/Docker build flows for Node 20/22/24 and
prebuild-based binary publishing. - Upgrade bundled SQLite (3.41.1 → 3.52.0) and refresh various docs/metadata (including deprecation notice).
- Refactor C++ binding internals to use
std::unique_ptrfor row/value storage and consolidate async-work creation via a macro.
Reviewed changes
Copilot reviewed 22 out of 26 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/semver-check.js | Adds a Node engine compatibility check for dependencies during CI. |
| tools/BinaryBuilder.Dockerfile | Updates binary builder image to Node 24 / Debian bookworm and switches build to npm run prebuild. |
| tools/benchmark/select-data.sql | Expands benchmark dataset SQL (significant additional inserts). |
| test/support/helper.js | Simplifies fileExists helper implementation. |
| test/support/createdb.js | Treats zero-length DB files as invalid and recreates them. |
| src/statement.h | Refactors value/row ownership to std::unique_ptr and updates related destructors/state init. |
| src/statement.cc | Adapts statement execution/binding/row conversion to std::unique_ptr and macroized async-work creation. |
| src/macros.h | Introduces CREATE_WORK helper macro and removes manual field deletion macro. |
| src/gcc-preinclude.h | Adjusts glibc detection and includes <unistd.h> for build environment compatibility. |
| src/database.h | Modernizes definitions and moves to in-class member initialization and macroized work definitions. |
| src/database.cc | Uses CREATE_WORK, modernizes code style, and tweaks event emission behavior. |
| src/backup.h | Switches internal queue storage to std::unique_ptr and uses in-class member initialization. |
| src/backup.cc | Adapts queue processing and async-work creation to new queue/value ownership patterns. |
| src/async.h | Minor modernizations in async listener implementation. |
| README.md | Marks repository as deprecated/unmaintained; updates SQLite version and binary install docs. |
| package.json | Major version bump; switches dependencies to prebuild/prebuild-install; raises Node engine requirement. |
| LICENSE | Updates copyright header. |
| lib/sqlite3-binding.js | Changes native binding loader to use bindings. |
| deps/extract.js | Removes SQLite VERSION file after extraction to avoid case-insensitive FS conflicts. |
| deps/common-sqlite.gypi | Updates bundled SQLite version numeric constant. |
| binding.gyp | Adjusts module naming and removes post-build copy step. |
| .gitignore | Ignores prebuilds/ output directory. |
| .github/workflows/ci.yml | Updates build matrix/steps for newer Node and prebuild workflows (including musl builds). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| if (!fs.existsSync(packageJsonPath)) return; | ||
|
|
||
| const packageJson = JSON.parse(fs.readFileSync(packageJsonPath)); |
| } | ||
| } | ||
|
|
||
| const packageJson = JSON.parse(fs.readFileSync(path.join(__dirname, '..', 'package.json'))); |
Comment on lines
52
to
56
| "devDependencies": { | ||
| "eslint": "6.8.0", | ||
| "mocha": "7.2.0", | ||
| "node-pre-gyp-github": "1.4.4" | ||
| "eslint": "8.56.0", | ||
| "mocha": "10.2.0", | ||
| "prebuild": "13.0.1" | ||
| }, |
Comment on lines
4
to
8
| #include <cstdlib> | ||
| #include <cstring> | ||
| #include <string> | ||
| #include <queue> | ||
| #include <vector> |
Comment on lines
+114
to
+118
| INSERT INTO foo SELECT 0 + Num, 'Row ' || ( 0 + Num) FROM Nums WHERE Num BETWEEN 1 AND 1000; | ||
| INSERT INTO foo SELECT 1000 + Num, 'Row ' || ( 1000 + Num) FROM Nums WHERE Num BETWEEN 1 AND 1000; | ||
| INSERT INTO foo SELECT 2000 + Num, 'Row ' || ( 2000 + Num) FROM Nums WHERE Num BETWEEN 1 AND 1000; | ||
| INSERT INTO foo SELECT 3000 + Num, 'Row ' || ( 3000 + Num) FROM Nums WHERE Num BETWEEN 1 AND 1000; | ||
| INSERT INTO foo SELECT 4000 + Num, 'Row ' || ( 4000 + Num) FROM Nums WHERE Num BETWEEN 1 AND 1000; |
Comment on lines
+294
to
295
| Napi::Value info[] = { Napi::String::New(env, "close") }; | ||
| EMIT_EVENT(db->Value(), 1, info); |
Comment on lines
200
to
+202
| int status; | ||
| std::string message; | ||
|
|
||
| bool inited; | ||
| bool locked; | ||
| bool completed; | ||
| bool failed; | ||
| int remaining; | ||
| int pageCount; | ||
| bool finished; | ||
| std::queue<Call*> queue; | ||
| std::queue<std::unique_ptr<Call>> queue; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.