ci(vortex-duckdb): cross-platform build.rs and Windows DuckDB tests by joseph-isaacs · Pull Request #8223 · vortex-data/vortex

joseph-isaacs · 2026-06-02T17:30:08Z

What

Makes vortex-duckdb build on Windows (x86_64/aarch64-pc-windows-msvc) and enables DuckDB tests on the Windows CI leg.

`build.rs` — cross-platform, with each platform decision pulled into a named helper

Also maps the Windows release archives in download and adds the Windows targets to the artifact mapping.

Add Windows (x86_64 and aarch64 MSVC) support to the DuckDB build script and refactor the platform-specific logic out of scattered inline conditionals into named helpers, so each platform-dependent decision lives in one place: - `target_is_msvc` / `target_is_windows`: read CARGO_CFG_TARGET_ENV/OS (the build target, not the host as `cfg!()` would in a build script). - `configure_cpp_warnings`: MSVC `cl` gets `/W4` and a plain include (no `/WX`); GCC/Clang keep `-Wall -Wextra -Wpedantic -Werror` with DuckDB headers via `-isystem`. - `rpath_link_arg`: emits `-Wl,-rpath` only where the linker supports it; on Windows the DLL is resolved via PATH / the executable directory. - `symlink_duckdb_source`: cfg(unix) creates the convenience source symlink; no-op elsewhere (Windows directory symlinks need elevated privileges). Also map the Windows release archives in `download` and add `duckdb.dll` to `BUILD_ARTIFACTS`. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

The flat BUILD_ARTIFACTS const mixed macOS (.dylib), Linux (.so/.a), and Windows (.dll) library names in one place. Replace it with a `build_artifacts()` helper that returns only the files DuckDB emits on the current target, matching the other platform helpers in this build script. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Add a dedicated `cargo nextest run -p vortex-duckdb` step to the Windows leg of the rust-test-other matrix (previously vortex-duckdb was excluded there). Windows has no rpath, so the DuckDB shared library must be resolvable at run time from the executable directory or PATH. Teach build.rs to copy `duckdb.dll` into the Cargo profile dir and its `deps/` subdir (where nextest runs test executables) on Windows targets; this is a no-op elsewhere, where rpath and `*_LIBRARY_PATH` already handle library resolution. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

codspeed-hq · 2026-06-02T17:37:26Z

Merging this PR will not alter performance

⚠️

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 1275 untouched benchmarks

_{Comparing claude/serene-franklin-HU10k (1973e3b) with develop (81046d7)}

The custom-labels crate ships a customlabels.cpp that uses GCC/Clang-isms (__attribute__, __thread, ssize_t, volatile blocks) which MSVC's cl.exe rejects, breaking the Windows build of vortex-duckdb. vortex-io already gates this dependency behind cfg(unix); do the same here and gate its only usage (the thread-local labelset setup in table_function::init_local) so Windows skips the profiling-label integration. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

vortex-duckdb's build.rs runs bindgen, which needs libclang at build time. The Windows runner has no clang.dll/libclang.dll on PATH, so the build script panicked with "Unable to find libclang". Add a Windows-only step that uses the preinstalled LLVM (or installs it via Chocolatey) and exports LIBCLANG_PATH before building vortex-duckdb. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Two MSVC-only build failures in the vortex-duckdb C++ extras: - cpp/vector.cpp used __builtin_unreachable(), a GCC/Clang builtin MSVC's cl rejects (error C3861). Use __assume(false) under _MSC_VER instead. - The MSVC build lacked /EHsc, so standard C++ exception unwinding (which the DuckDB headers rely on) was disabled (warning C4530). Add /EHsc to the MSVC compiler flags. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

duckdb_vector_flatten and duckdb_vector_to_string declared their length parameter as `unsigned long`, which bindgen maps to c_ulong. c_ulong is 64-bit on Linux/macOS but 32-bit on Windows (LLP64), so the u64 the Rust callers pass only compiled off-Windows. Use DuckDB's idx_t (uint64_t) like the rest of the header, so bindgen emits a u64 parameter on every platform. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

The Windows DuckDB test step cannot link: vortex-duckdb's C++ extras call DuckDB's internal C++ API, but the prebuilt Windows duckdb.dll only exports the C API, so the link fails with LNK1120 unresolved externals. Resolving this would require building DuckDB from source on Windows with all symbols exported. Mark the step continue-on-error so it doesn't block the PR. It still exercises the cross-platform build.rs: dependency resolution, bindgen (via libclang), and the MSVC C++ compile all run before the link stage. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

0ax1 · 2026-06-03T09:25:02Z

+/// `-isystem` for the same reason.
+fn configure_cpp_warnings(build: &mut cc::Build, duckdb_include_dir: &Path) {
+    if target_is_msvc() {
+        build.flag("/W4").flag("/EHsc").include(duckdb_include_dir);


Do we ignore header warnings for Windows? Ideally, we could match the UNIX setup here fully.

0ax1 · 2026-06-03T09:25:48Z

-        .flag(duckdb_include_dir)
+    let mut build = cc::Build::new();
+    build.std("c++20").cpp(true);
+    configure_cpp_warnings(&mut build, duckdb_include_dir);


Any way to retain the builder pattern style?

0ax1 · 2026-06-03T09:26:54Z

+/// is resolved via `PATH` / the executable directory instead).
+fn rpath_link_arg(library_dir: &Path) -> Option<String> {
+    if target_is_windows() {
+        None


How do we ensure then that libduckdb is added to the path?

0ax1 · 2026-06-03T09:29:55Z

+    }
+}
+
+/// On Windows, copy `duckdb.dll` next to the built artifacts so the loader can find it at


For all comments, double check whether we can make them more concise and readable.

0ax1 · 2026-06-03T10:04:24Z

+fn configure_cpp_warnings(build: &mut cc::Build, duckdb_include_dir: &Path) {
+    if target_is_msvc() {
+        build.flag("/W4").flag("/EHsc").include(duckdb_include_dir);
+    } else {


Why not just build with Clang actually. We need Clang anyway bc of libclang and clang is abi compatible with msvc on windows.

0ax1 · 2026-06-03T10:14:56Z

        return FSSTVector::Validity(vector).Reset();
    default:
+#if defined(_MSC_VER)
+        __assume(false);


If we build with clang, we don't need any of these specializations.

0ax1 · 2026-06-03T10:16:33Z

 use std::fs;
 use std::io;
+#[cfg(unix)]
 use std::os::unix::fs::symlink;


If we don't use a symlink what do we do for Windows.

0ax1 · 2026-06-03T10:20:08Z

+          # preinstalled LLVM if present, otherwise install it via Chocolatey.
+          $llvm = "C:\Program Files\LLVM\bin"
+          if (-not (Test-Path "$llvm\libclang.dll")) {
+            choco install llvm --no-progress -y


chocolatey has low ceiling rate-limiting, and strictly speaking requires a commercial license (maybe we can get away here without for vortex) but given the number of builds we run the rate-limiting is a real risk (i've recall this from embark studios where the ci shut down bc of that).

0ax1 · 2026-06-03T10:22:52Z

+        run: |
+          # vortex-duckdb's build.rs runs bindgen, which needs libclang. Use the
+          # preinstalled LLVM if present, otherwise install it via Chocolatey.
+          $llvm = "C:\Program Files\LLVM\bin"


Is this a robust way to get the path, can we query chocolatey?

0ax1 · 2026-06-03T10:25:07Z

+    if !dll.exists() {
+        return;
+    }
+    // OUT_DIR is `<target>/<profile>/build/<pkg>/out`; the profile dir four levels up holds


This differs for arch local and cross-compilation:

local : target/<profile>/build/<pkg-hash>/out

cross: target/<triple>/<profile>/build/<pkg-hash>/out

joseph-isaacs added 3 commits June 2, 2026 17:31

joseph-isaacs force-pushed the claude/serene-franklin-HU10k branch from ad6342b to e01c37c Compare June 2, 2026 17:31

joseph-isaacs added the changelog/ci label Jun 2, 2026 — with Claude

joseph-isaacs added 5 commits June 2, 2026 17:45

joseph-isaacs requested a review from 0ax1 June 3, 2026 09:13

0ax1 reviewed Jun 3, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci(vortex-duckdb): cross-platform build.rs and Windows DuckDB tests#8223

ci(vortex-duckdb): cross-platform build.rs and Windows DuckDB tests#8223
joseph-isaacs wants to merge 8 commits into
developfrom
claude/serene-franklin-HU10k

joseph-isaacs commented Jun 2, 2026 •

edited

Loading

Uh oh!

codspeed-hq Bot commented Jun 2, 2026 •

edited

Loading

Uh oh!

0ax1 Jun 3, 2026

Uh oh!

0ax1 Jun 3, 2026

Uh oh!

0ax1 Jun 3, 2026

Uh oh!

0ax1 Jun 3, 2026

Uh oh!

0ax1 Jun 3, 2026

Uh oh!

0ax1 Jun 3, 2026

Uh oh!

0ax1 Jun 3, 2026

Uh oh!

0ax1 Jun 3, 2026

Uh oh!

0ax1 Jun 3, 2026

Uh oh!

0ax1 Jun 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

joseph-isaacs commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

build.rs — cross-platform, with each platform decision pulled into a named helper

Uh oh!

codspeed-hq Bot commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

joseph-isaacs commented Jun 2, 2026 •

edited

Loading

`build.rs` — cross-platform, with each platform decision pulled into a named helper

codspeed-hq Bot commented Jun 2, 2026 •

edited

Loading