ci(multi-version): add DuckDB v1.5.2 matrix row alongside v1.4.4 LTS (step 2)#167
Open
estebanzimanyi wants to merge 4 commits into
Open
ci(multi-version): add DuckDB v1.5.2 matrix row alongside v1.4.4 LTS (step 2)#167estebanzimanyi wants to merge 4 commits into
estebanzimanyi wants to merge 4 commits into
Conversation
…get + doc)
Step 1 of N — same pattern as MobilityDB-on-Postgres compiling against
PG 13-18 from one source tree, applied to DuckDB extension ABIs.
DuckDB extension ABI is version-specific: a binary built against v1.4.4
cannot load into v1.5.2 and vice versa. MobilityDuck therefore needs
to produce a separate binary per (duckdb_version, platform). The
selection is driven by the SHAs that the three submodules — duckdb,
duckdb-spatial, extension-ci-tools — point at when the build runs.
This PR adds the *foundation* that the rest of the multi-version story
builds on:
- scripts/select-duckdb-version.sh
Per-version manifest mapping a DuckDB version to matching
duckdb-spatial and extension-ci-tools refs. Switches the three
submodules in lockstep when invoked.
- Makefile
New variable DUCKDB_VERSION (default v1.4.4, the ecosystem
alignment target) and a 'make set-duckdb-version' target that
delegates to the script.
- doc/multi-duckdb-version.md
Full design: per-version manifest, build-flow diagram, how to add
a new supported version, relationship to the (future) CI matrix,
adopter install path.
- README.md
Single-sentence pointer telling adopters which versions are
supported and where the multi-version doc lives.
What this PR does NOT change:
- the committed submodule SHAs (still v1.4.4 — the default build is
identical to before)
- MainDistributionPipeline.yml (CI still builds for one version; the
matrix expansion is the next PR, stacked on this one)
- any C++ source (preprocessor conditionals for API divergences land
alongside the CI matrix once a second version's build surfaces them)
The result is a no-behaviour-change foundation: existing builds work
identically; the v1.5.x track is one 'make set-duckdb-version' away
from being buildable; the CI matrix PR turns it on in one place.
Step 2 of N, stacks on the foundation in MobilityDB#166. Adds duckdb-build-v1-5 alongside duckdb-build-v1-4 (renamed from duckdb-latest-build, postfixed -v1-4 to disambiguate artefacts). Each row calls a different extension-ci-tools workflow tag because `uses:` in GitHub Actions cannot be a matrix expression. The v1.5.2 job is expected red until step 3 lands: - The reusable workflow's `make set_duckdb_version` handles the `duckdb/` submodule on its own, but `duckdb-spatial` and `extension-ci-tools` stay at the parent's committed SHAs (v1.4 line on `main`). No hook inside the reusable workflow lets us interleave the foundation script's swap between checkout and build. - Source-side, any DuckDB v1.5 ABI divergences (ExtensionUtil -> ExtensionLoader rename, bundled-spatial-extension surface reshuffle, etc.) will surface as compile errors during this job. Those are the signals step 3 (preprocessor conditionals) is authored against. Adding the matrix row now is the explicit visibility step: the divergences become observable in CI rather than discovered later. The deploy step for v1.5 is held until duckdb-build-v1-5 is reliably green — pushing broken binaries to the community-extension repository would be worse than no v1.5 binary. Also widens the PR-trigger branch globs to include `ci/**`, `cleanup/**`, `chore/**` (alongside the existing `feat/**` and `fix/**`) so this PR and similar follow-ups trigger CI without manual workflow_dispatch.
The stage_icu helper mapped only the Linux uname values, so on the macOS arm64 test runner uname -m returned "arm64" and the icu extension was copied to .duckdb/extensions/v1.4.4/arm64 instead of .../osx_arm64, where DuckDB's autoload looks. The hub fallback is not reliably resolvable on that runner, so the osx_arm64 Test step failed to load the extension. Map the OS and architecture to the DuckDB platform string (linux_amd64, linux_arm64, osx_amd64, osx_arm64) so the locally built icu is staged at the path autoload expects on every tested platform; the Linux mapping is unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`meosType` (lower-case) is the **pre-consolidation** MEOS type name; `MeosType` (upper-case) is the **post-consolidation** target that the upstream rename sweep has not yet reached. The current vcpkg pin (`vcpkg_ports/meos/portfile.cmake` REF f11b7443ee98…) is still pre-consolidation: `meos/include/temporal/meos_catalog.h` line 121 declares the typedef as `} meosType;` and every MEOS API uses the lower-case spelling. MobilityDuck's source code consistently uses `meosType` to match — `grep -rn '\bMeosType\b' src/` finds the name only on the alias line and its comment, nowhere else. c8cad6d added `using meosType = MeosType;` as a forward-looking bridge for the eventual consolidation bump. That bridge points at `MeosType`, which the current pin does NOT yet expose, so it breaks every PR's Linux arm64 build with: /duckdb_build_dir/src/include/tydef.hpp:18:18: error: ‘MeosType’ does not name a type; did you mean ‘meosType’? The fix is to drop the premature alias and replace the misleading comment with one that documents the pre/post-consolidation distinction and the resume path for the next pin bump — at that point a reviewer can either restore the bridge (this time it'll be valid because `MeosType` will exist) or sweep the MobilityDuck source from `meosType` to `MeosType` in a single PR. Unblocks every in-flight PR's Linux arm64 build: MobilityDB#126, MobilityDB#130, MobilityDB#149, MobilityDB#158, MobilityDB#159, MobilityDB#160, plus the entire `feat/*_port_core` extended-type stack (MobilityDB#148/MobilityDB#150/MobilityDB#151/MobilityDB#153/MobilityDB#155/MobilityDB#156).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacks on #166. Until #166 merges this PR's diff shows two commits — the foundation from #166 plus the matrix expansion in this PR.
Adds the second matrix row so DuckDB v1.5.2 builds run in CI alongside the existing v1.4.4 LTS build. Each row calls a different
extension-ci-toolsworkflow tag becauseuses:in GitHub Actions cannot be a matrix expression.Jobs
duckdb-build-v1-4@v1.4.4-v1-4duckdb-deploy-v1-4@v1.4.4-v1-4duckdb-build-v1-5@v1.5-variegata-v1-5Also widens the PR-trigger branch globs to include
ci/**,cleanup/**,chore/**alongside the existingfeat/**andfix/**.Expected state
The v1.5.2 job is expected to fail in this PR. Two reasons:
make set_duckdb_versionstep checks out the right DuckDB tag inside theduckdb/submodule directory, but theduckdb-spatialandextension-ci-toolssubmodules remain at the parent repository's committed SHAs — which onmaintarget the v1.4 line. The reusable workflow has no hook to interleave the foundation script's swap between checkout and build.ExtensionUtil→ExtensionLoaderrename, bundled-spatial-extension surface reshuffle, etc.) will surface as compile errors during this job.Both are the signals step 3 (preprocessor conditionals + parent-side submodule swap mechanism) is authored against. Adding the matrix row now is the visibility step: the divergences become observable in CI rather than discovered later.
The deploy step for v1.5 is held until
duckdb-build-v1-5is reliably green — pushing broken binaries to the community-extension repository would be worse than no v1.5 binary.Next steps (stacked PRs)
make set-duckdb-versionand commits the swap to a temporary ref the reusable workflow checks out, so the v1.5.2 job sees the rightduckdb-spatialandextension-ci-toolssubmodules.test/sql/*.testfor any case where DuckDB's own behaviour changed between versions.