dlopen offload

[sgasho]

dlopen offload to dist it in dist-x86_64-linux without enabling llvm.offload=true in CI (#159064 (comment))

I tested that this works by following steps at https://rustc-dev-guide.rust-lang.org/offload/usage.html for both LLVM22 and 23(by switch base branch to #158734 temporarily), though It might be better to land this after LLVM 23.

r? @ZuseZ4

[rustbot]

This PR changes how LLVM is built. Consider updating src/bootstrap/download-ci-llvm-stamp.

[sgasho]

I got this error while testing on LLVM23

Related to this I guess.
llvm/llvm-project#199483

View changes since the review

[ZuseZ4]

cc @kevinsala
@Sa4dUs I vaguely remember you ran into this when trying LLVM 23 patches, right? Was this the right solution?

[rust-log-analyzer]

The job aarch64-gnu-llvm-21-1 failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

diff of stderr:

- error[E0277]: `{integer}` is not a tuple
-   --> $DIR/non_tuple_args.rs:7:36
+ error: rust offload backend not found in the sysroot: failed to find a `libRustOffload-21` in the sysroot candidates:
+        * $BUILD_DIR/aarch64-unknown-linux-gnu/stage2/lib
+        * $BUILD_DIR/aarch64-unknown-linux-gnu/stage2/lib
3    |
- LL |     core::intrinsics::offload::<_, _, ()>(kernel_0, [1, 1, 1], [1, 1, 1], 0, 42);
-    |                                    ^ the nightly-only, unstable trait `std::marker::Tuple` is not implemented for `{integer}`
-    |
- note: required by a bound in `offload`
-   --> $SRC_DIR/core/src/intrinsics/mod.rs:LL:COL
+    = note: it will be distributed via rustup in the future
9 
- error: aborting due to 1 previous error
- 
- For more information about this error, try `rustc --explain E0277`.
13 

Note: some mismatched output was normalized before being compared
-        * /checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib
-        * /checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib
+ error: rust offload backend not found in the sysroot: failed to find a `libRustOffload-21` in the sysroot candidates:
+        * $BUILD_DIR/aarch64-unknown-linux-gnu/stage2/lib
+        * $BUILD_DIR/aarch64-unknown-linux-gnu/stage2/lib
+    = note: it will be distributed via rustup in the future


The actual stderr differed from the expected stderr
To update references, rerun the tests and pass the `--bless` flag
To only update this specific test, also pass `--test-args offload/non_tuple_args.rs`

error: 1 errors occurred comparing output.
status: exit status: 1
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/tests/ui/offload/non_tuple_args.rs" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/checkout/vendor" "--sysroot" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2" "--target=aarch64-unknown-linux-gnu" "--check-cfg" "cfg(test,FALSE)" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Zwrite-long-types-to-disk=no" "-Cstrip=debuginfo" "--emit" "metadata" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/ui/offload/non_tuple_args" "-A" "unused" "-W" "unused_attributes" "-A" "internal_features" "-A" "incomplete_features" "-A" "unused_parens" "-A" "unused_braces" "-Crpath" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "-Zunstable-options" "-Zoffload=Device" "-Clto=fat"
stdout: none
--- stderr -------------------------------
error: rust offload backend not found in the sysroot: failed to find a `libRustOffload-21` in the sysroot candidates:
       * /checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib
       * /checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib
   |
   = note: it will be distributed via rustup in the future
------------------------------------------

---- [ui] tests/ui/offload/non_tuple_args.rs stdout end ----
---- [ui] tests/ui/offload/type_mismatch.rs stdout ----
Saved the actual stderr to `/checkout/obj/build/aarch64-unknown-linux-gnu/test/ui/offload/type_mismatch/type_mismatch.stderr`
diff of stderr:

- error: expected a function item for the offload kernel, found `i32`
-   --> $DIR/type_mismatch.rs:8:5
+ error: rust offload backend not found in the sysroot: failed to find a `libRustOffload-21` in the sysroot candidates:
+        * $BUILD_DIR/aarch64-unknown-linux-gnu/stage2/lib
+        * $BUILD_DIR/aarch64-unknown-linux-gnu/stage2/lib
3    |
- LL |     core::intrinsics::offload::<_, _, ()>(not_fn, [1, 1, 1], [1, 1, 1], 0, ());
-    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- 
- error: offload kernel expects 1 arguments, but 0 arguments were provided
-   --> $DIR/type_mismatch.rs:12:5
-    |
- LL |     core::intrinsics::offload::<_, _, ()>(kernel_1, [1, 1, 1], [1, 1, 1], 0, ());
-    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- 
- error: type mismatch in offload kernel argument 0: expected `f32`, found `f64`
-   --> $DIR/type_mismatch.rs:16:5
-    |
- LL |     core::intrinsics::offload::<_, _, ()>(kernel_1, [1, 1, 1], [1, 1, 1], 0, (42.0f64,));
-    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- 
- error: offload kernel return type mismatch: kernel returns `()`, but offload call expects `f64`
-   --> $DIR/type_mismatch.rs:20:18
-    |
- LL |     let _: f64 = core::intrinsics::offload::<_, _, f64>(kernel_0, [1, 1, 1], [1, 1, 1], 0, ());
-    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- 
- error: type mismatch in offload kernel argument 0: expected `f32`, found `f64`
-   --> $DIR/type_mismatch.rs:24:5
-    |
- LL |     core::intrinsics::offload::<_, _, ()>(kernel_2, [1, 1, 1], [1, 1, 1], 0, (42.0f64, 42.0f64));
-    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- 
- error: type mismatch in offload kernel argument 1: expected `f32`, found `f64`
-   --> $DIR/type_mismatch.rs:24:5
-    |
- LL |     core::intrinsics::offload::<_, _, ()>(kernel_2, [1, 1, 1], [1, 1, 1], 0, (42.0f64, 42.0f64));
-    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- 
- error: aborting due to 6 previous errors
+    = note: it will be distributed via rustup in the future
38 
39 

Note: some mismatched output was normalized before being compared
-        * /checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib
-        * /checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib
+ error: rust offload backend not found in the sysroot: failed to find a `libRustOffload-21` in the sysroot candidates:
+        * $BUILD_DIR/aarch64-unknown-linux-gnu/stage2/lib
+        * $BUILD_DIR/aarch64-unknown-linux-gnu/stage2/lib
+    = note: it will be distributed via rustup in the future


The actual stderr differed from the expected stderr
To update references, rerun the tests and pass the `--bless` flag
To only update this specific test, also pass `--test-args offload/type_mismatch.rs`

error: 1 errors occurred comparing output.
status: exit status: 1
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/tests/ui/offload/type_mismatch.rs" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/checkout/vendor" "--sysroot" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2" "--target=aarch64-unknown-linux-gnu" "--check-cfg" "cfg(test,FALSE)" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Zwrite-long-types-to-disk=no" "-Cstrip=debuginfo" "--emit" "metadata" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/ui/offload/type_mismatch" "-A" "unused" "-W" "unused_attributes" "-A" "internal_features" "-A" "incomplete_features" "-A" "unused_parens" "-A" "unused_braces" "-Crpath" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "-Zunstable-options" "-Zoffload=Device" "-Clto=fat"
stdout: none
--- stderr -------------------------------
error: rust offload backend not found in the sysroot: failed to find a `libRustOffload-21` in the sysroot candidates:
       * /checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib
       * /checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib
   |
   = note: it will be distributed via rustup in the future
------------------------------------------

---- [ui] tests/ui/offload/type_mismatch.rs stdout end ----

[ZuseZ4]

The LLVM 23 update unfortunately takes a little longer than expected, but glad to hear you got it to work with both 22 and 23. In that case I'd just merge it once it's ready and not wait, since it seems unlikely that we'll break the LLVM 23 update PR with this work here. Worst case, std::offload is broken on main for a couple of days if we missed something, but that shouldn't affect too many users yet.

It's a bit unfortunate that we need a new cmake file for the offload wrapper. Hopefully I'll find time to upstream our wrappers into LLVM, so we won't need that in the future. But it would probably be part of llvm 24, so we shouldn't wait for that.

[ZuseZ4]

We don't really have any new safety checks in place ,or? So I'd leave those three fucntions marked as unsafe.

View changes since the review