Support standalone builds of the offload test suite#1376
Conversation
This change allows the offload test suite to be built against a prebuilt
LLVM distribution which is built with the included CMake cache file.
With this build it takes ~10 seconds to build the offloader, api-query
and unit test binaries against the provided LLVM distribution.
To use this build LLVM with a command like so:
```
cmake -C <offload test>/cmake/caches/OffloadDistribution.cmake \
-DCMAKE_INSTALL_PREFIX=<path to install to> \
<other cmake options> <path to llvm>
ninja install-distribution
```
Then configure and build the offload test suite with a command like so:
```
cmake -DCMAKE_PREFIX_PATH=<path to llvm install>/lib/cmake/llvm \
-DLLVM_MAIN_SRC_DIR=<path to llvm-project>/llvm \
-DDXC_DIR=<path to folder containing dxc/dxv> \
-DOFFLOAD_TEST_TEST_CLANG=On \
-DGOLDENIMAGE_DIR=<path to images> \
<other cmake options> <path to ofload test suite>
ninja check-hlsl
```
|
Do we want to obsolete the in-tree HLSL offload distribution we have defined in the HLSL cmake cache in the llvm repo? Or do we want to maintain both distributions? |
|
The https://github.com/llvm/offload-test-suite/blob/main/docs/offload-distribution.md doc should be updated to describe the two different distributions: in-tree and the out-of-tree that depends on an externally-built llvm. |
| # Seek installed Lit. | ||
| find_program(LLVM_LIT | ||
| NAMES llvm-lit lit.py lit | ||
| PATHS "${LLVM_MAIN_SRC_DIR}/utils/lit" |
There was a problem hiding this comment.
Should we add an if guard to verify that this main_src variable is set correctly?
| option(OFFLOADTEST_ENABLE_VALIDATION "Enable available runtime validation layers (defaults Off)." OFF) | ||
|
|
||
| list(APPEND OFFLOADTEST_DEPS | ||
| api-query |
There was a problem hiding this comment.
There's a mix of dependencies here that are sourced from llvm and from the offload test suite.
[FileCheck split-file obj2yaml not] are from llvm.
Should we only append them if they are not already built?
There was a problem hiding this comment.
The nice thing about the way CMake exported targets work is that we don't need to conditionalize this because even when the tools are built as part of the external LLVM build they still have targets that we import into the standalone build and we can depend on those but not have any build-time penalty.
There isn't really a cost to that dependency one way or the other, so simplifying the build logic is nice.
| option(OFFLOADTEST_ENABLE_VALIDATION "Enable available runtime validation layers (defaults Off)." OFF) | ||
|
|
||
| list(APPEND OFFLOADTEST_DEPS | ||
| api-query |
There was a problem hiding this comment.
The nice thing about the way CMake exported targets work is that we don't need to conditionalize this because even when the tools are built as part of the external LLVM build they still have targets that we import into the standalone build and we can depend on those but not have any build-time penalty.
There isn't really a cost to that dependency one way or the other, so simplifying the build logic is nice.
| if (OFFLOADTEST_TEST_CLANG) | ||
| if (OFFLOADTEST_TEST_CLANG AND NOT OFFLOADTEST_BUILT_STANDALONE) | ||
| list(APPEND OFFLOADTEST_DEPS clang) | ||
| endif() |
There was a problem hiding this comment.
I should note that the reason I made this conditional even though it doesn't strictly need to be (because of exported targets being wonderful) is because I wanted to break the connection between OFFLOADTEST_TEST_CLANG and the existence of the Clang target. That way we could build the offload-test-suite, LLVM and Clang all separately if we wanted and just point to different build directories for different bits.
|
@Icohedron I didn't want to interfere with the existing split build stuff since that is integrated into pipelines and this is not. If this flow ultimately replaces the other we should do a cleanup to remove unused code. I added a section at the end of the offload-distribution doc explaining this flow. Again, if this becomes part of our CI flow, I think a larger update/rewrite of the doc would be appropriate. |
4 participants
This change allows the offload test suite to be built against a prebuilt LLVM distribution which is built with the included CMake cache file.
With this build it takes ~10 seconds to build the offloader, api-query and unit test binaries against the provided LLVM distribution.
To use this build LLVM with a command like so:
Then configure and build the offload test suite with a command like so:
Description of Changes
This updates the root CMakeLists.txt to be able to use it as a standalone CMake root. In that configuration in uses find_package to load LLVM and configures the project. When built standalone it needs an LLVM source checkout to find googletest and lit, but can otherwise use pre-built LLVM libraries and binaries.
The Lit suites are updated to allow the offload-test-suite tools and LLVM/Clang tools to come from different paths. This is done by the addition of the OFFLOADTEST_TOOLS_DIR CMake option which can either be the same as the LLVM tools directory or unique.
I also moved the OffloadMigration header into the include directory for the Support headers. Since it is used across the API and Support libraries we really should have always had it there instead of relying on relative pathing. This change was required because the relative paths behaved differently under the standalone build configuration.