Enable opt-in BuildXL (MSBuildCache) project caching#9980
Conversation
Wire up the BuildXL-backed Microsoft.MSBuildCache project-cache plugin behind an opt-in /p:MSBuildCacheEnabled=true flag (off by default, so normal builds are unaffected). - Directory.Packages.props: select backend (AzurePipelines in CI / Local elsewhere), pin version, add gated GlobalPackageReferences. - eng/projectcaching.props: repo-specific settings (cache universe, allowed post-project file accesses), imported from Directory.Build.props only when enabled. - eng/build-with-cache.ps1: helper that runs a cached local build with the x64 desktop MSBuild engine and required switches. - docs/build-caching.md: usage, requirements (x64 desktop MSBuild + -reportFileAccesses), and CI recipe. Validated locally with the Local backend: populate then 100% cache hit.
There was a problem hiding this comment.
Pull request overview
Adds opt-in BuildXL/MSBuildCache support for testfx builds.
Changes:
- Conditionally configures local or Azure Pipelines caching.
- Adds repository-specific cache settings and a build helper.
- Documents local and CI usage.
Show a summary per file
| File | Description |
|---|---|
Directory.Packages.props |
Adds conditional cache package references. |
Directory.Build.props |
Imports cache configuration when enabled. |
eng/projectcaching.props |
Defines repository cache settings. |
eng/build-with-cache.ps1 |
Runs cached graph builds. |
docs/build-caching.md |
Documents setup and troubleshooting. |
Review details
- Files reviewed: 5/5 changed files
- Comments generated: 4
- Review effort level: Medium
| # Locate the x64 desktop MSBuild.exe (Bin\amd64\MSBuild.exe). -reportFileAccesses is x64-only. | ||
| $vswhere = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe" | ||
| if (-not (Test-Path $vswhere)) { | ||
| throw "vswhere.exe not found. Visual Studio 17.9+ (with the MSBuild component) is required for cached builds." | ||
| } | ||
| $msbuild = & $vswhere -latest -prerelease -products * -requires Microsoft.Component.MSBuild -find MSBuild\**\Bin\amd64\MSBuild.exe | | ||
| Select-Object -First 1 | ||
| if (-not $msbuild) { | ||
| throw "Could not find x64 desktop MSBuild.exe (Bin\amd64\MSBuild.exe). Install/repair Visual Studio 17.9+." | ||
| } |
There was a problem hiding this comment.
Done in 75c0783. The helper now uses the Arcade InitializeVisualStudioMSBuild helper (which enforces the global.json VS requirement and selects the host-arch MSBuild) instead of a raw vswhere query, and then asserts the resolved path is the x64 \amd64\MSBuild.exe flavor (throwing otherwise, since -reportFileAccesses is x64-only).
| - **x64 desktop `MSBuild.exe`** (from Visual Studio 17.9+), specifically the x64 flavor at | ||
| `MSBuild\Current\Bin\amd64\MSBuild.exe`. |
There was a problem hiding this comment.
Done in 75c0783. The docs now distinguish the plugin's 17.9 floor from testfx's higher pinned requirement (global.json VS + Universal workload, and TestFx.slnx needing 17.13+), and note the helper enforces global.json via the Arcade helper rather than accepting any install.
| - **Cache misses that should be hits** — usually a global property that changes every build is part of the | ||
| fingerprint (e.g. a version stamp). Add it to `MSBuildCacheGlobalPropertiesToIgnore` in | ||
| `eng/projectcaching.props`. Inspect the logs under the `MSBuildCache` log directory to see the fingerprint | ||
| inputs. |
There was a problem hiding this comment.
Fixed in 75c0783. The troubleshooting guidance now explicitly says a version/build-number stamp must NOT be ignored (it is baked into assemblies/packages, so ignoring it would replay stale-version artifacts), and limits MSBuildCacheGlobalPropertiesToIgnore to properties proven not to affect any output.
| - **"file accessed after project finished" / duplicate output errors** — a detached process (telemetry, | ||
| compiler server) touched a file after the project completed, or two projects wrote the same file. Add the | ||
| path to `MSBuildCacheAllowFileAccessAfterProjectFinishFilePatterns` or | ||
| `MSBuildCacheIdenticalDuplicateOutputPatterns` respectively. |
There was a problem hiding this comment.
Fixed in 75c0783. The guidance now limits MSBuildCacheAllowFileAccessAfterProjectFinishFilePatterns to accesses proven irrelevant to outputs, and explicitly says compiler/tool inputs must stay tracked (that is what the included SharedCompilation package is for). The same caveat is repeated in eng/projectcaching.props.
The MSBuildCache plugin only engages when the real project graph is the MSBuild entry point; it does NOT load through Arcade's Build.proj (nested MSBuild task). So the Windows CI job now builds TestFx.slnx directly under the cache: - azure-pipelines.yml: Windows job split into Restore (Arcade) -> Build (BuildXL cache, x64 desktop MSBuild via eng/build-with-cache.ps1) -> Pack (Arcade). Adds EnablePipelineCache variable and maps SYSTEM_ACCESSTOKEN for the AzurePipelines cache backend. - eng/build-with-cache.ps1: made CI-safe (no tools.ps1 dependency) with -BinaryLog/-ExtraProperties/-Targets params; documents the graph-entry requirement. - docs/build-caching.md: document the 3-step CI structure and that first runs will need per-project file-access exclusions tuned in eng/projectcaching.props.
| displayName: Build (BuildXL cache) | ||
| env: | ||
| # Required by the Azure Pipelines cache backend to read/write Pipeline Caching content. | ||
| SYSTEM_ACCESSTOKEN: $(System.AccessToken) |
There was a problem hiding this comment.
Addressed in 75c0783. The three cached steps are now gated on ne(variables._IsFork, 'True') (using System.PullRequest.IsFork), and a fork build runs a single uncached Arcade build (the pre-caching behaviour) instead, so fork Windows jobs no longer hit the AzurePipelines backend / token requirement.
| -NoRestore | ||
| -Targets Build | ||
| -BinaryLog $(Build.SourcesDirectory)\artifacts\log\$(_BuildConfig)\BuildCache.binlog | ||
| -ExtraProperties FastAcceptanceTest=true |
There was a problem hiding this comment.
Addressed in 75c0783. The cached build now passes -ci to the helper, which applies -p:ContinuousIntegrationBuild=true, -warnaserror and -nr:false to the MSBuild invocation, restoring the CI semantics CIBuild.cmd provided.
| When adding to a list-typed setting, always append to the existing value ($(Setting);newvalue) so the | ||
| plugin's own defaults are preserved. See https://github.com/microsoft/MSBuildCache for the full list of | ||
| settings and their meanings. |
There was a problem hiding this comment.
Corrected in 75c0783. The comment now explains that Directory.Build.props is evaluated before the plugin package props, so the setting is empty here and assigning a defaulted list would suppress the plugin default; it instructs to only set no-default / fully-owned settings here (or set the complete value / use a post-package-props hook). The two settings used (MSBuildCacheCacheUniverse scalar, and ...FilePatterns which has no plugin default) are safe at this point.
| - **x64 desktop `MSBuild.exe`** (from Visual Studio 17.9+), specifically the x64 flavor at | ||
| `MSBuild\Current\Bin\amd64\MSBuild.exe`. |
There was a problem hiding this comment.
Addressed in 75c0783. Docs now state TestFx.slnx requires VS 17.13+ (per dev-guide.md) and that the helper enforces the global.json VS requirement via the Arcade helper; the 17.9 floor is called out as the plugin minimum, applicable only to individual project builds that do not need the full solution's newer VS.
| incremental builds are not supported, start from a clean tree: | ||
|
|
||
| ```powershell | ||
| git clean -xdf |
There was a problem hiding this comment.
Addressed in 75c0783. The helper now bootstraps the pinned SDK via the Arcade InitializeDotNetCli helper on each run (so it works right after git clean -xdf), and the docs add a note to run build.cmd -restore after a clean if invoking MSBuild directly instead of the helper.
- helper: use Arcade InitializeVisualStudioMSBuild (enforces global.json VS requirement) + InitializeDotNetCli (bootstraps .dotnet after git clean), assert x64 amd64 MSBuild; add -ci to apply CI semantics (ContinuousIntegrationBuild=true, -warnaserror, -nr:false). Rename -BinaryLog to -BinLogPath to avoid collision with tools.ps1 \. - azure-pipelines.yml: pass /p:MSBuildCacheEnabled=true to the Arcade restore so the plugin package is restored and imported (otherwise the build ran uncached); pass -ci to the cached build; add fork fallback (System.PullRequest.IsFork) that runs the normal uncached Arcade build since forks lack SYSTEM_ACCESSTOKEN; keep signing in the Pack step. - eng/projectcaching.props: correct the evaluation-order guidance (Directory.Build.props runs before the plugin package props, so appending does not preserve plugin defaults); clarify that only output-irrelevant accesses may be allow-listed and compiler inputs stay tracked. Fix invalid '--' in XML comments. - docs/build-caching.md: distinguish the plugin's VS 17.9 floor from testfx's pinned VS (global.json) and TestFx.slnx 17.13+; note git clean removes .dotnet and how it is re-bootstrapped; correct troubleshooting so version stamps are NOT ignored in the cache key and compiler-server accesses stay tracked; document the fork fallback.
What
Enables the BuildXL-backed Microsoft.MSBuildCache project-cache plugin and makes it the gating Windows CI build, plus a local path and docs.
Follows the plugin README (CPM + Azure Pipelines) and the microsoft/microsoft-ui-xaml example (which builds its solution directly, not through Arcade).
Key mechanism finding
MSBuildCache only engages when the real project graph is the MSBuild entry point, using x64 desktop
MSBuild.exewith-reportFileAccesses. I verified that:dotnet build/ Core MSBuild rejects-reportFileAccesses; the 32-bit desktop MSBuild errors "x64 flavor only".Build.proj(nested MSBuild task) does not load the plugin (empty cache logs).So the build must invoke x64 desktop MSBuild directly on
TestFx.slnx.Changes
Directory.Packages.props— backend selection (AzurePipelinesin CI /Locallocally), pinned version, gatedGlobalPackageReferences (off unless/p:MSBuildCacheEnabled=true).eng/projectcaching.props— repo-specific plugin settings; imported fromDirectory.Build.propsonly when enabled.eng/build-with-cache.ps1— CI-safe wrapper: locates x64 desktop MSBuild, points it at the repo SDK, runs-graph -m -reportFileAccesses -p:MSBuildCacheEnabled=true. Used locally and by CI.azure-pipelines.yml— Windows job now builds under the cache: Restore (Arcade) → Build (BuildXL cache) → Pack (Arcade), withEnablePipelineCache+SYSTEM_ACCESSTOKENfor the Azure Pipelines cache backend.docs/build-caching.md— usage, requirements, CI structure, troubleshooting.Validation done locally
MSBuildCacheAssemblyempty).ProjectCachePluginregisters with repo settings.azure-pipelines.ymlparses;docs/passes markdownlint; helper runs end-to-end.Expected: red until tuned⚠️
This is a first-cut gating integration. A full-repo cached build under the file-access sandbox surfaces per-project issues (files written after a project finishes, non-deterministic/duplicate outputs, cache-busting global properties) that only real CI runs reveal. Expect to add exclusions to
eng/projectcaching.props(MSBuildCacheIgnoredOutputPatterns,MSBuildCacheAllowFileAccessAfterProjectFinishFilePatterns,MSBuildCacheIdenticalDuplicateOutputPatterns,MSBuildCacheGlobalPropertiesToIgnore) over a few iterations. The publishedBuildCache.binlogandMSBuildCachelogs are the debugging inputs. Also note the Windows build was split into Restore/Build/Pack, which may need adjustment so the downstream Test/pack steps stay green.