Overlay: Use overlay-aware CLI version when analyzing PRs

[henrymercer]

When analyzing PRs, prefer CLI versions that have cached overlay-base databases to speed up analysis time. However to ensure we can effectively rollback new versions, do not use a CodeQL version whose feature flag is disabled, even if this means running without overlay analysis.

This will be shipped via two feature flags:

• overlay_analysis_match_codeql_version_dry_run logs a diagnostic when the overlay-aware version differs from the latest enabled version
• overlay_analysis_match_codeql_version uses the overlay-aware version when analysing PRs

Risk assessment

For internal use only. Please select the risk level of this change:

• Low risk: Changes are fully under feature flags, or have been fully tested and validated in pre-production environments and are highly observable, or are documentation or test only.

Which use cases does this change impact?

Workflow types:

• Advanced setup - Impacts users who have custom CodeQL workflows.
• Managed - Impacts users with dynamic workflows (Default Setup, Code Quality, ...).

Products:

• Code Scanning - The changes impact analyses when analysis-kinds: code-scanning.
• Code Quality - The changes impact analyses when analysis-kinds: code-quality.
• Other first-party - The changes impact other first-party analyses.

Environments:

• Dotcom - Impacts CodeQL workflows on github.com and/or GitHub Enterprise Cloud with Data Residency.

How did/will you validate this change?

• Unit tests - I am depending on unit test coverage (i.e. tests in .test.ts files).

If something goes wrong after this change is released, what are the mitigation and rollback strategies?

• Feature flags - All new or changed code paths can be fully disabled with corresponding feature flags.

How will you know if something goes wrong after this change is released?

• Telemetry - I rely on existing telemetry or have made changes to the telemetry.
  • Dashboards - I will watch relevant dashboards for issues after the release. Consider whether this requires this change to be released at a particular time rather than as part of a regular release.
  • Alerts - New or existing monitors will trip if something goes wrong with this change.

Are there any special considerations for merging or releasing this change?

• No special considerations - This change can be merged at any time.

Merge / deployment checklist

• Confirm this change is backwards compatible with existing workflows.
• Consider adding a changelog entry for this change.
• Confirm the readme and docs have been updated if necessary.

[Copilot]

Pull request overview

This PR updates how the Action selects the default CodeQL CLI version so that, when analyzing pull requests, it can prefer an enabled CLI version that already has cached overlay-base databases for the configured languages (to speed up overlay/incremental analysis), while still respecting feature-flag rollback constraints.

Changes:

• Extend the default CLI “version info” returned from feature flags to include a sorted list of enabled default versions (not just a single version).
• Add PR-aware logic in setup-codeql to optionally pick the highest enabled version that intersects with overlay-base DB cache entries (with dry-run telemetry support).
• Thread the new version-info shape through callers and update unit tests and the changelog entry.

Show a summary per file

File	Description
src/upload-lib.ts	Switches to the new getEnabledDefaultCliVersions API and passes rawLanguages through initCodeQL.
src/testing-utils.ts	Updates test fixtures/mocks for the new CodeQLDefaultVersionInfo.enabledVersions shape.
src/start-proxy.ts	Adapts to the new version-info shape by selecting enabledVersions[0] for proxy downloads.
src/start-proxy.test.ts	Updates stubbing to getEnabledDefaultCliVersions.
src/setup-codeql.ts	Implements overlay-aware default-version resolution for PR analyses (feature-flag gated) and threads rawLanguages.
src/setup-codeql.test.ts	Updates call sites for new rawLanguages parameter and adds unit tests for overlay-cache version filtering.
src/setup-codeql-action.ts	Updates to getEnabledDefaultCliVersions and passes rawLanguages (currently undefined).
src/init.ts	Threads rawLanguages through to setupCodeQL.
src/init-action.ts	Uses getEnabledDefaultCliVersions and passes rawLanguages derived from the languages input.
src/feature-flags.ts	Introduces CodeQLVersionInfo, changes default version info to enabledVersions[], and adds new overlay match feature flags.
src/feature-flags.test.ts	Updates tests to validate multi-version enablement ordering/fallback behavior.
src/codeql.ts	Threads rawLanguages into tool setup to support PR-aware version resolution.
src/codeql.test.ts	Updates tests for the new default-version info shape and new function signatures.
CHANGELOG.md	Adds an UNRELEASED entry describing the experimental overlay-aware default version selection.
lib/upload-sarif-action-post.js	Generated JS output (not reviewed).
lib/upload-lib.js	Generated JS output (not reviewed).
lib/start-proxy-action.js	Generated JS output (not reviewed).
lib/start-proxy-action-post.js	Generated JS output (not reviewed).
lib/resolve-environment-action.js	Generated JS output (not reviewed).
lib/init-action.js	Generated JS output (not reviewed).
lib/autobuild-action.js	Generated JS output (not reviewed).
lib/analyze-action.js	Generated JS output (not reviewed).
lib/analyze-action-post.js	Generated JS output (not reviewed).

Copilot's findings

Comments suppressed due to low confidence (1)

src/start-proxy.test.ts:1029

• The stub variable is named getDefaultCliVersion, but it actually stubs getEnabledDefaultCliVersions. Renaming the local variable (and the later assertion) would avoid confusion and better reflect what’s being tested.

      const getDefaultCliVersion = sinon
        .stub(features, "getEnabledDefaultCliVersions")
        .resolves({
          enabledVersions: [{ cliVersion: "2.20.1", tagName: expectedTag }],
        });
      const path = await startProxyExports.getProxyBinaryPath(logger, features);

      t.assert(getDefaultCliVersion.calledOnce);
      sinon.assert.calledOnceWithMatch(

• Files reviewed: 14/26 changed files
• Comments generated: 3

[Copilot]

The PR description says this change is "for internal use only" and fully gated behind feature flags, but the changelog entry says "We expect to roll this change out to everyone in May." Please reconcile the changelog wording with the intended rollout/audience (e.g., clarify it’s feature-flagged/experimental and avoid implying an unconditional rollout if that’s not accurate).