[agent-issue]: support per-job pool overrides for generated jobs

[krubenok]

Submission requirements

• I generated this issue with an agent that used .github/agents/ado-aw.agent.md.
• I reviewed the generated issue and confirm it is being filed directly in githubnext/ado-aw.

Problem summary

ado-aw currently resolves the top-level pool: once and clones it onto every agent-hosted canonical job: Setup, Agent, Detection, SafeOutputs, Teardown, and Conclusion. This forces lightweight downstream jobs to consume the same specialized capacity as the Agent job, even when only Agent needs an expensive self-hosted pool, attached disk, or uncommon capability.

In one measured standalone workflow, Agent needed a specialized pool for a full dependency installation, while Detection, SafeOutputs, and Conclusion only consumed pipeline artifacts or shallow checkouts. Keeping all jobs on the specialized pool produced 14m51s of sequential inter-job allocation gaps and about 36m21s from Agent start through Conclusion finish. A generated-lock experiment that kept Agent on the specialized pool and moved the three downstream jobs to vmImage: ubuntu-22.04 reduced allocation gaps to 59s and total execution to about 15m11s. The end-to-end run succeeded with the existing separate jobs, artifact handoffs, and security boundaries intact: roughly 13m52s less allocation waiting and 21 minutes less total latency.

Manual lock-file edits are not a durable workaround because recompilation overwrites them and integrity checking expects generated output to match source. This is a general resource-efficiency and queue-latency problem, not a request to combine security stages onto one agent.

This is complementary to #1450: that issue adds demands to a selected pool; this issue allows different generated jobs to select different pools. The two capabilities should compose.

Reproduction details

Compile a standalone workflow whose Agent job needs a specialized named pool:

pool:
  name: SpecializedLinuxPool

The compiler resolves that configuration once and emits the same pool for Agent, Detection, SafeOutputs, and Conclusion, plus Setup/Teardown when present. There is no source-front-matter representation for keeping the current pool as a default while overriding only lightweight jobs.

A backward-compatible shape could retain pool: as the default and add an explicit override map, for example:

pool:
  name: SpecializedLinuxPool
job-pools:
  detection:
    vmImage: ubuntu-22.04
  safe-outputs:
    vmImage: ubuntu-22.04
  conclusion:
    vmImage: ubuntu-22.04

The exact field name is open to design, but supported keys should cover the compiler-owned jobs that may exist: setup, agent, detection, safe-outputs, teardown, and conclusion. SafeOutputs_Reviewed should inherit the SafeOutputs override unless separately justified. ManualReview must remain agentless on pool: server and must not be overrideable.

Expected behavior: unspecified jobs inherit the existing top-level/default pool, specified jobs use their override, and the generated lock remains reproducible and passes ado-aw check.

Observed behavior: every agent-hosted generated job receives the same resolved pool, so right-sizing requires unsupported lock-file edits.

Target-specific considerations:

• standalone, job, and stage already emit job-level pool mappings and can lower overrides per canonical job.
• Job templates currently hoist one pool to extends.parameters.pool and suppresses per-job pool keys through templateContext. If the template cannot express job-specific pools, compilation should reject overrides for that target with a clear diagnostic rather than silently ignoring them; otherwise map them through the supported pipeline template mechanism.
• Existing PoolConfig validation and sanitization should apply independently to defaults and overrides, including mutual exclusion of name and vmImage; pool demands from [agent-issue]: support standard Azure Pipelines pool demands #1450 should remain usable in either location.

Proposed next step

Add a typed per-job pool override configuration with default inheritance, resolve it into the shared canonical pipeline context, and have each job builder select its effective pool. Preserve all existing job dependencies, artifact handoffs, conditions, token scoping, and the fixed server pool for ManualReview.

Document that selecting a different self-hosted pool changes the trust boundary for that job: pool administrators and agents must be trusted for the credentials and artifacts available to the job. This feature should not broaden permissions or move steps between jobs.

Add compilation tests for default inheritance and each override across standalone/job/stage, explicit 1ES support or rejection, optional Setup/Teardown/Conclusion jobs, reviewed SafeOutputs inheritance, invalid pool combinations, compatibility with demands, and lock integrity/recompilation.

[jamesadevine]

pool:
  name: SpecializedLinuxPool       # default for all jobs
  overrides:
    detection:
      vmImage: ubuntu-22.04        # lightweight Microsoft-hosted image
    safe-outputs:
      vmImage: ubuntu-22.04
    conclusion:
      vmImage: ubuntu-22.04

in 0.44.0!