Skip to content

Allow pinning hosts to every slice of a sliced job#565

Open
fernandorocagonzalez wants to merge 1 commit into
ctrliq:mainfrom
fernandorocagonzalez:feat-slice-pinned-hosts
Open

Allow pinning hosts to every slice of a sliced job#565
fernandorocagonzalez wants to merge 1 commit into
ctrliq:mainfrom
fernandorocagonzalez:feat-slice-pinned-hosts

Conversation

@fernandorocagonzalez

Copy link
Copy Markdown
Contributor

The problem

Job slicing distributes the inventory evenly across the slices, which breaks any playbook where a play targets a host the rest of the run depends on. The classic case is a preparatory play on localhost that gathers data or builds configuration the other hosts consume. Once sliced, only the slice that happened to receive localhost runs that play, and the remaining slices fail or do the wrong thing. Today the options are to not slice those playbooks at all, or to rewrite them so no play depends on another host.

What this adds

A new Job Template field, job_slice_pinned_hosts: a comma separated list of inventory host names that are taken out of the round robin distribution and included in every slice instead, keeping their group memberships and variables.

The goal here is specifically to fix that sliced jobs problem, and the scope is deliberately narrow: the field only guarantees that the named hosts are part of every slice's inventory. It does not decide what runs where. That control stays in the playbook, as usual: each play's hosts: pattern determines what actually executes on which host, so a pinned host only runs the plays that target it and the fleet plays keep running on each slice's share of the inventory.

Matching is by exact host name (groups and limit style patterns are not supported), names that do not exist in the inventory are ignored, and the field does nothing when the job is not sliced. Since pinned hosts are repeated in every slice they are not counted when capping the slice count to the number of hosts: an inventory of 4 hosts with 1 pinned yields at most 3 slices. The docs spell out the main caveat: plays targeting a pinned host run once per slice, concurrently, so this is meant for idempotent coordination plays, and run_once becomes once per slice.

Implementation notes

  • The field lives on JobOptions, so it exists on both the template and the job and is copied automatically at launch time.
  • Inventory.get_sliced_hosts excludes the pinned names from the round robin and adds them to each slice; get_script_data and the task inventory build pass the list down.
  • get_effective_slice_ct subtracts pinned hosts from the distributable host count. The sliced workflow relaunch check now calls get_effective_slice_ct too, instead of re deriving min(hosts, slice_count) on its own, so relaunching keeps working when pinned hosts reduce the node count.
  • get_hosts_for_fact_cache uses the same pinned aware slicing, keeping the per slice fact cache host set aligned with the inventory the slice actually ran against.
  • _get_task_impact accounts for pinned hosts running in every slice.
  • New "Pinned Hosts" section in docs/job_slicing.md.

UI

A "Job slice pinned hosts" text field in the job template form, shown only when Job Slicing is greater than 1, plus the corresponding entry in the details view.

Job template form

Job template details

Tests and verification

  • 8 new functional tests: slice distribution with pinned hosts, group membership preservation, unknown names ignored, effective slice count capping (including the all hosts pinned edge), template to job inheritance checked down to each slice's generated inventory, fact cache alignment, and relaunch of a pinned sliced workflow.
  • 1 new UI jest test for the conditional form field.
  • Verified end to end on a docker compose dev env with a playbook that has a localhost coordination play plus an assert on the fleet play that localhost is present in the slice. Without pinning, the workflow fails on the slice that did not receive localhost. With job_slice_pinned_hosts=localhost, every slice carries localhost plus its share of hosts and the whole workflow is green, and relaunching the sliced workflow works.

Sliced workflow green

Slice output with the coordination play

Heads up on the migration: this adds 0201_job_slice_pinned_hosts, which will collide in numbering with the 0201 migrations in #561/#563/#564 if any of those merges first; happy to renumber whenever needed.

Job slicing distributes the inventory evenly across slices, which breaks
playbooks where a play targets a host the rest of the run depends on. The
usual victim is a preparatory play against localhost: after slicing, only
the slice that happened to receive localhost runs it, and the other slices
miss whatever it set up.

This adds a job_slice_pinned_hosts field to job templates: a comma
separated list of inventory host names that are taken out of the round
robin distribution and included in every slice instead, keeping their
group memberships and variables. Names that do not match an inventory
host are ignored and the field does nothing when the job is not sliced.

Pinned hosts do not add to the work worth distributing, so they no longer
count when capping the slice count to the number of hosts: an inventory
of 4 hosts with one of them pinned yields at most 3 slices.

The docs spell out the tradeoff: plays targeting a pinned host run once
per slice, concurrently, so this is meant for idempotent coordination
plays (localhost being the typical case).
@cigamit cigamit self-assigned this Jul 5, 2026
@cigamit cigamit added the enhancement New feature or request label Jul 5, 2026
@cigamit cigamit requested a review from Copilot July 5, 2026 01:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for “pinned hosts” in job slicing so specific inventory hosts (e.g., localhost) are included in every slice’s generated inventory, avoiding sliced-run failures when plays depend on a coordinating host.

Changes:

  • Adds job_slice_pinned_hosts (template + job) and threads it through inventory slicing, fact-cache host selection, task inventory build, and sliced workflow relaunch validation.
  • Updates UI to configure and display pinned hosts on job templates (plus help text), and documents the behavior in docs/job_slicing.md.
  • Adds functional coverage for pinned slicing behavior (distribution, groups, unknown hostnames, effective slice capping, relaunch) and a UI test for conditional form rendering.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
docs/job_slicing.md Documents pinned hosts semantics and caveats for sliced jobs.
awx/ui/src/screens/Template/shared/JobTemplateForm.test.js Adds UI test asserting the pinned-hosts input appears only when slicing is enabled.
awx/ui/src/screens/Template/shared/JobTemplateForm.js Adds conditional pinned-hosts input to the Job Template form and wires initial form value.
awx/ui/src/screens/Template/shared/JobTemplate.helptext.js Adds tooltip/help text for the pinned-hosts field.
awx/ui/src/screens/Template/JobTemplateDetail/JobTemplateDetail.js Displays pinned-hosts value in template details.
awx/main/tests/functional/models/test_job.py Adds functional tests for effective slice count and pinned-host inheritance/fact-cache alignment.
awx/main/tests/functional/models/test_inventory.py Adds functional tests for pinned slicing distribution, group membership preservation, and unknown-name handling.
awx/main/tests/functional/api/test_workflow_job.py Adds functional test ensuring sliced workflow relaunch validation works with pinned-aware effective slice count.
awx/main/tasks/jobs.py Passes pinned-hosts list into inventory script params during slice inventory build.
awx/main/models/jobs.py Adds model field + parsing helper, pinned-aware effective slice capping, pinned-aware fact-cache host selection, and task-impact adjustment.
awx/main/models/inventory.py Extends slice logic to include pinned hosts in every slice; threads pinned list into get_script_data.
awx/main/migrations/0201_job_slice_pinned_hosts.py Migration adding job_slice_pinned_hosts to Job + JobTemplate.
awx/api/views/init.py Updates sliced workflow relaunch validation to use pinned-aware effective slice count.
awx/api/serializers.py Exposes job_slice_pinned_hosts via JobOptionsSerializer.

Comment thread awx/main/models/jobs.py
Comment on lines +784 to +789
pinned_ct = len(self.job_slice_pinned_hosts_list)
count_hosts = max(count_hosts - pinned_ct, 0)
# Integer division intentional
count_hosts = (count_hosts + self.job_slice_count - self.job_slice_number) // self.job_slice_count
# pinned hosts run in every slice on top of its share
count_hosts += pinned_ct
id="template-job-slice-pinned-hosts"
name="job_slice_pinned_hosts"
type="text"
label={t`Job slice pinned hosts`}
Comment on lines +315 to +317
const pinnedInput = await waitFor(() =>
document.getElementById('template-job-slice-pinned-hosts')
);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Development

Successfully merging this pull request may close these issues.

3 participants