Allow pinning hosts to every slice of a sliced job#565
Open
fernandorocagonzalez wants to merge 1 commit into
Open
Allow pinning hosts to every slice of a sliced job#565fernandorocagonzalez wants to merge 1 commit into
fernandorocagonzalez wants to merge 1 commit into
Conversation
Job slicing distributes the inventory evenly across slices, which breaks playbooks where a play targets a host the rest of the run depends on. The usual victim is a preparatory play against localhost: after slicing, only the slice that happened to receive localhost runs it, and the other slices miss whatever it set up. This adds a job_slice_pinned_hosts field to job templates: a comma separated list of inventory host names that are taken out of the round robin distribution and included in every slice instead, keeping their group memberships and variables. Names that do not match an inventory host are ignored and the field does nothing when the job is not sliced. Pinned hosts do not add to the work worth distributing, so they no longer count when capping the slice count to the number of hosts: an inventory of 4 hosts with one of them pinned yields at most 3 slices. The docs spell out the tradeoff: plays targeting a pinned host run once per slice, concurrently, so this is meant for idempotent coordination plays (localhost being the typical case).
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds support for “pinned hosts” in job slicing so specific inventory hosts (e.g., localhost) are included in every slice’s generated inventory, avoiding sliced-run failures when plays depend on a coordinating host.
Changes:
- Adds
job_slice_pinned_hosts(template + job) and threads it through inventory slicing, fact-cache host selection, task inventory build, and sliced workflow relaunch validation. - Updates UI to configure and display pinned hosts on job templates (plus help text), and documents the behavior in
docs/job_slicing.md. - Adds functional coverage for pinned slicing behavior (distribution, groups, unknown hostnames, effective slice capping, relaunch) and a UI test for conditional form rendering.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/job_slicing.md | Documents pinned hosts semantics and caveats for sliced jobs. |
| awx/ui/src/screens/Template/shared/JobTemplateForm.test.js | Adds UI test asserting the pinned-hosts input appears only when slicing is enabled. |
| awx/ui/src/screens/Template/shared/JobTemplateForm.js | Adds conditional pinned-hosts input to the Job Template form and wires initial form value. |
| awx/ui/src/screens/Template/shared/JobTemplate.helptext.js | Adds tooltip/help text for the pinned-hosts field. |
| awx/ui/src/screens/Template/JobTemplateDetail/JobTemplateDetail.js | Displays pinned-hosts value in template details. |
| awx/main/tests/functional/models/test_job.py | Adds functional tests for effective slice count and pinned-host inheritance/fact-cache alignment. |
| awx/main/tests/functional/models/test_inventory.py | Adds functional tests for pinned slicing distribution, group membership preservation, and unknown-name handling. |
| awx/main/tests/functional/api/test_workflow_job.py | Adds functional test ensuring sliced workflow relaunch validation works with pinned-aware effective slice count. |
| awx/main/tasks/jobs.py | Passes pinned-hosts list into inventory script params during slice inventory build. |
| awx/main/models/jobs.py | Adds model field + parsing helper, pinned-aware effective slice capping, pinned-aware fact-cache host selection, and task-impact adjustment. |
| awx/main/models/inventory.py | Extends slice logic to include pinned hosts in every slice; threads pinned list into get_script_data. |
| awx/main/migrations/0201_job_slice_pinned_hosts.py | Migration adding job_slice_pinned_hosts to Job + JobTemplate. |
| awx/api/views/init.py | Updates sliced workflow relaunch validation to use pinned-aware effective slice count. |
| awx/api/serializers.py | Exposes job_slice_pinned_hosts via JobOptionsSerializer. |
Comment on lines
+784
to
+789
| pinned_ct = len(self.job_slice_pinned_hosts_list) | ||
| count_hosts = max(count_hosts - pinned_ct, 0) | ||
| # Integer division intentional | ||
| count_hosts = (count_hosts + self.job_slice_count - self.job_slice_number) // self.job_slice_count | ||
| # pinned hosts run in every slice on top of its share | ||
| count_hosts += pinned_ct |
| id="template-job-slice-pinned-hosts" | ||
| name="job_slice_pinned_hosts" | ||
| type="text" | ||
| label={t`Job slice pinned hosts`} |
Comment on lines
+315
to
+317
| const pinnedInput = await waitFor(() => | ||
| document.getElementById('template-job-slice-pinned-hosts') | ||
| ); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Job slicing distributes the inventory evenly across the slices, which breaks any playbook where a play targets a host the rest of the run depends on. The classic case is a preparatory play on
localhostthat gathers data or builds configuration the other hosts consume. Once sliced, only the slice that happened to receivelocalhostruns that play, and the remaining slices fail or do the wrong thing. Today the options are to not slice those playbooks at all, or to rewrite them so no play depends on another host.What this adds
A new Job Template field,
job_slice_pinned_hosts: a comma separated list of inventory host names that are taken out of the round robin distribution and included in every slice instead, keeping their group memberships and variables.The goal here is specifically to fix that sliced jobs problem, and the scope is deliberately narrow: the field only guarantees that the named hosts are part of every slice's inventory. It does not decide what runs where. That control stays in the playbook, as usual: each play's
hosts:pattern determines what actually executes on which host, so a pinned host only runs the plays that target it and the fleet plays keep running on each slice's share of the inventory.Matching is by exact host name (groups and
limitstyle patterns are not supported), names that do not exist in the inventory are ignored, and the field does nothing when the job is not sliced. Since pinned hosts are repeated in every slice they are not counted when capping the slice count to the number of hosts: an inventory of 4 hosts with 1 pinned yields at most 3 slices. The docs spell out the main caveat: plays targeting a pinned host run once per slice, concurrently, so this is meant for idempotent coordination plays, andrun_oncebecomes once per slice.Implementation notes
JobOptions, so it exists on both the template and the job and is copied automatically at launch time.Inventory.get_sliced_hostsexcludes the pinned names from the round robin and adds them to each slice;get_script_dataand the task inventory build pass the list down.get_effective_slice_ctsubtracts pinned hosts from the distributable host count. The sliced workflow relaunch check now callsget_effective_slice_cttoo, instead of re derivingmin(hosts, slice_count)on its own, so relaunching keeps working when pinned hosts reduce the node count.get_hosts_for_fact_cacheuses the same pinned aware slicing, keeping the per slice fact cache host set aligned with the inventory the slice actually ran against._get_task_impactaccounts for pinned hosts running in every slice.docs/job_slicing.md.UI
A "Job slice pinned hosts" text field in the job template form, shown only when Job Slicing is greater than 1, plus the corresponding entry in the details view.
Tests and verification
localhostcoordination play plus an assert on the fleet play thatlocalhostis present in the slice. Without pinning, the workflow fails on the slice that did not receivelocalhost. Withjob_slice_pinned_hosts=localhost, every slice carrieslocalhostplus its share of hosts and the whole workflow is green, and relaunching the sliced workflow works.Heads up on the migration: this adds
0201_job_slice_pinned_hosts, which will collide in numbering with the 0201 migrations in #561/#563/#564 if any of those merges first; happy to renumber whenever needed.