feat(ci): implement three-layer cook pattern with torrust-cargo-chef --external-only by josecelano · Pull Request #1895 · torrust/torrust-tracker

josecelano · 2026-06-09T18:16:01Z

Summary

Implements the three-layer cook pattern using the torrust-cargo-chef fork (v0.1.78-torrust) with --external-only support, resolving the dependency-layer cache reuse issue.

Changes

Containerfile (`Containerfile`)

Switched from cargo-chef to torrust-cargo-chef@0.1.78-torrust (temporary fork with --external-only support; upstream PR #360 pending)
Dual recipe generation: recipe.json (full) + recipe-thirdparty.json (external-only)
New layers: dependencies_thirdparty and dependencies_thirdparty_debug — cook only third-party crates, immune to workspace Cargo.toml changes
Stacked: dependencies/dependencies_debug now inherit from the third-party layers, reusing cached artifacts when only workspace manifests change

Spec

docs/issues/open/1869-.../ISSUE.md — Updated with --external-only findings, marked T1/T2 DONE, T3 DONE
docs/issues/drafts/1840-.../ISSUE.md — Marked as superseded (resolved by --external-only)
docs/issues/open/1840-...epic/EPIC.md — Updated subissue row 10 to SUPERSEDED

Related Issues

Closes Improve dependency-layer cache reuse within each workflow #1869
Part of EPIC EPIC: Improve PR Workflow Performance #1840
Supersedes draft investigation in docs/issues/drafts/1840-workflow-performance-split-external-dep-cache-layer/

Verification

linter all passes
cargo test --doc --workspace passes
cargo machete passes
Manual verification: local container build with torrust-cargo-chef (pending)

…sede draft - Add --external-only flag update section (torrust-cargo-chef fork) - Mark T1/T2 as DONE (confirmed by draft analysis and fork) - Convert T3 from "Propose" to "Implement" with three-layer cook pattern - Add T5 cleanup task to remove superseded draft folder - Add M3/M4/M5 verification scenarios for the new binary and perf testing - Mark split-external-dep-cache-layer draft as superseded - Update EPIC row 10 to SUPERSEDED

…--external-only Replace cargo-chef with the torrust-cargo-chef fork (v0.1.78-torrust) that supports --external-only. The Containerfile now generates both a full recipe and a third-party-only recipe. Two new intermediate Docker layers (dependencies_thirdparty and dependencies_thirdparty_debug) cook only external crates, making the third-party layer immune to workspace Cargo.toml changes. The existing dependencies stages now inherit from these third-party layers, reusing cached artifacts when only workspace manifests change. See upstream PR LukeMathWalker/cargo-chef#360.

Copilot

Pull request overview

This PR updates the repository’s container build to improve Docker layer cache reuse for Rust dependencies by adopting a three-layer “cook” approach (third-party-only cook → full cook → build), leveraging a temporary torrust-cargo-chef fork with --external-only support. It also updates the related workflow-performance specs/EPIC documentation to reflect the new approach and mark the prior draft investigation as superseded.

Changes:

Switch Containerfile tooling from upstream cargo-chef to torrust-cargo-chef@0.1.78-torrust, generating both full and third-party-only recipes and adding new third-party cook layers.
Update issue/EPIC specs to record the --external-only findings and mark the earlier draft investigation as superseded.
Add thirdparty to project-words.txt.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`Containerfile`	Implements dual recipe generation and adds third-party-only cook layers to improve dependency-layer cache reuse.
`docs/issues/open/1869-1840-workflow-performance-dependency-layer-cache-reuse/ISSUE.md`	Documents the `--external-only` approach and updates task statuses/progress notes.
`docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md`	Marks the split-cook-layer draft investigation as superseded in the EPIC subissue table and updates the log.
`docs/issues/drafts/1840-workflow-performance-split-external-dep-cache-layer/ISSUE.md`	Adds a superseded banner and updates frontmatter/date to reflect resolution by the new approach.
`project-words.txt`	Adds `thirdparty` to the spellchecker allowlist.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

codecov · 2026-06-09T18:37:01Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.15%. Comparing base (4cafafe) to head (4a424d9).

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #1895   +/-   ##
========================================
  Coverage    79.15%   79.15%           
========================================
  Files          327      327           
  Lines        22836    22836           
  Branches     22836    22836           
========================================
  Hits         18075    18075           
  Misses        4503     4503           
  Partials       258      258

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The crates.io version is 0.1.78 (without -torrust suffix). The -torrust suffix is only the git tag. Also update version references in the spec and add PR torrust#1895 reference to T3 notes. Manual verification: release container build succeeded (693/693 tests passed). Image: torrust-tracker:pr1895 (173MB).

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Resolves all 6 Copilot review suggestions: 1. Containerfile L187: softened comment about external-only recipe stability — now correctly notes it's immune to workspace path changes but still sensitive to external dep metadata/feature flags. 2. EPIC L81: removed 'torrust#1869' label from row 10 (draft) — uses '#[To be assigned]' with note it's superseded by torrust#1869. 3. EPIC frontmatter: bumped last-updated-utc to 2026-06-09. 4. Draft frontmatter: changed status from 'superseded' to 'done' (allowed value per skill convention). 5. Draft banner: tied removal to torrust#1869 closure (not just impl). 6. M2 scenario wording: constrained to workspace-only changes that don't alter the external dependency graph. Manual verification (M1/M5): - Made app-code-only change to src/lib.rs, rebuilt targeting release. - All dependencies_thirdparty layers confirmed CACHED (COPY recipe-thirdparty.json CACHED, cargo chef cook CACHED). - Full dependencies and build stages also fully cached.

Mark AC1, AC2, AC3 as DONE with evidence from local testing. Mark M1 and M5 as DONE (third-party layer caching confirmed).

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

…rding - EPIC row 7 (torrust#1869): updated from TODO to DONE with implementation notes. - M1 expected result: clarified that full dependencies/build stages can also be fully cached when only .rs files change (recipe.json unchanged). The third-party layer provides isolation specifically when workspace Cargo.toml files change. - EPIC progress log: added entry for row 7 status update.

josecelano · 2026-06-09T20:27:24Z

All review comments addressed in commits f7b8e1af and 7cbd4e3d. Summary:

Containerfile comment (x2): softened — immune to workspace path changes, still sensitive to external dep metadata/feature flags.
Spec version reference: consistent (crates.io v0.1.78, git tag v0.1.78-torrust).
EPIC row 10: changed to '#[To be assigned]' with superseded-by note.
EPIC row 7 (Improve dependency-layer cache reuse within each workflow #1869): updated from TODO to DONE with implementation evidence.
Draft frontmatter: status 'done' (allowed value per convention; SUPERSEDED retained in body).
Draft banner: removal tied to Improve dependency-layer cache reuse within each workflow #1869 closure (not just implementation).
M2 wording: constrained to workspace-only changes not affecting external dep graph.
M1 expected result: clarified that full dependencies/build may also be cached when recipe.json unchanged.
EPIC last-updated-utc: bumped to 2026-06-09.
Manual verification: M1, M3, M4, M5 all DONE (third-party layer CACHED, 693/693 tests).

josecelano · 2026-06-09T21:00:27Z

ACK 4a424d9

Move issue specs that have been closed on GitHub: - torrust#1869: dependency-layer cache reuse (PR torrust#1895) - torrust#1882: extract torrust-metrics to standalone repo (PR torrust#1892) - torrust#1884: move bittorrent-peer-id to torrust-bittorrent (PR torrust#1887) - torrust#1885: extract torrust-net-primitives to standalone repo (PR torrust#1893) - torrust#1889: migrate from bittorrent-primitives to torrust-info-hash (PR torrust#1891) - torrust#1894: extract torrust-located-error to standalone repo (PR torrust#1897) - torrust#1898: document security analysis process (PR torrust#1899) Update frontmatter headers: status -> done, spec-path -> closed/, add missing related-pr and branch fields, bump last-updated-utc.

…ame validation guidance c5f6add fixup: address Copilot PR review comments on branch-name validation (Jose Celano) c17de04 chore(docs): add branch-name validation guidance across skills, agents, and hook script (Jose Celano) 39b4b4c chore(docs): archive 7 closed issue specs from open/ to closed/ (Jose Celano) Pull request description: ## Summary Two independent changes on this branch: ### 1. Archive closed issue specs (commit 73263e9) Move 7 issue specs from `docs/issues/open/` to `docs/issues/closed/` that have been closed on GitHub, and update their frontmatter to reflect the closed state: | Issue | Title | PR | |-------|-------|----| | #1869 | Improve dependency-layer cache reuse within each workflow | #1895 | | #1882 | Extract `torrust-metrics` to standalone repo | #1892 | | #1884 | Move `bittorrent-peer-id` to `torrust/torrust-bittorrent` | #1887 | | #1885 | Extract `torrust-net-primitives` to standalone repo | #1893 | | #1889 | Migrate from `bittorrent-primitives` to `torrust-info-hash` | #1891 | | #1894 | Extract `torrust-located-error` to standalone repo | #1897 | | #1898 | Document security analysis process | #1899 | ### 2. Branch-name validation guidance (commit 756284f) Add validation guidance across the dev toolchain to prevent committing under a wrong, closed, or non-existent issue number: - **`create-feature-branch/SKILL.md`** — validation step before creating a branch - **`run-pre-commit-checks/SKILL.md`** — manual check for branch name validity - **`committer.agent.md`** — step 2 validates branch name against open specs - **`pre-commit.sh`** — TODO referencing #1843 for future Rust automation - **`1843-migrate-git-hooks-scripts-from-bash-to-rust.md`** — added T22 + AC22, cross-reference related artifacts ## Verification - [x] `cargo machete` — pass - [x] `linter all` — pass - [x] `cargo test --doc --workspace` — pass - [x] Pre-push hook — pass ACKs for top commit: josecelano: ACK c5f6add Tree-SHA512: 7b18e5963bfa4a6e2e53b47fe264847b17cffea3fd1b35361a56346be71399b135f6cea137d4716da192731d1907d637c24e7c195197653c2ebc777c3daa6cf8

josecelano added 2 commits June 9, 2026 19:02

Copilot AI review requested due to automatic review settings June 9, 2026 18:16

josecelano temporarily deployed to coverage June 9, 2026 18:16 — with GitHub Actions Inactive

Copilot started reviewing on behalf of josecelano June 9, 2026 18:16 View session

Copilot AI reviewed Jun 9, 2026

View reviewed changes

josecelano temporarily deployed to coverage June 9, 2026 18:56 — with GitHub Actions Inactive

josecelano self-assigned this Jun 9, 2026

josecelano added Continuous Integration Workflows and Automation - Developer - Torrust Improvement Experience Optimization Make it Faster Automation labels Jun 9, 2026

josecelano requested a review from Copilot June 9, 2026 19:06

Copilot started reviewing on behalf of josecelano June 9, 2026 19:06 View session

Copilot AI reviewed Jun 9, 2026

View reviewed changes

josecelano requested a review from Copilot June 9, 2026 19:27

Copilot started reviewing on behalf of josecelano June 9, 2026 19:28 View session

josecelano temporarily deployed to coverage June 9, 2026 19:28 — with GitHub Actions Inactive

docs(1869): update acceptance criteria with local verification evidence

a959482

Mark AC1, AC2, AC3 as DONE with evidence from local testing. Mark M1 and M5 as DONE (third-party layer caching confirmed).

josecelano temporarily deployed to coverage June 9, 2026 19:30 — with GitHub Actions Inactive

Copilot AI reviewed Jun 9, 2026

View reviewed changes

josecelano temporarily deployed to coverage June 9, 2026 19:34 — with GitHub Actions Inactive

chore: fmt trailing whitespace in EPIC and issue spec tables

4a424d9

josecelano temporarily deployed to coverage June 9, 2026 20:57 — with GitHub Actions Inactive

josecelano merged commit 6feef6e into torrust:develop Jun 10, 2026
17 checks passed

josecelano mentioned this pull request Jun 10, 2026

chore(docs): archive closed issue specs and add branch-name validation guidance #1901

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(ci): implement three-layer cook pattern with torrust-cargo-chef --external-only#1895

feat(ci): implement three-layer cook pattern with torrust-cargo-chef --external-only#1895
josecelano merged 7 commits into
torrust:developfrom
josecelano:1869-workflow-performance-dependency-layer-cache-reuse

josecelano commented Jun 9, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Jun 9, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

josecelano commented Jun 9, 2026

Uh oh!

josecelano commented Jun 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

josecelano commented Jun 9, 2026

Summary

Changes

Containerfile (Containerfile)

Spec

Related Issues

Verification

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

josecelano commented Jun 9, 2026

Uh oh!

josecelano commented Jun 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Containerfile (`Containerfile`)

codecov Bot commented Jun 9, 2026 •

edited

Loading