Migrate all references to the newer avocado.yaml structure (ENG-2182) - #23
Migrate all references to the newer avocado.yaml structure (ENG-2182)#23nicksinas wants to merge 9 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Migrates reference avocado.yaml files to the newer schema structure (removing deprecated keys, updating SDK image templating, and moving root-password configuration into top-level permissions profiles), and adds repo-wide tooling/ignores to support repeatable reference build verification without committing build state.
Changes:
- Update reference
avocado.yamlfiles to usepermissions+rootfs/initramfsand remove deprecatedcli_requirementandconfig/config-devconfext usage. - Standardize SDK image to
docker.io/avocadolinux/sdk:{{ config.distro.release }}. - Add a reference-by-reference build verification harness and expand
.gitignoreto exclude Avocado-generated state.
Reviewed changes
Copilot reviewed 31 out of 32 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| zephyr-imx8mp-evk/avocado.yaml | Removes deprecated keys and introduces top-level permissions + rootfs/initramfs; updates SDK image template. |
| webkit-ui/avocado.yaml | Moves root password config into permissions and updates SDK image template; removes config-dev runtime reference. |
| shell-heartbeat/avocado.yaml | Drops config confext usage, adds permissions + rootfs/initramfs, updates SDK image template. |
| scripts/build-check-references.sh | New harness to clean/unlock/install/build each reference and produce a live markdown report. |
| rust-vitals/avocado.yaml | Removes config-dev confext and adds permissions + rootfs/initramfs; updates SDK image template. |
| rubicon/avocado.yaml | Removes config-dev confext and adds permissions + rootfs/initramfs; updates SDK image template. |
| ros2-ufactory-lite6/avocado.yaml | Removes deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| react-dashboard/avocado.yaml | Removes config-dev runtime refs across targets; adds permissions + rootfs/initramfs; updates SDK image template. |
| qemu-quickstart/avocado.yaml | Drops config runtime/confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| python-yolo/avocado.yaml | Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| python-whisper/avocado.yaml | Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| python-multiversion-uv/avocado.yaml | Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| python-mqtt/avocado.yaml | Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| python-flask/avocado.yaml | Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| pi-metrics-exporter/avocado.yaml | Removes config-dev confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| nvidia-gstreamer-yolo/avocado.yaml | Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| nvidia-deepstream/avocado.yaml | Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| nodejs-dashboard/avocado.yaml | Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| linux-custom-kernel/avocado.yaml | Removes config-dev runtime/confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| java-hello/avocado.yaml | Removes config-dev confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| iphone-travel-router/avocado.yaml | Moves root password config into permissions while preserving non-empty password; updates SDK image template. |
| imx8mp-npu-pose/avocado.yaml | Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| imx8mp-npu-nnstreamer/avocado.yaml | Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| icam-540/avocado.yaml | Updates default target/board mapping and migrates to permissions + rootfs/initramfs; updates SDK image template. |
| elixir-phoenix/avocado.yaml | Removes config-dev runtime/confext and a now-empty confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| docker-save/avocado.yaml | Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| docker-registry/avocado.yaml | Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| dev/avocado.yaml | Removes config-dev confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| cpp-tui-dashboard/avocado.yaml | Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| c-gpio/avocado.yaml | Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template. |
| astra-1680-deepx/avocado.yaml | Migrates to permissions + rootfs/initramfs; updates SDK image template; also removes a local distro repo override. |
| .gitignore | Adds repo-wide ignores for Avocado lock/state files. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
jetm
left a comment
There was a problem hiding this comment.
Reviewed the avocado.yaml schema migration. It's clean and consistent across all references: no dangling config/config-dev extension references, the SDK image tag change (dropping the channel) is applied uniformly, and every reference got the new permissions: block. One minor cross-file consistency nit inline on icam-540.
| @@ -1,6 +1,8 @@ | |||
| default_target: icam-540 | |||
| default_target: jetson-orin-nx | |||
There was a problem hiding this comment.
icam-540/README.md frontmatter target is now stale. This renames the target to jetson-orin-nx (with icam-540 moved to default_target_board), but icam-540/README.md:4 frontmatter still lists `targets:
- icam-540
. Every other reference README uses the target slug (matching itsdefault_target), so a catalog keyed on target slug files this one undericam-540, which is no longer a valid target. Suggest updating the README frontmatter tojetson-orin-nx`.
There was a problem hiding this comment.
This is a new change on the backend. This particular target is now used as documented this reference. A docs update will accompany this. This is apart of the bigger change towards target and target_board. This is already documented in docs in the supported hardware table, but a more detailed page will be added to docs regarding this differentiation.
Update all 30 references to the current avocado.yaml layout:
- Remove the deprecated cli_requirement key
- Point the SDK image at docker.io/avocadolinux/sdk:{{ config.distro.release }}
(drop the -channel suffix; use the config.* namespace)
- Move the root password out of the config/config-dev confext into a
top-level permissions profile
- Add top-level rootfs and initramfs keys referencing that profile
- Drop the now-empty config/config-dev extension and its runtime references
icam-540 additionally retargets to jetson-orin-nx (its underlying SoM,
with default_target_board: icam-540), which resolves in the package feed.
Add top-level .gitignore rules so per-build generated files are never committed from any reference: - avocado.lock - .avocado-desktop.json - .avocado/ - .avocado-state Previously only 1 of 30 references ignored these locally.
Reference-by-reference build check: for each reference runs clean/unlock/install/build, then tears down to a clean git state and strips build artifacts. Writes a live markdown report after each step (scripts/build-report.md), continues past failures, and prints a PASS/FAIL summary.
- elixir-phoenix: restore the avocado-ext-connect-config extension that `avocado connect clean` stripped from avocado.yaml during a verification run (unintended side effect, not part of the migration). Verified the reference still builds with it in place. - build-check-references.sh: snapshot avocado.yaml per reference and restore it after teardown so `avocado connect clean` can no longer leak edits into tracked config; fix the REPORT_FILE default doc to match the code (scripts/build-report.md).
The reference shipped without the spec-required docs. Add a frontmatter README (language / targets / topics) and a full getting_started guide (prerequisites → install → build → deploy → verify → customize), sourced from the reference's avocado.yaml and files/ scripts.
Bring the three remaining references into compliance with the docs spec (spec_readme.md / spec_getting_started.md): - imx8mp-npu-nnstreamer, imx8mp-npu-pose: prepend README frontmatter (language / targets / topics); README bodies unchanged. - zephyr-imx8mp-evk: prepend README frontmatter and add the missing getting_started.md. All references now pass the spec (frontmatter README + getting_started.md).
Teardown previously ran `git clean -fdx`, which deletes ALL untracked files under a reference (including a developer's local work-in-progress). Switch to `-X` (ignored-only): build state is fully gitignored so it's still removed, while untracked non-ignored files are left in place and flagged by the residual-changes check. Addresses PR review feedback.
- getting_started.md (dev, rubicon, webkit-ui, qemu-quickstart): drop the `- config` / `- config-dev` line from the "add/remove extensions" examples. The migration deleted those extensions, so the copy-paste blocks pointed at a non-existent extension. - build-check-references.sh: the terminal summary piped the loop to `tee`, running it in a subshell so the pass/fail counters were lost (always printed PASS: 0 FAIL: 0). Redirect to the summary file and `cat` it, keeping the counters in the parent shell. The markdown report totals were already correct.
jetm
left a comment
There was a problem hiding this comment.
Reviewed cold at 7cf4106. The restructure is mechanically clean, and I want to say that first because most of what follows is edge cases: all 30 avocado.yaml files parse, all 30 carry rootfs: + initramfs: + a matching permissions.dev profile, every runtime extension name resolves, there are no config/config-dev/cli_requirement leftovers in YAML, no duplicate top-level keys, and the SDK image line is byte-identical across all 30 and matches the CLI's own scaffold (avocado-cli/configs/default.yaml:93). default_target_board is correct for CLI 0.37.0+ and keeps avocado-bsp-{{ avocado.target.board }} resolving to avocado-bsp-icam-540. The summary-counter fix in 7cf4106 is genuinely correct - a redirected for ... done > file does not subshell in bash, which I checked rather than assumed.
Five findings are inline. Separately, I independently landed on the two issues already raised in open threads here, and traced up some extra evidence worth having on each:
On the cli_requirement removal thread. Agreed, and it is the most serious thing in the PR. Two details that sharpen it. cli_requirement is not deprecated - it is enforced on every config load (avocado-cli/src/utils/config.rs:1415, called at :1732 and :3441), and the CLI's own avocado init template still emits cli_requirement: ">=0.41.0" (configs/default.yaml:1, asserted by src/commands/init.rs:1149). So if the PR body describes the key as deprecated, that premise is wrong. And the breakage is actually two distinct windows, not one: on CLI 0.26-0.35 the new blocks are silently dropped (no deny_unknown_fields anywhere in config.rs at any version), the build reports success, and the image ships root:*: in /etc/shadow so nobody can log in - but on 0.36-0.40.2 it is a hard error instead, because permissions is missing from the rootfs singleton whitelist at that version (git show 0.40.2:src/utils/config.rs:148-159), so rootfs: {permissions: dev} takes the named-map branch and dies with invalid type: string ... expected struct ImageConfig. Cryptic, but at least loud. Restoring cli_requirement: ">=0.41.0" across all 30 covers both.
On the icam-540 retarget thread. Agreed on the README/spec mismatch. Two corrections to the rationale, though, if it is going in the commit message: the icam-540 package feed does still resolve (2024/edge/target/icam-540/repodata/repomd.xml returns 200, revision 2026-05-01), it is just about two months staler than jetson-orin-nx at 2026-07-08 - so "stale" is accurate and "does not resolve" would not be. And I could not confirm avocado-bsp-icam-540 is installable against the jetson-orin-nx feed; no avocado-bsp-* packages showed up in any feed I could enumerate, so that one wants an actual avocado install against the reference rather than a reasoned argument.
Three more findings that I could not anchor inline, two of them because the files are not in this diff:
docker-registry/getting_started.md:78andnvidia-deepstream/getting_started.md:99still say "the defaultconfigextension sets an empty root password for development". 7cf4106 swept the YAML-snippet form of this out of thedev,qemu-quickstart,rubicon, andwebkit-uidocs but missed the prose form in these two. A reader grepsavocado.yamlforconfig, finds nothing, and cannot work out where the empty root password comes from.astra-1680-deepx/getting_started.md:93has the wording you want.docker-save/getting_started.md:221anddocker-registry/getting_started.md:367both still rundocker.io/avocadolinux/sdk:2024-edgewhile their ownavocado.yamlnow resolves todocker.io/avocadolinux/sdk:2024. The config change is right (it matchesconfigs/default.yaml:93, anddocs/.../0.41.0.md:94documents dropping the channel suffix from the generated tag;distro.channelstays load-bearing because it feeds the dnf$releasever). Both tags currently resolve to the same digest, so nothing breaks - the docs are just stale.- Worth one sentence somewhere: moving the root password out of a confext and into the rootfs (
avocado-cli/src/commands/rootfs/image.rs:309-322edits$ROOTFS_WORK/etcin place pre-mkfs) means changing that password is no longer a ~KB confext delta shipped viaavocado deploy- it is now part of the full.aosOS bundle. Still OTA-updatable, just a much bigger payload. No reference's getting-started doc mentions this, and the "Customize -> Permissions" bullets imply the old cheap path.
Traced and deliberately dropped, so nobody re-audits them: the iphone-travel-router password: "avocado" divergence (deliberate, documented in-file for Cockpit PAM); the removed distro.repo.url: http://localhost:8080 in astra-1680-deepx (that is a fix); the {{ avocado.distro.* }} to {{ config.distro.release }} switch (both namespaces are valid, and the config.* form fails loudly on a missing key, which is better); the dead-but-defined extensions deepx-app, htop, avocado-ext-connect (all pre-existing); and the redundancy between the new root .gitignore and the 28 per-reference ones that already carry .avocado/ and .avocado-state.
|
|
||
| # Avocado build state — generated per build, never commit these. | ||
| # Matches in every reference directory (patterns without a slash apply recursively). | ||
| avocado.lock |
There was a problem hiding this comment.
Gitignoring avocado.lock contradicts the CLI's tracked-lockfile design. avocado-cli/src/commands/init.rs:851-853 says explicitly: "avocado.lock is intentionally NOT ignored - it's a tracked, committed pin file (like Cargo.lock)", and init.rs:889 writes the literal comment # Avocado scratch + state (avocado.lock is tracked) into every generated .gitignore. docs/src/docs-guides/lockfiles-and-build-stamps.md:17 sells those pins as the reason "two team members ... get byte-identical sysroots". scripts/build-check-references.sh:146 reinforces the ignore with rm -f avocado.lock.
Effect: no reference ever commits its pins. Two users running avocado init --reference python-yolo a month apart resolve different package versions off the edge feed and get different images, and a reference that built last week can silently stop building when an upstream package moves.
If this is a deliberate policy for a references repo (arguably reasonable - you may want references to track the feed), the comment above should say so. As written, "generated per build, never commit these" asserts the opposite of the CLI's documented contract, so it reads as a mistake rather than a decision.
| avocado.lock | ||
| .avocado-desktop.json | ||
| .avocado/ | ||
| .avocado-state No newline at end of file |
There was a problem hiding this comment.
This .avocado-state rule is a no-op for the two files that actually violate it. docker-registry/.avocado-state and docker-save/.avocado-state are both tracked, and gitignore never affects tracked files. They currently hold another developer's local state: a source_path of /Users/nick/work/repos/avocado/references/docker-registry plus a docker volume UUID.
Root cause of the original leak: docker-registry/.gitignore:2 and docker-save/.gitignore:2 use .avocado-state/ (trailing slash, directory-only) while the other 28 references use the correct .avocado-state. And scripts/build-check-references.sh:151-155 explicitly restores both files with a checkout after teardown, cementing them.
So anyone who clones and builds docker-registry hands the CLI a state file naming a docker volume and a /Users/nick/... path that does not exist locally. Fixing this needs an index removal (--cached) on both files plus the trailing-slash fix in those two per-reference ignores; this root rule alone changes nothing for them.
| --- | ||
| language: C++ | ||
| targets: | ||
| - grinn-astra-1680-sbc |
There was a problem hiding this comment.
Three new README frontmatter blocks use target ids the repo spec rejects. This one (grinn-astra-1680-sbc), plus imx8mp-npu-nnstreamer/README.md:4 and imx8mp-npu-pose/README.md:4 (both ucm-imx8m-plus). None of the three appears in spec_readme.md's "Valid target identifiers" table, and that spec's Validation item 3 requires targets to be "*" or a list of valid identifiers.
So the docs build either rejects these three references at parse time or silently drops them from the index. The fix is one of two things, and I can't tell which you intend: add the three boards to the spec table (they're real hardware - they're the supported_targets in these same files), or the frontmatter is wrong. The fourth new block, zephyr-imx8mp-evk/README.md, uses imx8mp-evk and is fine.
| out="$(avocado config show 2>/dev/null)" | ||
| dt="$(printf '%s\n' "$out" | sed -n 's/^[[:space:]]*default_target:[[:space:]]*//p' | tr -d '"' | head -1)" | ||
| if [ -n "$dt" ] && [ "$dt" != "*" ]; then printf '%s' "$dt"; return; fi | ||
| st="$(printf '%s\n' "$out" | sed -n 's/.*supported_targets:[[:space:]]*\[\([^]]*\)\].*/\1/p' \ |
There was a problem hiding this comment.
resolve_target's supported_targets fallback never matches, so it silently builds qemuarm64. This sed only matches a YAML flow sequence (supported_targets: [a, b]), but every reference declares a block list, so st comes back empty and line 126 falls through to the hardcoded qemuarm64.
Concretely this fires for ros2-ufactory-lite6, whose default_target is "*" and whose supported targets are imx8mp-evk, imx91-frdm, imx93-evk, imx93-frdm, raspberrypi5 - none of them qemuarm64. The sweep then either reports FAIL for a target the reference never claimed, or PASS on one that proves nothing. Silent either way: the report's Target column shows qemuarm64 with no indication it was a guess.
| # | ||
| # Environment: | ||
| # LOG_DIR directory for per-reference logs (default: a fresh mktemp dir) | ||
| # REPORT_FILE markdown report path (default: $REPO_ROOT/scripts/build-report.md, |
There was a problem hiding this comment.
Two smaller issues in this script's reporting. First, this header calls $REPO_ROOT/scripts/build-report.md "the tracked master report", but no such file is tracked at head and it isn't gitignored either. A default run therefore leaves an untracked file that the -fdX clean at line 150 won't remove and the per-reference residual check (line 209, scoped -- "$ref") won't flag, so the user is told the tree is clean when it isn't.
Second, line 209's grep -vE '^ M .*/avocado.yaml$' only matches unstaged modifications. Porcelain emits M (staged) when the edits are added to the index, so a maintainer who stages the config edits before running the sweep gets "residual git changes" on all 30 references and quickly learns to ignore the warning.
jetson-trt landed on main after the ENG-2182 migration and still carried `cli_requirement: ">=0.41.0"`. Remove it so no reference pins a CLI floor (consistent with the other 30). The reference was otherwise already in the new structure (top-level permissions/rootfs/initramfs, config.* SDK image).
7cf4106 to
5998b65
Compare
What
Migrates all 30 references to the current
avocado.yamlstructure (ENG-2182).Per-file changes:
cli_requirementkeydocker.io/avocadolinux/sdk:{{ config.distro.release }}(drop the-channelsuffix; use theconfig.*namespace)config/config-devconfext into a top-levelpermissionsprofilerootfsandinitramfskeys referencing that profileconfig/config-devextension and its runtime referencesOverlays, packages, services,
on_merge/on_unmerge,docker_images, SDK args, and all real extensions are untouched.iphone-travel-routerkeeps its non-emptyavocadopassword (Cockpit/PAM).icam-540additionally retargets tojetson-orin-nx(its underlying SoM,default_target_board: icam-540) so it resolves in the feed.Also included:
.gitignore: repo-wide rules so per-build state is never committed (avocado.lock,.avocado-desktop.json,.avocado/,.avocado-state) — previously only 1 of 30 references ignored these.scripts/build-check-references.sh: a reference-by-reference build harness (clean→unlock→install→build→teardown-to-clean-git-state), with a live markdown report and PASS/FAIL summary.Build verification
Every reference was run through
install+buildvia the new harness. Each migrated file also validates against the avocado schema (ok:true).24 / 30 build clean. The remaining failures were investigated and are not caused by this migration — they fail either before the migrated sections are consumed (feed fetch) or inside code compilation (host toolchain), and 24 references using the identical migration pattern build green:
avocado-ext-devmissing from feed forgrinn-astra-1680-sbcavocado-ext-devmissing from feed forucm-imx8m-plusavocado-ext-devmissing from feed forucm-imx8m-plusHOSTCC= target cross-compiler → host tool can't run when cross-building x86 kernel on an arm64 host (builds on x86_64 hosts)avocado-ext-clipackage build exits non-zero after compiling + installing the CLIc-gpioinitially failed on a transient clock-skew flake and passes on a clean rebuild.zephyr-imx8mp-evkwas still building at PR time (Zephyr pulls dozens of module repos).Follow-ups for maintainers (not blocking this PR): publish
avocado-ext-devforgrinn-astra-1680-sbcanducm-imx8m-plusto2024/edge; investigate theavocado-ext-clipackage build (CLI1.0.0-rc.1).