Add the backfill census: changelog backfill inventory - #3764
Conversation
products.yml alone cannot say which products have release-note surfaces because the release-notes feature defaults to enabled. The census enumerates every release-notes product, merges the hand-maintained source seed, and writes the versioned inventory document planning consumes. Products the seed does not cover stay visible as source-unresolved entries with a warning, so "we looked and decided no" is always distinguishable from "we never looked" and an unresolved scope can never silently produce empty bundles. Stack-versioned semver sources default to the epic's 9.0.0 cutoff; unresolved products get a scheme derived from their versioning system, always paired with a note so a guess never reads as confirmed fact. Attributed repositories are checked against the local assembler.yml allowlist (planning re-validates against the deployed identity). The inventory contract now allows a null source repository/ref for source-unresolved entries only — the census records products whose location is unknown, which the previous required fields could not express. Part of elastic/docs-eng-team#673 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mpdreamz
left a comment
There was a problem hiding this comment.
Request changes: census/inventory is the wrong shape for a one-off
Same pushback as on #3681. Stacking a seed→enveloped-inventory compiler on the six-family contracts still does not move bundles to S3, and it does not discover what is published on elastic.co/docs/release-notes.
For a temporary changelog migrate-from-web (fetch published notes → existing bundle YAML → create-only S3 upload → print report → delete the command), we do not need an inventory artifact family, census service, or content-addressed output. A product/cutoff list as plain config (or CLI flags) is enough.
Paper trail, if any: one run report on the issue/PR — not a persisted inventory schema. Please retarget with #3681 toward that one-off path instead of growing the backfill document platform.
|
Closing per the re-scope agreed on #3681: scope and cutoff selection become CLI flags plus a small checked-in config list inside |
Why
Backfill planning needs a full census before any selective run:
config/products.ymlcannot identify release-note surfaces on its own because therelease-notesfeature defaults to enabled, and only products with published history or native artifacts should produce backfill actions — while everything else must stay visible so an unresolved scope can never silently produce empty bundles.Part of elastic/docs-eng-team#673 (sub-issue of the backfill epic elastic/docs-eng-team#656). Stacked on #3681 — the census writes the v1 inventory contract that PR introduces.
What
A new
docs-builder changelog backfill inventorycommand (read-only apart from the local output file) backed byInventoryCensusService:products.ymland merges a hand-maintained census seed (YAML) mapping products to their release-note sources — repository, ref, paths, target scheme, cutoff, substitutions, attributed repositories, adoption state, and classification.source-unresolvedentries with a warning; products can instead be deliberately deferred underunmapped, each with an auditable reason.source-unresolvedis not seedable — it is the census's own conclusion, never an operator claim.9.0.0cutoff when none is given; unresolved products get a target scheme derived from their versioning system (serverless/project → date, ech → monthly), always paired with a note so a guess never reads as confirmed fact.assembler.ymllink allowlist (advisory — planning re-validates against the deployed scrubber allowlist identity, docs-eng-team#671 / Publish and resolve the deployed scrubber allowlist identity #3763).BackfillDocuments: enveloped, schema-versioned, validated, content-addressed. Same inputs produce the same hash.One contract adjustment to the parent PR's
InventoryDocument:SourceRepository/GitRefare now null-able only forsource-unresolvedentries — the census records products whose location is unknown, which the previously required fields could not express. Validation still demands both for every other classification.Verified end-to-end against the real repo config: 87 release-notes products enumerated; with a sample seed, the elasticsearch source got the stack default cutoff and
elastic/cloudcorrectly reported off-allowlist.