fix(tools): restore drift-report visibility lost to helper refactors and parser gaps - #53
Merged
juemerson-at-purestorage merged 2 commits intoJul 25, 2026
Conversation
…ld not see tools/lib/PfbCmdletParamTools.ps1 resolved a parameter's REST wire name only from a literal `$body['k'] = $Param` / `$queryParams['k'] = $Param` index assignment in the cmdlet's own body. Two very common idioms were therefore invisible, and every parameter they cover was classified AttributesOnly/TypedUnresolved. Downstream, Get-PfbParameterCoverageGaps requires EVERY parameter on EVERY cmdlet calling an endpoint to be Surface='Typed', so one unresolved parameter pushed the whole endpoint into the notVerified bucket and out of gap analysis entirely. 1. The shared Private/Add-PfbCommonQueryParams.ps1 helper (issue dmann000#32/PR dmann000#49, extended to ~196 cmdlets by issue dmann000#33/PR dmann000#51) assigns -Filter/-Sort/-Limit/ -TotalOnly/-Names/-Ids on the caller's behalf, so migrated cmdlets contain no literal key at all. This made a pure mechanical refactor look like a 39% improvement: on PR dmann000#51's tree the headline missing-field count FELL 932 -> 571 while notVerified endpoints ROSE 194 -> 349. Added Get-PfbCommonQueryParamMap (a documented hardcoded mirror of the helper) plus Get-PfbCommonQueryParamHelperWireName. -Filter/-Sort/-Limit/-TotalOnly are matched by PARAMETER NAME (the helper reads them from the caller's $PSBoundParameters, and the call must actually forward it); -Names/-Ids by the variable passed to that argument, which the existing accumulator retry already traces from the parameter. Runs LAST so the non-generic keys those cmdlets deliberately kept as explicit lines (policy_names, file_system_names, ...) still win. 2. The hashtable-literal initializer `$queryParams = @{ 'names' = $Name }` -- the dominant shape across New-Pfb*/Remove-Pfb*/Update-Pfb* -- was only ever matched in its IndexExpressionAst form. Added Get-PfbHashtableLiteralWireNameForParameter, top-level pairs only: in `$body = @{ group = @{ name = $GroupName } }` the wire field is `group`, so crediting the inner `name` would mis-name the field and collide with every other sub-object's `name`. Also extracted the RHS shape test (Test-PfbWireValueIsParameter) and the parser's StatementAst unwrapping (Resolve-PfbSingleExpression) so both paths share one deliberately shape-exact matcher rather than duplicating it -- a pipeline transform is still refused rather than guessed at. Measured, generating the report from each tools/lib against each cmdlet tree (reporting only; no Public/, Private/, Data/ or Reports/ change): | gaps | notVerified | missing fields | uncovered baseline main | 254 | 194 | 932 | 117 baseline PR dmann000#51 | 132 | 349 | 571 | 117 fix 1 only main | 282 | 164 | 1019 | 117 fix 1 only PR dmann000#51 | 286 | 157 | 1029 | 117 both fixes main | 343 | 101 | 1370 | 117 both fixes PR dmann000#51 | 347 | 94 | 1380 | 117 The 155-endpoint main-vs-PR#51 spread closes to 7, zero endpoints newly become notVerified in any transition, and no already-resolved wire name changes. The residual 7 are cmdlets PR dmann000#51 gave query parameters to that had none on main (Get-PfbDns, Get-PfbAdminSetting, ...), which sidesteps a separate pre-existing bug in tools/lib/PfbApiDriftTools.ps1: `@($inventoryByCmdlet[$cmdletName])` yields @($null) for a cmdlet with zero inventory rows, wrongly marking its endpoint notVerified. That accounts for 37 of the remaining 101 and is left for its own change. Tests: 24 new cases in Tests/PfbCmdletParamTools.Tests.ps1 over synthetic Public/-shaped fixtures, including a mapping-table-vs-helper sync test that derives the contract from Private/Add-PfbCommonQueryParams.ps1's own AST so the hardcoded mirror cannot silently drift (mutation-verified: it fails when the helper gains a key). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e objects Two more instances of the same failure mode PR dmann000#53 opened for -- the drift report going quieter, which reads as better -- found while scoping the "make the drift report actionable" work. 1. Null-vs-empty conflation in Get-PfbParameterCoverageGaps. The per-cmdlet inventory lookup is a Group-Object -AsHashTable, which returns $null for an absent key, and @($null) is a ONE-element array whose element is $null. The surface loop therefore ran once with $row = $null, read $null.Surface -ne 'Typed' as true, and discarded the endpoint into notVerified. A cmdlet contributes zero inventory rows when every parameter it declares is -Array/-Attributes (both inventory-exempt) -- the exact shape of the -Attributes-only write cmdlets -- and nothing contradicting full mapping means vacuously fully mapped, not unmappable. 36 endpoints leave notVerified, including PATCH /arrays, PATCH /password-policies and PATCH /support with 18/19/12 previously invisible missing fields. 2. Nested single-key reference objects, `$body['account'] = @{ name = $Account }`, are now credited to their OUTER key. The capability map records top-level body properties only, so `account.name` is not a field it can track and `account` is the field such a parameter actually covers. Two parameters resolving to the same outer key (by name, by id) is correct, not a collision to suppress. Resolves 13 (cmdlet, parameter) pairs across 9 cmdlets, and via the notVerified gate 6 more endpoints. New resolver runs strictly after both direct-assignment passes, so it can only turn an unresolved parameter Typed, never rename an already-resolved wire name (measured: 0 wire-name changes, 0 Typed -> non-Typed regressions). Combined, notVerified 101 -> 59 with 0 newly-notVerified, reported missing fields 1370 -> 1538 with 0 fields removed, and the residual 7-endpoint notVerified spread against PR dmann000#51's tip closes to 0. Measured, not asserted; report regenerated to scratch paths only -- committed Data/ and Reports/ artifacts unchanged. Live-array testing not applicable: tools/ and Tests/ only, no runtime surface, no Public//Private/ change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
tools/lib/PfbCmdletParamTools.ps1'sGet-PfbWireNameForParameterresolved a cmdletparameter's REST wire name by finding exactly one AST shape — an index assignment into a
hashtable named
body/queryParams:The codebase uses at least four idioms for the same thing. A parameter matching none of
them gets
WireName = $nulland a non-Typedsurface, andGet-PfbParameterCoverageGapsthen discards the entire endpoint into its
notVerifiedbucket — so the drift reportsilently stops analysing it. A separate bug discarded endpoints whose cmdlet declares no
inventoried parameters at all.
Two consequences, both measured:
Get-Pfb*cmdlets onto the shared
Add-PfbCommonQueryParamshelper removed their literal$queryParams['names'] = ...lines. Those parameters are still declared and still reachthe wire — the inventory just can't see them.
notVerifiedwent 164 → 194.notVerifiedto 349 and dropping analysable endpoints to 132 of 481 (27%).The dangerous part is the direction of the error: the report's headline field count falls
932 → 571, which reads as a 39% improvement when it is actually blindness. The refactors are
correct; the tooling's heuristic was incomplete.
Fix
tools/+Tests/only — noPublic/,Private/,Data/,Reports/,.github/or.gitignorechanges.Idiom 2 — shared-helper delegation.
Get-PfbCommonQueryParamMapis a documented mirror ofPrivate/Add-PfbCommonQueryParams.ps1with two detection rules, because the helper learns itsinputs two ways: by parameter name via the caller's
$PSBoundParameters(
Filter/Sort/Limit/TotalOnly) and by explicit helper argument (-Names/-Ids).Get-PfbCommonQueryParamHelperWireNameresolves against call sites, requires a literal-Into <variable>soGet-PfbEndpointForVariablecan still trace the endpoint, and requires-BoundParametersto be literally$PSBoundParametersbefore trusting the by-name rule — theby-name mapping is only true because that dictionary is the caller's own bound parameters.
Idiom 3 — hashtable-literal initializer.
$queryParams = @{ 'names' = $Name }, common inNew-Pfb*. Handled at top level only.Idiom 4 — nested single-key reference object.
$body['account'] = @{ name = $Account },resolving to the outer key (
account). The capability map records top-level bodyproperties only, so
account.namedoes not exist as a field; two parameters both resolving toaccountis correct, not a collision. 13 pairs across 9 cmdlets. The resolver deliberatelyrefuses multi-key sub-objects, two levels of nesting, non-literal outer keys, and intermediate
variables — each has a test (see Known remaining gap below).
Resolution order is index-assignment → hashtable-literal → helper. Helper last is load-bearing:
per issue #32's design, cmdlets whose Name/Id-equivalents map to non-generic keys
(
policy_names,file_system_names,member_names, …) deliberately kept explicit linesafter the helper call, and those must win.
Null-vs-empty conflation,
tools/lib/PfbApiDriftTools.ps1.@($inventoryByCmdlet[$name])yields
@($null)— a one-element array whose element is$null— when the cmdlet declares onlyinventory-exempt parameters (
-Array/-Attributes), so the hashtable key is absent entirely.$null.Surface -ne 'Typed'was then true and the endpoint was discarded. Correct semantics: acmdlet with no inventory rows is vacuously fully mapped, not unmappable.
A sync test parses
Add-PfbCommonQueryParams's own AST and asserts the mirrored map stillmatches it, so the two cannot drift silently.
Verification
Report regenerated from each
tools/libagainst each cmdlet tree, all output to scratch paths.The acceptance criterion was that a pure mechanical refactor must be report-neutral. The
main-vs-PR#51
notVerifiedspread closes from 155 → 0, with identical endpoint sets onboth sides. The residual 21-field difference is a strict subset (21 fields main-only, 0
PR#51-only:
filter/limit/sorton cmdlets #51 migrated) — a genuine code difference from#51's own change, not tool blindness.
Field count rising 932 → 1538 is the intended effect: endpoints returning to analysis, not
new gaps appearing. Reviewers should expect this number to go up.
Contribution of each fix, measured in isolation (36 + 6 = 42, no overlap):
notVerified, includingPATCH /arrays,PATCH /password-policies,PATCH /supportwith 18 / 19 / 12 missing fields each.POST /buckets,POST /file-system-exports,POST /lifecycle-rules,POST /object-store-access-keys,POST /quotas/groups,POST /subnets.Regression checks by endpoint-set diff, not just counts:
Full 1763-row inventory diffed key-by-key: 0 keys added, 0 lost, 0
Typed→non-Typedregressions, 0 wire-name or endpoint/method changes on already-Typedrows, 0 fields removed from the gap list (strictly additive).
worktree rather than taken on trust: 436 / 434 / 0 / 2 — so the delta is exactly the 13 new
tests, all passing. (An earlier-quoted 6-skipped baseline was measured without the gitignored
tools/specscache, where 4 tests self-skip viaSet-ItResult.)$Into['destroyed']key to a scratch copy ofthe helper makes it fail with the expected diff. Not a vacuous assertion.
SHA256 for both
.jsonand.md).Add-PfbCommonQueryParamsis the only sharedhelper in
Private/that mutates a caller's body/queryParams hashtable.Assert-PfbApiCapabilityandInvoke-PfbApiRequesttake those hashtables but only read them.Live-array testing: not applicable, deliberately
This repo's standing rule is that changes are live-verified against a FlashBlade before a PR.
This change has no runtime surface — it touches only build-time reporting tooling, and no
Public//Private//Data/file. The equivalent verification is running the real generatorsagainst the real cached specs and two real cmdlet trees, which is the table above. Calling the
exemption out explicitly rather than skipping it silently.
One behaviour deliberately reversed
Tests/PfbCmdletParamTools.Tests.ps1previously asserted "does NOT descend into a nestedsub-object literal", and
Get-PfbHashtableLiteralWireNameForParameter's.DESCRIPTIONdocumented that as an intentional decision. Both are inverted here.
Worth flagging rather than burying: the original rationale was that crediting the inner key
(
name) would mis-name the field and make several parameters collide. Resolving to theouter key answers that objection rather than overriding it, so the doc comment was rewritten
to say so explicitly instead of being deleted.
Known remaining gap
The idiom-four resolver does not follow intermediate variables.
New-PfbFileSystem's$nfsBody/$smbBodyare not traceable to anInvoke-PfbApiRequestcall, soUpdate-PfbFileSystem's-NfsExportPolicy/-SmbSharePolicy/-SmbClientPolicyremainunresolved. That is the next parser gap — named here rather than silently absorbed, and each
refusal case has a test pinning the current behaviour.
🤖 Generated with Claude Code