fix(mcp): match fidelity globs with path.Match, not filepath.Match - #647
fix(mcp): match fidelity globs with path.Match, not filepath.Match#647tiendungdev wants to merge 2 commits into
Conversation
|
@tiendungdev IMO, it will be more beneficial to have a full test suite run for Windows as well as for Linux/MacOS - which is added in the #652 PR |
|
Agreed, and #652 is the better answer than what I proposed here. My "no cross-platform test is possible, so it needs a windows selector step" reasoning was solving the wrong problem — a full Windows shard makes the selector question moot, and my own selector lists were exactly the guesswork you describe. Two notes so this PR is easy to dispose of:
I have posted the full local Windows failure enumeration on #652, including which of them are my environment rather than the platform, since that draft is waiting on exactly that. |
zzet
left a comment
There was a problem hiding this comment.
Requesting changes at head 53ad34c.
The path.Match / path.Base substitution correctly fixes the targeted internal/*.go behavior on Windows. Two issues should be addressed before merge:
-
Blocking CI coverage gap — the Windows job does not run TestMatchFidelityGlob. Its -run selector omits FidelityGlob, so the only platform where the old and new implementations differ never executes the regression assertion. Linux and macOS pass both before and after this patch, which means the green checks do not validate the fix. PR #646 has now merged and current main still omits FidelityGlob from .github/workflows/ci.yml:118-124, so the earlier conflict rationale no longer applies. Rebase onto current main, add FidelityGlob to the Windows selector, and rerun CI.
-
Medium contract/documentation mismatch — internal/mcp/fidelity_globs.go:14 and the new matcher comment state that a single * never crosses /. However, matchSegmentGlob intentionally inherits matchPathPattern directory-prefix semantics: internal/* matches internal itself and its entire subtree. That compatibility behavior should not be removed in this patch, but the public schema and comment need to document the dir/* recursive exception, with a preservation test covering the directory, direct child, and nested child.
No security, authentication, secret-handling, or dead-code concern was found.
matchFidelityGlob normalizes both the pattern and the path to forward
slashes, then hands them to filepath.Match — whose separator is the
platform's. On Windows '/' is an ordinary character to that matcher, so a
single `*` crosses it:
matchFidelityGlob("internal/*.go", "internal/sub/x.go")
= true on windows, false on linux/macos
The file's own doc-comment states the assumption this breaks: "Go's
filepath.Match never crosses `/`". That holds on POSIX and is why the
linux/macos matrix has never seen it.
Everything around it already works in slash space — ToSlash on entry,
Split(rel, "/"), the `**` prefix and suffix handling — so path.Match and
path.Base are the matching primitives that space calls for. Identical to
filepath.Match on POSIX, where the separator already is '/'.
fidelity_globs is a public tool parameter on read_file and
get_editing_context, so on Windows a documented `internal/*.go` rule
silently applied to the whole subtree beneath internal/.
Whole package on windows: TestMatchFidelityGlob flips, newly-broken set
empty.
Review follow-up on the path.Match change. 1. The Windows selector did not run TestMatchFidelityGlob, so the only platform where filepath.Match and path.Match disagree never executed the regression assertion. Rebased onto current main — zzet#646 has landed, so the earlier conflict rationale is gone — and added FidelityGlob to the selector. It picks up 8 tests, including the two matcher tests and the five read_file / get_editing_context end-to-end cases. 2. matchSegmentGlob's directory-prefix shortcut is not glob matching, and a trailing `/*` goes through it: `internal/*` matches the directory and its entire subtree, exactly like `internal` and `internal/**`. That predates this change and is untouched by it, but the schema and the matcher comments claimed a blanket "a single `*` never crosses `/`". Both now state the exception, and TestMatchFidelityGlob_DirStarStaysRecursive pins the directory, a direct child, two nested depths, and the negative case that keeps the shortcut segment-anchored (`internalx/a.go` must not match). The schema wording is deliberately terse: tools/list has a byte gate and the core preset had 332 bytes of headroom (96168 against a 96500 ceiling). A fuller sentence cost 522 bytes — the description is used by two tool registrations — and failed TestToolsListByteCeilings at 96690. The wording that landed costs 144 and leaves 188 bytes of headroom. Verification (windows/amd64, go1.26.6, -count=1), against current main 812eb2d: internal/mcp 24 -> 23 failures, newly-broken set empty Sabotage-verified separately: reverting path.Match fails TestMatchFidelityGlob on `internal/*.go` vs `internal/sub/x.go`; removing the `/*` shortcut fails the new test on the directory itself and on a nested child. As before, neither assertion can fail on linux/macos — filepath.Match and path.Match return the same answer where '/' is the separator — so the Windows leg is the only thing that binds them.
53ad34c to
5302c87
Compare
|
Both correct. Verified each before changing anything, and the second one turned up a third problem of my own making. Pushed 1 — the selector gapConfirmed. Sabotage check that it now binds: reverting to 2 — the
|
| pattern | path | result |
|---|---|---|
internal/* |
internal |
true |
internal/* |
internal/a.go |
true |
internal/* |
internal/sub/x.go |
true |
internal/* |
internal/sub/deep/y.go |
true |
internal/* |
internalx/a.go |
false |
internal/*.go |
internal/sub/x.go |
false |
So internal/* has the same reach as internal and internal/**. matchSegmentGlob's prefix shortcut answers before path.Match ever sees a nested path — it is not glob matching at all, which is why the blanket "a single * never crosses /" in my comment and in the schema was wrong. The behavior predates this patch and is untouched by it.
Both comments and the schema now state it, and TestMatchFidelityGlob_DirStarStaysRecursive pins the directory, a direct child, two nested depths, and the negative that keeps the shortcut segment-anchored. Removing the /* block fails it on the directory itself and a nested child.
3 — my first wording broke the tools/list byte gate
Worth flagging since you would have hit it in review. The full sentence I wrote first cost 522 bytes — the description is shared by two tool registrations — and core has only 332 bytes of headroom:
core mode=defer bytes=96690 (baseline 96500) # FAIL, +190 over
Baseline on main is 96168. The terse wording that landed costs 144 and leaves 188 bytes of headroom:
core mode=defer bytes=96312 (baseline 96500) # PASS
I kept it under the ceiling rather than raising the ceiling — the gate is a deliberate diet and documentation bytes should not be the thing that relaxes it. If you would rather have the fuller explanation and think the ceiling can move, say so and I will swap them.
Measurement
windows/amd64, go1.26.6, -count=1, against 812eb2d8:
internal/mcp 24 -> 23 failures
Newly-broken set empty — the 23 are a strict subset of the 24, with TestToolsListByteCeilings and TestMatchFidelityGlob the only differences. gofmt and go vet clean.
Neither assertion can fail on linux/macos: where / is the separator, filepath.Match and path.Match return the same answer. The Windows leg is the only thing binding them, which is the same limitation as #646 and the reason I did not try to write a cross-platform version.
Problem
matchFidelityGlobnormalizes both the pattern and the path to forward slashes, then matches withfilepath.Match— whose separator is the platform's. On Windows/is an ordinary character to that matcher, so a single*crosses it:The file's own doc-comment states the assumption this breaks:
That is true on POSIX, which is exactly why the linux/macos matrix has never been able to see it.
fidelity_globsis a public tool parameter onread_fileandget_editing_context, so on Windows a documentedinternal/*.gorule silently applied its fidelity to the entire subtree beneathinternal/.Change
path.Match/path.BaseinmatchSegmentGlob. Everything around it already works in slash space —ToSlashon entry,Split(rel, "/"), the**prefix/suffix handling — so those are the primitives that space calls for. On POSIX they are identical to thefilepathversions, since the separator already is/.Verification
TestMatchFidelityGlobred → green on windows/amd64, go1.26.6.internal/mcppackage, failing test names diffed againstmain(c982cf52): newly-broken set empty.One result I am not claiming
The full-package run came back 27 → 25, but only one of those is mine.
TestNotesManager_SaveQueryDeletealso flipped to green in that run — and it is not fixed by this change: run in isolation on this very branch it fails 3 times out of 3. It is a pre-existing order/parallelism-dependent flake that happened to pass in that particular full-package run, so the honest attributable delta is one test, not two.No cross-platform test is possible
path.Matchandfilepath.Matchare the same function on POSIX, so an assertion here passes on linux/macos before and after. The guard belongs in the windowsTest native-separator store path comparisonsstep, whose package list already carriesinternal/mcp— but I have leftci.ymluntouched here because #646 is open against that exact-runline and I would rather not hand you a conflict. Happy to addFidelityGlobto it in a follow-up once #646 lands, or fold it into #646 if you prefer that order.Related, deliberately not included
internal/mcp/tools_generate_skill.gohas the mirror-image inconsistency:matchPathPatternthere receives a nativerel(filepath.Relat :134), so itsfilepath.Matchis correct, while:540testsstrings.HasPrefix(rel, prefix+"/")against that same native path. Different bug, different fix, own change.Branched from
mainatc982cf52. Windows 11, go1.26.6.