test: add signature near-miss fixtures - #82
Conversation
|
Hi @jpbelmo , I know this isn’t related to the code here, but could I DM you about something? If so, what’s the best place to reach you? |
jpbelmo
left a comment
There was a problem hiding this comment.
Three of these five are exactly what #27 wants pinned: the oauth-oidc string-literal miss, the ApplicationService inheritance shape, and the vitest mock surface. Keep those as-is. Four things before merge:
- The new github-actions negative is a byte-identical copy of the one already there (same path, same text). Swap it for a real near-miss, e.g.
uses: actions/checkout@v4inside a# comment, or workflow-shaped YAML living outside.github/workflows/. - The firebase-auth fixture is a trap for future us:
signInWithRedirectis a real firebase/auth API that just happens to be missing from the patterns today. As written, the fixture would block anyone who legitimately adds it to detection later. Make it a//comment line (so it tests the sanitizer, which is the interesting behavior) or use an API name that isn't a real detectable call. - Changelog: main released 0.11.0 yesterday, so your entry currently merges INSIDE the released 0.11.0 section and rewrites shipped notes: GitHub won't flag it because the merge is textually clean. Rebase, add a fresh
## [Unreleased]heading, and put the line under### Added(nothing is fixed here, and Added-before-Fixed is the file's convention). Closes #27→Part of #27. That issue is the standing hunt, and its definition of done is fixtures that fail current detection as false positives plus the tightening that makes them pass honestly. This PR is the "pin current behavior" half, welcome but not the close.
Small list, and the three good fixtures make the suite genuinely stronger. On the next push this merges through the usual gate.
|
@BharadwajKanneveti of course — DMs are open at x.com/jpbelmo, write me there anytime. |
420880e to
31df0cc
Compare
|
Merged. The reworked fixtures came back better than asked: the github-actions and firebase-auth negatives now pin the comment-stripping sanitizer itself, which is the behavior that actually protects against false positives in prose and docs. The hunt in #27 stays open whenever you want another pass at it. Eleven merged. |
What does this PR do, and why does it fit the north star?
This PR strengthens Tier 2 signature test coverage by adding genuine near-miss
negative fixtures for existing Firebase Auth, OAuth/OIDC, ActiveRecord,
GitHub Actions, and Jest signatures.
These fixtures represent code that looks similar to the corresponding
technology but should not match the signature. This helps verify that existing
signature detection remains precise and reduces the risk of false-positive
skill detection.
This fits the north star by making detected credentials more trustworthy and
verifiable through stronger fixture coverage.
Part of #27
Checklist
npm test)npm run typecheckpassesIf this adds or changes a detection signature
(the negative fixture is a genuine near-miss, not unrelated text)
taxonomy.json— if it's new, that'sa separate PR with a rationale, linked here: #
If this touches WHAT data leaves the machine, or WHERE it's sent
before this PR was opened): #
docs/schema.mdupdatedtest/privacy/If this adds a new runtime dependency
existing stack — commander/vitest — can't do it, what it adds to
the supply-chain surface)
Docs and changelog
CHANGELOG.mdentry added under[Unreleased](if user-facing)docs/added or updated, in EnglishAdditional context
The changes are limited to existing Tier 2 signature fixtures and do not
modify the detection engine, add new taxonomy slugs, change runtime
dependencies, or change the data boundary.
The negative fixtures were chosen as genuine near-misses for each signature,
rather than unrelated text, so they exercise cases that could plausibly be
mistaken for a match.