fix: safely suppress verified Hugging Face model-card image examples - #1791
fix: safely suppress verified Hugging Face model-card image examples#1791mldangelo-oai wants to merge 9 commits into
Conversation
…-readme-doc-false-positives-20260728
|
@codex review |
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
Pull request overview
This PR updates the TextScanner to downgrade network communication findings to informational only for a small set of revision-pinned Hugging Face model-card README payloads that exactly match expected size + SHA-256 digests (including LF/CRLF variants), reducing false positives while keeping existing “fail closed” behavior for incomplete analysis and policy overrides.
Changes:
- Add a digest-verified allowlist for three specific Hugging Face
timmmodel-card READMEs and downgrade only the correspondingurllib/urlopenfindings toINFOwhen whitelisting is enabled and analysis is complete. - Thread an explicit policy flag through network-finding classification so suppression can be disabled when scans are incomplete/limited or whitelisting is turned off.
- Add extensive regression coverage (whitelist modes, CLI flags, tampering/near-match, truncation, detector failure, finding limits, LF/CRLF) and add the pinned model-card fixtures.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
modelaudit/scanners/text_scanner.py |
Introduces digest-verified Hugging Face README recognition and conditional network-finding downgrade logic. |
tests/scanners/test_text_scanner.py |
Adds regression tests to ensure the downgrade is narrowly-scoped and fails closed under truncation/limits/failures and policy overrides. |
tests/assets/huggingface_model_cards/timm_mobilenetv3_small_100.lamb_in1k_README.txt |
Adds pinned model-card fixture used to validate the digest allowlist behavior. |
tests/assets/huggingface_model_cards/timm_convnext_femto.d1_in1k_README.txt |
Adds pinned model-card fixture used to validate the digest allowlist behavior. |
tests/assets/huggingface_model_cards/timm_repvgg_a0.rvgg_in1k_README.txt |
Adds pinned model-card fixture used to validate the digest allowlist behavior. |
CHANGELOG.md |
Documents the new false-positive suppression behavior for the verified model cards. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| finding: dict[str, Any], | ||
| *, | ||
| allow_verified_huggingface_documentation: bool = True, | ||
| ) -> bool: |
Performance BenchmarksCompared Top regressions:
|
|
Security review completed. No security issues were found in this pull request. Reviewed commit: ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
Summary
timmmodel-card READMEs with matching complete-file sizes and SHA-256 digests, documented Apache-2.0 or MIT provenance, and the expected official image URL.--no-whitelistand--strictpolicy, and fail closed for modified or truncated cards, detector failures, reporting limits, and actionable network behavior.mainnetwork and cache protections.Validation
pytest tests/scanners/test_text_scanner.py tests/detectors/test_network_comm_detector.py -q -n 4 --disable-warnings: 1,742 passed.--python-version 3.13 --disable-error-code unreachable; the unsuppressed diagnostic concerns an unchanged baseline test.