-
Notifications
You must be signed in to change notification settings - Fork 0
๐ก๏ธ Sentinel: [CRITICAL] Fix incomplete DSN secret redaction and over-redaction #745
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
seonghobae
wants to merge
36
commits into
main
Choose a base branch
from
sentinel/dsn-redaction-fix-14431549202957900548
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
36 commits
Select commit
Hold shift + click to select a range
823e0c3
๐ก๏ธ Sentinel: [CRITICAL] Fix incomplete DSN secret redaction and over-โฆ
seonghobae 296ba90
test(security): cover DSN plus and boundary semantics
seonghobae c81beac
fix(security): preserve URL userinfo semantics
seonghobae 4b50e12
chore: remove superseded agent journal entry
seonghobae f2343ac
docs(security): define DSN redaction semantics
seonghobae 52ed2b1
docs(changelog): record DSN redaction hardening
seonghobae 7832d5b
docs(changelog): preserve existing release notes
seonghobae b13dfdc
๐ก๏ธ Sentinel: [CRITICAL] Fix incomplete DSN secret redaction and over-โฆ
seonghobae 5c2eb66
test(security): preserve DSN decoding domains
seonghobae ed2a216
fix(security): separate DSN decoding domains
seonghobae 74a8b57
docs: remove transient DSN agent journal entry
seonghobae 05c683d
๐ก๏ธ Sentinel: [CRITICAL] Fix incomplete DSN secret redaction and over-โฆ
seonghobae bdb70d9
test(security): expose DSN decoding and boundary regressions
seonghobae 3627ccb
fix(security): preserve URL userinfo semantics and Unicode boundaries
seonghobae e58872a
chore: restore canonical Sentinel journal
seonghobae b8ae7c0
๐ก๏ธ Sentinel: [CRITICAL] Fix incomplete DSN secret redaction and over-โฆ
seonghobae 6425351
๐ก๏ธ Sentinel: [CRITICAL] Fix incomplete DSN secret redaction and over-โฆ
seonghobae 522c9e7
ci(pr745): rebuild focused DSN redaction fix
seonghobae 7c46d80
fix(security): restore focused DSN redaction boundary
github-actions[bot] b26638a
๐ก๏ธ Sentinel: [CRITICAL] Fix incomplete DSN secret redaction and over-โฆ
seonghobae b11c977
๐ก๏ธ Sentinel: [CRITICAL] Fix incomplete DSN secret redaction and over-โฆ
seonghobae 523d605
๐ก๏ธ Sentinel: [CRITICAL] Fix incomplete DSN secret redaction and over-โฆ
seonghobae d8091ed
๐ก๏ธ Sentinel: [CRITICAL] Fix incomplete DSN secret redaction and over-โฆ
seonghobae f9443e5
test(security): cover standalone and non-corrupting DSN redaction
seonghobae 11acfb2
fix(security): separate URL and form secret encodings
seonghobae 7930009
fix(deps): restore complete typed development contract
seonghobae b5ac3c8
fix(ci): restore immutable dependency lockfiles
seonghobae 5467f8c
docs(security): define DSN redaction evidence and contract
seonghobae 0d5ea7f
docs(changelog): record DSN secret redaction hardening
seonghobae 07f19f9
chore: merge protected main into DSN redaction branch
seonghobae e8d5e0b
Merge branch 'main' into sentinel/dsn-redaction-fix-14431549202957900548
opencode-agent[bot] 1e510c3
Merge main into sentinel/dsn-redaction-fix-14431549202957900548
seonghobae ed42bac
fix(security): preserve unrelated assignment keys
seonghobae 5924869
Merge branch 'main' into sentinel/dsn-redaction-fix-14431549202957900548
seonghobae 5887448
Merge branch 'main' into sentinel/dsn-redaction-fix-14431549202957900548
seonghobae 7c8afef
Merge branch 'main' into sentinel/dsn-redaction-fix-14431549202957900548
seonghobae File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| # DSN Secret Redaction Boundary | ||
|
|
||
| ## Incident class | ||
|
|
||
| Database drivers and connection libraries can repeat a database source name | ||
| (DSN), its user-information, or secret-bearing query parameters in an exception. | ||
| Passing those messages through unchanged can disclose passwords, tokens, API | ||
| keys, and connection strings in API responses, support bundles, or logs. | ||
|
|
||
| The risk is not limited to one textual representation. A driver can emit the raw | ||
| percent-encoded value, a decoded value, or a canonical re-encoding. A redactor | ||
| that handles only one form can leave equivalent credentials visible. Conversely, | ||
| using form-query decoding rules for URL user-information can create false | ||
| candidates and corrupt unrelated diagnostic text. | ||
|
|
||
| ## Decision | ||
|
|
||
| `backend/app/dsn_redaction.py` applies two ordered defenses: | ||
|
|
||
| 1. Recognized secret-key assignments such as `password=...`, `token: ...`, and | ||
| `api_key=...` are masked independently of DSN parsing. | ||
| 2. Additional candidates are derived from the supplied DSN and replaced in the | ||
| remaining message. | ||
|
|
||
| The candidate domains remain intentionally separate: | ||
|
|
||
| - URL user-information passwords use `urllib.parse.unquote`. A literal `+` | ||
| remains a plus, while percent-encoded octets such as `%20` are decoded. The | ||
| implementation adds the raw, decoded, and `quote(..., safe="")` forms. | ||
| - Query-string secret values use `urllib.parse.unquote_plus`, because HTML form | ||
| query semantics map `+` to a space. The implementation adds raw, decoded, | ||
| `quote`, and `quote_plus` forms only in this domain. | ||
|
|
||
| Candidates longer than four characters are replaced by exact, | ||
| case-insensitive occurrence. Short candidates use Unicode-aware | ||
| `(?<!\w)` and `(?!\w)` boundaries, which mask a standalone value without | ||
| corrupting a larger identifier or natural-language word that merely contains | ||
| it. Malformed authorities use best-effort extraction and never cause the | ||
| redaction boundary itself to raise. The complete diagnostic message is | ||
| preserved; redaction does not impose an unrelated truncation policy. | ||
|
|
||
| ## Verification | ||
|
|
||
| `backend/tests/test_dsn_redaction.py` covers: | ||
|
|
||
| - raw, decoded, and canonical encoded user-information passwords; | ||
| - query tokens with form-query decoding semantics; | ||
| - a literal-plus password without treating an unrelated space as equivalent; | ||
| - a `%20` user-information password without treating unrelated `a+b` text as | ||
| equivalent; | ||
| - standalone Unicode and punctuation-bearing short secrets while preserving | ||
| larger surrounding words; | ||
| - malformed and scheme-less DSNs; | ||
| - case-insensitive standalone candidates; | ||
| - assignment sanitization combined with DSN-derived candidates; and | ||
| - preservation of messages longer than 1,000 characters. | ||
|
|
||
| CI must install the immutable, hash-locked development dependency set and run | ||
| static typing and the full backend tests on the exact pull-request head. Security | ||
| Scan and Semgrep remain independent required gates. | ||
|
|
||
| ## Operational impact | ||
|
|
||
| The function returns a sanitized diagnostic string. It does not alter connection | ||
| establishment, driver selection, DSN storage, authentication, or database | ||
| behavior. Downstream code should still avoid logging raw DSNs and should keep | ||
| structured secret fields out of log records. Redaction is a defense-in-depth | ||
| boundary, not permission to collect plaintext credentials. | ||
|
|
||
| ## Research basis | ||
|
|
||
| Krause et al. (2023) found that accidental secret leakage is common in source | ||
| code workflows and that prevention and remediation mechanisms need low adoption | ||
| cost. Applying automatic redaction at the common error-message boundary reduces | ||
| the number of callers that must implement secret-specific handling correctly. | ||
|
|
||
| OWASP identifies passwords, access tokens, database connection strings, | ||
| credentials, and similar values as data that should not be recorded in plaintext | ||
| logs. Its secrets-management guidance explicitly requires encryption or masking | ||
| when a secret could otherwise reach a log. Python's `urllib.parse` | ||
| documentation is authoritative for the semantic distinction that | ||
| `unquote_plus` maps plus signs to spaces for form values, while `unquote` does | ||
| not. | ||
|
|
||
| The peer-reviewed paper is linked to the publisher-maintained open-access copy | ||
| rather than vendored as a binary, preserving provenance and avoiding a stale or | ||
| license-ambiguous repository copy. | ||
|
|
||
| ## References | ||
|
|
||
| Krause, A., Klemmer, J. H., Huaman, N., Wermke, D., Acar, Y., & Fahl, S. | ||
| (2023). Pushed by accident: A mixed-methods study on strategies of handling | ||
| secret information in source code repositories. In *32nd USENIX Security | ||
| Symposium (USENIX Security 23)* (pp. 2527โ2544). USENIX Association. | ||
| https://www.usenix.org/conference/usenixsecurity23/presentation/krause | ||
|
|
||
| Open Worldwide Application Security Project. (n.d.). *Logging cheat sheet*. | ||
| OWASP Cheat Sheet Series. Retrieved August 6, 2026, from | ||
| https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html | ||
|
|
||
| Open Worldwide Application Security Project. (n.d.). *Secrets management cheat | ||
| sheet*. OWASP Cheat Sheet Series. Retrieved August 6, 2026, from | ||
| https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html | ||
|
|
||
| Python Software Foundation. (n.d.). *urllib.parseโParse URLs into components*. | ||
| Python documentation. Retrieved August 6, 2026, from | ||
| https://docs.python.org/3/library/urllib.parse.html |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.