Summary
_collect_pr_newswire stamps each NewsDocument.identity through the private _build_identity, which returns news:pr-newswire:<sha256("pr-newswire\x1f" + payload_or_url)>. The public build_news_identity helper returns a different scheme for the same document: wire:<sha256(payload_or_url)> — different prefix, and a different hash input (no "pr-newswire\x1f" domain-separator prefix).
Because the collector does not use the public builder, a caller who reads doc.identity gets one string, but a caller who recomputes the identity for the same payload/URL via the public build_news_identity(...) gets a different string. Two "stable" identities exist for one document, and the public builder is not the one the collector actually stamps. Public API users cannot reproduce the collector's identity from the public surface.
Pinned at aafa89040693f99721baa45ef3d60e36dd700dc7:
- public
build_news_identity: quantmind/preprocess/news.py:343-362 -> wire:<sha256(payload_or_url)>
- collector call site:
quantmind/preprocess/pr_newswire.py:560-568 -> uses _build_identity
- private
_build_identity: quantmind/preprocess/pr_newswire.py:691-698 -> news:pr-newswire:<sha256("pr-newswire\x1f" + payload_or_url)>
Repro
- Collect a PR Newswire document via the public
collect_news(...) flow and read doc.identity -> starts with news:pr-newswire:.
- Recompute the identity for the same payload/URL with the public
build_news_identity(...) -> starts with wire:.
- The two strings differ in both prefix and hash input, so a consumer cannot re-derive
doc.identity from the public builder.
Expected
The identity a consumer can recompute from the public API equals the identity the collector stamps onto NewsDocument.identity. There should be a single source of truth. Two reasonable fixes:
- have the collector stamp identities via the public
build_news_identity (one publicly reproducible scheme), or
- promote the collector's domain-separated scheme into the public
build_news_identity so both agree.
Either way, the public builder and the collector should not return different identities for the same document.
Summary
_collect_pr_newswirestamps eachNewsDocument.identitythrough the private_build_identity, which returnsnews:pr-newswire:<sha256("pr-newswire\x1f" + payload_or_url)>. The publicbuild_news_identityhelper returns a different scheme for the same document:wire:<sha256(payload_or_url)>— different prefix, and a different hash input (no"pr-newswire\x1f"domain-separator prefix).Because the collector does not use the public builder, a caller who reads
doc.identitygets one string, but a caller who recomputes the identity for the same payload/URL via the publicbuild_news_identity(...)gets a different string. Two "stable" identities exist for one document, and the public builder is not the one the collector actually stamps. Public API users cannot reproduce the collector's identity from the public surface.Pinned at
aafa89040693f99721baa45ef3d60e36dd700dc7:build_news_identity:quantmind/preprocess/news.py:343-362->wire:<sha256(payload_or_url)>quantmind/preprocess/pr_newswire.py:560-568-> uses_build_identity_build_identity:quantmind/preprocess/pr_newswire.py:691-698->news:pr-newswire:<sha256("pr-newswire\x1f" + payload_or_url)>Repro
collect_news(...)flow and readdoc.identity-> starts withnews:pr-newswire:.build_news_identity(...)-> starts withwire:.doc.identityfrom the public builder.Expected
The identity a consumer can recompute from the public API equals the identity the collector stamps onto
NewsDocument.identity. There should be a single source of truth. Two reasonable fixes:build_news_identity(one publicly reproducible scheme), orbuild_news_identityso both agree.Either way, the public builder and the collector should not return different identities for the same document.