Skip to content

bug: PR Newswire collector identity diverges from public build_news_identity #110

Description

@keli-wen

Summary

_collect_pr_newswire stamps each NewsDocument.identity through the private _build_identity, which returns news:pr-newswire:<sha256("pr-newswire\x1f" + payload_or_url)>. The public build_news_identity helper returns a different scheme for the same document: wire:<sha256(payload_or_url)> — different prefix, and a different hash input (no "pr-newswire\x1f" domain-separator prefix).

Because the collector does not use the public builder, a caller who reads doc.identity gets one string, but a caller who recomputes the identity for the same payload/URL via the public build_news_identity(...) gets a different string. Two "stable" identities exist for one document, and the public builder is not the one the collector actually stamps. Public API users cannot reproduce the collector's identity from the public surface.

Pinned at aafa89040693f99721baa45ef3d60e36dd700dc7:

  • public build_news_identity: quantmind/preprocess/news.py:343-362 -> wire:<sha256(payload_or_url)>
  • collector call site: quantmind/preprocess/pr_newswire.py:560-568 -> uses _build_identity
  • private _build_identity: quantmind/preprocess/pr_newswire.py:691-698 -> news:pr-newswire:<sha256("pr-newswire\x1f" + payload_or_url)>

Repro

  1. Collect a PR Newswire document via the public collect_news(...) flow and read doc.identity -> starts with news:pr-newswire:.
  2. Recompute the identity for the same payload/URL with the public build_news_identity(...) -> starts with wire:.
  3. The two strings differ in both prefix and hash input, so a consumer cannot re-derive doc.identity from the public builder.

Expected

The identity a consumer can recompute from the public API equals the identity the collector stamps onto NewsDocument.identity. There should be a single source of truth. Two reasonable fixes:

  • have the collector stamp identities via the public build_news_identity (one publicly reproducible scheme), or
  • promote the collector's domain-separated scheme into the public build_news_identity so both agree.

Either way, the public builder and the collector should not return different identities for the same document.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: preprocessDeterministic acquisition, parsing, cleaning, and source handlingtype: bugExisting behavior violates its contract or documented expectation

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions