Support Sourcepoint GPP consent for EC generation by ChristianPavilonis · Pull Request #642 · IABTechLab/trusted-server

ChristianPavilonis · 2026-04-15T21:29:39Z

Summary

Add client-side Sourcepoint JS integration that auto-discovers _sp_user_consent_* from localStorage and mirrors GPP consent into __gpp / __gpp_sid cookies
Extend server-side GPP decoding to extract sale_opt_out from US GPP sections (IDs 7–23)
Add GPP US consent branch to allows_ec_creation() between existing TCF and us_privacy checks

Closes #640

Test plan

Rust tests pass (cargo test --workspace — 992 tests including 8 new)
JS tests pass (npx vitest run — 288 tests including 6 new)
Clippy clean
Verify EC generation succeeds for a Sourcepoint-only site in a regulated US state with GPP consent present
Verify GPC still blocks EC when GPP US section is permissive
Verify existing TCF and us_privacy EC gating behavior unchanged

🤖 Generated with Claude Code

aram356

Summary

Sourcepoint GPP consent is a small, well-motivated slice — the GPP-US decoder, the allows_ec_creation gating branch, and their tests are solid. But as shipped, the feature is non-functional (the JS module is never served to browsers), the PR fails CI, and it bundles several large unrelated changes that should be separate PRs.

Blocking

🔧 wrench

Sourcepoint JS module is built but never served. crates/trusted-server-core/src/integrations/registry.rs:790-792 hardcodes JS_ALWAYS = &["creative"]. The design spec (docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md:38) says the integration follows the same "JS-only, no Rust registration" pattern as creative, but that pattern requires JS_ALWAYS to list the module. integrations/mod.rs has no sourcepoint::register and JS_ALWAYS has no "sourcepoint" entry, so IntegrationRegistry::js_module_ids() will never include it — /static/tsjs=… will never concatenate tsjs-sourcepoint.js into a served bundle. End-to-end, no browser will ever execute mirrorSourcepointConsent(). Fix: add "sourcepoint" to JS_ALWAYS and a test asserting it ships in js_module_ids_immediate().
Orphaned dead-code file crates/trusted-server-core/src/ec/admin.rs. 380 lines implementing POST /_ts/admin/partners/register, but ec/mod.rs contains no pub mod admin; declaration (grep -rn "mod admin\|ec::admin::" crates/ returns zero hits). The base branch (feature/edge-cookies-final) replaced the KV-backed partner registry with config-based partners in commit 049a501e; that change dropped admin.rs but the rebase on this branch reinstated it unreferenced. Fix: delete crates/trusted-server-core/src/ec/admin.rs.
Clippy -D dead_code failures block CI. Three dead symbols introduced by the same rebase and carrying no callers:
- crates/trusted-server-core/src/platform/test_support.rs:174 — recorded_request_headers method
- crates/trusted-server-core/src/platform/test_support.rs:336 — build_services_with_http_client function
- crates/integration-tests/tests/common/runtime.rs:56 — PartnerRegistrationFailed variant (this is the actual failing CI log message)
Confirmed locally: cargo clippy --workspace --all-targets --all-features -- -D warnings fails in the PR's worktree. Fix: delete them all — nothing references any of the three.
Undisclosed scope: creative iframe sandbox lockdown. crates/js/lib/src/core/render.ts:7-18 removes allow-scripts, allow-same-origin, and allow-forms from the creative iframe sandbox= attribute, and crates/trusted-server-core/src/creative.rs:361-367 strips <iframe> elements entirely during HTML sanitization. Together, any creative relying on JavaScript (viewability pixels, click tracking, rich media, VPAID) or on third-party ad-tag iframes will stop rendering. This is a major ad-tech behavioral change wholly unrelated to Sourcepoint GPP consent, and it is not mentioned in the PR body. It needs its own PR with a threat model, product sign-off, and a rollout plan. Fix: extract these two changes to a dedicated PR.

❓ question

Why is Prebid User ID Module support shipping inside this PR? The title and body are "Support Sourcepoint GPP consent for EC generation," but the diff also contains ~132 lines of build-all.mjs User-ID-submodule generation, a new _user_ids.generated.ts, ~290 lines of new Prebid test coverage, a 212-line design spec (specs/2026-04-16-prebid-user-id-module-design.md), and a 599-line implementation plan. Two independent features in one review is hard to evaluate. Can Prebid User ID Module move to its own PR?

Non-blocking

📌 out of scope

No re-mirror after mid-session CMP interaction. mirrorSourcepointConsent() runs once when the module is parsed. If a user opens the Sourcepoint CMP and updates consent mid-session, __gpp / __gpp_sid won't reflect the new state until the next page load — meaning the same session can continue to block EC creation even after consent is granted. Follow-up: subscribe to __gppapi events (or a storage listener) and re-run on change.

ChristianPavilonis · 2026-04-21T13:46:35Z

Addressed the requested review feedback in 3bbdb1d.

Summary:

added sourcepoint to the JS-only bundle list and covered it with a registry test
removed the orphaned ec/admin.rs file
removed the dead-code clippy offenders in test support / integration tests
scoped this PR back down by reverting the unrelated creative-lockdown and Prebid user ID changes
aligned Sourcepoint cookie mirroring with the approved session-cookie spec
fixed the cookie test-name typo and the Sourcepoint test helper nit

Validation run locally:

cargo fmt --all -- --check
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace
cd crates/js/lib && npx vitest run

I also replied to and resolved the inline threads above.

aram356

Summary

Follow-up review focused on areas not covered in the prior pass. Sourcepoint flow (mirror + GPP US decoding + EC gating) is functional and well-tested, but two doc/file-stale issues from the scoping commit (8a6df3af) need correction before merge, and the always-shipped Sourcepoint module has cross-CMP failure modes worth designing around.

PR effective scope note: vs main the diff is 80 commits / 111 files, because the intended base (feature/edge-cookies-final) has merged. The Sourcepoint-specific changes are a small subset (~14 files); reviewers landing on this PR via GitHub should read the description with that in mind.

Blocking

🔧 wrench

Stale Prebid User ID docs reference removed TSJS_PREBID_USER_IDS env var (docs/guide/integrations/prebid.md:226-261)
_user_ids.generated.ts claims to be auto-generated but is now hand-edited (crates/js/lib/src/integrations/prebid/_user_ids.generated.ts:1)

Non-blocking

🤔 thinking

Sourcepoint hardcoded as always-shipped will clobber __gpp set by other CMPs (registry.rs:792 + sourcepoint/index.ts:60-72)
First page load race: mirror runs before Sourcepoint CMP populates localStorage (sourcepoint/index.ts:91-93)
TCF presence silently overrides explicit GPP US sale_opt_out=Yes in US states (consent/mod.rs:506-515)
Session-scoped cookie + run-once mirror leaves stale __gpp after mid-session retraction (sourcepoint/index.ts:39)

♻️ refactor

Stale comment displaced by us_sale_opt_out insertion (consent/gpp.rs:74-75)
Make clearing logic CMP-safe by tracking write source (sourcepoint/index.ts:42-46)

⛏ nitpick

Unused/inconsistent default export (sourcepoint/index.ts:95)

CI Status

fmt: PASS
clippy: PASS
rust tests: PASS (1001 lib + 21 misc)
js tests: PASS (294)

…in Prebid 10)

Prebid's liveIntentIdSystem.js uses a dynamic require() inside a build-flag-guarded branch that their gulp pipeline dead-codes via constant folding. esbuild leaves the require() in the output, causing ReferenceError: require is not defined at browser runtime. Remove from the bundle until we add an esbuild resolver plugin (or switch to Prebid's own build pipeline) — tracked as a follow-up in the design spec.

Introduces TSJS_PREBID_USER_IDS env var (mirroring TSJS_PREBID_ADAPTERS) to control which Prebid User ID submodules are bundled. The hardcoded imports in index.ts are replaced with a generated file written by build-all.mjs at build time, defaulting to the same 13-submodule set. - build-all.mjs: generatePrebidUserIds() validates names, denylists liveIntentIdSystem, and writes _user_ids.generated.ts. Existence check also probes dist/src/public/ to handle modules shipped as .ts in sources (sharedIdSystem). - index.ts: replaces 13 hardcoded submodule imports with import './_user_ids.generated' - _user_ids.generated.ts: committed default with all 13 submodules - Tests: updated mocks and regression guard; added 9 syncPrebidEidsCookie behavior tests - Docs: new "User ID Modules" section in prebid.md with TSJS_PREBID_USER_IDS usage; spec follow-up #1 marked complete

__gpp and __gpp_sid are read by the Rust server over HTTPS; they must be Secure. Also sets Max-Age=86400 (matching ts-eids) so stale consent state doesn't outlast the session, and replaces the legacy expires= deletion pattern with Max-Age=0.

… and Move logging initialization into Fastly adapter (#610) * Rename crates to trusted-server-core and trusted-server-adapter-fastly Rename crates/common → crates/trusted-server-core and crates/fastly → crates/trusted-server-adapter-fastly following the EdgeZero naming convention. Add EdgeZero workspace dependencies pinned to rev 170b74b. Update all references across docs, CI workflows, scripts, agent files, and configuration. * Add platform abstraction layer with traits and RuntimeServices Introduces trusted-server-core::platform with PlatformConfigStore, PlatformSecretStore, PlatformKvStore, PlatformBackend, PlatformHttpClient, and PlatformGeo traits alongside ClientInfo, PlatformError, and RuntimeServices. Wires the Fastly adapter implementations and threads RuntimeServices into route_request. Moves GeoInfo to platform/types as platform-neutral data and adds geo_from_fastly for field mapping. * Address platform layer review feedback - Defer KV store opening: replace early error return with a local UnavailableKvStore fallback so routes that do not need synthetic ID access succeed when the KV store is missing or temporarily unavailable - Use ConfigStore::try_open + try_get and SecretStore::try_get throughout FastlyPlatformConfigStore and FastlyPlatformSecretStore to honour the Result contract instead of panicking on open/lookup failure - Encapsulate RuntimeServices service fields as pub(crate) with public getter methods (config_store, secret_store, backend, http_client, geo) and a pub new() constructor; adapter updated to use new() - Reference #487 in FastlyPlatformHttpClient stub (PR 6 implements it) - Remove unused KvPage re-export from platform/mod.rs - Use super::KvHandle shorthand in RuntimeServices::kv_handle() * Reject host strings containing control characters in BackendConfig * Fix clippy error * Validate scheme and host for control characters in BackendConfig * Address review findings on platform abstraction layer * Address review findings on platform abstraction layer * Add config store read path and storage module split - Split fastly_storage.rs into storage/{config_store,secret_store,api_client,mod}.rs - Add PlatformConfigStore read path via FastlyPlatformConfigStore::get using ConfigStore::try_open/try_get - Add PlatformError::NotImplemented variant; stub write methods on FastlyPlatformConfigStore and FastlyPlatformSecretStore - Add StoreName/StoreId newtypes with From<String>, From<&str>, AsRef<str> - Add UnavailableKvStore to core platform module - Add RuntimeServicesBuilder replacing 7-arg constructor - Migrate get_active_jwks and handle_trusted_server_discovery to use &RuntimeServices - Update call sites in signing.rs, rotation.rs, main.rs - Add success-path test for handle_trusted_server_discovery using StubJwksConfigStore - Fix test_parse_cookies_to_jar_empty typo (was emtpy) * Harden legacy config-store reads and align Fastly adapter stubs * Address storage review feedback * Resolved github-advanced-security bot problems * Address PR review feedback on platform abstraction layer - Make StoreName and StoreId inner fields private; From/AsRef provide all needed construction and access - Add #[deprecated] to GeoInfo::from_request with #[allow(deprecated)] at the three legacy call sites to track migration progress - Enumerate the six platform traits in the platform module doc comment - Extract backend_config_from_spec helper to remove duplicate BackendConfig construction in predict_name and ensure - Replace .into_iter().collect() with .to_vec() on secret plaintext bytes - Remove unused bytes dependency from trusted-server-adapter-fastly - Add comment on SecretStore::open clarifying it already returns Result (unlike ConfigStore::open which panics) * Add PR 4 design spec for secret store trait (read-only) * Clarify test scope and deferred branches in PR 4 spec * Add implementation plan for PR 4 secret store trait * Add test for get_secret_bytes open-failure path * Add NotImplemented tests for FastlyPlatformSecretStore write stubs * Inline StoreId binding and add section comment in write-stub tests * Remove plan * Add PR 6 design spec for backend and HTTP client traits * Address spec review findings on PR 6 design * Implement PlatformHttpClient and thread RuntimeServices through proxy layer - Add PlatformHttpClient trait with send(), send_async(), and select() methods - Add PlatformBackend trait with predict_name() and ensure() methods - Add PlatformResponse wrapper around EdgeZero HTTP responses - Add PlatformPendingRequest and PlatformSelectResult for auction fan-out - Thread RuntimeServices through IntegrationProxy::handle(), IntegrationRegistry::handle_proxy(), and all first-party proxy endpoints so handlers can reach the HTTP client - Add StubHttpClient and StubBackend test stubs with build_services_with_http_client helper - Add proxy_request_calls_platform_http_client_send integration test - Fix proxy_with_redirects to stay within 7-arg clippy limit via ProxyRequestHeaders struct - Document Body::Stream limitation in edge_request_to_fastly with warning log - Document intentional duplication of platform_response_to_fastly across proxy and orchestrator - Remove spec file (promoted to plan + implementation) * Address pr review findings * Resolve pr review findings * Add PR7 design spec for geo lookup + client info extract-once Documents the call site migration plan: five Fastly SDK extraction points in trusted-server-core replaced by RuntimeServices::client_info reads, following Phase 1 injection pattern from the EdgeZero migration design. * Fix spec review issues in PR7 design doc - Correct erroneous claim about generate_synthetic_id being called twice via DeviceInfo; it is called once (line 91 for fresh_id), DeviceInfo.ip is a separate req.get_client_ip_addr() call fixed independently - Add before/after snippet for handle_publisher_request call site in main.rs - Add noop_services import instruction for http_util.rs test module - Clarify _services rename (drop underscore, not add new param) in didomi.rs - Clarify nextjs #[allow(deprecated)] annotations are out of scope (different function) * Update PR7 spec to address all five agent review findings - Change RequestInfo::from_request signature to &ClientInfo (not &RuntimeServices) so prebid can call it with context.client_info - Scope SDK-call acceptance criteria to active non-deprecated code only - List all six AuctionContext construction sites including two production sites in orchestrator.rs and three test helpers in orchestrator/prebid - Add explicit warn-and-continue pattern for publisher.rs geo lookup - Correct testing table: formats.rs and endpoints.rs have no test modules; add orchestrator.rs and prebid.rs test helper update rows * Add PR7 implementation plan and address plan review findings Plan covers 6 tasks in compilation-safe order: AuctionContext struct change first, then from_request signature, then synthetic.rs cascade, then publisher geo, then didomi. Includes two new copy_headers unit tests (Some/None). Spec fixes: clarify injection pattern exceptions for &ClientInfo and Option<IpAddr>; reword acceptance criterion to reflect that provider-layer reads flow through AuctionContext.client_info. * Fix three plan review findings and two open questions - Finding 1 (High): Add missing publisher.rs test call site at line ~695 for get_or_generate_synthetic_id — was omitted from Task 3 Step 6 - Finding 2 (Medium): Remove crate::geo::GeoInfo import from endpoints.rs rather than replacing it — type is not used by name after the change, keeping any import fails clippy -D warnings - Finding 3 (Low): Replace interactive git add -p in Task 6 with explicit file staging instruction - Open Q1: Add Task 2 step to update stale handle_publisher_request signature in auction/README.md - Open Q2: Add Task 2 step to update from_request doc comment to reflect ClientInfo-based TLS detection instead of Fastly SDK calls * Broaden two low-severity doc cleanup steps in PR7 plan - Step 7: cover all four stale Fastly-SDK-specific locations in http_util.rs (SPOOFABLE_FORWARDED_HEADERS doc, RequestInfo struct doc, from_request doc, detect_request_scheme doc) - Step 8: replace the whole routing snippet in auction/README.md, not just the one handle_publisher_request line — handle_auction and integration_registry.handle_proxy are also stale in that snippet * Fix two remaining low findings in PR7 plan - Add missing Location 2 (RequestInfo.scheme field doc, line ~67) to Step 7; renumber subsequent locations 3-5 - Replace &runtime_services with runtime_services in Step 5 and README snippet — runtime_services is already &RuntimeServices in route_request * Fix count drift in Step 7: four → five locations * Add client_info field to AuctionContext and fix all construction sites * Change RequestInfo::from_request to take &ClientInfo, thread services into handle_publisher_request * Add Task 2 follow-up coverage and README route fixes * Add services param to generate_synthetic_id, remove Fastly IP/geo calls in formats and endpoints * Revert premature publisher geo change from Task 3 * Replace deprecated GeoInfo::from_request in publisher.rs with services.geo().lookup() * Remove Fastly IP extraction from Didomi copy_headers, use ClientInfo instead * Move IpAddr import to test module level in didomi.rs * Apply rustfmt formatting to didomi.rs, publisher.rs, and synthetic.rs Fix multi-line function call style in didomi.rs, line-break wrapping in publisher.rs test, and import ordering in synthetic.rs test module. * Add test coverage for generate_synthetic_id with concrete client IP Adds noop_services_with_client_ip helper to test_support and a new test that verifies the client_ip path through generate_synthetic_id by asserting the HMAC differs when the IP changes. * Align geo lookup warn log format with codebase convention ({e} not {e:?}) * Apply Prettier formatting to PR7 plan and spec docs * Document content rewriting as platform-agnostic in platform module * Document html_processor as platform-agnostic * Document streaming_processor as platform-agnostic * Fix unresolved doc link: replace EdgeRequest with edgezero_core::http::Request * Add plan for content rewriting * Add plan for PR9: wire signing to store primitives * Add build_services_with_config_and_secret to test_support * Add FastlyManagementApiClient to adapter * Implement FastlyPlatformConfigStore and FastlyPlatformSecretStore write methods via management API Replace FastlyApiClient with FastlyManagementApiClient in the put/delete methods of FastlyPlatformConfigStore and the create/delete methods of FastlyPlatformSecretStore. Remove the now-unused FastlyApiClient import. * Add services to AuctionContext; remove deprecated from_config shim Thread RuntimeServices into AuctionContext so auction providers can access platform stores directly. Update PrebidAuctionProvider to use RequestSigner::from_services(context.services) instead of the now- removed from_config() shim. All construction sites and test helpers updated accordingly. This satisfies the final acceptance criterion of #490: no FastlyConfigStore/FastlySecretStore construction remains in the request_signing/ modules. * Fix rotate/delete atomicity, HTTP verb, idempotent deletes, and weak tests - Revert proxy.rs merge artifact: restore per-request allowed_domains at both redirect_is_permitted call sites; remove dead_code allow and stale comment — integration proxies defaulting to &[] get open mode again as documented - Drop unused trusted-server-js dep from adapter Cargo.toml - Fix check_response: gate body read behind error branch so 2xx paths do not buffer and discard the response body - Remove self-referential SECRET_UPSERT_METHOD test - Reorder write-cost doc so outbound HTTPS round-trip leads; handle-open caching noted as negligible - Refactor make_request to take fastly::http::Method; drop string match and unreachable arm; remove SECRET_UPSERT_METHOD const - Add SigningStoreIds named struct in endpoints.rs; update both call sites to destructure by name * Move logging initialization into Fastly adapter (PR 10) (#610) Move logging initialization into Fastly adapter (PR 10) (#610) - Reverted gratuitous _message rename and record.args() usage in logging.rs, returning to the idiomatic message parameter inside the fern format closure. - Refactored target_label to use .rsplit_once("::") rather than .split("::").last(). This provides a more explicit and robust way to extract the final module segment. - Expanded target_label test coverage to explicitly test edge cases such as inputs without :: separators, empty strings, and inputs with trailing ::.

ChristianPavilonis marked this pull request as draft April 16, 2026 16:42

ChristianPavilonis marked this pull request as ready for review April 16, 2026 21:49

ChristianPavilonis force-pushed the feature/edge-cookies-final branch from 7009838 to 43df212 Compare April 16, 2026 22:14

ChristianPavilonis force-pushed the edge-cookie-sourcepoint-consent branch from 0e5c07c to a8b5b1c Compare April 16, 2026 22:22

aram356 assigned ChristianPavilonis Apr 17, 2026

aram356 requested changes Apr 20, 2026

View reviewed changes

ChristianPavilonis requested a review from aram356 April 21, 2026 13:46

aram356 requested a review from prk-Jr April 22, 2026 03:08

ChristianPavilonis force-pushed the edge-cookie-sourcepoint-consent branch from 3bbdb1d to 8a6df3a Compare April 22, 2026 20:33

ChristianPavilonis force-pushed the feature/edge-cookies-final branch from 9261993 to d8c943d Compare April 27, 2026 16:26

aram356 requested changes Apr 30, 2026

View reviewed changes

ChristianPavilonis added 18 commits May 5, 2026 07:15

Add design spec for Sourcepoint GPP consent support (#640)

1c14b3d

Add implementation plan for Sourcepoint GPP consent support (#640)

fca8979

Add us_sale_opt_out field to GppConsent

a1bc657

Decode US sale opt-out from GPP sections

1964872

Recognize GPP US sale opt-out in EC consent gating

fbb2457

Add Sourcepoint JS integration for GPP consent cookie mirroring

ad6e790

Add design spec for Prebid User ID Module support

e025198

Add implementation plan for Prebid User ID Module support

6450637

Fix ESM path resolution in Prebid User ID plan regression guard

a760c4c

Add Vitest coverage for Prebid ts-eids cookie sync

c8abb71

Bundle Prebid User ID core and submodules in Prebid integration

eee99f0

Correct Prebid User ID plan + spec — drop pubCommonIdSystem (removed …

bd366ad

…in Prebid 10)

Clear stale consent cookies and aggregate US GPP opt-outs

67e55d5

support ec partners map for env overrides

08440a6

Scope Sourcepoint consent PR and address review feedback

8e3ff80

prk-Jr and others added 2 commits May 5, 2026 07:19

Remove generated Prebid user ID shim

24d4515

ChristianPavilonis force-pushed the edge-cookie-sourcepoint-consent branch from 8643550 to 24d4515 Compare May 5, 2026 12:30

Address Sourcepoint consent review feedback

a2c6d83

ChristianPavilonis requested a review from aram356 May 5, 2026 12:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support Sourcepoint GPP consent for EC generation#642

Support Sourcepoint GPP consent for EC generation#642
ChristianPavilonis wants to merge 21 commits intofeature/edge-cookies-finalfrom
edge-cookie-sourcepoint-consent

ChristianPavilonis commented Apr 15, 2026

Uh oh!

aram356 left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ChristianPavilonis commented Apr 21, 2026

Uh oh!

aram356 left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ChristianPavilonis commented Apr 15, 2026

Summary

Test plan

Uh oh!

aram356 left a comment

Choose a reason for hiding this comment

Summary

Blocking

🔧 wrench

❓ question

Non-blocking

📌 out of scope

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ChristianPavilonis commented Apr 21, 2026

Uh oh!

aram356 left a comment

Choose a reason for hiding this comment

Summary

Blocking

🔧 wrench

Non-blocking

🤔 thinking

♻️ refactor

⛏ nitpick

CI Status

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants