Skip to content

fix(vcr): anonymize secrets in VCR test cassettes (#4150)#4321

Open
Toshika-Kamble wants to merge 1 commit into
traceloop:mainfrom
Toshika-Kamble:fix/vcr-secret-leak
Open

fix(vcr): anonymize secrets in VCR test cassettes (#4150)#4321
Toshika-Kamble wants to merge 1 commit into
traceloop:mainfrom
Toshika-Kamble:fix/vcr-secret-leak

Conversation

@Toshika-Kamble

@Toshika-Kamble Toshika-Kamble commented Jun 26, 2026

Copy link
Copy Markdown

Description

This PR implements the best-practice recommendation raised in the original issue. The changes address findings from their AI PatchLab security scan, which flagged several Git-leaks matches (such as generic API keys and JWTs) being recorded in plaintext within the VCR test cassettes.

To resolve this, the VCR configurations have been updated to proactively anonymize and mask these sensitive headers and query parameters before recording.

Changes

  • Modified conftest.py across 26 instrumentation packages.
  • Added explicit configurations for filter_headers and filter_query_parameters to strip out sensitive credentials (like api-key, x-api-key, and access tokens) to keep the recorded cassettes clean and secure.

  • I have added tests that cover my changes.
  • If adding a new instrumentation or changing an existing one, I've added screenshots from some observability platform showing the change.
  • PR name follows conventional commits format: feat(instrumentation): ... or fix(instrumentation): ....
  • (If applicable) I have updated the documentation accordingly.

Closes #4150

Summary by CodeRabbit

  • Bug Fixes
    • Improved handling of sensitive request data in recorded HTTP interactions.
    • Expanded redaction coverage for API keys and related credentials, including request headers and query values.
    • Reduced the chance of secrets appearing in test recordings, helping keep fixtures safer and more consistent.

@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Multiple test VCR fixtures now filter additional API-key header variants, and many also redact the api_key query parameter. A few fixtures only add a missing header filter while leaving the rest of the VCR configuration unchanged.

Changes

VCR redaction updates

Layer / File(s) Summary
Header-only filter additions
packages/opentelemetry-instrumentation-agno/tests/conftest.py, packages/opentelemetry-instrumentation-langchain/tests/conftest.py, packages/opentelemetry-instrumentation-openai/tests/traces/conftest.py
These fixtures add missing API-key header filters to existing VCR header lists.
Header and query redaction
packages/opentelemetry-instrumentation-alephalpha/tests/conftest.py, packages/opentelemetry-instrumentation-anthropic/tests/conftest.py, packages/opentelemetry-instrumentation-bedrock/tests/conftest.py, packages/opentelemetry-instrumentation-cohere/tests/conftest.py, packages/opentelemetry-instrumentation-google-generativeai/tests/conftest.py, packages/opentelemetry-instrumentation-groq/tests/traces/conftest.py, packages/opentelemetry-instrumentation-haystack/tests/conftest.py, packages/opentelemetry-instrumentation-llamaindex/tests/conftest.py, packages/opentelemetry-instrumentation-mistralai/tests/conftest.py, packages/opentelemetry-instrumentation-openai-agents/tests/conftest.py, packages/opentelemetry-instrumentation-openai/tests/conftest.py, packages/opentelemetry-instrumentation/openai/tests/metrics/conftest.py, packages/opentelemetry-instrumentation-pinecone/tests/conftest.py, packages/opentelemetry-instrumentation-replicate/tests/conftest.py, packages/opentelemetry-instrumentation-sagemaker/tests/conftest.py, packages/opentelemetry-instrumentation-together/tests/conftest.py, packages/opentelemetry-instrumentation-vertexai/tests/conftest.py, packages/opentelemetry-instrumentation-voyageai/tests/conftest.py, packages/opentelemetry-instrumentation-watsonx/tests/metrics/conftest.py, packages/opentelemetry-instrumentation-watsonx/tests/traces/conftest.py, packages/opentelemetry-instrumentation-weaviate/tests/conftest.py, packages/traceloop-sdk/tests/conftest.py, packages/traceloop-sdk/tests/guardrails/conftest.py
These fixtures expand VCR filtering to additional API-key header variants and api_key query-parameter redaction.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

Suggested reviewers

  • doronkopit5
  • nina-kollman
  • galzilber

Poem

🐰 I hopped through cassettes with a grin,
and tucked api keys safely within.
Headers and URLs, all cozy and tight,
stay hidden from tests in the moonlit night.
thump! says the bunny, “recordings are bright.”

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 3.85% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: anonymizing secrets in VCR cassettes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
packages/opentelemetry-instrumentation-langchain/tests/conftest.py (1)

146-182: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy lift

Consider centralizing the shared VCR redaction defaults.

This header/query/body masking pattern now appears across many instrumentation packages in the same cohort. Pulling the common vcr_config() pieces into a shared test helper would reduce drift and make future secret-filter updates a one-file change. As per coding guidelines, **/tests/**/*.py: Ensure VCR cassettes never contain secrets or PII; scrub them using VCR filters....

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/opentelemetry-instrumentation-langchain/tests/conftest.py` around
lines 146 - 182, Centralize the repeated VCR redaction setup used by
vcr_config() so the shared header, query, and body masking defaults live in one
test helper instead of being duplicated per package. Update this conftest.py to
consume the common helper while preserving the existing secret filters for
authorization/api-key headers, api_key query/body scrubbing, and IMDS
ignore_hosts behavior so cassette sanitization stays consistent across
instrumentation tests.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/opentelemetry-instrumentation-langchain/tests/conftest.py`:
- Around line 146-182: Centralize the repeated VCR redaction setup used by
vcr_config() so the shared header, query, and body masking defaults live in one
test helper instead of being duplicated per package. Update this conftest.py to
consume the common helper while preserving the existing secret filters for
authorization/api-key headers, api_key query/body scrubbing, and IMDS
ignore_hosts behavior so cassette sanitization stays consistent across
instrumentation tests.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 49aa68ec-5e1b-4e74-9373-c0ee060646df

📥 Commits

Reviewing files that changed from the base of the PR and between fb292d0 and 073ba1e.

📒 Files selected for processing (26)
  • packages/opentelemetry-instrumentation-agno/tests/conftest.py
  • packages/opentelemetry-instrumentation-alephalpha/tests/conftest.py
  • packages/opentelemetry-instrumentation-anthropic/tests/conftest.py
  • packages/opentelemetry-instrumentation-bedrock/tests/conftest.py
  • packages/opentelemetry-instrumentation-cohere/tests/conftest.py
  • packages/opentelemetry-instrumentation-google-generativeai/tests/conftest.py
  • packages/opentelemetry-instrumentation-groq/tests/traces/conftest.py
  • packages/opentelemetry-instrumentation-haystack/tests/conftest.py
  • packages/opentelemetry-instrumentation-langchain/tests/conftest.py
  • packages/opentelemetry-instrumentation-llamaindex/tests/conftest.py
  • packages/opentelemetry-instrumentation-mistralai/tests/conftest.py
  • packages/opentelemetry-instrumentation-openai-agents/tests/conftest.py
  • packages/opentelemetry-instrumentation-openai/tests/conftest.py
  • packages/opentelemetry-instrumentation-openai/tests/metrics/conftest.py
  • packages/opentelemetry-instrumentation-openai/tests/traces/conftest.py
  • packages/opentelemetry-instrumentation-pinecone/tests/conftest.py
  • packages/opentelemetry-instrumentation-replicate/tests/conftest.py
  • packages/opentelemetry-instrumentation-sagemaker/tests/conftest.py
  • packages/opentelemetry-instrumentation-together/tests/conftest.py
  • packages/opentelemetry-instrumentation-vertexai/tests/conftest.py
  • packages/opentelemetry-instrumentation-voyageai/tests/conftest.py
  • packages/opentelemetry-instrumentation-watsonx/tests/metrics/conftest.py
  • packages/opentelemetry-instrumentation-watsonx/tests/traces/conftest.py
  • packages/opentelemetry-instrumentation-weaviate/tests/conftest.py
  • packages/traceloop-sdk/tests/conftest.py
  • packages/traceloop-sdk/tests/guardrails/conftest.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Best-practice: anonymize secrets in VCR test cassettes (+ minor SDK jinja2 trust-boundary comment)

1 participant