Add metadata field to LLMRequest for simplified user-specific data handling in callbacks

[donggyun112]

Is your feature request related to a problem? Please describe.

In multi-user applications, it is often necessary to inject user-specific context into the system prompt on a per-request basis to personalize responses. A common example of this is injecting long-term memory, such as conversation summaries. However, the current methods for accessing this context within callbacks are brittle and lead to boilerplate code.

• Brittle Context Access: Accessing user context inside callbacks (e.g., before_model_callback) requires repeated parsing of context.session.user_id with null checks, which is error-prone.
• Multimodal Input Issues: JSON-wrapping inputs to carry metadata breaks when using audio or other multimodal inputs.
• Unsuitable RunConfig: The RunConfig object is not a suitable context carrier for arbitrary per-request data due to its schema (extra='forbid').

These issues cause boilerplate code, subtle personalization failures, and slow down development.

Proposed solution

We propose adding a canonical, per-request context channel on the input side of the ADK.

• Add a metadata field to LlmRequest:
  • LlmRequest: metadata: dict[str, Any] = Field(default_factory=dict)
• Update the Runner methods to accept this metadata:
  • Runner.run_async(..., metadata: dict | None = None)
  • Runner.run_live(..., metadata: dict | None = None)
• The Runner will merge the provided metadata into the underlying LlmRequest, allowing callbacks to reliably access per-request context:
  • llm_request.metadata.get("user_id")
  • llm_request.metadata.get("memctx_key")

Conceptual API

# In models/llm_request.py
class LlmRequest(BaseModel):
    ...
    metadata: Dict[str, Any] = Field(
        default_factory=dict, 
        description="Per-request context for callbacks/memory"
    )
    
    model_config = ConfigDict(extra='ignore') # Keep ignoring unknown fields

# In Runner
async def run_async(self, ..., metadata: dict | None = None, ...):
    # If provided, merge into request.metadata
    ...

async def run_live(self, ..., metadata: dict | None = None, ...):
    ...

Minimal Reproduction Case

• Today's Problem: A before_model_callback that needs user_id for memory injection must perform null checks everywhere and fails when a session is missing.
• Example Scenario:
  1. Define a before_model_callback that requires a user_id.
  2. Call an agent with an audio input (where JSON-wrapped metadata is not possible).
  3. Observation: It's not possible to reliably pass the user_id into the callback without brittle wiring or misusing RunConfig.

Alternatives Considered

• Parse context.session.user_id in every callback: This is repetitive and brittle.
• JSON-wrap inputs with metadata: This approach breaks for audio/multimodal inputs and adds unnecessary parsing overhead.
• Subclass RunConfig: The schema forbids arbitrary fields, and the config object is not designed to be a per-request context carrier.
• Use LlmResponse.custom_metadata: This is on the output side only and does not solve the input-side context problem.

Backward Compatibility

This is a non-breaking change:

• The new LlmRequest.metadata field defaults to an empty dictionary ({}).
• The Runner methods accept an optional metadata parameter; omitting it results in no change to the current behavior.
• Extra fields remain ignored where applicable (ConfigDict(extra='ignore')).

Security & Privacy

• The request metadata can carry user-scoped keys (e.g., user_id, memctx_key).
• Applications should avoid storing sensitive PII in the metadata itself. We recommend using short-TTL keys for any Redis-backed memory contexts.

Performance Impact

• The performance impact is negligible.
• It involves an O(1) dictionary merge with no additional network calls.
• No measurable overhead was observed in our local tests (100+ executions on ADK v1.2.0).

Test Plan (Upstream)

• Unit: Verify that metadata round-trips correctly into LlmRequest on run_async and run_live.
• Unit: Ensure before_model_callback can see request.metadata for both normal and streaming flows.
• Unit: Confirm isolation across concurrent runs (no metadata bleed).
• Unit: Check that metadata defaults to {} when omitted.
• Integration: Test a memory injection callback using metadata.memctx_key in a full streaming session.

Open Questions

• Should metadata be propagated to all nested flows and tools automatically?
• Should there be any size limits or filtering for the metadata dictionary?

How We Solved It Locally (Workaround)

We have shipped a safe, runtime-only workaround that mirrors this proposal without modifying ADK source code, demonstrating its viability.

• ContextVar Carrier (app/adk/memctx.py): Uses current_run_metadata to hold context.
• Web Server Propagation (app/api/custom_adk_web_server.py): Wraps runner methods to set and reset current_run_metadata from request headers.
• LlmRequest Monkey Patch (app/adk/llm_request_patch.py): Ensures request.metadata exists and merges the ContextVar on LlmRequest construction.
• Callback Implementation (app/agents/common/dynamic_callback.py): Reads llm_request.metadata['memctx_key'], loads data from Redis, and injects it into the system_instruction.
• Memory Count Policy (app/agents/prompts/prompt_loader.py): Respects a memory_max_items count from context so that backend-selected items are injected as-is.

Environment

• Python 3.12+
• ADK v1.2.0

We are happy to contribute a PR that aligns with the proposal above.

[Jacksunwei]

@donggyun112 We've recently add custom_metadata field to RunConfig. Would that suffice your use case?

[donggyun112]

Hi @Jacksunwei, thank you for considering this feature and adding RunConfig.custom_metadata!

I analyzed the ADK v1.17.0 source code to understand how this would work, and I have some concerns that RunConfig.custom_metadata alone may not be sufficient for our use case. Here's what I found:

---

Key Issues

1. No automatic propagation to LlmRequest

I traced the execution path in v1.17.0:

runner.run_async(run_config=config)
  ↓ runners.py:364
InvocationContext(run_config=config)
  ↓ invocation_context.py:196
invocation_context.run_config stores it
  ↓ base_llm_flow.py:821-823
callback(callback_context, llm_request)  # Only 2 params

Problem: custom_metadata stays at the InvocationContext.run_config level but:

• Callbacks receive (callback_context, llm_request) - no config parameter
• ReadonlyContext (parent of CallbackContext) doesn't expose a run_config property
• LlmRequest doesn't receive the metadata

Question: Does the new version automatically copy run_config.custom_metadata → llm_request.metadata? If not, how should callbacks access it?

---

2. Callback signature limitation

Our current callback:

def dynamic_context_callback(callback_context, llm_request):
    # How do we access run_config.custom_metadata here?
    metadata = ?

Possible solutions:

• Add callback_context.run_config property (requires ADK change)
• Add 3rd parameter run_config (breaking change for existing callbacks)
• Auto-copy to llm_request.metadata (cleanest solution)

Question: Which approach will the new RunConfig.custom_metadata use?

---

3. Semantic concern: Request data vs Configuration

Configuration (RunConfig):
  - streaming_mode, max_llm_calls, speech_config
  - "How to run"

Request context (what we need):
  - user_id, memctx_key, trace_id
  - "What to process"

Per-request context like memctx_key is semantically request data, not configuration.

Additionally, LlmResponse.custom_metadata already exists (response side), so LlmRequest.metadata would mirror it for symmetry.

---

What Would Work

Option A (Ideal): Add LlmRequest.metadata as originally proposed

await runner.run_async(..., metadata={"memctx_key": "xyz"})

def callback(callback_context, llm_request):
    llm_request.metadata.get("memctx_key")  # Clean access

Benefits:

• No callback signature changes
• Symmetry with LlmResponse.custom_metadata
• Naturally flows through nested agents and streaming

---

Option B (Alternative): Auto-propagate RunConfig.custom_metadata → LlmRequest

config = RunConfig(custom_metadata={"memctx_key": "xyz"})
await runner.run_async(..., config=config)

# ADK internally does:
llm_request.metadata = config.custom_metadata

def callback(callback_context, llm_request):
    llm_request.metadata.get("memctx_key")  # Also works

---

Option C (Less ideal): Expose run_config in callbacks

def callback(callback_context, llm_request):
    callback_context.run_config.custom_metadata.get("memctx_key")  # Verbose

Concern: Semantically awkward - request context stored in configuration object.

---

Our Current Workaround

We're using runtime patching to inject metadata into LlmRequest:

• Monkey-patch LlmRequest.__init__ to add a metadata field
• Use ContextVar for per-request propagation
• Patch BaseLlmFlow.run_live to inject context metadata

This works in production but is fragile and hard to maintain across ADK version updates.

---

Test Scenario

To clarify how RunConfig.custom_metadata works, could you confirm this?

config = RunConfig(custom_metadata={"test_key": "test_value"})
await runner.run_async(..., config=config)

def callback(callback_context, llm_request):
    # Which of these works?
    llm_request.metadata.get("test_key")  # Option A/B
    callback_context.run_config.custom_metadata.get("test_key")  # Option C

---

Summary

We're excited about RunConfig.custom_metadata being considered! However, we need clarification on:

1. Does it auto-propagate to llm_request.metadata?
2. If not, how should callbacks access it?
3. Would you consider LlmRequest.metadata as the canonical solution?

I'm happy to contribute a PR once we align on the approach! I have a working implementation (currently as a workaround) that I can adapt to whichever design you prefer.

Thank you for considering this! 🙏

[rcleveng]

@donggyun112 We've recently add custom_metadata field to RunConfig. Would that suffice your use case?

If you are using the /run and /run_sse methods as recommended, I don't see how the run_config can be set on that runner (so deploying to vertex would not work, nor using the recommended endpoints). Can you please explain how that flow would work?

[donggyun112]

@donggyun112 We've recently add custom_metadata field to RunConfig. Would that suffice your use case?

If you are using the /run and /run_sse methods as recommended, I don't see how the run_config can be set on that runner (so deploying to vertex would not work, nor using the recommended endpoints). Can you please explain how that flow would work?

We hit this same issue in production.

Our /run_sse endpoint receives metadata in HTTP requests, but there's no way to pass it to callbacks through RunConfig.

We ended up patching LlmRequest.metadata to bridge: HTTP endpoint → LlmRequest → callback

Works well, but native support would be much cleaner!

[rcleveng]

Actually that last question is for @Jacksunwei -

[klateefa]

@donggyun112 Thank you for reporting this. I understand the request is currently under review. Please keep me posted if anything else is needed. Thanks for your patience.