[bot] OpenAI Batch API is not instrumented (mis-tagged as a generic LLM span)

[braintrust-bot]

Summary

The OpenAI instrumentation module (openai_2_15_0) generically intercepts every HTTP call via TracingHttpClient (swapped into ClientOptions.httpClient/originalHttpClient), so a call to client.batches().create(...) (or .retrieve()/.list()/.cancel()) does produce a span — but the shared tagging logic in InstrumentationSemConv has no awareness of the Batch API's request/response shape, so the span is actively mis-tagged rather than simply absent: it's marked span_attributes.type = "llm" as if it were a real model call, given a low-information span name ("batches"), and gets no model metadata, no input_json, and no output_json/metrics.

The OpenAI Batch API lets you submit up to 50,000 chat-completion/embeddings/responses/moderation requests as a single async job (POST /v1/batches); the job's eventual output file contains one JSONL line per request with the same generative output (token usage, model output) that this SDK already spans for synchronous calls — but none of that ever gets a Braintrust span today.

What is missing

In braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java:

• getSpanName() (lines 518–529) switches on providerName + ":" + lastPathSegment. For POST /v1/batches, the last path segment is "batches", which matches neither the openai:completions nor openai:embeddings case, so it falls through to default -> lastSegment, yielding the literal span name "batches" instead of something descriptive like "openai.batches.create".
• tagOpenAIRequest() (lines 110–141) unconditionally sets span_attributes = {"type":"llm"} (line 119) even though a batch-create call isn't itself a model invocation. It only reads metadata.model when requestJson.has("model") (line 129) and input_json from messages or an array-typed input (lines 133–137) — but a BatchCreateParams request body has none of these; it has input_file_id, endpoint (e.g. /v1/chat/completions), and completion_window. All of that is silently dropped.
• tagOpenAIResponse() (lines 143–208) looks for choices or output for output_json (lines 149–153) and a top-level usage object for metrics (line 160) — a Batch object (returned by create/retrieve/list) has neither; it has id, status, output_file_id, error_file_id, request_counts, and timestamps. None of this is captured, and there is no instrumentation at all of retrieving/parsing the completed batch's output file (where the actual per-request custom_id + generative response.body results, including usage, become available) — so even a fully successful, completed batch job produces zero spans reflecting its actual generative work.
• No test or example anywhere in the repo exercises client.batches() in any form (confirmed via grep for batch under braintrust-sdk/instrumentation/openai_2_15_0/ — zero matches).

Braintrust docs status: not_found

Checked https://www.braintrust.dev/docs/integrations/ai-providers/openai across all per-language sections (TypeScript, Python, Ruby, Go, Java, .NET): no mention of "batch" or "batches" anywhere. Its "What Braintrust traces" tables list only Chat Completion, Embedding, Moderation, openai.responses.create/parse/compact, Transcription, Translation, Speech, and Image Generation/Edit/Variation — the Batch API is absent for every language, not just Java. A broader site search only surfaces unrelated uses of "batch" (eval batches, UI batch labeling, batch-ingested span timestamps in the changelog).

Note: this repo's own gap-audit history already treats "Batch API not instrumented, mis-tagged as a generic LLM span" as a valid, in-scope finding — see the already-filed and still-open #155 for Anthropic's Message Batches API, which this issue mirrors for the OpenAI provider (a distinct upstream API/SDK, not a duplicate).

Upstream sources

• Official OpenAI Batch API reference: https://platform.openai.com/docs/api-reference/batch (create/retrieve/list/cancel sub-pages) and guide: https://platform.openai.com/docs/guides/batch — submit up to 50,000 requests (200MB input file) for async processing, typically within 24h, at a 50% cost discount vs. synchronous calls.
• Official openai-java SDK exposes this directly: openai-java-core/src/main/kotlin/com/openai/services/blocking/BatchService.kt and BatchServiceImpl.kt (https://github.com/openai/openai-java) define create(BatchCreateParams): Batch (POST /batches), retrieve(batchId), list(), and cancel(batchId) (POST /batches/{batch_id}/cancel). Also documented at https://developers.openai.com/api/reference/java/resources/batches.
• Request/response shape: create request has input_file_id, endpoint (/v1/chat/completions, /v1/embeddings, /v1/responses, or /v1/moderations), completion_window (currently only "24h"); output JSONL lines (fetched via output_file_id) have custom_id, response.body (the real chat-completion/embedding/responses result including usage), and error — structurally analogous to Anthropic's Message Batches results. Source: https://developers.openai.com/api/docs/guides/batch.

Local repo files inspected

• braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java — lines 110–141 (tagOpenAIRequest), 143–208 (tagOpenAIResponse), 518–529 (getSpanName)
• braintrust-sdk/instrumentation/openai_2_15_0/src/main/java/dev/braintrust/instrumentation/openai/v2_15_0/BraintrustOpenAI.java and TracingHttpClient.java — generic transport-swap; produces a span for any OpenAI HTTP call including /v1/batches, with no batch-specific logic
• Repo-wide grep for batch/Batch under braintrust-sdk/instrumentation/openai_2_15_0/ — zero matches (no test or example exercises this API)