Summary
EventToSpanEventBridgeProcessor (proposed in #4309) bridges log-record events onto the live Span referenced by the log record's resolved context. It needs a LogRecordProcessor that can:
- read the resolved context (
Context/live Span) at OnEmit() time
- read the log record's own data (body, attributes) without taking exclusive ownership away from downstream processors/exporters
The current LogRecordProcessor interface doesn't support this shape of "non-exporting", read/mutate pipeline processor, so the bridge processor can't be implemented correctly without new SDK architecture.
Background
From review on #4309:
Processors like the bridge processor will require some new architectural components and changes to existing components to be spec compliant and maintain reasonable performance. This is in addition to the Logger API level EmitLogRecordWithContext change, which is needed.
The architecture components / changes that seem to be required include:
- Standardize the SDK built-in exporters (OTLP, ostream) on a recordable with a read-write interface. Pipeline processors need to read data from the record and not duplicate it.
- Create a new processor interface for composable pipeline processors that can read and mutate data but do not own the recordable life-cycle.
- Create a new root level processor that supports pipeline processors
- Update the
LoggerContext to support setting a new root processor.
The concrete problem: with today's LogRecordProcessor interface, a configuration like
logger_provider:
processors:
my_logs_filter:
my_log_record_sanitizer:
event_to_span_bridge:
batch:
exporter:
otlp_http:
fans out via MultiLogRecordProcessor instead of forming a pipeline: each processor gets an independent copy of the record via OnEmit(), which transfers ownership. That means N copies of the record (N times the recording cost) and the exporter at the end receives the raw, unfiltered/unsanitized record rather than the output of the upstream stages, since mutations made by an earlier stage are made on a copy that is then discarded.
See the full discussion: #4309 (review) and #4309 (comment)
Scope
- Standardize SDK built-in exporters (OTLP, ostream) on a
ReadWriteLogRecord-style recordable.
- New processor interface for composable pipeline processors that mutate data in place without owning the recordable lifecycle.
- New root-level processor implementing a pipeline of these composable processors.
LoggerContext support for installing this new root processor.
- Once the above lands, implement
EventToSpanEventBridgeProcessor against the new interface and reintroduce it into declarative configuration (currently the config model/parser accepts the event_to_span_bridge processor block, but SdkBuilder only logs a warning that it is not yet supported; see the FIXME-SDK comment in sdk_builder.cc).
Status
Pending design and implementation of the components above. #4309 was scoped down to just the configuration model and YAML parser per this discussion, with the actual processor implementation deferred until this issue is resolved.
Summary
EventToSpanEventBridgeProcessor(proposed in #4309) bridges log-record events onto the liveSpanreferenced by the log record's resolved context. It needs aLogRecordProcessorthat can:Context/liveSpan) atOnEmit()timeThe current
LogRecordProcessorinterface doesn't support this shape of "non-exporting", read/mutate pipeline processor, so the bridge processor can't be implemented correctly without new SDK architecture.Background
From review on #4309:
The concrete problem: with today's
LogRecordProcessorinterface, a configuration likefans out via
MultiLogRecordProcessorinstead of forming a pipeline: each processor gets an independent copy of the record viaOnEmit(), which transfers ownership. That means N copies of the record (N times the recording cost) and the exporter at the end receives the raw, unfiltered/unsanitized record rather than the output of the upstream stages, since mutations made by an earlier stage are made on a copy that is then discarded.See the full discussion: #4309 (review) and #4309 (comment)
Scope
ReadWriteLogRecord-style recordable.LoggerContextsupport for installing this new root processor.EventToSpanEventBridgeProcessoragainst the new interface and reintroduce it into declarative configuration (currently the config model/parser accepts theevent_to_span_bridgeprocessor block, butSdkBuilderonly logs a warning that it is not yet supported; see theFIXME-SDKcomment insdk_builder.cc).Status
Pending design and implementation of the components above. #4309 was scoped down to just the configuration model and YAML parser per this discussion, with the actual processor implementation deferred until this issue is resolved.