Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions docs/introduction/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,28 @@ Additional parameters:
- `opentelemetry_excluded_urls` - extra regexes for URLs excluded from tracing; the metrics path and (unless health-check spans are enabled) the health-check path are excluded automatically. Litestar searches them in the request path, FastAPI and FastMCP in the full URL. FastAPI and FastMCP raise `ConfigurationError` at config construction if an entry contains a comma because OpenTelemetry uses commas to separate patterns. Litestar accepts commas in config entries. The environment variable OpenTelemetry defines for the framework (`OTEL_PYTHON_LITESTAR_EXCLUDED_URLS`, `OTEL_PYTHON_FASTAPI_EXCLUDED_URLS` or `OTEL_PYTHON_STARLETTE_EXCLUDED_URLS` for FastMCP), falling back to `OTEL_PYTHON_EXCLUDED_URLS`, adds to them.
- `opentelemetry_metrics_endpoint` - will be passed to `OTLPMetricExporter` as endpoint, and turns on the metrics signal. Its own field rather than a flag on `opentelemetry_endpoint`, so upgrading never starts exporting metrics you did not ask for. Under `opentelemetry_exporter_protocol="http"` this is a full URL (e.g. `http://collector:4318/v1/metrics`); `opentelemetry_exporter_protocol` and `opentelemetry_insecure` are shared with traces.

### Tracing without span processors

lite-bootstrap installs an SDK `TracerProvider` only when it adds a span processor:
Pyroscope span correlation, `opentelemetry_log_traces=True`, or a trace endpoint
with its exporter installed. Metrics-only configuration does not create an SDK
tracer provider. A trace endpoint with a missing exporter still warns, but uses
no-op tracing if no other span processor is configured.

Without span processors, framework and custom instrumentors receive a local
`NoOpTracerProvider` unless an external tracer provider is already installed.
An existing external provider is preserved and is never shut down by lite-bootstrap.
Install an external provider before bootstrap if it should record framework spans.
The no-op path leaves the global provider registration slot available, but installing
an external provider afterward does not replace the no-op provider already passed
to framework and custom instrumentors.

Code that previously added a span processor to `get_tracer_provider()` after a
metrics-only bootstrap must instead create and install its own SDK provider before
bootstrap. For metrics-only requests without external tracing, spans do not record
and JSON logs omit `tracing`; there are no `span_id` or `trace_id` values to inject.
Metrics collection is unchanged.

### Metrics

Left unset, no `MeterProvider` is installed and the metrics signal stays off. Set it, and the
Expand Down
16 changes: 16 additions & 0 deletions docs/migration/to-2.x.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,19 @@ warnings.filterwarnings("error", category=InstrumentDependencyMissingWarning)

Unconfigured instruments continue to be skipped silently. A configured instrument
with a missing optional dependency emits `InstrumentDependencyMissingWarning`.

## Metrics-only OpenTelemetry tracing

Metrics-only configuration no longer installs an SDK `TracerProvider` or records
unexported framework spans. The same applies when a trace endpoint's exporter is
missing and no other span processor is configured; the missing-exporter warning remains.
JSON logs from these requests omit `tracing` unless external tracing records a span.

If you add span processors yourself, create an SDK tracer provider, attach your
processors, and install it with `opentelemetry.trace.set_tracer_provider()` before
bootstrap. lite-bootstrap preserves that external provider and leaves its shutdown
to your application. Installing a provider after bootstrap does not activate tracing
for framework and custom instrumentors already given a no-op provider.

Configurations with a working trace exporter, stdout span logging, or Pyroscope
span correlation continue to use an SDK tracer provider.
3 changes: 1 addition & 2 deletions lite_bootstrap/bootstrappers/fastapi_bootstrapper.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@

if import_checker.is_opentelemetry_installed:
from opentelemetry.metrics import get_meter_provider
from opentelemetry.trace import get_tracer_provider

if import_checker.is_fastapi_opentelemetry_installed:
from opentelemetry.instrumentation.asgi import OpenTelemetryMiddleware
Expand Down Expand Up @@ -213,7 +212,7 @@ def bootstrap(self) -> None:
super().bootstrap()
FastAPIInstrumentor.instrument_app(
app=self.bootstrap_config.app,
tracer_provider=get_tracer_provider(),
tracer_provider=self.tracer_provider,
meter_provider=get_meter_provider(),
excluded_urls=",".join(self._build_excluded_url_patterns("FASTAPI")),
exclude_spans=self.bootstrap_config.opentelemetry_exclude_spans,
Expand Down
3 changes: 1 addition & 2 deletions lite_bootstrap/bootstrappers/fastmcp_bootstrapper.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@

if import_checker.is_fastmcp_opentelemetry_installed:
from opentelemetry.metrics import get_meter_provider
from opentelemetry.trace import get_tracer_provider
from starlette.routing import BaseRoute, Match, Mount, Route
from starlette.types import Scope

Expand Down Expand Up @@ -241,7 +240,7 @@ def _instrument_http_app(self, http_application: "StarletteWithLifespan") -> "St
),
excluded_urls=",".join(self._build_excluded_url_patterns("STARLETTE")),
exclude_spans=self.bootstrap_config.opentelemetry_exclude_spans,
tracer_provider=get_tracer_provider(),
tracer_provider=self.tracer_provider,
meter_provider=get_meter_provider(),
)
setattr(http_application, _OPENTELEMETRY_INSTRUMENTED_MARKER, True)
Expand Down
4 changes: 2 additions & 2 deletions lite_bootstrap/bootstrappers/faststream_bootstrapper.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ async def after_processed(
if import_checker.is_opentelemetry_installed:
from opentelemetry import trace
from opentelemetry.metrics import Meter, MeterProvider, get_meter_provider
from opentelemetry.trace import TracerProvider, get_tracer_provider
from opentelemetry.trace import TracerProvider

tracer: typing.Final = trace.get_tracer(__name__)

Expand Down Expand Up @@ -227,7 +227,7 @@ def bootstrap(self) -> None:
config.application.brokers, config.opentelemetry_middleware_cls, "opentelemetry_middleware_cls"
):
broker.add_middleware(
middleware_cls(tracer_provider=get_tracer_provider(), meter_provider=get_meter_provider())
middleware_cls(tracer_provider=self.tracer_provider, meter_provider=get_meter_provider())
)


Expand Down
3 changes: 1 addition & 2 deletions lite_bootstrap/bootstrappers/litestar_bootstrapper.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,6 @@

if import_checker.is_opentelemetry_installed:
from opentelemetry.metrics import get_meter_provider
from opentelemetry.trace import get_tracer_provider

if import_checker.is_structlog_installed:
import structlog
Expand Down Expand Up @@ -382,7 +381,7 @@ def bootstrap(self) -> None:
application_config.plugins.append(
OpenTelemetryPlugin(
LitestarOpenTelemetryConfig(
tracer_provider=get_tracer_provider(),
tracer_provider=self.tracer_provider,
meter_provider=get_meter_provider(),
scope_span_details_extractor=build_litestar_route_details_from_scope,
# An empty list compiles to a pattern that matches, and so skips, every path
Expand Down
61 changes: 45 additions & 16 deletions lite_bootstrap/instruments/opentelemetry_instrument.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,15 @@
SimpleSpanProcessor,
SpanExporter,
)
from opentelemetry.trace import Span, format_span_id, get_tracer_provider, set_tracer_provider
from opentelemetry.trace import (
NoOpTracerProvider,
ProxyTracerProvider,
Span,
format_span_id,
get_tracer_provider,
set_tracer_provider,
)
from opentelemetry.trace import TracerProvider as APITracerProvider

if import_checker.is_otlp_grpc_exporter_installed:
# opentelemetry-api can be present without the grpc otlp exporter package (e.g.
Expand Down Expand Up @@ -177,7 +185,11 @@ def on_end(self, span: "ReadableSpan") -> None:
class OpenTelemetryInstrument(BaseInstrument[OpenTelemetryConfigT]):
"""OpenTelemetry tracing instrument.

Lifecycle note: ``bootstrap()`` calls ``opentelemetry.trace.set_tracer_provider``,
Without span processors, framework and custom instrumentors use an existing external
provider or a local no-op provider. The global registration slot stays untouched.

Lifecycle note: when span processors are configured, ``bootstrap()`` calls
``opentelemetry.trace.set_tracer_provider``,
which the OTel SDK enforces as **set-once per process** (subsequent calls log
"Overriding of current TracerProvider is not allowed" and have no effect).
``teardown()`` calls ``shutdown()`` on the provider, which flushes batched
Expand All @@ -194,6 +206,7 @@ class OpenTelemetryInstrument(BaseInstrument[OpenTelemetryConfigT]):
_tracer_provider: "TracerProvider | None" = dataclasses.field(
default_factory=lambda: None, init=False, repr=False, compare=False
)
tracer_provider: "APITracerProvider" = dataclasses.field(init=False, repr=False, compare=False)
_meter_provider: "MeterProvider | None" = dataclasses.field(
default_factory=lambda: None, init=False, repr=False, compare=False
)
Expand Down Expand Up @@ -337,7 +350,7 @@ def _bootstrap_metrics(self, resource: "resources.Resource") -> None:
)
self._meter_provider = meter_provider

def _apply_instrumentors(self, tracer_provider: "TracerProvider") -> None:
def _apply_instrumentors(self, tracer_provider: "APITracerProvider") -> None:
for one_instrumentor in self.bootstrap_config.opentelemetry_instrumentors:
if isinstance(one_instrumentor, InstrumentorWithParams):
one_instrumentor.instrumentor.instrument(
Expand All @@ -347,25 +360,41 @@ def _apply_instrumentors(self, tracer_provider: "TracerProvider") -> None:
else:
one_instrumentor.instrument(tracer_provider=tracer_provider)

def _build_span_processors(self) -> "list[SpanProcessor]":
config = self.bootstrap_config
processors: list[SpanProcessor] = []
if import_checker.is_pyroscope_installed and getattr(config, "pyroscope_endpoint", None):
processors.append(PyroscopeSpanProcessor())
if config.opentelemetry_log_traces:
processors.append(SimpleSpanProcessor(ConsoleSpanExporter(formatter=_format_span)))
if config.opentelemetry_endpoint and (span_exporter := self._build_span_exporter()):
processors.append(BatchSpanProcessor(span_exporter))
return processors

def bootstrap(self) -> None:
config = self.bootstrap_config
self._silence_otel_loggers()
resource = self._build_resource()
tracer_provider = TracerProvider(resource=resource, sampler=config.opentelemetry_sampler)
set_tracer_provider(tracer_provider)
if get_tracer_provider() is not tracer_provider:
warn_at_caller(
"a TracerProvider is already installed; the configured exporter, sampler and resource will not be used"
processors = self._build_span_processors()
if processors:
tracer_provider = TracerProvider(resource=resource, sampler=config.opentelemetry_sampler)
self._tracer_provider = tracer_provider
for processor in processors:
tracer_provider.add_span_processor(processor)
set_tracer_provider(tracer_provider)
if get_tracer_provider() is not tracer_provider:
warn_at_caller(
"a TracerProvider is already installed; "
"the configured exporter, sampler and resource will not be used"
)
self.tracer_provider = get_tracer_provider()
else:
existing_provider = get_tracer_provider()
self.tracer_provider = (
NoOpTracerProvider() if isinstance(existing_provider, ProxyTracerProvider) else existing_provider
)
self._tracer_provider = tracer_provider
if import_checker.is_pyroscope_installed and getattr(config, "pyroscope_endpoint", None):
tracer_provider.add_span_processor(PyroscopeSpanProcessor())
if config.opentelemetry_log_traces:
tracer_provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter(formatter=_format_span)))
if config.opentelemetry_endpoint and (span_exporter := self._build_span_exporter()):
tracer_provider.add_span_processor(BatchSpanProcessor(span_exporter))
self._bootstrap_metrics(resource)
self._apply_instrumentors(tracer_provider)
self._apply_instrumentors(self.tracer_provider)

def teardown(self) -> None:
with collect_teardown_errors() as teardown_errors:
Expand Down
98 changes: 97 additions & 1 deletion tests/instruments/test_opentelemetry_instrument.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from unittest.mock import patch

import pytest
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter as OTLPGrpcMetricExporter
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter as OTLPHttpMetricExporter
from opentelemetry.instrumentation.instrumentor import BaseInstrumentor
Expand All @@ -16,6 +17,8 @@
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.sdk.trace import TracerProvider as SDKTracerProvider
from opentelemetry.sdk.trace import sampling
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
from opentelemetry.trace import set_tracer_provider

import lite_bootstrap.instruments.opentelemetry_instrument as otel_module
Expand All @@ -26,7 +29,12 @@
OpenTelemetryConfig,
OpenTelemetryInstrument,
)
from tests.conftest import CustomInstrumentor, emulate_package_missing_with_module_reload, warning_source_files
from tests.conftest import (
CustomInstrumentor,
emulate_package_missing_with_module_reload,
in_memory_opentelemetry_metric_reader,
warning_source_files,
)


def test_opentelemetry_instrument() -> None:
Expand Down Expand Up @@ -716,3 +724,91 @@ def test_bootstrap_warns_when_a_meter_provider_is_already_installed() -> None:
"a MeterProvider is already installed; the configured metrics exporter and resource will not be used"
]
assert warning_source_files(caught, UserWarning) == [__file__]


class TracerCapturingInstrumentor(BaseInstrumentor):
provider: trace.TracerProvider

def instrumentation_dependencies(self) -> typing.Collection[str]:
return []

def _instrument(self, **kwargs: object) -> None:
self.provider = typing.cast("trace.TracerProvider", kwargs["tracer_provider"])

def _uninstrument(self, **kwargs: object) -> None:
pass


def test_metrics_only_tracing_is_noop_without_installing_an_sdk_provider() -> None:
custom = TracerCapturingInstrumentor()
instrument = OpenTelemetryInstrument(
bootstrap_config=OpenTelemetryConfig(
opentelemetry_metrics_endpoint="localhost:4317",
opentelemetry_instrumentors=[custom],
)
)
with in_memory_opentelemetry_metric_reader():
try:
instrument.bootstrap()
assert not isinstance(trace.get_tracer_provider(), SDKTracerProvider)
with custom.provider.get_tracer(__name__).start_as_current_span("discarded") as span:
assert not span.is_recording()
assert isinstance(get_meter_provider(), SDKMeterProvider)
finally:
instrument.teardown()


def test_metrics_only_tracing_preserves_an_external_provider_and_its_lifetime() -> None:
external = SDKTracerProvider()
exporter = InMemorySpanExporter()
external.add_span_processor(SimpleSpanProcessor(exporter))
trace.set_tracer_provider(external)
custom = TracerCapturingInstrumentor()
instrument = OpenTelemetryInstrument(
bootstrap_config=OpenTelemetryConfig(
opentelemetry_metrics_endpoint="localhost:4317",
opentelemetry_instrumentors=[custom],
)
)
try:
with in_memory_opentelemetry_metric_reader(), warnings.catch_warnings(record=True) as caught:
warnings.simplefilter("always")
instrument.bootstrap()
assert not caught
assert custom.provider is external
with custom.provider.get_tracer(__name__).start_as_current_span("external"):
pass
instrument.teardown()
with external.get_tracer(__name__).start_as_current_span("after-teardown"):
pass
assert trace.get_tracer_provider() is external
assert [span.name for span in exporter.get_finished_spans()] == ["external", "after-teardown"]
finally:
instrument.teardown()
external.shutdown()


@pytest.mark.parametrize("protocol", ["grpc", "http"])
def test_configured_exporter_still_records_and_exports_spans(protocol: typing.Literal["grpc", "http"]) -> None:
exporter = InMemorySpanExporter()
custom = TracerCapturingInstrumentor()
instrument = OpenTelemetryInstrument(
bootstrap_config=OpenTelemetryConfig(
opentelemetry_endpoint="localhost:4317",
opentelemetry_exporter_protocol=protocol,
opentelemetry_instrumentors=[custom],
)
)
exporter_name = "OTLPGrpcSpanExporter" if protocol == "grpc" else "OTLPHttpSpanExporter"
try:
with patch.object(otel_module, exporter_name, return_value=exporter):
instrument.bootstrap()
provider = trace.get_tracer_provider()
assert isinstance(provider, SDKTracerProvider)
assert custom.provider is provider
with custom.provider.get_tracer(__name__).start_as_current_span("exported") as span:
assert span.is_recording()
provider.force_flush()
assert [span.name for span in exporter.get_finished_spans()] == ["exported"]
finally:
instrument.teardown()
Loading
Loading