Logs: optimize performance

[jpnurmi]

In sentry-unreal, UE_LOG can be wired into sentry-native logs. That means the native SDK may receive not only explicit game/application logs, but also high-volume Unreal Engine logs from many engine and game threads.

This makes log ingestion a hot path: in busy Unreal-like workloads, every log reads the global scope to apply attributes and then passes through the native log batcher. Under bursty multi-threaded logging, that exposes two main bottlenecks:

• logger/producer threads: global scope lock for applying attributes (#ff0000)
• batcher/consumer thread: log item serialization (#0000ff) and cleanup (#00ff00)

[linear-code]

NATIVE-200

[sentry-pierre]

Confirmed both bottlenecks in the current implementation:

• construct_log takes the global options lock (SENTRY_WITH_OPTIONS) and the global scope lock (SENTRY_WITH_SCOPE in sentry__apply_attributes) on every producer-thread call.
• sentry__batcher_flush serializes and cleans up log items on the single consumer thread.

The flame graph analysis lines up with the code. A maintainer can take it from here.

[jpnurmi]

Potential solutions

• Batcher: move/post-pone the cost of serialization to (an)other thread(s). Processing of batches becomes blazing fast and no longer prevents them from being queued. Could be a dedicated thread pool for all of telemetry, for example. Notice that moving the work to the transport thread might not be ideal, because it could create a congestion there and block more important envelopes. The very least, we would need a separate telemetry queue or introduce bgworker task priorities to let higher priority envelopes bypass the queue.

  

• Scope lock: replace the global scope mutex with a read-write lock to allow parallel reads