feat(native): symbolicated stacktraces for non-crashed threads on Linux#1747
Open
jpnurmi wants to merge 30 commits into
Open
feat(native): symbolicated stacktraces for non-crashed threads on Linux#1747jpnurmi wants to merge 30 commits into
jpnurmi wants to merge 30 commits into
Conversation
Add libunwind remote unwinding support to the native backend's crash daemon, enabling DWARF-based stack walking and symbol name resolution for all threads (not just the crashing thread) on Linux. - Re-add upstream libunwind src/ptrace/ (_UPT_* accessors) - Add `unwind_remote` CMake target with G-prefix + ptrace sources, only built when SENTRY_BACKEND=native on Linux - New sentry_remote_unwind.c using unw_init_remote() + unw_get_proc_name() - Integrate into daemon's build_stacktrace_for_thread() with fallback to pre-captured backtrace and FP-walking Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
Only free the captured stack buffer once a remote-unwind stacktrace is actually returned. If remote unwinding yields frames but all are filtered, the fallback stacktrace paths still need to own and clean up the buffer.
Avoid reserving the remote unwind frame buffer on the daemon stack. If the allocation fails, remote unwinding is skipped and the existing fallback stacktrace paths continue to run.
Move the Linux remote libunwind implementation under src/unwinder and route it through sentry_unwinder.c, matching the existing local unwinder backend selection pattern. Expose a backend-neutral sentry__unwind_stack_from_thread() entry point for the daemon while keeping the libunwind-specific implementation private to the backend file.
Skip remote thread unwinding for the crashed Linux thread when the signal handler already captured a backtrace. This keeps the signal-frame-aware ucontext unwind as the primary stacktrace and avoids replacing it with a potentially partial remote unwind from the handler wait state. Make sentry_unwinder.h self-contained now that sentry_unwinder.c includes it for remote unwinder dispatch.
Document that frame trust tracks the unwind source rather than the emitted frame index, so filtered initial frames do not cause subsequent CFI frames to be labeled as context frames.
Skip daemon-side ptrace unwinding for the crashed Linux thread so PTRACE_DETACH cannot resume it while crash processing is still running. The crashed thread continues to use the saved fault context and any pre-captured backtrace.
After attaching to a thread, require waitpid() to report a stopped tracee before entering libunwind's remote unwinder. This avoids calling unw_init_remote() when the attach/wait race reports a non-stopped thread.
Add the arm32 generic libunwind sources and target definitions to the vendored unwind_remote target so daemon-side remote unwinding is built consistently with the local arm32 unwinder.
The regular sentry target compiles the Linux thread-unwind dispatcher without the daemon-only remote unwinder backend enabled. Mark the dispatcher arguments as intentionally unused so Clang -Werror builds do not fail in that configuration.
Non-crashed Linux threads discovered by the daemon only have a TID and a zero-filled ucontext. Do not attach a registers object to successfully remote-unwound stacktraces for those threads.
Capture register values from the initial libunwind cursor for remote Linux stack walking and attach them to the produced stacktrace. Keep the remote thread unwinder API aligned with sentry_unwind_stack by returning the frame count directly, with registers passed as optional side metadata.
jpnurmi
changed the title
supervacuus
approved these changes
May 27, 2026
Collaborator
supervacuus
left a comment
supervacuus
left a comment
There was a problem hiding this comment.
LGTM. Just one documentation feedback and one build-cleanup.
…/libunwind-remote
jpnurmi
changed the title
…/libunwind-remote
When thread_idx == 0, tid was unconditionally set to ctx->crashed_tid, causing is_crashed_thread to be true. But the caller only invokes build_stacktrace_for_thread(ctx, 0) for non-crashed threads, so remote DWARF unwinding was incorrectly skipped for the first thread in the list whenever it wasn't the crashing thread. Remove the || thread_idx == 0 condition so thread 0 resolves its own tid from platform.threads[0].tid like every other non-crashed thread.
When SENTRY_LIBUNWIND_SYSTEM is used, the daemon's remote DWARF unwinding code needs _UPT_* ptrace accessor symbols from the separate libunwind-ptrace library, but only the core libunwind was linked. Add a pkg_check_modules for libunwind-ptrace and link it alongside PkgConfig::LIBUNWIND.
…hardcoding The vendored libunwind's config.h.cmake.in hardcoded all HAVE_DECL_PT_* symbols to 0 and added a comment claiming they were "not available on Linux". On modern Linux kernels with CONFIG_COMPAT, PT_GETREGS is in fact available via <sys/ptrace.h>, so the cmake build should detect it. - Replace hardcoded `#define ... 0` with `#cmakedefine01` in config.h.cmake.in so the values come from cmake detection - Add a check_c_source_compiles loop in CMakeLists.txt for the PT_* declarations, matching the existing PTRACE_* pattern - Switch _UPT_access_reg.c from `#if defined(HAVE_DECL_PT_GETREGS)` to `#if HAVE_DECL_PT_GETREGS` so that a value of 0 correctly disables the code path (defined() treats any definition, even 0, as true)
The blocking waitpid(tid, ..., __WALL) after PTRACE_ATTACH can hang indefinitely if the target thread is in uninterruptible sleep (D state). Replace it with a WNOHANG-based polling loop (50 × 100ms = 5s timeout) so the daemon logs a warning and continues instead of blocking forever.
jpnurmi
changed the title
- **Normal case**: waitpid returns on the first iteration (microseconds), so 20 retries add zero overhead vs 50. - **Transient D state**: I/O or page-fault wait usually resolves in well under 1s. 2s covers pathological cases without compounding latency for the user. - **Crash report latency**: The daemon holds all other threads in SIGSTOP while unwinding. Every extra second frozen delays the crash report and makes the perceived hang worse.
jpnurmi
changed the title
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 79f17f0. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add libunwind remote unwinding support to the native backend's crash daemon, enabling DWARF-based stack walking and symbol name resolution for non-crashed threads on Linux. Since #1764, only the crashing thread received a stacktrace; all other threads appeared without one.
src/ptrace/(_UPT_*accessors)unwind_remoteCMake target with G-prefix + ptrace sources, only built whenSENTRY_BACKEND=nativeon Linuxsentry_remote_unwind.cusingunw_init_remote()+unw_get_proc_name()build_stacktrace_for_thread()with fallback to pre-captured backtrace and FP-walkingSee also: