feat(symbols): key Android uploads by R8's map id, and upload dSYMs from an Xcode build - #769
Open
abelonogov-ld wants to merge 12 commits into
Open
feat(symbols): key Android uploads by R8's map id, and upload dSYMs from an Xcode build#769abelonogov-ld wants to merge 12 commits into
abelonogov-ld wants to merge 12 commits into
Conversation
`ldcli symbols upload --type android` needed a --path pointing at a directory holding mapping.txt and an --app-version matching what the app reports, so every project had to add build script code to stage the mapping somewhere and plumb its version into CI. The Android Gradle Plugin already writes both: R8's mapping at <module>/build/outputs/mapping/<variant>/mapping.txt, and the version it packaged in output-metadata.json beside the APK. Read them, and the command works from an Android project root with no flags and no build script at all. Several obfuscated variants is an error naming them rather than a guess, since only one of them is the build being shipped. An Android mapping is now also stored as mapping.txt however deep it was found. Symbolication reads it at <lane>/mapping.txt, so a mapping uploaded from a nested path was previously keyed somewhere nothing looks. Co-authored-by: Cursor <cursoragent@cursor.com>
A build that stamps a content-derived symbols id into assets/ld_symbols_id.txt had to also stage a mapping.txt.symbolsid sidecar next to a copy of the mapping, purely so the upload could be keyed by the same id the app reports. The packaged APK/AAB already carries that asset, and it is the app that will run: what it carries is exactly what will be reported. Reading it there needs nothing handed over by the build and leaves no way for the two to disagree, so the staging step goes away and a stamped build uploads on the Symbols Id Lane with no flags. An id that does not match the 32-hex-char shape the SDK reports is ignored rather than keyed on, since it names a lane nothing would ever ask for. Co-authored-by: Cursor <cursoragent@cursor.com>
A release R8 mapping is tens of megabytes of text and every build pushes it again: the e2e app's is 61.3 MB, which gzips to 5.0 MB in under half a second. React Native source maps and Apple symbol maps compress on the same order. Each object is marked Content-Encoding: gzip, so it stays self-describing — storage returns the header on read, an HTTP client inflates it in transit, and the backend inflates whatever still arrives compressed. Artifacts held in memory are compressed before an upload URL is asked for rather than at the point of sending, because the digest that proves an object is already stored is compared against the ETag of the stored bytes; hashing the artifact instead would never match and every source bundle would be re-sent. Files are compressed as they are read, streamed through the compressor so a large mapping is never held in memory whole. Anything that comes out of gzip no smaller is sent as it is, so a .srcbundle, which already gzips its own entries, is not wrapped a second time. Co-authored-by: Cursor <cursoragent@cursor.com>
… mapping An Android upload sent R8's mapping.txt as it was, tens of megabytes of text, and the backend parsed it into a random-access index on the first crash of every build that ever crashed. The mapping is here, on the machine that just produced it, so the index is built here too and the text is never stored: `upload` and `generate` now write mapping.v1.index to the Symbols Id and Version lanes. The mapping is streamed through r8index.EncodeFrom rather than parsed into memory, so a build machine does not have to find hundreds of megabytes to produce a few. A mapping that yields no index is an error: symbolication reads only the index, so storing anything else would leave a build looking like it has symbols while every crash arrives obfuscated. internal/symbols/r8index is a verbatim copy of the backend's package, the way dsymmap and srcbundle already are, and srcbundle.Builder now compresses on add so that a per-class bundle costs its compressed size rather than its raw one. Co-authored-by: Cursor <cursoragent@cursor.com>
The r8index package exists twice, here and in the backend that reads what this writes, and an index is only interchangeable if the two copies agree byte for byte. Two copies cannot import each other's tests, so what they share is a fixture: a readable mapping and the exact index bytes it encodes to, identical in both repos. Either side drifting now fails here rather than in production, where the symptom would be an unreadable index or a silently wrong frame. Co-authored-by: Cursor <cursoragent@cursor.com>
The golden fixture proves the encoder here still produces the bytes both repos agreed on; this proves the artifact a build machine wrote answers like the mapping it came from, which is the guarantee symbolication actually rests on. Opt-in via R8_MAPPING and R8_INDEX, since it needs a real release build. Co-authored-by: Cursor <cursoragent@cursor.com>
Keeps the r8index copy in step with the backend's, where reading these paths from R8_MAPPING and R8_INDEX fails a CI check that bans os.Getenv anywhere in the backend. A test flag is what the package already reaches for anyway (-update, for the golden fixture). end-of-file-fixer runs here too, through pre-commit, and it appended a newline to the backend's copy of the golden index — a byte that moves the footer the reader locates the index by, and breaks the byte-for-byte agreement the fixture exists to prove. It cannot tell a binary file from a text one, so *.index is excluded. Co-authored-by: Cursor <cursoragent@cursor.com>
abelonogov-ld
force-pushed
the
andrey/android-r8-map-id
branch
from
July 31, 2026 21:17
cf2ead4 to
b73c02b
Compare
abelonogov-ld
force-pushed
the
andrey/ios-xcode-upload
branch
from
July 31, 2026 21:20
307157b to
e31288e
Compare
Vadman97
approved these changes
Jul 31, 2026
#767 was squash-merged, so main carries as one commit what this branch has as three, and git saw both sides as having changed cmd/symbols/upload.go. The squash is byte-identical to what this branch started from and main has nothing else new, so the branch's own version stands and the merge changes no file. Co-authored-by: Cursor <cursoragent@cursor.com>
Generating an Android source bundle reads sourcePathFlag, which generate never registered: --source-path was rejected outright, so sources could only be scanned from the working directory. The read answered anyway, because both commands bind the same viper keys and upload's flag was left to supply the default — a value the caller of generate could neither see nor change. Also stop describing --include-sources as Apple-only, since an Android mapping carries sources now too. Co-authored-by: Cursor <cursoragent@cursor.com>
…pping An Android build only reached the Symbols Id Lane if something had stamped an id into the app for it, which meant a Gradle task in every project that wanted one. R8 has been recording an id for its own mapping all along — "# pg_map_id:" in the header — and from AGP 8.12 it stamps that same id into each class, so the shipped app already reports it on every frame of every crash. Read it from the header when no id was given or found in the packaged app, and key the upload by it. A project that adds nothing to its build now uploads to the lane its crashes arrive on; an id the app stamped still wins, since that is a build saying what it will report, and the only answer for one too old for R8 to stamp anything itself. Co-authored-by: Cursor <cursoragent@cursor.com>
abelonogov-ld
force-pushed
the
andrey/android-r8-map-id
branch
from
August 1, 2026 00:03
b73c02b to
e782b6a
Compare
A dSYM is best uploaded by the build that made it, which means a Run Script phase — and every project writing one wrote the same two pieces of shell first: a --path pointing at DWARF_DSYM_FOLDER_PATH, and a guard skipping the configurations that make no dSYM, without which a Debug build fails on "no .dSYM bundles found". Both are debugged inside a build phase, where the way you find out is a failed build. Read the folder from the build environment when no --path was given, and treat finding no dSYM there as nothing to do rather than as an error. What a phase has to say is now the project it uploads to. An explicit --path still wins, so uploading a dSYM from an archive or from CI is unchanged. Co-authored-by: Cursor <cursoragent@cursor.com>
abelonogov-ld
force-pushed
the
andrey/ios-xcode-upload
branch
from
August 1, 2026 00:03
e31288e to
5e93e97
Compare
#768 was squash-merged, so cmd/symbols/android_upload.go arrives from both sides and git could not tell that this branch's copy is main's plus one commit: the id R8 records for a mapping, read here and used to key the upload. Main's copy is byte-identical to this branch's before that commit, so ours stands and the merge changes no file. Co-authored-by: Cursor <cursoragent@cursor.com>
abelonogov-ld
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two independent changes, both about a build being able to upload what it just produced without being told anything.
An Android upload is keyed by the id R8 recorded for the mapping. A build only reached the Symbols Id Lane if something had stamped an id into the app, which meant a Gradle task in every project that wanted one. R8 has been recording an id for its own mapping all along —
# pg_map_id:in the header — and from AGP 8.12 it stamps that same id into each class, so the shipped app already reports it on every frame of every crash. It is read from the header when no id was given or found in the packaged app, so a project that adds nothing to its build uploads to the lane its crashes arrive on. An id the app stamped still wins: that is a build saying what it will report, and the only answer for one too old for R8 to stamp anything itself.A dSYM is uploaded by the Xcode build that made it. That means a Run Script phase, and every project writing one wrote the same two pieces of shell first: a
--pathpointing atDWARF_DSYM_FOLDER_PATH, and a guard skipping the configurations that make no dSYM, without which a Debug build fails on "no .dSYM bundles found". Both get debugged inside a build phase, where the way you find out is a failed build. The folder is now read from the build environment when no--pathwas given, and finding no dSYM there is nothing to do rather than an error — a Debug build keeps its debug information in the binary, and the message says how to change that. An explicit--pathstill wins, so uploading from an archive or from CI is unchanged.Ordering
The map-id half needs its backend counterpart deployed first: launchdarkly/observability#1620 teaches symbolication the
r8-map-id-…marker, and without it the CLI would upload to a lane nothing looks in. The dSYM half depends on nothing.Test plan
go test ./cmd/symbols/ ./internal/symbols/...pg_map_idis read from the mapping header, and an id the app stamped takes precedenceDWARF_DSYM_FOLDER_PATHis used when--pathis absent and ignored when it is givenMade with Cursor