Enable symbolication of native stack frames in ANR events #4061
+306
−34
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lealanko-rt
requested review from
adinauer,
romtsn,
stefanosiano and
markushi
as code owners
lealanko-rt
force-pushed
the
feat/anr-native-symbolication
branch
5 times, most recently
from
2435bd9 to
d828dde
Compare
added 4 commits
January 18, 2025 03:09
Add a button to trigger ANR by holding a lock too long in native code. This can be used to test native stack frames in ANR events.
Handle offsets and deleted files, recognize "???" as a marker for unknown functions. Use named capturing groups for better readability and editability.
Use the "native" attribute of stack frames to indicate JNI invocation frames, like SentryStackTraceFactory does.
lealanko-rt
force-pushed
the
feat/anr-native-symbolication
branch
from
d828dde to
eebfcdf
Compare
added 3 commits
January 20, 2025 10:32
The images are parsed from the build ids and filenames in the thread dump's stack frames.
The instruction addresses of native stack frames in thread dumps are relative to the image file from which the code is loaded, and there are no absolute mapping addresses of images available. So explicitly inform the Sentry server about the correct images by using a relative "addr_mode" attribute. Also add the attribute to the SentryStackFrame class since it was not yet supported by it. The field documentation is converted from event.schema.json in the sentry server repo.
lealanko-rt
force-pushed
the
feat/anr-native-symbolication
branch
from
eebfcdf to
b1f58bf
Compare
|
@lealanko-rt thanks for opening this up, that's amazing! We'll have a closer look at this asap. |
lealanko-rt
commented
Jan 22, 2025
| @@ -270,9 +269,9 @@ private void reportAsSentryEvent( | |||
| event.setMessage(sentryMessage); | |||
| } else if (result.type == ParseResult.Type.DUMP) { | |||
| event.setThreads(result.threads); | |||
| if (!result.debugImages.isEmpty()) { | |||
| if (result.debugImages != null) { | |||
There was a problem hiding this comment.
This won't ever be null, I think?
Hi, thanks for working on this. What is the workflow for PRs here? My own preference is to absorb all fixes into appropriate commits to produce a clean history, but I'm not sure if that's feasible with multiple people contributing to the branch... |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
📜 Description
Add instruction address information and debug image information to AnrV2 events. This enables symbolication of native stack frames in stripped libraries.
Before
After
💡 Motivation and Context
Earlier, it was nearly impossible to debug ANRs caused by native code in stripped libraries, since no address information was passed in the events. The introduction of
io.sentry.anr.attach-thread-dumps(#3791) at least made the raw data available for manual investigation, but that was toilsome.Symbolication of a stack frame requires an instruction address and a way of identifying a debug image. Both of these are available in the Android stack dumps that AnrV2 reads, the latter in the form of a BuildId. By sending this data and annotating the stack frames correctly, the Sentry server is able to symbolicate them.
💚 How did you test it?
Unit tests were modified to test all added features. Moreover, a button for triggering an ANR via native code was added to the sample Android application. The results can be seen in the screenshots above.
📝 Checklist
sendDefaultPIIis enabled.🔮 Next steps
Functionally this is pretty complete, I think. The rendering for managed frames in a hybrid stack (as seen in the second screenshot) is a bit ugly, and omits the module (i.e. package+class), so that could perhaps be improved a little.
A natural continuation of this work would be tackling #3295, which is fundamentally about the same thing, but needs to deserialize a protobuf tombstone instead of a textual dump, so there is not much code to be reused.