Override dlsym instead of dlopen to correctly honour RPATH/RUNPATHS #525

We need to override dlopen/dlclose to account for new shared libraries being loaded in the process memory space. This is needed so we can correctly track allocations in those libraries by overriding their PLT entries and also so we can properly map the addresses of the symbols in those libraries when we resolve later native traces. Unfortunately, we can't just override dlopen directly because of the following edge case: when a shared library dlopen's another by name (e.g. dlopen("libfoo.so")), the dlopen call will honor the RPATH/RUNPATH of the calling library if it's set. Some libraries set an RPATH/RUNPATH based on $ORIGIN (the path of the calling library) to load dependencies from a relative directory based on the location of the calling library. This means that if we override dlopen, we'll end up loading the library from the wrong path or more likely, not loading it at all because the dynamic loader will think the memray extenion it's the calling library and the RPATH of the real calling library will not be honoured. To work around this, we override dlsym instead and override the symbols in the loaded libraries only the first time we have seen a handle passed to dlsym. This works because for a symbol from a given dlopen-ed library to appear in a call stack, *something* from that library has to be dlsym-ed first. The only exception to this are static initializers, but we cannot track those anyway by overriding dlopen as they run within the dlopen call itself. Signed-off-by: Pablo Galindo <[email protected]>

Now that we're patching `dlsym`, we need to be careful to always pick up the global symbol named `dlsym`, to avoid lldb considering our own namespaces and finding the wrong symbol's address. Signed-off-by: Matt Wozniski <[email protected]>

LLDB wants to resolve `&::malloc` to `memray::hooks::malloc` for reasons that I haven't fully managed to make sense of, but we can work around this by calling our hook `memray::hooks::memray_malloc` instead. Wrap that up in a macro to make it harder to forget the way to compute the hook object name from the hooked function's name. Signed-off-by: Matt Wozniski <[email protected]>

To avoid deadlocks and infinite recursion, we need to avoid patching symbols inside the linker shared objects. We have been diligently doing this with the GNU linker ("ld-linux") but not for the muslc linker ("ld-musl"). This has been working by chance because we use our recursion guards in many of the symbols we patch, including dlopen. The fact that we override dlopen and we use the guard was preventing a deadlock when using native traces in muslc, as fetching native traces ends adquiring a lock that it's shared by dlopen itself. Now that we do not patch dlopen, there is nothing preventing memray to try to fetch native traces in the inner calls to calloc() and that tries to adquire the dlopen lock which is not re-entrant. The proper fix is to avoid patching inside the muslc linker as we do for Linux. Signed-off-by: Pablo Galindo <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Override dlsym instead of dlopen to correctly honour RPATH/RUNPATHS #525

Override dlsym instead of dlopen to correctly honour RPATH/RUNPATHS #525

Commits on Feb 8, 2024