Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CMPCOV breaks compilation on LLVM16/18 in some cases #2079

Closed
0xricksanchez opened this issue May 10, 2024 · 9 comments
Closed

CMPCOV breaks compilation on LLVM16/18 in some cases #2079

0xricksanchez opened this issue May 10, 2024 · 9 comments

Comments

@0xricksanchez
Copy link

0xricksanchez commented May 10, 2024 •
edited

All experiments were conducted on the following setup:

  • OS Ubuntu 22.04.4 LTS
  • Compiler: afl-clang-fast + CMPCOV (s. below)
  • Python: 3.10.12
  • AFL++: 4.21a (commit: ad0d0c7)
  • LLVM: 16, 17, and 18

Note:: AFL++ was freshly compiled after switching to a different LLVM version!

Note: CMPLOG doesn't seem to show the below described behavior.

Setup

The different LLVM versions were installed side-by-side via the llvm.sh script as offered on the LLVM website
Setting the different LLVM versions to be the default was achieved by my littler helper script

Fetching libxml2

libxml2 has been downloaded from the RO-mirror on GitHub in the form of X.tar.gz releases, like so:

LIBXML2_VER = "2.9.14"
wget https://github.com/GNOME/libxml2/archive/refs/tags/v${LIBXML2_VER}.tar.gz
tar -xzvf v${LIBXML2_VER}.tar.gz
rm -rf v${LIBXML2_VER}.tar.gz

Compilation

cd libxml2-v${LIBXML2_VER}
autoreconf -fvi
// Edit: exported the AFL_* vars 
export AFL_LLVM_LAF_ALL=1
export AFL_HARDEN=1
CC=afl-clang-fast ./configure --disable-shared --prefix="$(pwd)/install" --without-ftp --without-http --without-legacy --without-python
make -j$(nproc)

Observations

  • Gist: (An) Older version(s) of libxml2 fail(s) to compile with CMPCOV and throws a stack-trace.
  • It seems in particular libxml2 versions v2.9.XX are affected, while libxml2 v2.10.0 seems to work just fine!
    • Diff between libxml2-2.19.14 and libxml2-2.10.0 here
  • The behavior was observed on LLVM 16 and LLVM 18. LLVM 17 does not exhibit this behavior as (parts of) CMPCOV were disabled due to bugs (s. below).

As for which libxml2 v2.9.x versions are affected, I tried multiple ones from v.2.9.14 down to v2.9.4 and all of them showed the same behavior to me.

Details

Find below the details. If useful, I can attach the mentioned XXX.sh and YYY.c files as suggested by the stack-trace

libxml2-2.19.14 | AFL 4.21a | LLVM 16

[+] Instrumented 5564 locations with no collisions (hardened mode) of which are 88 handled and 0 unhandled selects.
[+] Instrumented 3001 locations with no collisions (hardened mode) of which are 22 handled and 0 unhandled selects.
 #0 0x00007ff5995edc16 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/lib/llvm-16/bin/../lib/libLLVM-16.so.1+0xfbec16)
 #1 0x00007ff5995ebdb0 llvm::sys::RunSignalHandlers() (/usr/lib/llvm-16/bin/../lib/libLLVM-16.so.1+0xfbcdb0)
 #2 0x00007ff59952c600 (/usr/lib/llvm-16/bin/../lib/libLLVM-16.so.1+0xefd600)
 #3 0x00007ff598103520 (/lib/x86_64-linux-gnu/libc.so.6+0x42520)
 #4 0x00007ff5981579fc __pthread_kill_implementation ./nptl/pthread_kill.c:44:76
 #5 0x00007ff5981579fc __pthread_kill_internal ./nptl/pthread_kill.c:78:10
 #6 0x00007ff5981579fc pthread_kill ./nptl/pthread_kill.c:89:10
 #7 0x00007ff598103476 gsignal ./signal/../sysdeps/posix/raise.c:27:6
 #8 0x00007ff5980e97f3 abort ./stdlib/abort.c:81:7
 #9 0x00007ff5980e971b _nl_load_domain ./intl/loadmsgcat.c:1177:9
#10 0x00007ff5980fae96 (/lib/x86_64-linux-gnu/libc.so.6+0x39e96)
#11 0x00007ff595dd9058 /home/fuzz/tools/AFLplusplus/instrumentation/split-compares-pass.so.cc:0:0
#12 0x00007ff595dd4c8d llvm::detail::PassModel<llvm::Module, (anonymous namespace)::SplitComparesTransform, llvm::PreservedAnalyses, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) /usr/lib/llvm-16/include/llvm/IR/PassManagerInternal.h:89:5
#13 0x00007ff599756219 llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (/usr/lib/llvm-16/bin/../lib/libLLVM-16.so.1+0x1127219)
#14 0x00007ff5a1720396 (/usr/lib/llvm-16/bin/../lib/libclang-cpp.so.16+0x1aea396)
#15 0x00007ff5a1718208 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>) (/usr/lib/llvm-16/bin/../lib/libclang-cpp.so.16+0x1ae2208)
#16 0x00007ff5a1a73a4f (/usr/lib/llvm-16/bin/../lib/libclang-cpp.so.16+0x1e3da4f)
#17 0x00007ff5a08037e2 clang::ParseAST(clang::Sema&, bool, bool) (/usr/lib/llvm-16/bin/../lib/libclang-cpp.so.16+0xbcd7e2)
#18 0x00007ff5a241b565 clang::FrontendAction::Execute() (/usr/lib/llvm-16/bin/../lib/libclang-cpp.so.16+0x27e5565)
#19 0x00007ff5a2398754 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/usr/lib/llvm-16/bin/../lib/libclang-cpp.so.16+0x2762754)
#20 0x00007ff5a2493fe4 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/usr/lib/llvm-16/bin/../lib/libclang-cpp.so.16+0x285dfe4)
#21 0x000055bf4fe2bf4c cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/usr/lib/llvm-16/bin/clang+0x14f4c)
#22 0x000055bf4fe291f0 (/usr/lib/llvm-16/bin/clang+0x121f0)
#23 0x00007ff5a20749a2 (/usr/lib/llvm-16/bin/../lib/libclang-cpp.so.16+0x243e9a2)
#24 0x00007ff59952c39c llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (/usr/lib/llvm-16/bin/../lib/libLLVM-16.so.1+0xefd39c)
#25 0x00007ff5a20740c3 clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const (/usr/lib/llvm-16/bin/../lib/libclang-cpp.so.16+0x243e0c3)
#26 0x00007ff5a203f922 clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&, bool) const (/usr/lib/llvm-16/bin/../lib/libclang-cpp.so.16+0x2409922)
#27 0x00007ff5a203fb7e clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&, bool) const (/usr/lib/llvm-16/bin/../lib/libclang-cpp.so.16+0x2409b7e)
#28 0x00007ff5a205a2fd clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&) (/usr/lib/llvm-16/bin/../lib/libclang-cpp.so.16+0x24242fd)
#29 0x000055bf4fe288b5 clang_main(int, char**) (/usr/lib/llvm-16/bin/clang+0x118b5)
#30 0x00007ff5980ead90 __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#31 0x00007ff5980eae40 call_init ./csu/../csu/libc-start.c:128:20
#32 0x00007ff5980eae40 __libc_start_main ./csu/../csu/libc-start.c:379:5
#33 0x000055bf4fe25855 _start (/usr/lib/llvm-16/bin/clang+0xe855)
clang: error: clang frontend command failed with exit code 134 (use -v to see invocation)
Ubuntu clang version 16.0.6 (++20231112100510+7cbf1a259152-1~exp1~20231112100554.106)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-16/bin
[+] Instrumented 3170 locations with no collisions (hardened mode) of which are 19 handled and 0 unhandled selects.
Running split-switches-pass by [email protected]
Running compare-transform-pass by [email protected], extended by [email protected]
Replacing 10 calls to strcmp/memcmp/strncmp/strcasecmp/strncasecmp
clang: note: diagnostic msg:
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang: note: diagnostic msg: /tmp/xmlunicode-f77579.c
clang: note: diagnostic msg: /tmp/xmlunicode-f77579.sh
clang: note: diagnostic msg:

********************
Split-compare-newpass by [email protected], extended by [email protected] (splitting icmp to 8 bit)
make[2]: *** [Makefile:1286: xmlunicode.lo] Error 1
make[2]: *** Waiting for unfinished jobs....
Split-floatingpoint-compare-pass: 0 FP comparisons split
881 comparisons found

libxml2-2.19.14 | AFL 4.21a | LLVM 17

Compilation runs until the end and produces a binary as expected.

Note: The WARNING part is easily missed and should IMHO result in an abort. I missed this initially..

SanitizerCoveragePCGUARD++4.21a
[+] Instrumented 7694 locations with no collisions (hardened mode) of which are 253 handled and 0 unhandled selects.
Running split-switches-pass by [email protected]
Running compare-transform-pass by [email protected], extended by [email protected]
Replacing 33 calls to strcmp/memcmp/strncmp/strcasecmp/strncasecmp
Split-compare-newpass by [email protected], extended by [email protected] (splitting icmp to 8 bit)
WARNING: AFL++ splitting integer comparisons is disabled in LLVM 17 due bugs, switch to 16 or 18!
Split-floatingpoint-compare-pass: 66 FP comparisons split
66 comparisons found
SanitizerCoveragePCGUARD++4.21a

libxml2-2.19.14 | AFL 4.21a | LLVM 18

[+] Instrumented 2390 locations with no collisions (hardened mode) of which are 45 handled and 0 unhandled selects.
Running split-switches-pass by [email protected]
xpath.c:10499:9: warning: variable 'len' set but not used [-Wunused-but-set-variable]
 10499 |     int len = 0, l;
       |         ^
Running compare-transform-pass by [email protected], extended by [email protected]
Replacing 27 calls to strcmp/memcmp/strncmp/strcasecmp/strncasecmp
Split-compare-newpass by [email protected], extended by [email protected] (splitting icmp to 8 bit)
Split-floatingpoint-compare-pass: 0 FP comparisons split
 #0 0x00007f7e344a0256 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x3be9256)
 #1 0x00007f7e3449e210 llvm::sys::RunSignalHandlers() (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x3be7210)
 #2 0x00007f7e343efe70 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x3b38e70)
 #3 0x00007f7e3038b520 (/lib/x86_64-linux-gnu/libc.so.6+0x42520)
 #4 0x00007f7e303df9fc __pthread_kill_implementation ./nptl/pthread_kill.c:44:76
 #5 0x00007f7e303df9fc __pthread_kill_internal ./nptl/pthread_kill.c:78:10
 #6 0x00007f7e303df9fc pthread_kill ./nptl/pthread_kill.c:89:10
 #7 0x00007f7e3038b476 gsignal ./signal/../sysdeps/posix/raise.c:27:6
 #8 0x00007f7e303717f3 abort ./stdlib/abort.c:81:7
 #9 0x00007f7e3037171b _nl_load_domain ./intl/loadmsgcat.c:1177:9
#10 0x00007f7e30382e96 (/lib/x86_64-linux-gnu/libc.so.6+0x39e96)
#11 0x00007f7e2e030177 /usr/lib/llvm-18/include/llvm/Support/Casting.h:578:3
#12 0x00007f7e2e02bc3d llvm::detail::PassModel<llvm::Module, (anonymous namespace)::SplitComparesTransform, llvm::PreservedAnalyses, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) /usr/lib/llvm-18/include/llvm/IR/PassManagerInternal.h:89:5
#13 0x00007f7e3461c104 llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x3d65104)
#14 0x00007f7e3a4bb9a5 (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x26349a5)
#15 0x00007f7e3a4b41ea clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>, clang::BackendConsumer*) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x262d1ea)
#16 0x00007f7e3a843832 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x29bc832)
#17 0x00007f7e39474256 clang::ParseAST(clang::Sema&, bool, bool) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x15ed256)
#18 0x00007f7e3b2a4ef5 clang::FrontendAction::Execute() (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x341def5)
#19 0x00007f7e3b21e4e4 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x33974e4)
#20 0x00007f7e3b31f6be clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x34986be)
#21 0x0000562e3e443a25 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/usr/lib/llvm-18/bin/clang+0x15a25)
#22 0x0000562e3e440e25 (/usr/lib/llvm-18/bin/clang+0x12e25)
#23 0x00007f7e3aed6999 (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x304f999)
#24 0x00007f7e343efc0c llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x3b38c0c)
#25 0x00007f7e3aed632e clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x304f32e)
#26 0x00007f7e3ae9e8c1 clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&, bool) const (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x30178c1)
#27 0x00007f7e3ae9eb0e clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&, bool) const (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x3017b0e)
#28 0x00007f7e3aebb0ed clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x30340ed)
#29 0x0000562e3e440794 clang_main(int, char**, llvm::ToolContext const&) (/usr/lib/llvm-18/bin/clang+0x12794)
#30 0x0000562e3e44e286 main (/usr/lib/llvm-18/bin/clang+0x20286)
#31 0x00007f7e30372d90 __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#32 0x00007f7e30372e40 call_init ./csu/../csu/libc-start.c:128:20
#33 0x00007f7e30372e40 __libc_start_main ./csu/../csu/libc-start.c:379:5
#34 0x0000562e3e43d895 _start (/usr/lib/llvm-18/bin/clang+0xf895)
clang: error: clang frontend command failed with exit code 134 (use -v to see invocation)
Ubuntu clang version 18.1.3 (++20240322073153+ef6d1ec07c69-1~exp1~20240322193300.86)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-18/bin
Running split-switches-pass by [email protected]
Running split-switches-pass by [email protected]
688 comparisons found
SanitizerCoveragePCGUARD++4.21a
@0xricksanchez 0xricksanchez changed the title CMPCOV breaks compilation on LLVM16/18 CMPCOV breaks compilation on LLVM16/18 in some cases May 10, 2024
@0xricksanchez
Copy link
Author

I didn’t have the time to investigate other AFL++ versions yet or find other software packages that show the same behavior. Maybe putting it out here will bring some more information to light eventually

@vanhauser-thc
Copy link
Member

just quickly as I am on vacation: llvm 17, 18 and 19 worked for me on this specific libxml2 version with LAF_ALL without any errors

@vanhauser-thc
Copy link
Member

can you try versions from apt.llvm.org (with llvm.sh installed) please?

@0xricksanchez
Copy link
Author

0xricksanchez commented May 11, 2024 •
edited

I’ll try to purge all the LLVM versions again and test again..

@vanhauser-thc
Copy link
Member

actually I was wrong, because your how-to was wrong :)

make clean
autoreconf -fvi
CC=afl-clang-fast ./configure --disable-shared --without-ftp --without-http --without-legacy --without-python
export AFL_LLVM_LAF_ALL=1
make

I can reproduce this now

@vanhauser-thc
Copy link
Member

AFL_LLVM_LAF_SPLIT_COMPARES=1 'afl-clang-fast' '-DHAVE_CONFIG_H' '-I.' '-I./include' '-I./include' '-pedantic' '-Wall' '-Wextra' '-Wshadow' '-Wpointer-arith' '-Wcast-align' '-Wwrite-strings' '-Waggregate-return' '-Wstrict-prototypes' '-Wmissing-prototypes' '-Wnested-externs' '-Winline' '-Wredundant-decls' '-Wno-long-long' '-Wno-format-extra-args' '-D_REENTRANT' '-g' '-O2' '-MT' 'xmlunicode.lo' '-MD' '-MP' '-MF' '.deps/xmlunicode.Tpo' '-c' 'xmlunicode.c' '-o' 'xmlunicode.o'

@0xricksanchez
Copy link
Author

0xricksanchez commented May 12, 2024 •
edited

actually I was wrong, because your how-to was wrong :)

make clean
autoreconf -fvi
CC=afl-clang-fast ./configure --disable-shared --without-ftp --without-http --without-legacy --without-python
export AFL_LLVM_LAF_ALL=1
make

I can reproduce this now

Ah wow okay my bad.. of course the AFL_* environment vars need to be applied to the make step as well. Sorry, my bad! Should just attached my Makefile.toml (ref) :D …

Glad i wasn’t there imagining things though..

@vanhauser-thc
Copy link
Member

I pushed a fix to the dev branch

@0xricksanchez
Copy link
Author

0xricksanchez commented May 13, 2024 •
edited

Fix seems to work, as I was successfully able to compile multiple versions that didn't work before.
Tested with:

  • libxml2 v2.9.14 (failed before)
  • libxml2 v2.9.4 (failed before)
  • libxml2 v2.12.6 (worked before and still works)

I consider this issue resolved. Thanks for the prompt fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vanhauser-thc @0xricksanchez