Ngram8 and laf-intel crashes on some targets during compilation

[tokatoka]

IMPORTANT

1. You have verified that the issue to be present in the current dev branch.

yes
3. Please supply the command line options and relevant environment variables,
e.g., a copy-paste of the contents of out/default/fuzzer_setup.

Thank you for making AFL++ better!

Describe the bug
Ngram & laf-intel crashes on some targets when compiling fuzzbench targets.

To Reproduce

1. clone fuzzbench. git clone [email protected]:google/fuzzbench.git
2. Apply these two patches to make two fuzzers. laf.txt and ngram8.txt. these patches will make two new fuzzer setup, aflplusplus_fuzzbench_laf and aflplusplus_fuzzbench_ngram8
3. run make debug-builder-aflplusplus_fuzzbench_laf-libxml2_xml to trigger laf-intel crashes
4. when it finishes building the fuzzer, type fuzzer_build to build the fuzzer
5. it crashes

+ make seed/xml.stamp
/afl/afl-clang-fast -DHAVE_CONFIG_H -I. -I..  -I../include -I../include  -pedantic -Wall -Wextra -Wshadow -Wpointer-arith -Wcast-align -Wwrite-strings -Waggregate-return -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Winline -Wno-long-long -Wno-format-extra-args -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -pthread -Wl,--no-as-needed -Wl,-ldl -Wl,-lm -Wno-unused-command-line-argument -O3 -MT genSeed.o -MD -MP -MF .deps/genSeed.Tpo -c -o genSeed.o genSeed.c
mv -f .deps/genSeed.Tpo .deps/genSeed.Po
/bin/bash ../libtool  --tag=CC   --mode=link /afl/afl-clang-fast -pedantic -Wall -Wextra -Wshadow -Wpointer-arith -Wcast-align -Wwrite-strings -Waggregate-return -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Winline -Wno-long-long -Wno-format-extra-args -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -pthread -Wl,--no-as-needed -Wl,-ldl -Wl,-lm -Wno-unused-command-line-argument -O3   -o genSeed genSeed.o fuzz.o ../libxml2.la 
libtool: link: /afl/afl-clang-fast -pedantic -Wall -Wextra -Wshadow -Wpointer-arith -Wcast-align -Wwrite-strings -Waggregate-return -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Winline -Wno-long-long -Wno-format-extra-args -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -pthread -Wl,--no-as-needed -Wl,-ldl -Wl,-lm -Wno-unused-command-line-argument -O3 -o genSeed genSeed.o fuzz.o  ../.libs/libxml2.a -lz -llzma -lm -pthread
./genSeed xml '../test/*' '../test/errors/*.xml' '../test/errors10/*.xml' '../test/namespaces/*' '../test/recurse/*.xml' '../test/SVG/*.xml' '../test/valid/*.xml' '../test/VC/*' '../test/VCM/*' '../test/xmlid/*'
make: *** [Makefile:855: seed/xml.stamp] Segmentation fault (core dumped)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/src/fuzzers/aflplusplus_fuzzbench_laf/fuzzer.py", line 178, in build
    utils.build_benchmark()
  File "/src/fuzzers/utils.py", line 81, in build_benchmark
    subprocess.check_call(['/bin/bash', '-ex', build_script], env=env)
  File "/usr/local/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/bin/bash', '-ex', '/src/build.sh']' returned non-zero exit status 2.
root@1f97bfc9ae95:/src/libxml2# exit

6. Same for ngram8, run debug-builder-aflplusplus_fuzzbench_ngram8-php_php-fuzz-parser_0dbedb, type fuzzer_build
7. it fails to pass tests

Build complete.
Don't forget to run 'make test'.

+ sapi/cli/php sapi/fuzzer/generate_all.php
=================================================================
==129165==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7ffe92e41220 at pc 0x000002239ff0 bp 0x7ffe92e40d90 sp 0x7ffe92e40d88
WRITE of size 8 at 0x7ffe92e41220 thread T0
    #0 0x2239fef in register_class_Error /src/php-src/Zend/zend_exceptions_arginfo.h:288:133
    #1 0x2236b6a in zend_register_default_exception /src/php-src/Zend/zend_exceptions.c:756:18
    #2 0x2392587 in zend_register_default_classes /src/php-src/Zend/zend_default_classes.c:35:2
    #3 0x1cdb562 in zm_startup_core /src/php-src/Zend/zend_builtin_functions.c:38:2
    #4 0x1c2171a in zend_startup_module_ex /src/php-src/Zend/zend_API.c:2217:7
    #5 0x1c26d88 in zend_startup_module_zval /src/php-src/Zend/zend_API.c:2232:10
    #6 0x1cb1ee3 in zend_hash_apply /src/php-src/Zend/zend_hash.c:2006:13
    #7 0x1c25f66 in zend_startup_modules /src/php-src/Zend/zend_API.c:2343:2
    #8 0x16bd17a in php_module_startup /src/php-src/main/main.c:2253:2
    #9 0x27ba11d in php_cli_startup /src/php-src/sapi/cli/php_cli.c:410:9
    #10 0x27ac4e0 in main /src/php-src/sapi/cli/php_cli.c:1300:6
    #11 0x7fb36dd65082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)
    #12 0x60120d in _start (/src/php-src/sapi/cli/php+0x60120d)

Address 0x7ffe92e41220 is located in stack of thread T0
SUMMARY: AddressSanitizer: stack-use-after-scope /src/php-src/Zend/zend_exceptions_arginfo.h:288:133 in register_class_Error
Shadow bytes around the buggy address:
  0x1000525c01f0: 00 00 cb cb cb cb cb cb ca ca ca ca 00 00 cb cb
  0x1000525c0200: cb cb cb cb ca ca ca ca 00 00 cb cb cb cb cb cb
  0x1000525c0210: ca ca ca ca 00 00 cb cb cb cb cb cb ca ca ca ca
  0x1000525c0220: 00 00 cb cb cb cb cb cb ca ca ca ca 00 00 cb cb
  0x1000525c0230: cb cb cb cb ca ca ca ca 00 00 cb cb cb cb cb cb
=>0x1000525c0240: ca ca ca ca[f8]f8 cb cb cb cb cb cb ca ca ca ca
  0x1000525c0250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000525c0260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000525c0270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000525c0280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 cb
  0x1000525c0290: cb cb cb cb cb cb cb cb cb cb cb cb 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==129165==ABORTING
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/src/fuzzers/aflplusplus_fuzzbench_ngram8/fuzzer.py", line 197, in build
    utils.build_benchmark(env=new_env)
  File "/src/fuzzers/utils.py", line 81, in build_benchmark
    subprocess.check_call(['/bin/bash', '-ex', build_script], env=env)
  File "/usr/local/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/bin/bash', '-ex', '/src/build.sh']' returned non-zero exit status 1.

Expected behavior
It should build successfully

Screen output/Screenshots
If applicable, add copy-paste of the screen output or screenshot that shows the issue. Please ensure the output is in English and not in Chinese, Russian, German, etc.

Additional context
Add any other context about the problem here.
laf.txt
ngram8.txt

[tokatoka]

I'll examine laf

[vanhauser-thc]

with your patch I do not think it is possible that ngram is used.

there is a hardcoded part in fuzzer.py:

    if build_flags.find(
            'array-bounds'
    ) != -1 and 'qemu' not in build_modes and 'classic' not in build_modes:
        if 'gcc' not in build_modes:
            build_modes[0] = 'native'

that is because a few bug benchmarks had issues with our pcguard implementation. that is now fixed though.

[tokatoka]

so you mean i should use 'tracepc' now? sure i can try

[tokatoka]

yeah ctx has the same asan crash.
it is possible that this is due to classic instrumentation

[tokatoka]

no.. i can't use it
I get this

[-] PROGRAM ABORT : CALLER, CTX and NGRAM instrumentation options can only be used with the LLVM CLASSIC instrumentation mode.
         Location : main(), src/afl-cc.c:2449

[vanhauser-thc]

build_modes = ['classic', 'ngram8']

[vanhauser-thc]

hmm ngram8 + php worked fine for me, build cleanly.

and assembly shows that ngram8 was used to instrument:

   0x0000000002622b07 <+199>:	pxor   %xmm1,%xmm2
   0x0000000002622b0b <+203>:	movdqa %xmm2,%xmm1
   0x0000000002622b0f <+207>:	psrld  $0x10,%xmm1
   0x0000000002622b14 <+212>:	pxor   %xmm2,%xmm1
   0x0000000002622b18 <+216>:	pextrw $0x0,%xmm1,%ecx
   0x0000000002622b1d <+221>:	mov    $0x302ccf8,%rdx
   0x0000000002622b24 <+228>:	mov    (%rdx),%rsi
   0x0000000002622b27 <+231>:	xor    $0xd5f3,%ecx
   0x0000000002622b2d <+237>:	mov    (%rsi,%rcx,1),%dl
   0x0000000002622b30 <+240>:	add    $0x1,%dl
   0x0000000002622b33 <+243>:	adc    $0x0,%dl
   0x0000000002622b36 <+246>:	mov    %dl,(%rsi,%rcx,1)

tried twice

[tokatoka]

build_modes = ['classic', 'ngram8']

I'm trying this. still building

[tokatoka]

yes it built
😅
my setting is wrong
thanks

[vanhauser-thc]

hmm with your setup it crashes but I do not understand why. the location that asan reports makes no sense.

the only real difference is that you install a clang-15 from llvm and are not using the existing one. the afl++ commit does not make a difference.

btw this in your setup does not work no_cmplog = True # no cmplog, wrong location. (what I mean is: cmplog is built but you do not use it. just prolongs build time)

[vanhauser-thc]

btw it requires ['classic', 'cmplog'] to crash, ngram is not required.

[vanhauser-thc]

the crash is this:

  if ( *(_BYTE *)(((unsigned __int64)&v45 >> 3) + 0x7FFF8000) )
  {
LABEL_96:
    ((void (__fastcall *)(zend_string *, __int64))_asan_report_store8)(p_ptr1, v6);
    goto LABEL_97;
  }

asan use after scope crash.

I do not get it. just classic instrumentation is no issue, just cmplog is no issue, the combination is an issue however this code part has neither. the whole function has only a coverage instrumentation at the function entry point and nowhere else, and there is no cmplog in this function.