index.md

title
Report

{% capture template %}

Experiment Configuration

• Duration: 8h
• Trials: 10
• Version: v1.0

Fuzz Targets

This report summarizes the results of fuzzing the following targets:

• libpng
• libtiff
• libxml2

Evaluated Fuzzers

The fuzzers used in this evaluation are listed below:

• afl
• afl_asan
• aflplusplus_lto_asan
• moptafl_asan
• pst_aflplusplus
• pst_aflplusplus_recent_commit

Experiment Summary

Total Unique Bugs Triggered

• Mean and Standard Deviation

  This plot shows the mean number of unique bugs triggered by every fuzzer against every target, and the standard deviation bar, across all campaigns.

• Stastistical Significance

  This matrix summarizes the p-values of the pairwise Mann-Whitney U tests calculated against the total bug count sample sets collected for every fuzzer across all campaigns. Cells with a green shade indicate that the number of bugs triggered by a fuzzer is statistically different.

Mean Survival Time

• Ranking of Bugs and Fuzzers

  This table shows the restricted mean survival time for every bug being reached or triggered over the duration of the campaign, using the Kaplan-Meier non-parametric survival function estimator. Green-shaded cells indicate the best performance (lowest time) for a bug metric across all fuzzers. Yellow-shaded cells indicate the worst performance (highest time) for a bug metric across all fuzzers. Red-shaded cells indicate that the bug survived being reached or triggered by the fuzzer throughout the campaign duration.

color_lens Change Colormap

<script type="text/javascript"> $('#btn-colormap').click(function() { stl = $('#survival_stylesheet'); var href = stl.prop('href'); if (href.includes('hiliter')) { stl.prop('href', 'css/survival_heatmap.css'); } else { stl.prop('href', 'css/survival_hiliter.css'); } }); </script> <style type="text/css"> #T_survival_table tr:nth-child(even) { background-color: #ececec; } #T_survival_table tbody tr:hover { background-color: #d9edfd; } #T_survival_table thead tr:not(:last-child) .index_name { visibility: hidden; } #T_survival_table thead tr:nth-child(3) { display: none; } #T_survival_table tbody td { padding: 2px 5px 2px 15px; text-align: right; } #T_survival_table th { padding: 2px 5px 2px 5px; text-align: center; } #T_survival_table th { border-top: none; border-bottom: none; border-radius: 0px; } #T_survival_table td { border-top: none; border-bottom: none; border-radius: 0px; } #T_survival_table tr { border-top: none; border-bottom: none; border-radius: 0px; } #T_survival_table thead tr:first-child th:not(:last-child) { border-right: 1px solid #888888; } #T_survival_table th:nth-child(odd):not(:last-child) { border-right: 1px solid #888888; } #T_survival_table td:nth-child(odd):not(:last-child) { border-right: 1px solid #888888; } #T_survival_table { border-collapse: collapse; } </style>

Fuzzer	moptafl_asan		pst_aflplusplus		afl		aflplusplus_lto_asan		pst_aflplusplus_recent_commit		afl_asan
Metric	R	T	R	T	R	T	R	T	R	T	R	T
Bug ID
TIF007	24s	56s	12s	3m	2m	1h	18s	1m	10s	5m	2m	1h
PNG003	10s	15s	10s	21s	10s	15s	8h	8h	10s	34s	10s	15s
XML017	10s	3h	10s	25s	10s	20s	15s	4m	8h	8h	10s	20s
TIF012	10s	17m	5s	1h	10s	3h	10s	1h	5s	2h	10s	4h
TIF014	24s	29m	12s	1h	2m	6h	18s	1h	10s	1h	2m	7h
XML009	1h	4h	15s	5h	15s	8h	10s	1h	8h	8h	15s	8h
TIF006	5h	5h	6h	6h	6h	6h	7h	7h	6h	6h	8h	8h
PNG007	15s	4h	15s	6h	15s	6h	8h	8h	15s	7h	15s	8h
XML003	1h	8h	15s	2h	15s	8h	8h	8h	8h	8h	15s	8h
TIF009	6h	6h	6h	6h	8h	8h	7h	7h	7h	7h	8h	8h
TIF002	7h	7h	8h	8h	21m	8h	8h	8h	8h	8h	30m	8h
XML012	1h	8h	15s	8h	15s	8h	8h	8h	8h	8h	15s	8h
TIF010	43m	8h	8h	8h	2h	8h	8h	8h	8h	8h	2h	8h
TIF001	7h	8h	8h	8h	7h	8h	8h	8h	8h	8h	8h	8h
PNG006	15s	8h	15s	8h	15s	8h	8h	8h	15s	8h	15s	8h
XML001	1h	8h	15s	8h	15s	8h	8h	8h	8h	8h	15s	8h
PNG005	15s	8h	15s	8h	15s	8h	8h	8h	15s	8h	15s	8h
XML006	1h	8h	15s	8h	15s	8h	8h	8h	8h	8h	15s	8h
XML008	3h	8h	25s	8h	19s	8h	8h	8h	8h	8h	20s	8h
PNG004	16s	8h	15s	8h	15s	8h	8h	8h	15s	8h	15s	8h
XML011	1h	8h	15s	8h	15s	8h	8h	8h	8h	8h	15s	8h
TIF003	15s	8h	8h	8h	15s	8h	8h	8h	8h	8h	15s	8h
PNG001	15s	8h	15s	8h	15s	8h	8h	8h	15s	8h	15s	8h

{% endcapture %} {{ template | replace: ' ', ''}}