A Usability Evaluation of AFL and libFuzzer with CS Students

A Usability Evaluation of AFL and libFuzzer with CS Students
Stephan Plöger, Mischa Meier, Matthew Smith

CHI 2023: The ACM CHI Conference on Human Factors in Computing Systems
Session: Digital Safety

In top-tier companies and academia, fuzzing has established itself as a valuable tool for finding bugs.
It is a tool created by experts for experts, and a lot of research is being invested into improving the power of fuzzing.
However, the usability of fuzzing has not received much attention yet.
To alleviate this, we evaluated the usability of two popular fuzzers: AFL and libFuzzer.
In our fuzzing study, 47 computer science students each worked up to 20 hours in total.
We found significant usability challenges for both fuzzers leading to only 17 participants who were able to finish all tasks.
Even the successful participants struggled with some of the necessary steps and found them complex and confusing.
While on the whole, AFL fared better than libFuzzer, both fuzzers have strengths and weaknesses and can be improved based on our results.

Pre-recorded presentation videos for papers at CHI 2023