Inferring Loop Invariants through Gamification

Inferring Loop Invariants through Gamification
Dimitar Bounov, Anthony DeRossi, Massimiliano Menarini, William G. Griswold, Sorin Lerner

CHI '18: ACM CHI Conference on Human Factors in Computing Systems
Session: Gamification, Personalization, Tracking

Abstract
In today's modern world, bugs in software systems incur significant costs. One promising approach to improve software quality is automated software verification. In this approach, an automated tool tries to prove the software correct once and for all. Although significant progress has been made in this direction, there are still many cases where automated tools fail. We focus specifically on one aspect of software verification that has been notoriously hard to automate: inferring loop invariants that are strong enough to enable verification. In this paper, we propose a solution to this problem through gamification and crowdsourcing. In particular, we present a puzzle game where players find loop invariants without being aware of it, and without requiring any expertise on software verification. We show through an experiment with Mechanical Turk users that players enjoy the game, and are able to solve verification tasks that automated state-of-the-art tools cannot.

Recorded at the ACM CHI Conference on Human Factors in Computing Systems in Montréal, Canada April 21-26, 2018