HTTP Error 500 52 URL Rewrite Module Error

When installing a local copy of WordPress, some of my students get the HTTP Error 500.52: URL Rewrite Module Error (you might also see IIS Detailed Error 500 Duplicate Entry). This seems to be an issue with WebMatrix 3 and the famous 5-minute install.