100% Code Coverage but 0% Regression Bug Protection?! | Unit Testing | Mutation Testing | PIT Test

100% Code Coverage alone does not mean that we are being protected against regression bugs. That's why we need Mutation Testing.

Many companies set code coverage targets - for example, 80% coverage, 90% code coverage, and 100% code coverage.

The problem is that even though we can have 100% code coverage, it does NOT actually mean that our tests are protecting us against regression bugs.

CONNECT WITH ME:

CHAPTERS:

0:00 Intro
0:51 Starting with 0% Code Coverage
1:55 Reaching 100% Code Coverage
3:15 But there is no regression bug protection
4:45 Mutation Testing comes to the rescue
6:59 Reaching 100% Mutation Coverage
8:53 Summary

#tdd #unittesting #softwaretesting #softwareengineering

[gilzrdBJJ]

At the beggining I though that you would sum integer.max values. It would fail even with assertion. Great video!

[thelonearchitect]

True!
Interesting fact : code written in TDD tends to be highly resistant to mutants.
The more TDD cycles are short and close to the Red/Green/Refactor practice, the less mutants there are

[anastasiaIvanova]

I agree, code coverage shouldn't be the only thing developers measure quality against.

[theanswer]

So do I get it that code coverage tools should basically just check if assertion is in the test or not?

[raduking]

What's the audience this video intended for because it's clearly not for developers, anyone who would write a test as you gave in your first example should not call themselves a developer... I've been using mutation testing in Java since the 1st version of pitest and I think this is a very poor example to illustrate the real power of this tool. It still has a lot of problems for example not having annotation mutations (since I think those are just as important as any other line of code if you use for example Spring or EJB) and others which I won't enumerate here...