The Lies Of 100% Code Coverage | Prime Reacts

Recorded live on twitch, GET IN

Become a backend engineer. Its my favorite site

This is also the best way to support me is to support yourself becoming a better backend engineer.

MY MAIN YT CHANNEL: Has well edited engineering videos

Discord

Hey I am sponsored by Turso, an edge database. I think they are pretty neet. Give them a try for free and if you want you can get a decent amount off (the free tier is the best (better than planetscale or any other))

[KvapuJanjalia]

Code coverage: Goodhart's Law all over again. "When a measure becomes a target, it ceases to be a good measure"

[nv]

Tests are not for detecting bugs. Tests are for checking if a function does Z when given XY.
But in most of the times it cant detect bad programming, race conditions. Also tests are written from the same person which has maybe wrote the bug in first place.

Write tests, but don't expect them to detect bugs.

[RichHeart]

100% agree with the "focus on integration tests over unit tests" opinion in the article. Unit tests often feel like testing for the sake of testing whereas my clunky and bulky robot_framework test suite finds stuff to fix or improve quite often

[ayehavgunne]

The answer is obviously > 200% coverage.

[mike]

What I've seen in relatively high integrity code is that code coverage hovers around 70-80%, mainly for one reason: defensive programming. It's good to be careful and future-proof by having plenty of basic checks (null pointer, bounds checking, integer overflow) and you can easily end up with like 20% of the lines of code being "dead code" because they check conditions within helper functions that are not called anywhere in a way that could trigger those errors. But that could change in the future, making those checks useful in the long run. Trying to achieve test coverage of all these checks is so tedious and high-maintenance that it discourages you from writing them in the first place.

[omarmachin]

It’s crazy how integration tests were not even mentioned. Unit tests are not design to test end to end functionality of your application. It suppose to test individual “units” of it. In reasonable well designed applications 100% coverage would not be even possible because guess what? You have an entry point where you create all your dependencies and that hard and inefficient to mock. That’s when integration tests come in the picture. You should have a set of test that will call real dependencies. For example call the real weather api and verify that the output you get its correct. Units tests by design are not meant to test this type of end to end scenarios. If you find yourself in a situation like this, something in the architecture of your app/ service can be improved.

[SamArmstrong-DrSammyD]

The best code coverage is 100% if it’s achieved via tree shaking removal of all code that isn’t covered in an integration test.

[AlLiberali]

I'd like to see a mock testing framework _mock_ old school human testers; pun intended. Walk to a pub. Ask for 1 beer, -1 beer, 0 beer, 2^32 beer, 0.5 a beer. Come in through the window, exit through the back door. Breakdance inside and then defenestrate yourself. Introduce yourself as null, empty string, a null terminated string, CRLF, etc.

[bigfishoutofwater]

This kind of policy just makes the test suite run all the lines of code but it doesn't make you test anything.

[remirth_bergström]

I'll tell you what, I'm covering something else to a 100% if you catch my meaning

[orterves]

19:20 "The best way to have a well running project is to have tests easier to write and run to test your project than running the project itself" - this is the core takeaway from the video and it is absolutely true.

Identifying and managing the path of least resistance is the core of driving human behaviour in general.

[SimGunther]

100% value delivered "to the people" > 100% not happy path code coverage > 100% happy path code coverage

[SasuraRdm]

I swear to god as someone who has been doing this professionally for less than 6 years. So many of the titles of your videos make me chuckle internally( and sometimes externally ). Because it's all things that, the realization of how bullshit some of these are, is still so fresh in my mind.

[ShadowKestrel]

9:00 chatter seemingly doesn't realise: by US law, 100% code coverage, even if it's 100% by useful tests, *is not enough*. avionics have to be mathematically proven correct at every level from the hardware to compiler to actual code itself, even taking a whole range of failures into account.

[adama]

300% coverage, bro. You got to pump those numbers up !

[alexaneals]

The problem with unit tests and code coverage is that it should only be part of the testing solution, not the entire testing solution as many companies want. As developers we are terrible at testing code because we only want to see the code work. A good tester likes it when he can cause your app to crash the machine. He tests for how to break the code in addition to "happy path" testing. The problem is that unit tests generally only test for happy path and then rest of the testing is passed on to the users. Most users will only test happy path and everything will seem hunky dory until someone finds that one evasive bug that brings down the entire system or worst yet spills all of your sensitive data out to the dark web.

[Sande]

Goodhart's law: "When a measure becomes a target, it ceases to be a good measure"
Easy to create shitty tests just because you have a shitty target.

High code coverage does not catch all bugs. One of the most common bugs is that the software doesn't do what it is supposed to do. You might get 100% code coverage because you are just missing the code that you are supposed to have and you aren't testing it either. Maybe a function is supposed to call an external service as one of the steps, doesn't care about the result. And if the function doesn't do it and the unit test doesn't check for it either, everything seems OK.

Better to just write tests for the more important stuff. Don't waste time on covering every edge case. Testing it manually if often enough. Tradeoff of spending time vs shipping faster.

[fb-guer]

Most unit tests are actually useless. I only add unit tests for specific functions that contain business logic. I have found to be better to write down every single possible scenario to go wrong and try to break it

[wannabelikegzus]

Hot take: Unit tests are, by definition, lazy tests. The only reason people do unit tests instead of benchmarks and end2end tests is that unit tests are quick and integration+load tests take some forethought. However, at the end of the day, what really matters is that your application is meeting functional requirements in a timely manner, not that you've covered every single (or even any) of your catch blocks. Moreover, as you get better at programming to meet functional tests, it will become easier and easier for you to spend your mental energy on thinking of possible failure modes FOR THE APPLICATION and writing the integration tests to cover those modes.

The only exception to this is when you are writing a piece of code that is too hard for you to validate in your head.

[kartirnium]

I just started my first job at an extremely high security sector and the project i've been assigned needs to have as high as possible coverage or hundreds of lives can be lost due to one bug. So it depends a lot on what you're actually doing.