Why I Don’t Unit Test

Keywords: TESTING E2E UNIT TESTS CI CD END TO END INTEGRATION TESTING REACT JAVASCRIPT TYPESCRIPT

LIVE EVERY WEDNESDAY AT ROUGHLY 1PM PT

And obv shoutout Idez as always 🙏

[RasmusSchultz]

I actually agree with most of your observations. I don't fully agree with your conclusion though.

I have a similar but different view of unit tests: as a development tool. I write unit tests as part of a process, to verify my own assumptions - will this work, will the API be appropriate in client code, and so on. This isn't extra work for me - I can code something faster by verifying each step of the way, not fumbling my way through the dark. I work faster and produce better work with tests.

I usually check them into source control as well. But I am always ready to throw them away, at a moments notice - and I make a point of telling everyone, if it gets in the way of making a change, throw it away.

I think, If that's your position, unit tests are perfectly harmless - they can provide value as a development tool, and as documentation, for as long as they're still viable. As long as you don't view them as a safety net, you can safely throw them away as soon as they no longer provide value. If that's after writing the client code, before even checking them in, or once you have E2E or integration tests covering the important parts, great, toss them. But they can still be part of an effective development process.

[simitron]

i agree when it comes to UI stuff, but for example if you write a parser unit test are super usefull because the feedback loop for changes is much shorter

[moonlifeSW]

The feedback I give my team on any test is

1. Add unit tests to make it faster to write complex functions (like doing some algorithm).
2. Add unit tests when the cost of something breaking is very high. (In our case, our APIs have 99.99% uptime expectations from our integrators).
3. Add unit tests around code that is difficult to understand for whatever reason.

Otherwise, prefer higher level tests.

Also, tests are not free, both the build, maintain and run. Consider the value of the test before writing one. We still end up with around 50% branching coverage because of being in a regulated industry, but it's not all or nothing.

[ssshenkie]

I’ve been doing TDD now for a year and can honestly say that it helped our team deploy quicker and with near zero defects.

Part of TDD is trying to break your code, so naturally you will catch loads of edge cases.

Writing tests after the fact is not very productive, but the safety is useful for refactoring to ensue things still work

[benjaminsanglitan]

Agree with not testing UI stuff, it's just a hassle. But for even some react hooks)/anything similar, I beg to differ. Just the thought of having a bug solved before appear again since it was not handled by a unit test is just a pain in the butt.

Saw your discussion with Prime about testing though, and I really enjoyed that! Both of you really made valid points.

[peteredmonds]

I don't think unit tests are designed to slow engineers down. I think the vast majority of engineers don't understand how to write good tests. Partially that's because writing good tests is difficult, but mostly because the word "unit" makes engineer think "test the smallest possible part of code". Usually that ends up being a function or a component, which really shouldn't be the target of a test. If you are testing features and not implementation details, tests can be really valuable when refactoring, but again the hard part is defining logical tests. I agree 100% that code coverage is a poor metric though.

[larryd]

For me unit tests are up to date documentation. And they provide me the safety net to refactor code at any time without taking into consideration tens of thousands lines of code.

[MachineYearning]

Type-checking and unit-testing are both ways of proving behavioral properties about your code's programming interfaces. When seen through this lens a lot of these arguments fall apart:

"Typescript is just a way of slowing down engineers"

"Catch/fix type bugs quickly in production - safety nets, not guard rails"

"Type annotations just unnecessarily cement the JS data types in place and make them difficult to change later"

Look, there's a very real possibility that the code Theo works on doesn't benefit from unit tests. Just like some JavaScript applications probably don't benefit from taking the time to setup and maintain static type safety. But in my opinion the way these half-truth arguments and vague examples are phrased is just gonna enable a lot of people struggling with understanding automated testing to sink back into their comfort zone, without really elucidating why you would or would not want unit tests in a given scenario.

[DarrylHebbes]

I always found that engineers who did not know how to test or code for testing were more likely to find reasons to avoid them

[gp]

03:45: Fair enough, but what about bugs that don't get reported? What about users that are negatively impacted by the bug, notice it, then leave and don't come back? I feel like the latter point is especially relevant when your brand is not well known to the public and lacks the "street credit" to afford having user-facing bugs in prod.

[AlexZurek]

Automated tests (unit, integration, and e2e) are tools to help ensure your code is working as expected and to help catch possible regressions as you make future changes. Just because a tool exists doesn't mean you have to use it, but I would like to point out that the "be able to quickly patch bugs in production" workflow doesn't work for industries such as aerospace, medical devices, etc. where reliability of the software is crucial. If you're not working in these types of industries then you can probably get away with what Theo is suggesting in this video.

[Brodeon]

I see unit tests as a means to defend a code against the future self who could modify a function in such a way that breaks other functions that use this modified function, but I also think the code coverage should not be enforced. Much more important are still integration tests which should detect any errors that dev could do. I personally believe that if you are working on MVP, the last thing you focus on are unit tests. When working on MVP there is high possibility that code in many parts of an app is going to change and making old unit tests useless

[jww]

summary: do things that you need, don't just do just because...

[chrisseamon]

Cheap as possible to fail... sounds like a job for a unit test or integration test.

Tests can be used as a form of documentation.

Unit tests should test inputs and outputs of methods.

100% code coverage is definitely not needed. 80% is a good start.

I agree that typescript can help you find issues due to compiling.

[Renoistic]

I create unit tests for both Api and UI, but it's important to only test important and worthwhile use cases and not blindly look at code coverage.

[ArkDEngal]

I think it would be more helpful to explain what safety nets you employ and how you design/create them. I think a lot of people see TDD as a safety net.

[rhode]

With FB, the lack of unit testing is because of a *myriad* of other things are blocking and slowing down other engineers, and that FB maintains no other environments besides production. IMO, if the CI/CD process for landing a diff didn’t take days, it might be more possible. My current team deals with unit tests at Meta, and they are the least intrusive and often most informative part of processes. However, they are few, fast, and carefully chosen.

I agree and disagree that unit tests are there solely to block devs. I agree in that unit tests usually end up doing that because they’re overly and poorly implemented. I also agree with you that unit tests need to be viewed through the lens of “does this add value”. Treat unit tests as product code, and when adding them ask if they’re adding value to the product or just a blanket of false security.

I disagree in that when well implemented they can serve as living documentation - not just internally but externally. Also, when composed well and written to be fast, they can help catch issues before they might come up at potentially more painful (and longer running) stages like image/container builds, etc. It’s a lot easier to run a quick slew of fast unit tests in a minute then deploy a fix for a caught issue than wait 20-30 minutes for a service to redeploy every time a bug is reported or interrupts a build. There’s also that, even in types safe or strongly typed code, you can still have logic errors. Don’t get me wrong, typed languages can reduce a *lot* of common errors, but even in individual smaller units of code, it’s still possible to write perfectly typed but logically flawed code.

Finally, writing unit tests can serve as a canary for confusing or convoluted code. If a unit test takes longer than a second or two to execute or the mocking is difficult to implement, it’s a signal to me that the code it’s testing might be overcomplicated/slow itself and I need to look at simplifying it. That brittle feature that has umpteen billion tests? Like you said, that’s a sign the brittleness needs to be fixed, which can be far more difficult to gauge sans unit tests. IMO - if writing unit tests feels like it’s blocking devs or slowing things down, look at *why* implementing those tests is difficult. The why is usually an issue with the implementation, size, or even architecture of that feature that needs to be fixed.

Sum total, you raise a *lot* of really great points Theo, and I deeply enjoy your perspective. See you in the next one!

[BrianVanderbusch]

The real value in tests is to make you think about how your code is designed. If your code can be unit tested, it's far more likely that it's designed in a way that is orthogonal and flexible. This means your whole app is easier to add features to and respond to changing needs of the users.

If you just see unit tests as a roadblock to production, then you're missing the point that the unit tests are telling you... that your code is designed poorly and is brittle. It might be cheaper to deploy a bug and then fix the bug than it would be to write a unit test... but when writing the unit test ensures the function is designed well, it's far less likely to have to be operated on.

When you're not designing functions to be tested, you're going to write brittle functions, with internal state and uncontrolled side-effects, and you're going to spend a LOT of time fixing production... and the cost adds up, and it is accumulated in an environment that has an opportunity cost because it impacts your users and your brand.

Your whole take on making a unit test work to cement the code is exactly what I'm talking about. That just shows the code was written without regard to how it would be unit tested. A unit of code that can't be unit tested is likely to be just as brittle and difficult to use by the other bits of code in your application. The fact that you see unit tests as conforming to code instead the other way around just highlights how you fail to see the real purpose of unit tests to begin with.

This line of thinking is selfish and short-sighted, and irresponsible.

[chrisatkeson]

I can't tell you how much time tests have saved me and my coworkers. Unit Tests might not be that useful but functional tests in my opinion are essential.
- Proof of Function
- Easier to Refactor Code
- Reduction of Errors
- Improved On boarding (It would be a nightmare to have new developers work on something with no test cases).
- Peace of Mind

[careymcmanus]

I work at a company that is extremely risk intolerent (which is justified) and so we have many layers of protections to prevent bugs going into production, including extensive unit testing. When I started here I was a strong proponent of unit testing (this is my first dev job) but now I am super skeptical. I have seen so many bugs that (fortunately) got caught by other forms testing that had unit tests 'covering' them but the unit test was bad because the dev wrote bad unit tests. This was not really the devs fault as there are so many reasons this occurs including:
- not being able to think of all the possible edge cases
- frustration at all the irrelevent stuff you have to consider to test poorly written legacy code
- tiredness or lack of motivation (because you've spent too much time writing unit tests)

The most egregious example of a bug that had a unit test that I've seen was for an Api call that literally did nothing where the unit test was literally just a test that the method exists

I 100% agree with the view that writing unit cements your code in cement as if you want to rework bad code most of the work comes from fixing the unit tests (especially on legacy code bases).

Your take on make safety nets not guard rails is spot on

fortunately my company doesn't have a code coverage requirement and the main person pushing for code coverage has left.