Stop Writing So Many Tests

Testing is hard and knowing what to test is even harder. In this video I talk about the 3 different types of tests and how they compare to one another. I also talk about why end to end tests are so great and why writing less tests can make your test coverage better.

📚 Materials/References:

🌎 Find Me Here:

⏱️ Timestamps:

00:00 - Introduction
00:40 - Types of tests
02:55 - Comparison between test types
03:52 - When to use unit tests
06:35 - When to use end to end tests
08:18 - When to use integration tests

#Testing #WDS #E2ETests

[gbroton]

From my experience, it is really hard to find good articles about unit testing. 99% of them are examples presenting silly "add" functions, which is completely useless for anyone who wrote at least one unit test. But in fact writing good unit tests is an art, and require some practice, but without any proper guidance it is hard to get there.

[_eric]

Needless to say this is a bit of a controversial video. While I am sure there are situations where your assessment is correct. There are equally counted situations where it may not be correct. While there is a place for all the types of tests you are talking about, Unit tests are still by a long shot, the most valuable tests. As many other viewers have commented, they run in a fraction of the time any other tests would run. When written properly, one can run 1000s of unit tests in a matter of seconds. Giving us instant feedback, faster builds, etc.
Furthermore some of your complaints about the value of unit tests are usually the results of a lack of design in the first place. What I mean by that is that when a unit is built, its interactions must be defined. When a new component is built in the UI, we need to define all its inputs and outputs, as well as describing all its interactions with other components or subsystems (like http calls). Part of that contract includes corner cases and exceptional situations. And those are the ones that get extremely hard and expensive to test at a higher level of test (integration or e2e). In your example of the ToDo tests, I would argue that the e2e value of doing basic tests for the 4 major operations (CRUD) is very low. Those tests cases are so core to your application that they can be tested by a human in no time. However, testing situations where the network is slow, there is a duplicate task, the user clicks twice, the database is busy for like 2s while a user is updating their list, etc. those are situations that are just as critical, if not more critical and are extremely hard to test in e2e or integration layers. However, they are trivial to test at the unit-test layer.
I have always enjoyed your videos and always welcome your ideas. I would just caution against making blanket statements to your growing audience that _unit tests are useless and you should only write e2e tests_ (I know it's not exactly what you said, but that's what some inexperienced viewers will retain from the video...)

[DanielMaldonadoPC]

Does console.log every time you make a change count as testing

[stojadinovicp]

And if your end2end fails? Good luck debugging...
I'll do my unit tests and go up - looking for "bad" combos of those units.
Bottom up makes way more sense than top down. And is cleaner, safer and in the end saves time.

[PrzemysawSech]

Generally, I love the content in this channel. However, this one is so wrong, I struggle where to start commenting.

The following is my personal opinion, based on over 10 years in QA automation.

E2E tests are genetic, on failure they say 'something is broken', where unit tests highlight the specific line of code and the reason.
E2E are prone to false positives, they suffer from network latency and 3rd party dependencies.
The cost of maintenance is much higher that unit or integration tests.

E2E involve backend thus an app must be deployed to an environment, which increases the cost. This also makes the feedback super slow, where unit, integration, contact tests are build-time test, which makes them cheaper and faster.

As a Software QA I spent years in E2E area. In my opinion the test pyramid pattern is right to apply. There should be as few E2E as possible. I treat these as post-deployment smoke and infrastructure tests.

If you want to assure quality of BE-FE integration, use contract tests.

[Pluvofor]

I've heard a lot of arguments around test type ratios at work, but I agree with you. The only thing I would say is that end to end tests are more difficult to work into daily work flow. Because they can take so long to run they are often not run on every commit, nor even each push. It's then harder to work out which commit caused the issue. And, earlier in a project, the UI may not be settled. This means that you'll spend a lot of time maintaining E2E tests. That time may not be factored into ticket points.

[calkelpdiver]

I've been in Software Testing for 35 years now. And it comes down to being smart about what you test, and what you automate.

There are different levels of testing and different way to automate the execution of those tests. It comes down to "risk" when determining what to test. Obviously the first thing to test is basic functionality (Smoke/Build Validation) of the system. Got to "plug it in" to make sure it doesn't go "POOF!" with a big cloud of smoke.

Next is to test the key functionality using the 80/20 rule. Test the 20% of functionality used 80% of the time. These can be Integration, Functional, Business Scenario (End-to-End) type tests. Next is Negative Testing / Fault Injection to make sure the system under test will handle error conditions properly and not give you the "Blue Screen of Death".

After that it is a question of what else can be squeezed into the schedule as additional testing tasks. Implementing Automation in Testing for the above should follow the same rules. Automate what makes sense and don't even attempt to get to 100% automation, it is a myth and a target you will never reach.

[Treborbobuk]

If you are having to mock/fake a lot of stuff to make a unit test work, then it's because the architecture is poor - too many dependencies etc... Unit tests are wonderful when it comes to refactoring code, that's the main purpose of them, not to replace manual QA / regression testing. Unit tests give devs confidence that changes to code have not changed the output of the code. E2E, visual testing etc... are replacing humans doing QA, not in any sense comparable to writing test code. Good video, but the different types of test are not either/or choices, they are all necessary and serve different purposes.

[Nil-jsbf]

Writing trivial tests is soul draining. I absolutely abhor the unit tests that mock everything. Especially when it's testing a function that integrates a whole bunch of other functions. The test ends up looking like - assert function 1 was called 1 time with these params, assert function 2 was called 1 time with these params, assert function 3 wasn't called.

It's so incredibly stupid and the test is essentially rewriting the implementation.

[FiveTwo_Nick]

Wow, the last time I was here, Kyle was still an unmarried man

[BornEditHD]

I wrote something like 168 tests for my latest project and it worked perfectly. I always knew exactly what was wrong when the app stops working.

[tdombui]

love how you have a single guitar and a pillow that says 'family' in the background

[raymondbyczko]

Good work! I am just about in total agreement with your conclusions. I would add that while unit tests can be reserved for critical pieces, they can also be used for just plain complicated ones. Also while function A works well on its own, and B works well too on its own, the execution of A and B (maybe A calls B) explores boundary conditions etc. There can be side effects that occur, that were unanticipated. A good end to end helps find these things. Again great work!

[mountakhabi]

Cypress ? 😂 Just try Playwright bro it will blow your mind

[daniellewis]

I've actually come to embrace the idea that tests are great for verifying that a method "unit" you're writing does what you want. Want to know what your add method does? Throw 0, 1, NegativeInfinity, null, undefined, and "bob" into it.

When you write code units that can be tested this way, you can test this way. When you don't, trying to pigeon-hole that code into unit tests sucks. Not everything can be, or maybe even should be unit tested, but it's great to do where you can because those units can be more or less pre-proven and that reduces the code surface over which bugs are likely to happen.

As a developer, I like using them, but if a company had a mandate on them I'd have to listen harder on why they're doing it and how their devs are actually finding it.

[garcipat]

The mix is important. Havig only e2e tests will give you a huge effort figuring out if one of them doesnt work whats actually wrong, where unit tests pinpoint you very exactly where the error is. e2e are expensive to maintain also. Coverage alone is not the goal.

[david-youtube-channel]

I concur. Behavior-driven development (BDD) is an effective approach for writing tests in this manner.

[TOracle]

Integration tests are fragile, slow, require maintenance, have intermittent failures E2E is just another integration test with horrible scalability. The proper use of integration testing is when you need to verify something that cannot be readily unit tested. Also, the reason why you have to mock so much, is because you are using libraries that have poor testability.

[oliverkelly]

Dude, throughout the entire video you are using the singular for "test" instead of "tests"

You don't say "your integration test". You say "your integration tests"

It made the video so hard to listen to.

[ygeb]

I still think that unit tests play an important role.
Some developers avoid covering their essential core business models with unit tests, with the claim that the only relevant tests are E2E and integration. They claim that unit tests don't prove much and have little worth.
I think this claim is wrong, because a unit test can point to where something is broken. You get an early signal that a certain part of the software is no longer working as expected. And you also get the exact location in the codebase where something is no longer working.
An E2E test will tell you that a user flow is broken. But you can't always rely on it to tell you why the user flow no longer works and where the issue may be. And if the system is complex and composed of many subsystems, the problem is bigger.
Unit tests help us debug issues at an earlier stage.