When Test Driven Development Goes Wrong

Test Driven Development is one of the best ways that we have to amplify our talent as software developers, maybe software engineers. This Software Engineering practice is one of the best ways to improve the quality of your code, but it is difficult to do it well, and it often goes wrong. The interesting thing is that when it goes wrong, it may be at its most valuable. TDD is a cornerstone of Continuous Delivery, BDD and DevOps. Using it to give us valuable, efficient feedback on the quality of our designs is at the heart of its value but is often missed by people who are new to it. Dave explores these ideas with some real code examples to demonstrate the value of TDD.

In this episode, Dave Farley explains 5 common ways that TDD goes wrong, how to fix them, and what we can learn from them.

Some of the ideas in this video were inspired by this blog post:

-------------------------------------------------------------------------------------

🎓 CD TRAINING COURSES 🎓

📚 BOOKS:

📖 Dave’s NEW BOOK "Modern Software Engineering" is now available on
In this book, Dave brings together his ideas and proven techniques to describe a durable, coherent and foundational approach to effective software development, for programmers, managers and technical leads, at all levels of experience.

📖 "Continuous Delivery Pipelines" by Dave Farley

📖 The original "Continuous Delivery" book by Dave Farley and Jez Humble

📧 JOIN CD MAIL LIST 📧
Keep up to date with the latest discussions, free "How To..." guides, events and online courses.

-------------------------------------------------------------------------------------

[thought-provoker]

Gotta steal that sentence, "If our tests are difficult to write, it's telling us something about our design."
Yeah. Exactly. That's what I observe, too. Over, and over - and over again.

When we end up with a test setup of 72 steps and a teardown of 90 steps for a single database update, that's not a test problem.

[rafellus]

-- Anti-patterns --
4:03 - The Lair
5:52 - Excessive Setup
9:54 - The Giant
12:57 - The Mockery
16:22 - The Inspector

[drcl]

My university tried to teach unit testing to us by giving us already written software and asked us to apply unit tests to it. At the time I remember hating it because it was mind bending trying to get the tests to work due to the ridiculous amount of coupling meaning stupendous amount of setup.

I didn't at the time really understand why I was finding it so hard but I knew something wasn't right. I couldn't imagine anyone would do this in a job and there had to be a better way. The tests that me and others designed got to about 80% coverage and that was seen as acceptable for assessment - it was probably impossible to go much higher.

More than a year later they reintroduced the the concept of TDD (sort of) in a module called advanced software concepts which cited Martin, Fowler, Beck and the Gang of Four but it was all just piled in together - not making it clear how each part fit with the others and that TDD is basically the lynchpin to all the other ideas or emphasising the importance of the cycle and fail first.

It's no wonder enterprise doesn't know how to implement it. The way this was all taught almost made me not want to pursue work in the industry if this was what even some of my days were going to be like.

[PlayRiteProductions]

0:00 - Intro
2:39 - TDD
4:03 - The Liar
5:52 - Excessive Setup
9:51 - The Giant
12:54 - The Mockery
16:19 - The Inspector

Amazing video with excellent points! I made some timestamps for review.

[kguentube]

This video is gold, as usual in this Channel.

[SteinGauslaaStrindhaug]

One of my first experiences with debugging someone elses test, was a test with lots of mocks that happened to fail when I improved the code it was "testing". I moved an expensive repeated call to a procedure on an argument from inside the loop to outside the loop; as it was just recalculating the same value every time anyway. The test then suddenly failed because it expected it to call that procedure 8 times and got only 1. Once I managed to understand the "test" it turned out it mocked the database, the message objects and the webserver etc. and the only part of the actual code it were actually testing was that the loop were written inefficiently.
This is the main issue I have with TDD, or at least "mock TDD" is that when I'm working with projects that have lots of "tests" written by TDD-enthusiasts (that rarely understands it) is that the tests often don't catch bugs, it just enforces existing bugs; so almost always when I cause a test to fail when refactoring it turns out the test was asserting bad code; and when I actually introduce bugs it normally passes just fine because the "tests" mostly just tests that the mocks mock and the fakes fake and very little else.
So many articles about "TDD" just wastes so much time on fancy ways of mocking the entire environment in order to have "test coverage" rather than just assuming some parts of the environment should just work.. If you're not making a database, just using a 3rd party database, why would you waste your time testing that SELECT selects and INSERT inserts? If the database doesn't do that it's pretty worthless, and even worse testing that a mock-database mock inserts when you ask it to insert something is just stupid. If the code you're testing simply takes some data, validates it and forwards it to the database; just test the validation if it's complex enough to warrant it, and don't bother testing the database.
I've also seen "unit tests" that test getters and setters on message objects... wtf? Are they trying to verify that language features work? That variable assignments assigns and return statements return? Are they even aware that the testing framework is written in the same language and presumes those features works in order to work. And even more stupid, is code that basically test getters and setters on a mock objec.

[rpopov]

Clear, practical, not dogmatic. I like it. And I like also that Dave considers there is some design before writing the tests. TDD is not an alternative to design - the tests actually validate the design before implementing it.

[Stevexupen]

The most annoying thing of developer with tdd is there's some that has become so (falsely) over-confident of their code because all their test case passed that they use it to (wrongly) argue that the fault is not in their code while more often than not it is..

[JanMagnusson]

This is what I am saying as well when I try to encourage taking up TDD. That TDD is not about ensuring that you have good coverage or some such. The main benefit is that it gives you feedback on your design before you have spent lots of time implementing it. If your design is hard to write tests for, its probably not a good design, but since you didn't yet implement it, going back and improving the design is cheap. TDD is a way to test design in addition to code so it pays off in better design, better tests AND better more error free code.

[nickj]

Deming said it (maybe) - "when a measure becomes a target, it stops being a measure."

[tonyjaradev]

Dave, as someone living somewhere without much access to information like this, or not surrounded at all by developers, I greatly value this videos. Thank you!

[aly-bocarcisse]

👏🏿👏🏿👏🏿
Not a heavy practitioner of TDD here however even I did see those mistakes over and over. This is gold, on top of gold on top of reflection & care. Thank you !

[EwaldDieser]

Learned a lot that I can apply in my work. While doing code reviews I often have difficulties to explain what I don’t like. A list like that is really helpful!

[satyrkrieg]

Nice video. A suggestion, when you put code on the screen, please place it in a way that is clearly visible.

[BenDavis]

Great video! Many of the pitfalls explained here are due to not using a TDD approach. I would love to see some advice for developers tasked with adding tests to existing projects that were not originally developed with TDD.

[TheChodex]

Having bad tests in project that pass reminds me of that meme where dog is sitting inside a burning house and saying "this is fine"

[leandronunes]

I really enjoyed the video but I think that the Jenkins example was unfortunate. By transforming the functional test on Jenkins into a unit test on JenkinsUrl (or whatever the class name was) we’ve also moved from testing the requirement (“Jenkins should not have the word localhost in its url”) to testing an implementation detail. I’ve experienced times and times again that implementation details like these are not actually used in the use case I was working on (in the Jenkins example, what guarantees do I have that JenkinsUrl is actually being used?).

[tordjarv]

On the Inspector anti-pattern. I am a nuclear physicist and are developing a large scale simulation code for atomic nuclei, and I do use TDD. However, in my code I have functions that produce transformation matrices which can be very large (with very large I mean that they can take several GB just to store the non-zero matrix elements as single-precision floating point values) in production code, so naturally I need test cases that cover these larger cases. The problem is that because they are so large and that I need to check very many versions of them it is unpractical to store examples of each and every test case, so instead I need to check that they fulfill specific mathematical properties. However, in the production code there should never be a need to check these properties, since it would be a waste of expensive computing time to check it in production runs when it should be guaranteed if the code is correct. This means that the code to check these properties are specifically written for the tests and only used in the tests. This to me sounds like what you describes as the Inspector anti-pattern, but I don't see any practical way to get around it.
Other than that I think this is a great video and I really think that I learn good stuff from all your videos, so thank you.

[justinbehnke]

I am shocked at how spot on the symptoms and causes of these anti patterns are. Well done you read me like an open book!

[riwjin]

First thing first, that t-shirt is very cool and I really want one now.
Yes, priorities.