Jonathan Blow on unit testing and TDD

I made this to test out some editing software. I might do more, let me know if there's any quotes you specifically want to see.

[MrOhioutod]

Jon Blow: "Don't take me out of context, testing is very important, right? If you don't test your software, then your software doesn't work. What I'm saying is, I don't think that this obsessive unit testing that happens in many cases is the right way to test. I think there are more efficient, better ways to test that capture more bugs. And we want to do those things."


That's a very different statement than what most people are criticizing.

[thebrd]

If you've watched Jon working on his compiler, you've seen that he runs a battery of "unit tests" after significant changes, he's just not religious about it. There's a point of diminishing returns, which dogmatists proudly ignore.

[romaingauthier]

To be fair about pair programming: I used to know a company that would d pair programming for all juniors and mid-level programmers, while the founders (guys who started in the 80s and were all killers) would program solo. They found that their less experienced programmers would produce higher quality code and improve faster when working in pairs. It seemed to work for them. This wasn't a game company by the way, but an AI company.

[willmcpherson]

For me, testing is something that grows naturally, like any other part of the program.
You start with a dumb program, manual compilation and manual testing. Over time your program takes form, your build system becomes more sophisticated and you develop an automated test suite.

[bigmofo]

Asking Jonathan Blow about unit testing is a meme at this point.

[wombologist]

Part of the problem is that we adopt these strategies with an all-or-nothing mentality, when the best way to go is adopt as needed. All these mentalities give people false sense of security - we feel good in "believing" in something like a religion.

[Elrinth]

@ 05:40 -> Pair programming is about sharing the knowledge of the code. Not become more efficient. It's never going to be more or of equal efficiency. But he is totally correct, it's all about costs. Very nice talk, and nice hearing his opinion.

[MechMK]

I remember being an intern in a company, and I was brought along to a discussion regarding unit tests. And the manager wanted unit tests for *everything*. Like, every tiny bit of code had to be tested obsessively, to the point where trivial functions (like getters and setters) needed unit tests.

The programmers complained, saying that they will end up spending countless hours writing code, which ultimately never get any value to the customer.

The discussions after the meeting were very insightful, because they essentially drove the point home "test where it matters". Like, if you have some complex business logic, make sure that there are tests for that. If you have some code, that writes files into a specific format, make sure that the data produced by those functions adheres to the specified format.

But everything else? It doesn't matter. What really opened my eyes was "You don't need to test whether or not some internal logic works. If it doesn't, then that will result in some observable failure - and *that* is what you should test for."

Ultimately, I sadly don't know how the whole "we need to test everything" story ended, as my internship ended not too long after.

[canthandlet]

Many interesting and valid comments. One thing that hasn’t been mentioned and is a bit quirky is how TDD contributes a bit of gamification in how you write the software. Given the knowledge of how gamification and productivity relate one could conclude that TDD is a productivity tool. So, where’s the gamification element in TDD? A struggle of software development is the delayed feedback of your effort. Sometimes you do a lot without seeing the immediate benefit. With TDD you develop against your unit test trying to get it green. The feedback cycle is more immediate and the task at hand is more targeted. A core element of gamification is that the goal is clear—TDD is basically the definition of the goal.

[haruruben]

I think there's a ratio of test code to app code where the testing becomes a net gain on productivity because you don't have to go back and manually retest things and you can quickly see if your new stuff has had unexpected ramifications

[wjjam]

I think, like most things in programming, TDD is appropriate for some things and not for others.
If you're making a video game, TDD is probably overkill.
If you're writing software for a manned space mission, making absolutely certain that everything works 100% correctly is probably a good idea.

[johnwalkerlee]

When project managers aim for 100% coverage "I found a metric for my spreadsheet!"

[stangeorge]

If you listen closely you can hear Uncle Bob screaming.

[HartleySan]

Jonathan's bit at the end about ignoring costs is right on the money (pun intended). I often struggle trying to make that exact point at places I work.

[JasonDoege]

Unit tests are useful when the interface is very well defined and the thing being tested is intended to be passed to someone else to use.

[thomasmolby]

I do not think this is completely black and white, however I did work in a large web development team where 100% test coverage and pair programming were mandatory.

Writing tests took precedence over exploring the problems we had been tasked, and everything took forever to make. As Jon talks about in the video, people were more focused on the tests than the problems. Once some of my colleagues spent 4 days figuring out how to test that the color of a button was actually the color that it was supposed to be and THEN half an hour to make the button change color. I wish i was joking...

Generally, when my colleagues paired up, I often noticed that their back and forth would lead them into making really weird choices. Once two guys talked each other into making elaborate abstraction layers around a library for doing graphs and charts, barely consulting the actual documentation. They were very proud of it too. However the end result was a graph that looked nothing like the design and only read right at certain resolutions... but another two sprint cycles would help get it within acceptable parameters...

Enforcing TDD and pair programming made sure people never got to sit down and just figure stuff out. Im pretty sure my colleagues with very little html, css and javascript experience would get a much better chance at actually getting that experience if they had been allowed (and allowed themselves) to sit down and try out some stuff.

I'm open to the possibility that tests and pairing up could help a lot in certain cases.
If you have a big complicated data transformation, why wouldn't you test out a large portion of inputs and outcomes to ensure that it does what it's supposed to? And why wouldn't you want other people to give their perspective on your work and your process? and the other way around? But not allowing people space and freedom to explore and consider a problem because you found the one true way to do software is just very, very dumb.

[artiumnihamkin]

From my personal experience, exploring solutions to a problem or debugging with another person is very productive, more than 2x. I never thought that pair programming means two people should share a keyboard and do the actual programming together.

[DaleReidy]

I guess my rebuttal is - there's always a trade-off, and experience guides when to do X, and when to do Y. If the acceptance criteria of a piece of new functionality is expressed via a suite of unit tests - that's cool - If there's ever a regression then the test(s) will fail before the broken code makes its way into a live release, saving QA time reproducing the issue, and the Customer/Player's frustration of using a buggy product. If the acceptance criteria changes on a basis that's regular enough that these tests become painful, then there's something fucky going on. Maybe the level of churn on this piece of functionality is so regular such that hard-coding it and covering it with unit tests is a criminally terrible idea - so one option is implement a scripting language... or some other configurable mechanism - and then have a scripted bot/play-through test to verify that the mechanics work in-game, and bin off the unit tests - they were never around long enough to pay for themselves. As for pair programming - I find it absolutely invaluable for spreading knowledge of the code-base being working on - people shouldn't work in isolation. You don't pair program on noddy stuff - that's a complete waste of time - but for the tricky stuff, having somebody to throw ideas around with is great. Investigating tricky bugs, and introducing new team members to the code are two use cases. I've no idea why I wrote this, but it has been cathartic.

[aichpvee]

Every time someone says “pair programming, ” all I can hear is “pear programming.”

[wforbes]

"if you're doing anything interesting" *laughs in business application*