Build bigger in less time: code testing beyond the basics - Predrag Gruevski | EuroRust 2024

Modern software projects are huge: hundreds of thousands of lines of code across hundreds of dependencies. How do they avoid falling apart due to an endless stream of bugs?

The answer: advanced testing techniques — much more than just _clippy_, unit tests, and end-to-end CI.

Better testing allows us to build bigger in less time, while minimizing bug triage and debugging needs. Let’s dig into some of those advanced techniques and see how they power the projects that we use every day!

*About Predrag Gruevski*
Predrag has spent the last 8 years working on query engines for almost every kind of data imaginable: from relational and graph databases to APIs and unstructured data files. He built Trustfall, a universal query engine that can query any data source(s), and cargo-semver-checks, a semantic versioning linter for Rust.

Previously, Predrag was a principal engineer at Kensho Technologies, did performance engineering research at MIT, and competed in international math and programming competitions.

When not in front of a screen, Predrag enjoys skiing, ice hockey, and board games. He is also a huge space nerd. Ask him about the time he went to watch a giant rocket explode!

--------------------

[porglezomp]

The insight that snapshot testing can be used to make a test suite *faster* was fascinating. I had never considered that.

[claasdev]

I’ve heard about deterministic testing but this is the first time that I actually see what it means concretely. Thank you

[ammaraz]

I think this was of of the most valuable talks in EuroRust this year 🙌

[AdamChalmers]

Really great talk, I've already put it to use and built deterministic simulation tests for my project. OK the deterministic part is a bit aspirational. But it's definitely helped a lot.

[arthurararuna]

Great talk!!! A slight oops though...

5:58 The statement in the comment is only true if you can assert that the data *is* previously sorted. It's a necessary condition, and not a sufficient one.

In fact, you can have a non-sorted array that always yields true to those checks and still returns None, despite containing a wanted value:

data = [ 1, 2, 3, 4, 0 ]
item = 0

You would assert `1 <= 3 && 3 <= 4`, update `right` and lose track of the 0.

All this to say that the BinarySearch algorithm has the minimum precondition (cannot be loosened) that the array is fully sorted (not just partially).

[Mawkler]

This whole talk was very interesting. I would love to see a deep dive into deterministic simulation testing with a more concrete but also simple example, but I can't find any good resource online. I get the general idea, but I'm a little confused on the specifics. How do I go from a random seed to a test case for my app? Should I write specific tests for specific user input myself, or is it more like fuzzing with random input? If it's the former, what's a good technique for creating a specific test case? Do I turn my simulator into a giant builder pattern?

[malteneuss]

Invariant testing sounds a lot like property-based testing (in functional programming languages), and snapshot testing is also known as golden test (in frontend dev) or contract testing (between different systems). You may find additional guides and tools when searching for these keywords.

[Vagelis_Prokopiou]

Excellent insights into testing.
Thanks a lot.

[polares]

Amazing talk. Would love to use this paradigm in my development

[ketzunet]

My main problem with snapshot testing, is snapshots containing more than the specific feature you test. This can lead to many tests failing for a change, leaving many developers with simply regenerating all affected snapshots, as individually checking every affected test can be overwhelming. But regenerated snapshots are automatically correct, possibly hiding bugs.

[maf_aka]

why is invariant testing only done in debug mode? are we saying the assertions will always be true in prod? what/ who enforces that?