CppCon 2018: William Clements “Save $$ Testing Code the Playback-Based Way”

—
—
Your actively growing library or component is inadequately tested! Good enough maintenance is nearly impossible, because you cannot run all of your clients' software for regressions.

Enter Playback-Based Testing (PBT)!!

Record artifact during client live session, playback in your lab with just your library. Curate "recordings" from clients for automated integration testing.The benefit/cost ratio can be spectacular, as library scales. (A cost is adapting or doubling the API layer).

Consider leveraging these artifacts to improve author-client collaboration.

The talk: General architecture and strategy of PBT tools described. Impactful PBT is arguably most attainable through a Grow-Your-Own effort. Since it works by exploiting data otherwise thrown away, PBT is like a second cousin of Time Travel Debugging (TTD).

Specific experience: Having been immersed in PBT mindset (a cornerstone of Autodesk Revit's success), William's team attacked flakiness in a Revit integration of provider's cloud data storage framework -- by cobbling together a PBT tool.

—
William Clements, Sr. Principal Engineer
Autodesk

Coding design software for almost 40 years on East coast. Accomplishments include feats of "infrastructure" -- graphics, data archiving, multi-threading. Principle author of a UI-based test recorder that has produced over 15,000 tests for a premier CAD application.
—

*-----*
*-----*