SREcon19 Europe/Middle East/Africa - Bigtable: A Journey from Binary to Service and the Lessons...

Bigtable: A Journey from Binary to Service and the Lessons Learned along the Way

Brendan Gleason and Gaurav Prabhu Gaonkar, Google

In this talk, we'll examine the development of a global multi-tenant "Bigtable Service" based on Bigtable, a highly scalable wide column store originally developed for single user, single cluster instances. Because SREs value deduplication of effort, this type of service development work is often undertaken by SREs, but building a service is far more complicated than just wrapping "deploy" in a for loop. We'll discuss the challenges of correctly defining your "product", the revelation that the service layer wrapped around the core is a complex distributed system itself, some common traps that SREs fall into when designing services, and the challenges of migrating users to a central service. Finally, we will describe how the relationship between the core product development team and the SRE team has evolved and highlight best practices and anti-patterns for the developer: SRE relationship that we've learned on our journey.