LISA18 - Managing OS Release Transitions at Netflix Scale

No environment is static. Ed Hunter describes the approach Netflix took when migrating its fleet of virtual machines from one release of Ubuntu to another which can be applied to other large-scale changes.

Managing OS Release Transitions at Netflix Scale

Edward Hunter, Netflix

Netflix runs over 150k+ instances of Ubuntu inside the Amazon cloud (AWS) supporting hundreds of micro-services to serve over 125m customers worldwide. A small team of engineers is responsible for maintaining and evolving the base OS (BaseAMI) on which every service depends. Over the past year or so we have migrated the majority of the fleet from Ubuntu's Trusty release to Xenial. When Bionic released we were ready to start moving services very shortly after the release date.

Our goals with the migration were simple:

Don't break Netflix
Minimize developer pain/complexity during the migration
Be ready for the next release of Ubuntu as soon as practical after it's release
Meeting these goals required changes to packaging, tools and processes. This talk will reveal some of what we do to manage the OS and allow Netflix to deploy it quickly to thousands of VMs on a daily basis. It will also look at what it takes to stay up-to-date with patches and other changes in the ecosystem all while supporting our users, both internal and external, 24x7.