RedisConf 2021: Lessons learned operating Redis Clusters at Roblox

RedisConf 2021
Speaker: Jan Berktold
Track: Industry Solutions
Roblox relies heavily on its Redis and Memcached caching layers to support the experiences of its 150 million monthly active users. Early in 2020, the company experienced reliability and infrastructure management bottlenecks with its caching deployments and decided to invest in building a fully automated way to operate open-source Redis Clusters via Docker on top of Hashicorp's Nomad scheduler. As of today, Roblox engineers have used its self-service APIs to create hundreds of Redis clusters to serve 8 million QPS at peak times while drastically simplifying management of the infrastructure and making the company resilient against large-scale hardware failures. This talk walks through Roblox's requirements for its Redis deployments, telling the story of building its Redis orchestration services and sharing lessons learned along the way.