Scaling Prometheus: How We Got Some Thanos Into Cortex - Thor Hansen, HashiCorp & Marco Pracucci

Scaling Prometheus: How We Got Some Thanos Into Cortex - Thor Hansen, HashiCorp & Marco Pracucci, Grafana Labs

Cortex is a long term storage for Prometheus, designed for scalability, multi-tenancy and high-availability. It can reliably ingest and query millions of time series per second with sub-second latency. The current storage design uses a NoSQL store to index series and an object store for compressed time series data - two dependencies, and one with significant cost implications. In this talk we will show the new experimental Cortex blocks storage, based on Thanos and Prometheus TSDB, aiming to reduce the Cortex operational cost without compromising scalability and performances. We’ll cover the trade-off between the standard chunks storage and the new blocks storage, and share lessons learned running Cortex at scale. Cortex is a CNCF sandbox project.