Leveraging Prometheus’ TSDB for conprof (Continuous Profiling) - Matthias Loibl

Leveraging Prometheus’ TSDB for conprof (Continuous Profiling) - Matthias Loibl

Prometheus’ time-series database (TSDB) stores tuples of timestamp and float64 value, for each time series and thus, is optimized for storing these as efficiently as possible. conprof, on the other hand, needs to store tuples of timestamp and the raw, recorded profile that can be arbitrarily large. We needed to adapt the TSDB for a completely new use case.

Every conprof workflow starts by looking at the timestamps of each series and only afterwards individual profiles are requested. We explored optimizing for timestamp-reads to ignore the profiling data to save CPU and memory. Separating timestamps and profiles would make it easier and maybe we could improve the compression for profiles?
This talk will take us on a journey to see how we've answered these questions so far!"

[alexanderkachur]

Really cool stuff! Have you considered integration with Flame Commander and their protocol?