Python profiling and performance tuning in production (Joe Gordon)

Pinterest decreased latency and shrunk their front-end fleet by over 40% with less than 100 lines of python. This was done by understanding how a significant amount of latency and consumed resources were the result of gevent’s cooperative multitasking. 4 different open source python profilers, were then used together to identify and fix bottlenecks and inefficiencies in the code base. Most profilers are too slow to use in production, so profiling of web servers often occurs in synthetic environments. But to get an accurate picture of what is happening you need production profiling data, so a production ready Linux signal (SIGVTALRM) based python profiler was used. This talk will cover how Pinterest shrunk their front-end fleet, the techniques and tooling used to improve performance and how they can be applied elsewhere.