filmov
tv
Use a SQL DB For User-Facing Analytics? Apache Pinot!
Показать описание
I just learned about Apache Pinot. This video is short demo and description of the business problems I think it solves (quite well).
Most of the work I've done in analytics and ML has been batch use cases: data gets aggregated into a big data warehouse each night, a bunch of business questions run while everyone sleeps in the form of SQL and python jobs, and then execs and LOB managers read reports in the morning.
But when the end user/consumer of your interesting business question is an actual customer/web app/mobile app user, overnight jobs on day-old data don't cut it. And you don't have 10 or 50 or 100 customers, but maybe 10k or 10M+
In those scenarios, typically heavy engineering effort is invested to build indexes in large key-value stores to meet the latency and freshness requirements. But it sure would be easier to just use a database and write some SQL. And that's where Pinot is filling a gap. I'm just running through a quick start dataset but even with old MLB stats this looks really promising
Most of the work I've done in analytics and ML has been batch use cases: data gets aggregated into a big data warehouse each night, a bunch of business questions run while everyone sleeps in the form of SQL and python jobs, and then execs and LOB managers read reports in the morning.
But when the end user/consumer of your interesting business question is an actual customer/web app/mobile app user, overnight jobs on day-old data don't cut it. And you don't have 10 or 50 or 100 customers, but maybe 10k or 10M+
In those scenarios, typically heavy engineering effort is invested to build indexes in large key-value stores to meet the latency and freshness requirements. But it sure would be easier to just use a database and write some SQL. And that's where Pinot is filling a gap. I'm just running through a quick start dataset but even with old MLB stats this looks really promising