Building a scalable, open source application data platform using Apache Iceberg (DemandBase)

Relational databases are the backbone of many applications and the go-to for data storage of all kinds but what do you do when you need to access this data for analytics? Oftentimes relational databases are ill suited for analytical use cases and getting data out in bulk can be costly and resource intensive thus making access for your data teams near impossible. By utilizing a change data capture approach to stream data into Apache Iceberg you can achieve low latency data available for analytics at a relatively low cost. Additionally, by creating this general purpose data platform that other disparate sources can be synced to you have the ability to utilize all of your data together. Now you have a data platform that is up to date with your application in near real time that can power all kinds of backend analytics, reporting and ETL use cases without taking down or increasing load on the RDBMS that serves your customers. Key Talk Points: - Challenges of data analytics on application data from relational databases - Apache Iceberg as the unifying layer for your data - Change Data Capture from RDBMS to Iceberg - Putting it all together (RDBMS + CDC + Iceberg) to get value for data analytics, data science, etc.