Adopting Apache Iceberg on LINE Data Platform - 2021 English version -

Data #linedevday
Apache Iceberg is an emerging table format which tracks files in a table over time with the concept of snapshots which are created atomically.
The advantage of this design is that it is expected to help us address challenges in Data Platform, such as building large scale low-latency data pipelines and supporting mutating tables reliably.
In this session, we will outline the motivation and expected use cases of adopting Apache Iceberg at LINE, and present an ongoing project of revamping our log data pipeline architecture with Apache Iceberg.

■ Speaker
Tomoyuki Saito
LINE / IU Tech Forward team / IU Tech Forward team
Tomoyuki Saito joined LINE as a new graduate in 2015 and has been engaged in the development and operations of log collection infrastructures using Apache Kafka and Apache Flink and Elasticsearch as log storage. Currently, as a senior software engineer in Data Platform Department, Saito is responsible for research and development for the introduction of new technologies and architectures, as well as leading development projects.

Takeshi Ono
LINE / Data Engineering1 / Software Engineer
Takeshi Ono has been engaged in Hadoop-related projects mainly as a software engineer for over 10 years. Ono worked for a consulting firm and joined LINE as an engineer in March 2019. Currently, Ono develops Ingestion Pipeline in Data Platform Department.

■ Website

■ Slide

■ Other language Movie