Building Data Discovery and Classification at Scale - Elizabeth Nammour & Pinyao Guo

As a company scales, keeping track of user data becomes an increasingly hard problem to solve, as data is constantly generated and propagated across different data stores. With the rise of new privacy laws such as GDPR and CCPA, tackling this problem is more important than ever before. To address this challenge, we built a platform for data discovery and classification across all of our data stores, such as S3, MySQL and Hive, providing powerful privacy and security engineering capabilities. In this talk, we are going to share the experience we had building and operating this platform for Airbnb. We will present the high level architecture and technical specifics of the platform that allow it to leverage traditional algorithms and machine learning to scan petabytes of user data against growing numbers of data types, every single day.

Elizabeth Nammour
Software Engineer, Airbnb
Elizabeth Nammour is a Software Engineer at Airbnb, where she builds tools to enable data security and privacy across the company. Prior to that, she earned her undergraduate degree in Computer Science from the University of Pennsylvania.

Pinyao Guo
Software Engineer, Airbnb
Pinyao Guo is a Software Engineer at Airbnb working on building data security and privacy tooling and infrastructure. Previously, he worked on building the phishing detection pipeline for Airbnb. Prior to that, he received a Ph.D. from Pennsylvania State University.