How does BigQuery store data?

How does BigQuery’s internal storage work? In this episode of BigQuery Spotlight, we share how BigQuery stores data so you can make informed decisions on how to optimize your BigQuery storage. We’ll also talk about partitioning, as well as clustering and how it allows for efficient lookups.

Timestamps:
0:00 - Intro
0:27 - Overview
0:50 - Columnar Storage
1:46 - Capacitor File Format
2:35 - Colossus Distributed File System
4:30 - Storage Optimization
5:00 - Partitioning
6:23 - Clustering
7:12 - Partitioning + Clustering

#BigQuerySpotlight

product: Cloud - Data Analytics - BigQuery; re_ty: Publish;

[andreanardone]

The video maybe contains an error at 1:35 - it says "OLTP" while it should probably say "OLAP" for Columnar Storage

[Caspar]

Oh man. Finally a clear explanation of clustering. I've been studying for my Data Engineer cert for the last couple of months and this is the best explanation of clustering that I've read. Thanks!

[GoonCity]

Thank you for explaining how this operates with the tables and capacitor file format

[AkshayAradhya]

Anyone else noticed that she switched the places of the paintings in the background ?

[tobiasvogler]

Does a Capacitor store a whole table in a columnar layout or does one capacitor store one column?

[popvic]

partitioning is a condition to clustering...