Best practices from experts to maximize BigQuery performance (featuring Twitter)

You’ve made the decision to run your data analytics on BigQuery’s serverless platform. As you deploy complex workloads on your data, you want to maximize the performance of all data operations from data loading to data analytics.

Learn the performance best practices from speeding up your data ingest into BigQuery to learning the tips and tricks from the BigQuery engineering team to maximize query performance of your data warehouse.

Speakers: Jagan Athreya, Gary Steelman

Watch more:

#GoogleCloudNext

DA201
product: BigQuery; fullname: Jagan Athreya;

event: Google Cloud Next 2020; re_ty: Publish; product: Cloud - Data Analytics - BigQuery; fullname: Jagan Athreya, Gary Steelman; event: Google Cloud Next 2020;

[zeleravideos]

At 2:15, Jagan says:
"Data is stored in Colossus, which is BigQuery's columnar storage which is encrypted, replicated and distributed making it highly durable against failures"

He is mixing two concepts: Colossus (Google's DFS) and Capacitor (BigQuery's columnar storage).

Jagan should've said:
"Data is stored in Capacitor, which is BigQuery's columnar storage. In turn, Capacitor files are stored in Colossus which is Google's encrypted, replicated and distributed file system making it highly durable against failures"

[intouch]

it’s nice to have the power point representation but what about with technical approach on how we can implement this in real time ?

[jankrynauw]

At 11:41, you mention 'Ingest from GCS or HTTP POST...' Could you please explain what you mean by or give an example of the 'HTTP POST' method?