An introduction to Apache Parquet

In this video, we learn all about Apache Parquet, a column-based file format that's popular in the Hadoop/Spark ecosystem. We use pyarrow and parquet-cli to make sense of some Parquet files from the NYC Taxis dataset.

Resources:

[evefreeman]

Mark, I have to say, you are one of the few youtube people I don't feel like I need to set 1.5 or 2x speed on to watch. :) Thanks for the videos!

[HashemAlDhaheri]

Thanks Mark. You're not only explaining the usefulness of using Parquet in about 5m, but giving us extra tips on using some very useful tools as well.

[EmergingStar]

Succinct and to the point. Just what you need to go from "dumb" to "dangerous." Thanks!

[madelineDaMiddle]

Thank you for edification that doesn't waste time. Well done, Sir!

[justinxia]

Very great video, achieved something that numerous cloud providers couldn't achieve with their lengthy documentations within 5 minutes.

[abhishekprakash]

Thanks, Mark, for this awesome video and this is very informative

[_truthful_q_]

Wow, succinct, to the point, and above all useful!

[snoreking]

this was so easy to understand thanks mark

[kutra]

perfectly explained.. didn't need too much verbosity and you did a great job

[ntswfwd]

Hi Mark, great video. Could you pls cover Parquet Modular encryption topic also.

[sidilekhalifa]

Hello, I have many .parquet files of the same type and I would like to display these files as a 'select * from ...many_file.parquet', how can I do this with parq please?

[TheSlurton]

This was like drinking from a fire hose. I liked it but would like to see a much more detailed video where you download the file describe it and then go over the tools needed to make use of the file.

[coopernik]

Good video but u need to slow down your speech mate

[alx]

Every binary format will be more size effective than CSV, YAML, JSON or god forgive me XML. Just because the latter are very bloated when it comes to size.

[TheHitessh]

Why are you so fast? It's overwhelming for the people who doesn't know a specific topic.

[williamknox]

🫤 this would have been more helpful if you'd had significantly less caffeine.