AWS Glue: Write Parquet With Partitions to AWS S3

This is a technical tutorial on how to write parquet files to AWS S3 with AWS Glue using partitions. This will include how to define our data in aws glue catalog on write

timestamps
00:00 Introduction
00:30 Remap Columns in dataframe
02:57 Write to Parquet - getSink Method

[companionprose]

Love this! FYI it might be a good idea if you're referencing a previous video to put a link in the description for us to easily find it.

[AntonioJiménez-ox]

Thank you for the tutorial! Could I personalize the parquet partition name?

[JavierHernandez-xonb]

Exellent video.... I wish that you make one of AWS Quicksight automatization....😊😊

[joelluis]

Hi ! I've heard that you have the AWS Analytics Speciality Certification.. That's right? Could you please post one video with some advices or resources to prepare this exam or advices ?

I found your chanel today and really liked it !

[jogeshrajiyan]

Hi! I just wanted to know is creating database in glue catalog is a pre-requisite before converting to parquet file or it can be created automatically as you refered for the table in setCatalogInfo() function??

[sanishthomas]

what is this Interface, how we have opened and installed this and connect from AWS, account. can u show something for beginners

[asishb]

Hi, how can I write the Transformed data into a Data Catalog table of AWS Glue, WITHOUT writing the data to S3 ?
Please help !!

[udaynayak]

can you please create a video wherein you read the data from redshift tables under aws glue pyspark(spark.sql)