AWS EMR Serverless - What is it? [FULL TUTORIAL in 25mins]

00:37 - What is EMR Serverless? Part 1
00:58 - What is EMR?
01:34 - What is EMR Serverless? Part 2
02:30 - EMR Vs EMR Serverless
03:21 - Glue Vs EMR Serverless
04:40 - Tutorial: Setup Work
13:52 - Tutorial: Create EMR Studio
17:02 - Tutorial: Create Spark App
19:20 - Tutorial: Create Hive App

In this video we take a look AWS EMR Serverless which is a new service from AWS that allows users to run Spark and Hive applications on demand. EMR Serverless removes the barriers to entry of EMR as a user no longer has to manage the underlying infrastructure that comes with EMR. We compare EMR Serverless to AWS EMR and AWS Glue. Then we create are own Spark and Hive apps on the AWS Console with a full tutorial.

😎 About me
I have spent the last decade being immersed in the world of big data working as a consultant for some the globe's biggest companies.My journey into the world of data was not the most conventional. I started my career working as performance analyst in professional sport at the top level's of both rugby and football. I then transitioned into a career in data and computing. This journey culminated in the study of a Masters degree in Software

[BabaiChakraborty-sspt]

This is great video to lean. Thanks Johnny

[skywalkerful]

Great Video .. Much needed one. Now many clients are asking if they can try EMR serverless for their systems instead of EMR clusters.

[ReenanOFC]

Very good video. What configs should I do to read a file in other aws account s3 bucket ?

[chconnect]

Hi, what are the advantages from an spark job vs glue spark job?. Meanwhile the "application" inside the studio is running there is a cost? or only per job?

[pradeepm]

Hi Johnny, thanks for the EMR Server less tutorial, i request could you please share us how to upgrade an EMR cluster from 6.5.0 to 6.6.0 and how we can install a software that i missed to opt during EMR cluster creation lets say sqoop or Flink.

Hoping your best knowledge session continues as usual, Keep rocking :-)

[satishmajji]

Can you please make a video on unit testing for sql query in AWS using step functions??

[masterashu]

Hi Johnny, thanks for this amazing video. I tried same approach to run my spark job but getting Access Denied on S3 service. I have tried with full S3 access as well as Administrative access, but no luck. Please let me know, what could go wrong?

[mdabdulmujeebmalik]

Hi Johnny. Firstly your videos are awesome. However, I have been getting this error as shown in the screen shot below.
Failed to open logs for hive-job (00faoga565cv480a). PersistentAppUI isn't available for jobs that never ran.

[srirajvasireddy]

How do you deploy or run spark jobs on emr serverless when python script has dependency with external libraries like pandas, kafka?

[sumitdavid]

Hi Johnny, I am trying to run a SpringBoot application on EMR Serverless, despite providing jar and the main file name, getting the error Failed to load, calling shutdown hook. Any idea?

[ViswaThatha]

can we schedule pyspark script in EMR serverless?

[fatenlouati]

how to install python libreries please