Intro to Amazon EMR Toolkit

See how to install and use the Amazon EMR Toolkit for VS Code.

- Browse your EMR on EC2, EMR on EKS, and EMR Serverless resources.
- Explore your Glue Data Catalog and view table details.
- Create a local PySpark development container based on EMR.
- Deploy PySpark jobs to EMR Serverless

Table of Contents:

00:00 - Intro
00:40 - Installing the EMR Toolkit
01:22 - EMR Explorer
02:25 - Glue Data Catalog Explorer
03:04 - EMR local development container
05:12 - PySpark shell in EMR DevContainer
06:22 - Jupyter notebook in EMR DevContainer
07:31 - Deploy PySpark to EMR Serverless

[tysong]

Hi Dacort, I can’t finish the “Reopen in Container” step, as after it downloads the docker image, an error pops up saying “yum doesn’t have enough cache data to continue” at step “RUN yum install -y sudo && “Hadoop ALL=(ALL) NOPASSWD:ALL”..

Wondering if you might know any reason behind? Thank you so much!

[sahilbhatia]

Hi Decort. Thank you for the video but somehow I am unable to connect this with my aws account as we use the SSO (single Sing On) authentication. But I am able to connect via AWS toolkit using the 'aws configure sso' command. Could you please help what can I do in this case?

[panchao]

Thanks for creating this nice tool. In the Glue catalog view, can we also see the partition keys of a table?

[atoz]

Thanks for the video Dacort..Please share how to connect my own S3 from Docker container. I try to add AWS keys in docker file as ARG. but it did not worked after rebuild the container. Please advise..

[arjunshah]

This is great Damon. Just a quick question. Does this need a connection to AWS similar to the AWS Toolkit in Visual Studio? Somehow even after having AWS local profile and the necessary permissions, the explorers does not load

[itzikpaz]

Hi, thanks for the great video 😊
I am searching for an article or any direction to build a development pipeline on top of emr-eks from local/dev to production.