Top AWS Services A Data Engineer Should Know

This is a high-level overview that the top services on AWS a data engineer should know in order to solve their data engineering challenges. I explain it by using an example of integrating 2 different data sources to create a central data repository to enable our hypothetical analytics team to perform their own self-service analytics. This video is broken down into Data Ingestion, Data Lake, Transformation, Data Warehouse, Data Analytics, Application Integration, Data Pipeline Orchestration, and Monitoring.

Timeline

00:00 Introduction
01:05 Data Ingestion
04:24 Storage - S3
05:10 Transformation
05:47 Data Catalog
06:44 Data Warehouse
07:13 Data Analytics
09:08 Application Integration
10:28 Orchestration
11:57 Monitoring


useful links:

#AWS
#dataengineering

[cringe]

Woah woah
I know nothing about AWS
Then why the heck did i totally totally understand this video
It was crystal clear
I usually don't subscribe to channel but this time it was not even a question 👍🏾
Man I wish you could teach me all about data engineer

[renzcarillo]

As a self taught data engineering student, figuring out what services to start with aws is very hard - this indeed uncomplicates everything!

[jihanzhang]

Great video. Something to add here: S3 Select can be used for quick and adhoc querying dealing with single S3 file. Athena can also work directly with S3 files if you just need some quick data understanding and investigation. EMR Serverless can address the headache for managing EMR cluster and in the meantime gives your more power for ML.

[rahulsrivastava]

The concepts in this video went inside my brain like a hot knife going in butter. Great video for someone like me who comes from a functional background. Great work...really appreciated.

[vibhavmishra]

I am glad I found this video. Brilliant overview. cheers !!

[mehmetkaya]

Such a great video! Summarized basic AWS services for data engineering very nicely! One of the best! Thanks!

[go]

It is so far the most helpful video I saw about aws services for DE. I hope there are more likewise. Thanks a lot for sharing!

[shreyaroraa]

One of the best AWS explanation I saw so far

[Omer]

Your channel is a god send. Data Engineering channels are rare on youtube and those that do exist are tailored towards Indian Students. Thank you for the content and you've got a new subscriber.

[chriscrocker]

I'm currently working on my AWS certification and will be referencing the diagram from this video often. Thanks for the clear and concise walk through of the context of each of these services!

[saiduluchintha]

Excellent video.. the sequence you have covered this in is seamless.
I am surely having this for quick reference.

[Ghillieye]

Great overview and I think your method of slowly explaining the diagram section by section is brilliant! A follow up video of a real use case would be even better. Subbed!

[victoriwuoha]

@DataEng Uncomplicated. This has to be one of the best explanation of how I can use AWS for my data analytics engineering workloads. Thank you for the detailed summary of the various services.

[bqphimq]

Thanks for creating this video. You explained the concepts very clearly.

[jasminew]

Great video Adriano! It helped me understand all the AWS services better.

[obiebbw]

As someone else commented. I'm learning to be a Data engineer and learning what each application is used for has been a struggle. I'm learning the Azure system, but seeing this visual helped. New sub.

[bartstough]

Still a great overview. Makes everything a lot clearer. Thank you.

[endpermia]

Thank you so much for this video! I have an interview tomorrow and this boosted my understanding and confidence. Great explanations!

[LittleDetours]

Love your clarity on the topic. Subscribed! Can't wait to explore all your videos👀

[ijihqpe]

Thank u so much. Your tutorial helps me a lot.