AWS Serverless Data Lake Architecture

preview_player
Показать описание
#AWS
What AWS services can be leveraged to build a data lake on aws through a serverless architecture is what this video will cover. The video is broken down into its various components including data ingestion, storage, transformation, data cataloging, and analytics.

Relevant Resources:

Citations
-Photo by dirk von loen-wagner on Unsplash

timestamps:
00:00 Introduction
01:07 Data Source Layer
01:29 Data Ingestion
02:52 Storage
03:19 Transformation
04:28 Data Catalog
05:12 Serverless Data Analysis
06:18 Pipeline Orchestration
07:28 Logging
7:53 Lake Formation
Рекомендации по теме
Комментарии
Автор

Would love to see a whole playlist on setting up a serverless data lake with local development. With data coming from DynamoDB, permissions managed with. Lake house, processed with Glue jobs and query the data from Athena

StephenRayner
Автор

Thanks for the video. I have 2 (newbie) questions:
1. How is this a serverless?
2. Where do all these AWS services reside? Inside an EC2 instance?

kannanlg
Автор

Great Introduction, can you pls create a set full length videos or full 10 hours or less I mean a use case for insurance data to build a data lake video

sjvr
Автор

If you needed to reprocess data in your raw zone to your process zone how would you do that? Is there a video on reprocessing data in a serverless data lake?

samerabusaleh
Автор

Are you saying directly ingest data into glue from source and then storing data in s3

ravidawade
Автор

How can i directly ingest data in glue from source, is it necessary to first store data in s3 and then Giving the path of s3, is there any way that i can directly ingest data into glue

ravidawade
welcome to shbcf.ru