Design a Data Warehouse | System Design

preview_player
Показать описание

Many large companies wind up with several sources of data and want to use all of them together to make business decisions. This video dives into the low-level details of designing a data warehousing system to bring in these data sources, transform them, and analyze them in aggregate.

Table of Contents:
0:00 - Introduction
0:35 - Requirements
1:33 - Data Sources
2:28 - Batch Ingestion System
4:40 - Batch Ingestion Strategies
7:31 - Streaming Ingestion
8:16 - Third-Party Services
9:40 - Transformation
11:09 - SQL Analysis
11:30 - Overview
12:22 - Overview using Snowflake

Socials:
  1. Interview Pen
Рекомендации по теме
Designing Your Data 1:01:21

Designing Your Data Warehouse from the Ground Up

Design a Data 0:14:08

Design a Data Warehouse | System Design

Database vs Data 0:05:22

Database vs Data Warehouse vs Data Lake | What is the Difference?

10 ETL Design 0:04:48

10 ETL Design Patterns (Data Architecture | Data Warehouse)

Data Modeling Tutorial: 0:16:34

Data Modeling Tutorial: Star Schema (aka Kimball Approach)

Let's Compare the 0:05:16

Let's Compare the Kimball and Inmon Data Warehouse Architectures

Data Architecture 101: 0:05:48

Data Architecture 101: The Modern Data Warehouse

Introduction to Data 0:08:39

Introduction to Data Warehouse (Data Architecture | Data Warehouse)

Using Data to 0:53:38

Using Data to Create Liveable Cities - Rachel Lim

What is a 0:09:37

What is a Data Warehouse - Explained with real life example | datawarehouse vs database (2020)

Creating your first 0:06:56

Creating your first Data Warehouse in Microsoft Fabric

Designing DataWarehouse from 2:41:21

Designing DataWarehouse from Scratch | End to End Data Engineering

[[ 3 HOURS 3:05:30

[[ 3 HOURS ]] Data Warehouse Complete Tutorial - SQL + SSIS + SSAS + Power BI - { End to End }

KNOW the difference 0:08:10

KNOW the difference between Data Base // Data Warehouse // Data Lake (Easy Explanation👌)

Preparing For A 0:11:56

Preparing For A Data Engineering Interview: How To Design A Data Warehouse For.A Food Delivery App

Create Data Warehouse 0:05:43

Create Data Warehouse | SQL | Budget Project | Data Warehousing

Data Modeling in 0:10:14

Data Modeling in the Modern Data Stack

Designing Your Data 1:01:21

Designing Your Data Warehouse from the Ground Up

Project A Data 1:54:23

Project A Data Modelling Best Practices Part I: How to Model Data in a Data Warehouse?

Data Warehouse Concepts 0:52:30

Data Warehouse Concepts | Data Warehouse Tutorial | Data Warehouse Architecture | Edureka

Dimensional Modeling 0:53:54

Dimensional Modeling

Introduction to Data 0:05:42

Introduction to Data Warehouse🏺 with Examples

Data warehouse design 0:24:59

Data warehouse design | Bottom-Up approach | Top-Down approach | Steps to build Data warehouse

Databases Vs Data 0:14:44

Databases Vs Data Warehouses Vs Data Lakes - What Is The Difference And Why Should You Care?

Комментарии
Автор

Years of experience all packed in to 14 min, Thank you Sir!
As with all your videos, great work

richardmccauley
Автор

The timing could not be better. I'm about to take on a new role literally about the contents of this video so thank you so much for making this!

Porkductions
Автор

I generally believe this should be lakehouse design instead of warehouse also the database replication into data lake is classified under change data capture (cdc). If you push your data both into queue and database it’s hard to ensure consistency between the datalake and the database ie cases like rollback or database failure. Typically the state of the art solution for this type of problem is to rely on the database journal for instance through the binlog or WAL

MahadirAhmad
Автор

In today's lesson we explain motor vehicles ... We will go over everything ... But the engine ....

DW means show OLTP Schema design vs OLAP

ramielkady
Автор

I'm wondering which app were you using for demo

gaberial
Автор

Surprised that dbt was not mentioned since we're talking about a modern approach👀. Appreciated the video otherwise

decrypt_key