Azure Databricks - Building a Lakehouse - Medallion Architecture - Data Pipeline - Reporting- Part 3

Join Kasam Shaikh, leader of the Azure India community, for the live recording of the 17th session of the #GetAzureandAiReady series, to work with Azure Databricks in this informative video, and also where you can learn about Azure and AI in the "Let's Learn Azure and AI with Azure" 2023 series.

Topic covered: Azure Databricks for Cloud Professionals

This is a 4-part series on Azure Databricks for Cloud professionals.

Building a Lakehouse using Azure Databricks and Azure Data Factory

For the 3rd part of this series, Wilson will guide you through:

- Introduce Lakehouse and Delta Lake.
- Explain the Medallion architecture.
- A demo of using PySpark and SQL to create a data pipeline. Bring data from the source to Databricks’ SQL warehouse.

At the end of this session, you will understand Lakehouse and be ready to build your own data pipeline to ingest, transform load the data into Databricks' SQL warehouse.

Speaker - Wilson Mok - Senior Data Architect - Aavande

For all upcoming sessions, register here for once, and explore the sessions with one single joining link throughout the series.

Bookmark the YouTube Playlist - Get Azure and AI Ready - 2023 -

Kindly like, share the video, and subscribe to the channel, hitting the bell icon for future notifications.

Best Regards,
Kasam Shaikh

[Join Kasam Shaikh, leader of the Azure India community, for the live recording of the 17th session of the #GetAzureandAiReady series, to work with Azure Databricks in this informative video, and also where you can learn about Azure and AI in the "Let's Learn Azure and AI with Azure" 2023 series.]

[dactoanho]

Great Presentation Wilson. and Thank you

[robertotosta]

What a class!.

greetings from brazil

[yurioleynikov]

Would love to get link to your demo notebooks

[apoorvag]

Video content is good. Video itself is not very clear, can you pls share GIT link for the code walk through ?

[AmitSharma-]

Hey great video, can you share the Git link?

[gunasekharraju]

Excellent video on medallion architecture, Thank you.
If there is Git Link for these notebooks that would be awesome. Thanks

[TheDataArchitect]

You have used bronze_processing_date, how to make it automated, that only latest records should be processed from bronze to silver, without providing the date filter?