How to Build Python Data Engineering Pipelines with Snowpark

preview_player
Показать описание
Join Jeremiah Hansen, Principal Data Platform Architect at Snowflake, for a virtual hands-on lab that walks you step-by-step through the process of building Python data engineering pipelines on Snowflake using Snowpark Python. The video includes a brief overview of Snowpark and the benefits it offers by allowing users to work with Python, Scala, and Java natively in Snowflake.

00:00–Introduction
01:01–An Overview of Snowpark and Its Key Features
09:40–The Snowpark Python Quickstart Guide
10:33–An Overview of Topics to be Covered in the Hands-on Lab
14:11–Hands-on Lab Begins
15:01–Prerequisites for Participating in the Lab
17:16–Setting Up Snowflake
19:50–A Look at the Snowflake Extension
20:38–The Work Begins: Loading Data
26:41–Collecting Data from the Data Marketplace
30:44–Creating Views and Streams on Views
33:35–Creating a Python UDF
40:12–Building the First Pipeline
48:16–Creating a Second Stored Procedure to Populate Weather Data Table
51:38–Joining Data Together
53:53–The Deployment
55:35–Orchestrating the Stored Procedures
59:30–Running the Pipelines Incrementally
1:01:39–Viewing the Query History
1:06:25–Deploying Through a DevOps Pipeline using GitHub Actions
1:13:42—Reviewing What We’ve Done

To connect with Jeremiah Hansen of Snowflake:

❄ Join our YouTube community ❄

Learn how to build your application on Snowflake:

Join the Snowflake Community:
Рекомендации по теме
Комментарии
Автор

I think this video is helpful but appreciate if you can send other videos for detailed explanation or online documentation.I am asking about the architecture it starts from 07:19

sathwikemmadi
Автор

If the raw files in csv format, is there anyway to infer the schema without explicitly defining the schema StructType.

joerozario
Автор

Is there documentation on the toml file?

treyhannam
Автор

Do we need an external cloud storage account to implement this project

bharathraj
welcome to shbcf.ru