How to create Great Epxectations suite? Quality Checks for Data Pipelines | Data Quality

In this video we are going to cover how to create a Great Expecations suite for Data Quality testing. Previously we have created a custom suite as a json file. The Expectation library has built-in functions to carry out the data quality tests. With Great Expectations, you can assert what you expect from the data you load and transform, and catch data issues quickly – Expectations are basically unit tests for your data. Great Expectations also creates data documentation and data quality reports from those Expectations.

#dataquality #Python #greatexpectations

💥Subscribe to our channel:

📌 Links
-----------------------------------------
#️⃣ Follow me on social media! #️⃣

-----------------------------------------

Topics in this video (click to jump around):
==================================
0:00 Introduction Great Expectations
0:38 Notebook & Data Import
1:01 Install and configure Great Expectations
2:10 Create connection to data source
3:45 Create Great Expectations suite
4:37 Define & Run Data QualityTests
7:09 Automated Documentation
8:12 Edit & Update suite

[sivasubramanyam]

Thank you so much for sharing your knowledge, can you share some light on pyspark data testing in Databricks in your future videos