filmov
tv
Developing ML-enabled Data Pipelines on Databricks using IDE & CI/CD at Runtastic
Показать описание
Data & ML projects bring many new complexities beyond the traditional software development lifecycle. Unlike software projects, after they were successfully delivered and deployed, they cannot be abandoned but must be continuously monitored if model performance still satisfies all requirements. We can always get new data with new statistical characteristics that can break our pipelines or influence model performance.
All these qualities of data & ML projects lead us to the necessity of continuous testing and monitoring of our models and pipelines. In this talk we will show how CI/CD Templates can simplify these tasks: bootstrap new data project within a minute, set up CI/CD pipeline using GitHub Actions, implement integration tests on Databricks. All this is possible because of conventions introduced by CI/CD Templates which helps automate deployments & testing of abstract data pipelines and ML models.
The CI/CD templates are used by Runtastic for automating deployment processes of their Databricks pipelines. During this webinar Emanuele Viglianisi, Data Engineer at Runtastic will show how Runtasic is using CI/CD templates during their day to day development to run, test and deploy their pipelines directly from PyCharm IDE to Databricks. Emanuele will present the challenges Runtastic has faced and how they successfully solved them by integrating the CI/CD template in their workflow.
About:
Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Connect with us:
All these qualities of data & ML projects lead us to the necessity of continuous testing and monitoring of our models and pipelines. In this talk we will show how CI/CD Templates can simplify these tasks: bootstrap new data project within a minute, set up CI/CD pipeline using GitHub Actions, implement integration tests on Databricks. All this is possible because of conventions introduced by CI/CD Templates which helps automate deployments & testing of abstract data pipelines and ML models.
The CI/CD templates are used by Runtastic for automating deployment processes of their Databricks pipelines. During this webinar Emanuele Viglianisi, Data Engineer at Runtastic will show how Runtasic is using CI/CD templates during their day to day development to run, test and deploy their pipelines directly from PyCharm IDE to Databricks. Emanuele will present the challenges Runtastic has faced and how they successfully solved them by integrating the CI/CD template in their workflow.
About:
Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Connect with us:
0:44:13
Developing ML-enabled Data Pipelines on Databricks using IDE & CI/CD at Runtastic
0:28:55
CI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on Databricks
0:02:00
What are Data Pipelines?
0:05:25
What is Data Pipeline? | Why Is It So Popular?
0:29:06
Continuous Delivery of ML-Enabled Pipelines on Databricks using MLflow
0:00:58
Data Pipeline Overview
0:13:31
ML/AI Compute and How It Fits Into a Data Pipeline
0:28:33
Quick Introduction to AI and the development pipeline
1:53:35
End TO End Deep Learning project
0:22:36
How to Build Data Pipelines for ML Projects (w/ Python Code)
0:37:12
Building Metadata and Lineage Driven Pipelines on Kubernetes
0:32:52
Deep Learning Pipelines: Enabling AI in Production
0:15:26
How to successfully launch a streaming data pipeline on Google Cloud
0:11:22
Build unified batch and streaming pipelines on popular ML frameworks
0:32:33
Enabling Scalable Data Science Pipeline with Mlflow at Thermo Fisher Scientific
1:28:37
Using open source *Delta Lake* and *dbt* in your ML data pipelines [Demos + Webinar]
0:00:33
Accelerating MLflow 2.0 Pipelines for Production using Databricks
0:05:27
Data Pipeline for ML-enabled Guitar Appraiser
0:26:47
Implementing Machine Learninng Pipelines USsing Sklearn And Python
0:27:05
ML Production Pipelines: A Classification Model
0:09:42
#CONVERGE NY 2018: Agile Data Pipelines to Enable Machine Learning
0:46:10
How and Why To Test Data Pipelines
0:25:03
Accelerate Your ML Pipeline with AutoML and MLflow
0:19:35
Automate Your Machine Learning Pipelines
Комментарии