Orchestrating Machine Learning Workflows with Prefect // Kevin Kho // MLOps Meetup #94

Показать описание

MLOps Community Meetup #94! Last week, we talked to Kevin Kho, Orchestrating Machine Learning Workflows with Prefect, co-hosted by Ben Epstein

//Abstract
Prefect is a general-purpose workflow orchestration management system that enables users to build, run, and monitor data pipelines at scale.

In this talk, Kevin shows how to orchestrate a full ML hyperparameter grid search pipeline over a Dask cluster. Other tools included will be Pandera, Tune, and Evidently AI.

// Bio
Kevin Kho is an Open Source Community Engineer at Prefect, an open-source workflow orchestration management system. Previously, he was a data scientist for four years working in the energy and HR spaces. Outside of work, he is a contributor for Fugue, an abstraction layer for Pandas, Spark, and Dask. He also organizes the Orlando Machine Learning and Data Science Meetup.

// Related links

----------- ✌️Connect With Us ✌️-------------
Follow us on Twitter: @mlopscommunity

Timestamps:
[00:00] Introduction to Kevin Kho
[04:34] An Intro to Workflow Orchestration with Prefect
[04:51] Kevin's background
[05:16] Agenda
[05:44] Workflow Orchestration
[05:48] Machine Learning Pipeline
[06:14] Pipeline dependencies
[07:06] Points of failures
[08:57] Negative Engineering
[10:17] Introducing Prefect
[12:21] Current state of Prefect
[12:33] Prefect Orion
[13:59] Simple problem
[14:14] Motivation
[12:22] Coindesk
[14:49] Setting a Slack web block explicitly
[20:41] Put infinite loop in production
[22:30] Change Dodgecoin to Bitcoin
[23:58] Testing the detective
[24:22] Function as a flow
[32:13] Intrinsic data passing from task to task
[37:28] Open-source tools
[37:42] Pandera Tutorial
[44:39] Tune Tutorial
[49:45] Evidently.AI Tutorial
[56:16] ML Training
[1:01:40] Resources
[1:02:00] Prefect is hiring!
[1:02:46] MLOps Cool Merch!
[1:03:01] Networking event to match with the MLOps Community