Production ML Pipelines with Python SDK v2 of Azure Machine Learning

By the end of this video tutorial, you should be able to use Azure Machine Learning (Azure ML) python SDK new version (V2) to productionize your ML project.

This means you will be able to leverage the AzureML Python SDK V2 to:

-connect to your Azure ML workspace
-create Azure ML data assets
-create reusable Azure ML components
-create, validate and run Azure ML pipelines
-deploy the newly-trained model as an endpoint
-call the Azure ML endpoint for inferencing

-Reference codes utilized in this video:

-Azure ML SDK V2 new features:

🔔 Subscribe for more cloud computing, data, and AI analytics videos by clicking on subscribe button so you don't miss anything.

✅ You can contact me at:

#azureml #MLOps #MLPipeline #DevOps #MLEngineering #AI #ML #MachineLearningOperation #machinelearningengineer #cicd #azureml #azuremlops #AzureMLOpsV2 #MLOpsTemplate #PythonSDKV2 #AzurePipeline

[BRN]

I think V2 is heading in the right direction, but it is horrible and unclear to use. Every time i work with it takes me hours and hours of research to find how to do it in the SDK.

[kevon]

This was incredibly helpful. Great tutorial.

[Sunny-qeel]

I want to have a multi-output(more than one target) regression, but the prebuilt "Train model" in Azure ML only allows to set 1 target.
Thinking of creating a custom component for train model that allows multi-target regression.
Do you have any recommendation on this?
Thanks for your video by the way :)

[bilalsedef]

Hi, thank you for this helpful video. I wonder if we can consume SQL query data assets in v2? And how can we pass it through pipeline components as tabular or convert it to uri file/folder between nodes. It was doable in v1 and was helpful for me to use the latest datasets that data engineers prepared every week. But this v2 seems little bit static to me. Thank you in advance

[saurabhray]

Hello @MG, this was informative, but for some reason the ml_client create_or_update does not work. Can you confirm if this is a new issue inside azure itself

[marwanFX]

in your udemy course do you use sdk v2 or v1 ?

[yashgoel]

Can we save any csv file during our pipeline that will be stored in our working directory. For example you saved training and testing data in next steps but will that .to_csv() function will create a file in our directorty?

[shubhamchouhan]

Here let's say I want to access files from blob storage but when I am doing issues like public access is not allowed on this storage account Although it is the same storage account with machine learning workspace we have created here ??
Is there anybody else also getting issues like this as I downloaded the file used here and placed it in a storage account of this workspace

[arian_kazemi]

Hi, Could you explain how can we have child job in sdk2?, I'm using child job in sdk1 to predict my test data but I couldn't find it in sdk2

[vinupillai]

@MG - the course and concept is really good. But with hands on lot of debugging is required - Like now my pipeline is stuck at prep_data stage for last 1 hr 30 mins and in log it's reporting "Job is in progress. Execution status: Preparing.".
Not sure if there is a compute issue or something.

[akrsrivastava]

I noticed that when you built the pipeline, you did not register it to the AzureML workspace. Does SDK v2 not have this functionality? What happens if I want to run the pipeline again, either manually or on schedule or from some trigger? How can this be done?
And thanks for this video. A very timely video for me as I am new to AzureML and just starting off on a new Azure ML project. I have been going through your previous videos as well and they are very well made and informative. One request is that when you are working on some code, please zoom it a little bit for better visibility.

[giorgossartzetakis]

Hej MG. You got some great stuff going on. I have a question. How can I pass objects that are not data frames between components. For example how can I pass data_loaders, or TimeSeriesData sets?