Building a Machine Learning Pipeline with Python and Scikit-Learn | Step-by-Step Tutorial

Welcome to our comprehensive tutorial on building powerful machine learning pipelines using Python and Scikit-Learn! In this video, we will guide you through the entire process of creating a robust machine learning pipeline, from data preprocessing to model evaluation

Interested in discussing a Data or AI project? Feel free to reach out via email or simply complete the contact form on my website.

🍿 WATCH NEXT

MY OTHER SOCIALS:

WHO AM I?
As a full-time data analyst/scientist at a fintech company specializing in combating fraud within underwriting and risk, I've transitioned from my background in Electrical Engineering to pursue my true passion: data. In this dynamic field, I've discovered a profound interest in leveraging data analytics to address complex challenges in the financial sector.

This YouTube channel serves as both a platform for sharing knowledge and a personal journey of continuous learning. With a commitment to growth, I aim to expand my skill set by publishing 2 to 3 new videos each week, delving into various aspects of data analytics/science and Artificial Intelligence. Join me on this exciting journey as we explore the endless possibilities of data together.

*This is an affiliate program. I may receive a small portion of the final sale at no extra cost to you.

[RyanNolanData]

d2 = {'Genre':['Rock', 'Metal', 'Bluegrass', 'Rock', np.nan, 'Rock', 'Rock', np.nan, 'Bluegrass', 'Rock'],
np.nan, 1310000, np.nan,
'Sold_out':[1, 0, 0, 1, 0, 0, 0, 1, 0, 1]}

[dsmn]

This is by far the best tutorial I’ve come across on YT on pipelines and column transformers. Thank you Ryan

[abedijkstra]

Looks nice. Can you use k-fold cross validation on the pipeline and tune the hyperparameters of the model inside the pipeline using GrindSearchCV?

[deoz-ye]

This is the Best tutorial based on pipeline. Well explained.

[xnwgywg]

Good video, thanks. One of the best on this topic.

It would help if you explained the main idea more though. It always seemed mysterious how the output of one procedure/ function/pipeline component flowed into the next one. Also what conditions are required to be confident this process turns out right?

[Guidussify]

Do we need to scale for a decision tree?

[iunzgyl]

Thanks again. This is one great video. Very informative and demonstrate how to get it done w pipeline. I just summarize your content, and have a few questions. Let me know if I misunderstand some of your content. Using the cat/num pipeline example for the summary.

# 1 train_test_split raw data with NaN value
# 2 define num_cols and cat_cols for the num_pipeline, and cat_pipeline
# 3 make num_pipeline w Pipeline
# 4 make cat_pipeline w Pipeline
#5 use column transformer to combine num_pipeline and cat_pipeline
# 6 using make_pipeline to combine column_transformer (preprocessing) and DTC (classifier)

[modhua]

where can I find this notebook with all the python scripts in the video?

[arnabmukherjee]

First of all I want to thank you for making such a beautiful informative video on this topic and very neat and clean explanation.
I also want to know how to implement multiple Ml algorithms in the pipelines and choose the best algo according to the problem statement and dataset. Is there any way to do different tasks like 'nulifying multicollinearity(VIF)' and 'dimentionality reduction' within the pipeline?

[shahbazKHAN-wfyn]

best tutorial come across ...love from india

[iunzgyl]

here are my few question:
1) Do we test_train_split first before we do anything?
2) We use fit_transfrom for imputer in previous videos? Do you just need to fit the data in pipeline, and imputer will transform it in the pipeline?
3) what is n_jobs = -1 in the column_transformer?
4) pipeline going to save us time to retype all the code, so say if i want to use a different classifier (like random forest)? Do i just need to modify step 6 like this
RF = RandomForestClassifier()
pipefinal = make_pipeline (col_transformer, RF)
5) also -wonder if i run a different classifier, does that mean i also redo all the preprocessing step and then use a different classifier
6) can you explain how i can use the saved pipeline?

Thanks again, sorry for so many question, but you really provide a lot of good info on this topic for beginners. Thank you very much.

[rishidixit]

What is the difference between makepipeline and pipeline while importing