filmov
tv
Machine Learning Model Comparison with Bootstrap Resampling | sklearn Implementation
Показать описание
At some point in your machine learning analysis, you want to be able to say that classifier A is better than B and that this is statistically significant. In this video I will show you a technique to be able to make such a statement which make uses of Bootstrap Resampling.
Acknowledgement:
- music from the youtube library
- used seaborn for the violin plot
- thumbnail made with Canva
This is what Wikipedia as to say about Bootstrap Methods:
"Bootstrapping is any test or metric that uses random sampling with replacement, and falls under the broader class of resampling methods. Bootstrapping assigns measures of accuracy (bias, variance, confidence intervals, prediction error, etc.) to sample estimates. This technique allows estimation of the sampling distribution of almost any statistic using random sampling methods.
Bootstrapping estimates the properties of an estimator (such as its variance) by measuring those properties when sampling from an approximating distribution. One standard choice for an approximating distribution is the empirical distribution function of the observed data. In the case where a set of observations can be assumed to be from an independent and identically distributed population, this can be implemented by constructing a number of resamples with replacement, of the observed data set (and of equal size to the observed data set).
It may also be used for constructing hypothesis tests. It is often used as an alternative to statistical inference based on the assumption of a parametric model when that assumption is in doubt, or where parametric inference is impossible or requires complicated formulas for the calculation of standard errors. "
The technique can be summarized as this:
- Sample your data with replacement to create a bootstrap dataset (same length)
- Run your machine learning pipeline on that bootstrap dataset.
- add the performance metric to your distribution.
- repeat this N amount of time
At the end of this process you will have a distribution of your performance metric. You can then use that distribution to compare it with another model distribution. If the middle 95% of the distributions from model A and B don't overlap you can say that the improvement of model B is statistically significant with p smaller than 0.05 for a bootstrap resampling of n = 1000!
----
----
Follow Me Online Here:
___
Have a great week! 👋
Acknowledgement:
- music from the youtube library
- used seaborn for the violin plot
- thumbnail made with Canva
This is what Wikipedia as to say about Bootstrap Methods:
"Bootstrapping is any test or metric that uses random sampling with replacement, and falls under the broader class of resampling methods. Bootstrapping assigns measures of accuracy (bias, variance, confidence intervals, prediction error, etc.) to sample estimates. This technique allows estimation of the sampling distribution of almost any statistic using random sampling methods.
Bootstrapping estimates the properties of an estimator (such as its variance) by measuring those properties when sampling from an approximating distribution. One standard choice for an approximating distribution is the empirical distribution function of the observed data. In the case where a set of observations can be assumed to be from an independent and identically distributed population, this can be implemented by constructing a number of resamples with replacement, of the observed data set (and of equal size to the observed data set).
It may also be used for constructing hypothesis tests. It is often used as an alternative to statistical inference based on the assumption of a parametric model when that assumption is in doubt, or where parametric inference is impossible or requires complicated formulas for the calculation of standard errors. "
The technique can be summarized as this:
- Sample your data with replacement to create a bootstrap dataset (same length)
- Run your machine learning pipeline on that bootstrap dataset.
- add the performance metric to your distribution.
- repeat this N amount of time
At the end of this process you will have a distribution of your performance metric. You can then use that distribution to compare it with another model distribution. If the middle 95% of the distributions from model A and B don't overlap you can say that the improvement of model B is statistically significant with p smaller than 0.05 for a bootstrap resampling of n = 1000!
----
----
Follow Me Online Here:
___
Have a great week! 👋
0:05:01
All Machine Learning Models Explained in 5 Minutes | Types of ML Models Basics
0:05:49
AI vs Machine Learning
0:02:48
Classification and Regression in Machine Learning
0:21:33
Machine Learning Algorithm- Which one to choose for your Problem?
0:06:27
Supervised vs Unsupervised vs Reinforcement Learning | Machine Learning Tutorial | Simplilearn
0:07:08
Supervised vs. Unsupervised Learning
0:26:42
Comparing machine learning models in scikit-learn
0:10:05
How to evaluate ML models | Evaluation metrics for machine learning
0:33:46
Supervised Learning: Classification in Machine Learning | AIML End-to-End Session 44
0:03:30
7 Steps to Build a Machine Learning Model
0:02:35
Machine Learning Explained in 100 Seconds
0:07:50
Machine Learning vs Deep Learning
0:07:52
Machine Learning | What Is Machine Learning? | Introduction To Machine Learning | 2024 | Simplilearn
0:00:36
THIS is HARDEST MACHINE LEARNING model I've EVER coded
0:08:09
When M1 DESTROYS a RTX card for Machine Learning | MacBook Pro vs Dell XPS 15
0:16:30
All Machine Learning algorithms explained in 17 min
0:30:57
Build your first machine learning model in Python
0:09:45
AI VS ML VS DL VS Data Science
0:11:37
Machine Learning Tutorial 3 - Intro to Models
0:46:18
10 ML algorithms in 45 minutes | machine learning algorithms for data science | machine learning
0:00:59
Deep Learning vs. Machine Learning, which is better?
3:53:53
Machine Learning for Everybody – Full Course
0:00:26
I can't STOP reading these Machine Learning Books!
0:02:19
Support Vector Machine (SVM) in 2 minutes
Комментарии