What is AdaBoost (BOOSTING TECHNIQUES)

In this Video we will discussing about the ADABOOST algorithm which is basically a boosting technique.

Buy the Best book of Machine Learning, Deep Learning with python sklearn and tensorflow from below
amazon url:

You can buy my book on Finance with Machine Learning and Deep Learning from the below url

Connect with me here:

Subscribe my unboxing Channel

Below are the various playlist created on ML,Data Science and Deep Learning. Please subscribe and support the channel. Happy Learning!

You can buy my book on Finance with Machine Learning and Deep Learning from the below url

🙏🙏🙏🙏🙏🙏🙏🙏
YOU JUST NEED TO DO
3 THINGS to support my channel
LIKE
SHARE
&
SUBSCRIBE
TO MY YOUTUBE CHANNEL

[ashisharora]

Adaboost (Adaptive Boosting)
Adaboost combines multiple weak learners into a single strong learner.
This method does not follow Bootstrapping. However, it will create different decision trees with a single split (one depth), called decision stumps.
The number of decision stumps it will make will depend on the number of features in the dataset. Suppose there are M features then, Adaboost will create M decision stumps.
1. We will assign an equal sample weight to each observation.
2. We will create M decision stumps, for M number of features.
3. Out of all M decision stumps, I first have to select one best decision tree model. For selecting it, we will either calculate the Entropy or Gini coefficient. The model with lesser entropy will be selected (means model that is less disordered).
4. Now, after the first decision stump is built, an algorithm would evaluate this decision and check how many observations the model has misclassified.
5. Suppose out of N observations, The first decision stump has misclassified T number of observations.
6. For this, we will calculate the total error (TE), which is equal to T/N.
7. Now we will calculate the performance of the first decision stump.
Performance of stump = 1/2*loge((1-TE)/TE)
8. Now we will update the weights assigned before. To do this, we will first update the weights of those observations, which we have misclassified. The weights of wrongly classified observations will be increased and the weights of correctly classified weights will be reduced.
9. By using this formula: old weight * e performance of stump
10. Now respectively for each observation, we will add and subtract the updated weights to get the final weights.
11. But these weights are not normalized that is their sum is not equal to one. To do this, we will sum them and divide each final weight with that sum.
12. After this, we have to make our second decision stump. For this, we will make a class intervals for the normalized weights.
13. After that, we want to make a second weak model. But to do that, we need a sample dataset on which the second weak model can be run. For making it, we will run N number of iterations. On each iteration, it will calculate a random number ranging between 0-1 and this random will be compared with class intervals we created and on which class interval it lies, that row will be selected for sample data set. So new sample data set would also be of N observation.
14. This whole process will continue for M decision stumps. The final sequential tree would be considered as the final tree.

[pankaj]

My Suggestion will be that first arrange your playlist, so that we do not get confused of topics

[World-vfts]

This was the longest 14min video I have ever seen....
The content of the video is much much more than the displayed duration of video
Thanks a lot sir

[bhavikdudhrejiya]

This is a in-depth process of ad boosting algorithm.
Great explained by Krish Sir. Thank you for making such a wonderful video.
I have jotted down process step from this video:

This iteration is performed until all misclassification convert into correct classification
1. We have a dataset
2. Assigning equal weights to each observation
3. Finding best base learner
-Creating stumps or base learners sequentially
-Computing Gini impurity or Entropy
-Whichever the learner have less impurity will be selecting as base learner
4. Train a model with base learner
5. Predicted on the model
6. Counting Misclassification data
7. Computing Misclassification Error - Total error = sum(Weight of misclassified data)
8. Computing performance of the stumps - Performance of stumps = 1/2*Log-e(1-total error/total error)
9. Update the weights of incorrectly classified data - New Weight = Old Weight * Exp^performance of stump
Updating the weights of correctly classified data - New Weight = Old Weight * Exp^-performance of stump
10. Normalize the weight
11. Creating buckets on normalize weight
12. Algorithm generating random number equals to number of observations
13. Selecting where the random numbers fall in the buckets
14. Creating a new data

15. Running 2 to 14 steps above mentioned on each iteration until it each its limit
16. Prediction on the model with new data
17. Collecting votes from each base model
18. Majority vote will be considered as final output

[karangupta]

One of the best explanations of AdaBoost if I have seen so far... Keep up the good work Krish :)

[lohithv]

Each and every topics are there in your channel on DS, ML, DL and which is explained clearly.Because of you many of the students learn all these kinds of stuff, thanks for that.I assure no one can explain like this with such a content💯. once again thank u so... much....

[bhargavasavi]

Krish was mentioning 8 iterations for selecting the records for the next learner...there are really 7 records...it will choose a random bucket 7 times...and since the max weighted values mostly will be present in the larger bucket size, probability of rand(0, 1), most of the time the maximum bucket will be technique!!

[yuvrajpawar]

Watched all your videos but still always eager every day for next topic to learn

[rahulalshi]

At 8:13 3rd record is incorrectly classified, so shouldn't the updated weight value of 3rd instance be 0.349

[aination]

Indian Youtubers are the best. Always! To the point and clear explanation.

[sandipansarkar]

Great video once again. plies don't forget to watch it once more as things are getting a little bit more complicated. I will watch the same video again but not today. tomorrow. Thanks

[sitarambiradar]

One of the best explanatory video of AdaBoost. thank you sir!!

[somnathbanerjee]

@8:30 minutes of the video, it should be 0.349 for an incorrectly specified classifier. As we got updated weight for the correctly specified classifiers. I love your teaching. Adore.

[teslaonly]

You should have gotten more views for this video. Your explanation is excellent

[sergeypigida]

Hi Krish! Thanks for the quick and clear explanation. At 11:42 you missed one thing. When we got a new collection of samples we need give all samples equal weights again 1/n

[raghavendras]

@Krish Naik : Thank you very much for the video. Concepts are clearly explained and it is simply Excellent. One thing I wanted to highlight is --- In the Adaboost, final prediction is not the mode of the prediction given by the stump's. It is that value, whose group's total performance say is high

[MatheoXenakis-ry]

You just adaboosted my confidence my guy

[SUNNYKUMAR-vkng]

now i got better understanding of ensemble techniques, thanks sir

[seahseowpeh]

Thanks for the explanation. Good video.

[gnavarrolema]

Thank you for this great explanation 👍