Understanding Overfitting and Underfitting in Machine Learning (Variance vs Bias)

Показать описание

One of the very important concepts to understand in Machine learning is the concept of underfitting and overfitting. Let us understand underfitting and overfitting with a very simple example.

Say, there are two kids who have prepared for a mathematics examination. First kid Tom only learnt additions. He skipped subtractions, multiplications, and divisions.

Second kid, Mary has a really good memory and she memorized all the problems from the textbook

In the exam, Tom will only be able to solve questions related to additions and will fail in questions related to subtractions, divisions and multiplications.

On the other hand, Mary will only be able to answer questions if they were from the same textbook which she had memorized. Mary will falter if she encounters any question which was not there in the textbook or if the question comes from any other textbook.

Both Tom and Mary will not be able to perform well in the exam.

Machine Learning algorithms also have similar behaviour. Sometimes the models these algorithms generate are like Tom where they learn from only some part of the training data. In such cases, the model is called to be underfitting.