Machine Learning Interview Series#1-Asked in Interview

When should u select the loss function of mean squared error and mean absolute error in a regression problem

[xzsexqj]

Mean Squared Error (MSE) and Mean Absolute Error (MAE) are both commonly used metrics for evaluating the performance of regression models. They are used to quantify the differences between predicted values and actual (observed) values in a dataset. The choice between MSE and MAE depends on the specific characteristics of the problem and the preferences of the analyst. Here's when to use each metric:

Mean Squared Error (MSE):

MSE is the average of the squared differences between predicted and actual values.
It penalizes larger errors more heavily due to the squared term.
Use MSE when you want to give higher weight to large errors. This is useful when outliers have a significant impact on your analysis or when you want your model to focus on reducing larger errors.
Mathematical Formula:
MSE = Σ (predicted - actual)^2 / n

Mean Absolute Error (MAE):

MAE is the average of the absolute differences between predicted and actual values.
It treats all errors equally since it doesn't use squared terms.
Use MAE when you want to give equal weight to all errors. This is useful when you have a more balanced dataset and outliers are not as important.
Mathematical Formula:
MAE = Σ |predicted - actual| / n

Considerations:

MSE is more sensitive to outliers than MAE, as the squared term amplifies the impact of large errors. If your dataset has a few extreme outliers that you want to be particularly attentive to, MSE might be more appropriate.
MAE is more robust to outliers since it only considers the absolute differences, making it a better choice when your data is noisy or contains significant outliers.
Both MSE and MAE can be used together or in combination with other metrics depending on the context and goals of your analysis.

[subhadipghosh]

Mse is used because mse is continuous and differentiable at all points, and for the gradient descent algorithm to work, the loss function must be differentiable at all points. Whereas in MAE, the mathematical expression is mod of y_act - y_pred, and mod is not differentiable at all points which is why we choose Mse as our loss function.

[chiragverma]

MAE: when we have outliers we use MAE as MSE will give a very large amount of error for outliers.
MSE: But when we don't have outliers then we use MSE as MAE is not differentiated at 0 and also MSE gives an optimum amount of error and due to which loss is get reduced much faster in MSE as compared to MAE in this case.

[mihirnatani]

HUBER LOSS function can be used -- It is
1 ) MAE for outliers.
2 ) MSE for small values .
Hence giving us the benefit of using both functions .

[Sayakvids]

here i am waiting for others to answer xD

[mohammadmaaz]

MSE : This measures the squared average distance between the real data and the predicted data. Here, larger errors are well noted (better than MAE).but disadvantage is that it also squares up the units.

MAE :This measures the absolute average distance between the real data and the predicted data, but it fails to punish large errors

[durgeshmajeti]

We usually use MSE because the MSE penalizes the outliers. That way we are ensuring that the model learns from all the points. With MAE, the model doesn't penalize the outliers as much as MSE does. So, the model will not be learning much from outliers.

[PritishMishra]

Means absolute error will ignore the large error in the data as it just substracts.. but Mean squared error is more efficient when we does not have to ignore large errors...

That's why MSE is more generally used over MAE

Secondly, MAE also ignores outliers and more usally used when we have to ignore the outliers in our data..
While MSE also accomodate outliers in our data!

[arunmeghani]

the answer will be mean squared error probably. the reason is that whenever the differences between y true and y predicted is increasing, then the value of mean squared error will increase more rapidly than mean absolute error, thus mean squared error will punish our model more than mean absolute error, and thus the optimization will be faster. I hope this is the reason

[varunsingh]

Mae will be used if there is outliers in data else we will use MSE, as in MSE error due to outliers will increase, this type of loss function is called huber loss function.

[kripalsinghbamel]

MSE is generally preferred as it's differentiation is easy. Also it penalize more for large errors.

[shubhamchoudhary]

@Krish Naik As we know that outliers is big issue in Linear regression..
If our data has some outliers or extreme values then we use |MSE| error, coz in case of outliers the error difference will be high (even the model is predicting close to actual values) hence error terms will be very high, gradient descent will try to penalize those error in such way that the error will reduce and tend to zero(ideally) & if we used (MSE) at this situation, our best fit line is more towards the outliers side rather than original normal data points so it'll affect the final predictions significantly.
When the consideration of outliers are necessary then we can use MSE as a cost function.
MAE loss useful if training data corrupted with outliers, but MAE also have some disadvantages like gradient will be large even for small loss values, this is not good for learning .
MSE will perform better in this situation, the gradient of MSE is high for larger loss & vice versa ..
Meanwhile, if the outliers represent anomalies that are important in business then we can use MSE, else if outliers are useless then we can use MAE.



Make me correct & suggest some additional topics !!
Thanks for reading and helping out !!🙏

[sandyofficial]

When you require the magnitude not the direction and if the outliers present in the data use mean absolute error and when there is no outlier present in the data and the magnitude is more preferable then direction of the error we will go with mean squared error as the mse will squares the values by that if there is any outliers present in the data it will be heighly impact and the squared error will be very high result will be fail to give good model.

[RishSandMagic]

When we have outliers we should go for MAE. MSE actually penalize large errors much more than they actually are. Lets suppose, we have a financial related problem statement, we should go for MAE as a diff of 30Inr is better left at 30inr rather escalating it to 900inr in case of MSE

[amarpratapsingh]

MSE is more sensitive to outliers in our predictions so it diminishes the effect of smaller errors and one large error could mislead us into thinking that the whole of our model is not working as it is supposed to be. Also the error value it gives is not on the same scale as the data points(ie why rmse exists).
MAE has "equal" weightage to each error and is less prone to outliers. The value provided is on the same scale as our datapoints.
If our overall error is linear ie an error of 10 is twice as bad as an error of 5, we should use MAE otherwise MSE or RMSE.

[myfeelings]

MSE:Mean Squared Error is used if the numbers associated with the data points are not large enough to compute.Therefore normally it is used if computational complexity does not arise at a significant label.
Mean Absolute error is used if the data points are sufficiently large numerically.To reduce computational complexity, it is useful there.

[tjdyhuv]

in case of outliers we use MAE and if there is no outlier or less effects on model then we use MSE

[richaasenthil]

Based on the application in hand: When we want to penalize a lot for deviation, go for mse (more accurate; less generalized). When you don't want to penalize a lot (more generalization), go for mae

[mehulgoyal-yeariddcivilen]

For a feature having outliers the mae is better than the mse .since the mse value is greater for outliers than mae.for gradient descent optimisation we will use mse because it is quadratic equation and have a parabolic curve where the minimum point is our optimal loss and the parameters corresponding to that is our optimal parameters for our ml model which gives us best accuracy among all the set of parameters.
Mae has a curve of straight line and gradient descent technique is not suitable for that because there is no convergence point in that.

[edwinsamuel]

Mean absolute error is generally the best and simplest value of error,
but it fails to punish extreme values.
So when we are in need of punishing extreme values in our dataset, then MSE is used.