Learn Bias-Variance Trade-off
A Story of Applying Machine Learning Algorithms
First we'll understand both bias and variance individually and then we will jump into trade-off of bias-variance which most essential step while applying any machine learning algorithm.
When we choose any algorithm depending upon our initial data analysis or more specifically exploratory data analysis which we did on our data. At that time, with maintaining or having little variability our algorithm hit the target feature. In simple words, imagine you throwing a paper ball in dustbin from a certain distance, with every next throw we maintain the variability and change the trajectory of throw. After number of attempts the accuracy to throw that paper ball in dustbin increases.
With this little weird story of paper ball and you, now we'll look at the technical ascpect of the topic.
Bias and Variance are the measurement metrics which measure the difference between the prediction given by the model for the target feature and actual outcome.
Bias is an error occurred in the model due to simplicity learnt by model from data which is also termed as oversimplification. Generally, it underfits on the data (underfitting). That is target function predicts simplified outcomes which can be understood easily but anyways it doesn’t results good on the test data.
Variance is an error occurred in the model due to complexity of algorithm, target function learns much noise from training data and never performs good on test data. Which is termed as overfitting.
At this point of time, we are aware of what is bias and what is variance individually. And here comes the very important concept of this blog: Bias-Variance Trade-off.
Understand it with bullet points:
1. Keeping bias and variance in mind, as the complexity of the model increases. We can note the moderation in error which is smaller than earlier.
2. This can happen up until a certain point and then, as the complexity of model increases more than that certain point, it results in overfitting on the data.
3. Where with low bias, we suffer from high variance.
4. Many of the supervised algorithms are supposed to have low bias and low variance to attain the best prediction level but, unfortunately it is not applicable to all.
5. Point to remember is, we cannot escape from this indirect relationship of bias and variance. Increasing the bias will decrease the variance. Increasing the variance will decrease bias.
Thank you for reading!
More content at plainenglish.io
