- Statistics for Machine Learning
- Pratap Dangeti
- 304字
- 2021-07-02 19:05:55
Bias versus variance trade-off
Every model has both bias and variance error components in addition to white noise. Bias and variance are inversely related to each other; while trying to reduce one component, the other component of the model will increase. The true art lies in creating a good fit by balancing both. The ideal model will have both low bias and low variance.
Errors from the bias component come from erroneous assumptions in the underlying learning algorithm. High bias can cause an algorithm to miss the relevant relations between features and target outputs; this phenomenon causes an underfitting problem.
On the other hand, errors from the variance component come from sensitivity to change in the fit of the model, even a small change in training data; high variance can cause an overfitting problem:

An example of a high bias model is logistic or linear regression, in which the fit of the model is merely a straight line and may have a high error component due to the fact that a linear model could not approximate underlying data well.
An example of a high variance model is a decision tree, in which the model may create too much wiggly curve as a fit, in which even a small change in training data will cause a drastic change in the fit of the curve.
At the moment, state-of-the-art models are utilizing high variance models such as decision trees and performing ensemble on top of them to reduce the errors caused by high variance and at the same time not compromising on increases in errors due to the bias component. The best example of this category is random forest, in which many decision trees will be grown independently and ensemble in order to come up with the best fit; we will cover this in upcoming chapters:

- Building Modern Web Applications Using Angular
- WebAssembly實戰
- Python計算機視覺編程
- jQuery從入門到精通 (軟件開發視頻大講堂)
- Python Network Programming Cookbook(Second Edition)
- 零基礎學Python數據分析(升級版)
- Python面向對象編程:構建游戲和GUI
- SQL Server 2016數據庫應用與開發
- Raspberry Pi Home Automation with Arduino(Second Edition)
- PySide 6/PyQt 6快速開發與實戰
- 計算機應用基礎實踐教程
- Learning Apache Cassandra
- 搞定J2EE:Struts+Spring+Hibernate整合詳解與典型案例
- Fast Data Processing with Spark(Second Edition)
- Neo4j 3.x入門經典