官术网_书友最值得收藏!

Random forest

The random forest (RF) is a very powerful technique which is used frequently in the data science field for solving various problems across industries, as well as a silver bullet for winning competitions like Kaggle. We will cover various concepts from the basics to in depth in the next chapter; here we are restricted to the bare necessities. Random forest is an ensemble of decision trees, as we know, logistic regression has very high bias and low variance technique; on the other hand, decision trees have high variance and low bias, which makes decision trees unstable. By averaging decision trees, we will minimize the variance component the of model, which makes approximate nearest to an ideal model.

RF focuses on sampling both observations and variables of training data to develop independent decision trees and take majority voting for classification and averaging for regression problems respectively. In contrast, bagging samples only observations at random and selects all columns that have the deficiency of representing significant variables at root for all decision trees. This way makes trees that are dependent on each other, for which accuracy will be penalized.

The following are a few rules of thumb when selecting sub-samples from observations using random forest. Nonetheless, any of the parameters can be tuned to improve results further! Each tree is developed on sampled data drawn from training data and fitted as shown

About 2/3 of observations in training data for each individual tree

Select columns sqrt(p) -> For classification problem if p is total columns in training data

p/3 -> for regression problem if p is number of columns

In the following diagram, two samples were shown with blue and pink colors, where, in the bagging scenario, a few observations and all columns are selected. Whereas, in random forest, a few observations and columns are selected to create uncorrelated individual trees.

In the following diagram, a sample idea shows how RF classifier works. Each tree has grown separately, and the depth of each tree varies as per the selected sample, but in the end, voting is performed to determine the final class.

Due to the ensemble of decision trees, RF suffered interpretability and could not determine the significance of each variable; only variable importance could be provided instead. In the following graph, a sample of variable performance has been provided, consisting of a mean decrease in Gini:

主站蜘蛛池模板: 武冈市| 伊川县| 金平| 砀山县| 兴化市| 辉县市| 南和县| 泽普县| 九龙城区| 剑川县| 大冶市| 平利县| 平顺县| 台东县| 呈贡县| 土默特左旗| 罗源县| 武威市| 黄山市| 乌苏市| 西峡县| 河西区| 商城县| 井陉县| 巴东县| 衡东县| 高碑店市| 汤阴县| 阿克| 疏勒县| 新绛县| 昌宁县| 长岛县| 阆中市| 甘谷县| 永城市| 文登市| 友谊县| 商都县| 托里县| 临高县|