官术网_书友最值得收藏!

Random forest

The random forest (RF) is a very powerful technique which is used frequently in the data science field for solving various problems across industries, as well as a silver bullet for winning competitions like Kaggle. We will cover various concepts from the basics to in depth in the next chapter; here we are restricted to the bare necessities. Random forest is an ensemble of decision trees, as we know, logistic regression has very high bias and low variance technique; on the other hand, decision trees have high variance and low bias, which makes decision trees unstable. By averaging decision trees, we will minimize the variance component the of model, which makes approximate nearest to an ideal model.

RF focuses on sampling both observations and variables of training data to develop independent decision trees and take majority voting for classification and averaging for regression problems respectively. In contrast, bagging samples only observations at random and selects all columns that have the deficiency of representing significant variables at root for all decision trees. This way makes trees that are dependent on each other, for which accuracy will be penalized.

The following are a few rules of thumb when selecting sub-samples from observations using random forest. Nonetheless, any of the parameters can be tuned to improve results further! Each tree is developed on sampled data drawn from training data and fitted as shown

About 2/3 of observations in training data for each individual tree

Select columns sqrt(p) -> For classification problem if p is total columns in training data

p/3 -> for regression problem if p is number of columns

In the following diagram, two samples were shown with blue and pink colors, where, in the bagging scenario, a few observations and all columns are selected. Whereas, in random forest, a few observations and columns are selected to create uncorrelated individual trees.

In the following diagram, a sample idea shows how RF classifier works. Each tree has grown separately, and the depth of each tree varies as per the selected sample, but in the end, voting is performed to determine the final class.

Due to the ensemble of decision trees, RF suffered interpretability and could not determine the significance of each variable; only variable importance could be provided instead. In the following graph, a sample of variable performance has been provided, consisting of a mean decrease in Gini:

主站蜘蛛池模板: 龙口市| 永定县| 深圳市| 临汾市| 虹口区| 从化市| 广水市| 滨海县| 唐海县| 青浦区| 广灵县| 工布江达县| 湖北省| 涿鹿县| 宁河县| 阆中市| 荔波县| 平和县| 普格县| 西安市| 宜都市| 五华县| 平乐县| 渭源县| 怀安县| 林西县| 社旗县| 信丰县| 克什克腾旗| 永丰县| 南开区| 广东省| 武汉市| 涡阳县| 民和| 望谟县| 和龙市| 宁明县| 巩留县| 玉屏| 九江市|