官术网_书友最值得收藏!

Example of simple linear regression from first principles

The entire chapter has been presented with the popular wine quality dataset which is openly available from the UCI machine learning repository at https://archive.ics.uci.edu/ml/datasets/Wine+Quality.

Simple linear regression is a straightforward approach for predicting the dependent/response variable Y given the independent/predictor variable X. It assumes a linear relationship between X and Y:

β0 and β1 are two unknown constants which are intercept and slope parameters respectively. Once we determine the constants, we can utilize them for the prediction of the dependent variable:

Residuals are the differences between the ith observed response value and the ith response value that is predicted from the model. Residual sum of squares is shown. The least squares approach chooses estimates by minimizing errors.

In order to prove statistically that linear regression is significant, we have to perform hypothesis testing. Let's assume we start with the null hypothesis that there is no significant relationship between X and Y:

Since, if β1 = 0, then the model shows no association between both variables (Y = β0 + ε), these are the null hypothesis assumptions; in order to prove this assumption right or wrong, we need to determine β1 is sufficiently far from 0 (statistically significant in distance from 0 to be precise), that we can be confident that β1 is nonzero and have a significant relationship between both variables. Now, the question is, how far is far enough from zero? It depends on the distribution of β1, which is its mean and standard error (similar to standard deviation). In some cases, if the standard error is small, even relatively small values may provide strong evidence that β1 ≠ 0, hence there is a relationship between X and Y. In contrast, if SE(β1) is large, then β1 must be large in absolute value in order for us to reject the null hypothesis. We usually perform the following test to check how many standard deviations β1 is away from the value 0:

With this t value, we calculate the probability of observing any value equal to |t| or larger, assuming β1 = 0; this probability is also known as the p-value. If p-value < 0.05, it signifies that β1 is significantly far from 0, hence we can reject the null hypothesis and agree that there exists a strong relationship, whereas if p-value > 0.05, we accept the null hypothesis and conclude that there is no significant relationship between both variables.

Once we have the coefficient values, we will try to predict the dependent value and check for the R-squared value; if the value is >= 0.7, it means the model is good enough to deploy on unseen data, whereas if it is not such a good value (<0.6), we can conclude that this model is not good enough to deploy.

主站蜘蛛池模板: 夏邑县| 武陟县| 曲靖市| 桐乡市| 扎兰屯市| 克东县| 昌平区| 治县。| 津市市| 壶关县| 惠来县| 兰坪| 监利县| 张家港市| 峨眉山市| 巨野县| 绵竹市| 阳泉市| 滨州市| 兴宁市| 汽车| 北票市| 惠东县| 梅州市| 黔江区| 从江县| 茂名市| 儋州市| 武汉市| 禹城市| 会理县| 梨树县| 岗巴县| 大渡口区| 陆河县| 南昌县| 沈阳市| 沁源县| 三台县| 望奎县| 富川|