官术网_书友最值得收藏!

Example of simple linear regression from first principles

The entire chapter has been presented with the popular wine quality dataset which is openly available from the UCI machine learning repository at https://archive.ics.uci.edu/ml/datasets/Wine+Quality.

Simple linear regression is a straightforward approach for predicting the dependent/response variable Y given the independent/predictor variable X. It assumes a linear relationship between X and Y:

β0 and β1 are two unknown constants which are intercept and slope parameters respectively. Once we determine the constants, we can utilize them for the prediction of the dependent variable:

Residuals are the differences between the ith observed response value and the ith response value that is predicted from the model. Residual sum of squares is shown. The least squares approach chooses estimates by minimizing errors.

In order to prove statistically that linear regression is significant, we have to perform hypothesis testing. Let's assume we start with the null hypothesis that there is no significant relationship between X and Y:

Since, if β1 = 0, then the model shows no association between both variables (Y = β0 + ε), these are the null hypothesis assumptions; in order to prove this assumption right or wrong, we need to determine β1 is sufficiently far from 0 (statistically significant in distance from 0 to be precise), that we can be confident that β1 is nonzero and have a significant relationship between both variables. Now, the question is, how far is far enough from zero? It depends on the distribution of β1, which is its mean and standard error (similar to standard deviation). In some cases, if the standard error is small, even relatively small values may provide strong evidence that β1 ≠ 0, hence there is a relationship between X and Y. In contrast, if SE(β1) is large, then β1 must be large in absolute value in order for us to reject the null hypothesis. We usually perform the following test to check how many standard deviations β1 is away from the value 0:

With this t value, we calculate the probability of observing any value equal to |t| or larger, assuming β1 = 0; this probability is also known as the p-value. If p-value < 0.05, it signifies that β1 is significantly far from 0, hence we can reject the null hypothesis and agree that there exists a strong relationship, whereas if p-value > 0.05, we accept the null hypothesis and conclude that there is no significant relationship between both variables.

Once we have the coefficient values, we will try to predict the dependent value and check for the R-squared value; if the value is >= 0.7, it means the model is good enough to deploy on unseen data, whereas if it is not such a good value (<0.6), we can conclude that this model is not good enough to deploy.

主站蜘蛛池模板: 息烽县| 苍梧县| 林州市| 中牟县| 辽阳县| 通辽市| 石河子市| 毕节市| 常熟市| 金堂县| 关岭| 万山特区| 大厂| 盘山县| 宜黄县| 田林县| 夹江县| 两当县| 本溪| 天峻县| 论坛| 秦皇岛市| 汝州市| 启东市| 安吉县| 舞钢市| 密山市| 罗甸县| 白河县| 凤冈县| 老河口市| 钟祥市| 安丘市| 唐山市| 峨边| 廉江市| 信丰县| 天气| 永善县| 当涂县| 黑龙江省|