官术网_书友最值得收藏!

Maximum likelihood estimation

Logistic regression works on the principle of maximum likelihood estimation; here, we will explain in detail what it is in principle so that we can cover some more fundamentals of logistic regression in the following sections. Maximum likelihood estimation is a method of estimating the parameters of a model given observations, by finding the parameter values that maximize the likelihood of making the observations, this means finding parameters that maximize the probability p of event 1 and (1-p) of non-event 0, as you know:

probability (event + non-event) = 1

Example: Sample (0, 1, 0, 0, 1, 0) is drawn from binomial distribution. What is the maximum likelihood estimate of μ?

Solution: Given the fact that for binomial distribution P(X=1) = μ and P(X=0) = 1- μ where μ is the parameter:

Here, log is applied to both sides of the equation for mathematical convenience; also, maximizing likelihood is the same as the maximizing log of likelihood:

Determining the maximum value of μ by equating derivative to zero:

However, we need to do double differentiation to determine the saddle point obtained from equating derivative to zero is maximum or minimum. If the μ value is maximum; double differentiation of log(L(μ)) should be a negative value:

Even without substitution of μ value in double differentiation, we can determine that it is a negative value, as denominator values are squared and it has a negative sign against both terms. Nonetheless, we are substituting and the value is:

Hence it has been proven that at value μ = 1/3, it is maximizing the likelihood. If we substitute the value in the log likelihood function, we will obtain:

The reason behind calculating -2*ln(L) is to replicate the metric calculated in proper logistic regression. In fact:

AIC = -2*ln(L) + 2*k

So, logistic regression tries to find the parameters by maximizing the likelihood with respect to individual parameters. But one small difference is, in logistic regression, Bernoulli distribution will be utilized rather than binomial. To be precise, Bernoulli is just a special case of the binomial, as the primary outcome is only two categories from which all the trails are made.

主站蜘蛛池模板: 崇礼县| 苍山县| 民勤县| 特克斯县| 锦州市| 卓资县| 罗源县| 秦安县| 德令哈市| 永嘉县| 黔西县| 西城区| 沙坪坝区| 西贡区| 泰兴市| 高平市| 锦州市| 新巴尔虎左旗| 永宁县| 安化县| 烟台市| 嘉黎县| 桑日县| 保山市| 驻马店市| 封开县| 万山特区| 抚州市| 平塘县| 海伦市| 梓潼县| 高碑店市| 娱乐| 扎鲁特旗| 高青县| 栾川县| 辽阳市| 牙克石市| 马鞍山市| 开化县| 莱阳市|