Entropy calculation

The frequency table for the combination Outlook-Train outside is as follows:

 

Using these values, we get the entropy of two variables, as shown here in detail:

p(Sunny).S(Sunny)+p(Overcast).S(Overcast)+p(Rainy)*S(Rainy)=

5/14*(-3/5*log2(3/5)-2/5*log2(2/5)) +

4/14*(-4/4*log2(4/4)-0/4*log2(0/4))+

5/14*(-2/5*log2(2/5)-3/5*log2(3/5))=

0.693

Here, p(Sunny) = (#Yes+#No)/Total entries = (2+3)/14, p(Overcast) = (#Yes+#No)/Total entries = (4+0)/14, and p(Rainy) = (#Yes+#No)/Total entries = (2+3)/14. The entropy values S(v) are calculated using the corresponding probabilities, that is, #Yes or #No over the total #Yes+#No.

The frequency table for the combination Temperature-Train outside is as follows:

 

Using these values and an analogous calculation, the entropy is shown in detail here:

p(Hot).S(Hot)+p(Mild).S(Mild)+p(Cool)*S(Cool)=

4/14*(-2/4*log₂(2/4)-2/4*log₂(2/4)) +

6/14*(-4/6*log₂(4/6)-2/6*log₂(2/6))+

4/14*(-3/4*log₂(3/4)-1/4*log₂(1/4)) =

0,911

The frequency table for the combination Humidity-Train outside is as follows:

Using these values, we get the entropy as follows:

p(High).S(High)+p(Normal).S(Normal)=

7/14*(-3/7*log₂(3/7)-4/7*log₂(4/7)) +

7/14*(-6/7*log₂(6/7)-1/7*log₂(1/7))=

0,788

The frequency table for the combination Windy-Train outside is as follows:

 

Using these values, we get the entropy as follows:

p(True).S(True)+p(False).S(False)=

8/14*(-6/8*log₂(6/8)-2/8*log₂(2/8)) +

6/14*(-3/6*log₂(3/6)-3/6*log₂(3/6))

=0,892