Sigmoid activation

The output range of this function is from zero to one for all real number inputs. This is very important for generating probabilistic scores from neurons. The function is also continuous and non-linear and helps to preserve the non-linearity of outputs. Also, the gradient of curve is steep near the origin and saturates as we start moving away on the x-axis. This means significant change in output will occur for a small change in input around the origin. This characteristic aids in the classification task as it tries to keep the output close to either zero or one. Following is the equation for sigmoid activation against the input x:

The following is a plot of the sigmoid activation function: