官术网_书友最值得收藏!

The cost function of neural networks

We will now explore how can we evaluate the performance of a neural network by using the cost function. We will use it to measure how far we are from the expected value. We are going to use the following notation and variables:

Variable Y to represent the true value

Variable a to represent the neuron prediction

In terms of weight and biases, the formula is as follows:

We pass z, which is the input (X) times the weight (X) added to the bias (b), into the activation function of .

There are many types of cost functions, but we are just going to discuss two of them:

The quadratic cost function

The cross-entropy function

The first cost function we are going to discuss is the quadratic cost function, which is represented with the following formula:

In the preceding formula, we can see that when the error is high, which means the actual value (Y) is less than the predictive value (a), then the value of the cost function will be negative and we cannot use a negative value as cost. So, we are going to square the result, then the value of the cost function will be a positive value. But unfortunately, when we use the quadratic cost function, the way the formula works actually reduces the learning rate of the network.

Instead, we are going to use the cross-entropy function, which can be defined as follows:

This cost function allows faster learning because the larger the difference between y and a, the faster our neurons' learning rate. This means that if the network has a large difference between the predicted value and the actual value athe beginning of the model training process, then we can essentially move toward using a cost function because the larger the difference, the faster the neurons are going to learn.

There are two key components of how neural networks learn from features. First, there are neurons and their activation functions and cost functions, but we are still missing a key step: the actual learning process. So, we need to figure out how we can use the neurons and their measurement of error (the cost function) to correct our prediction or make the network learn. Up until now, we have tried to understand neurons and perceptrons, and then linked them to get a neural net. We also understand that cost functions are essentially measurements of errors. Now, we are going to fix the errors between the actual and predicted values using gradient descent and backpropagation.

主站蜘蛛池模板: 东至县| 桦南县| 谢通门县| 孟州市| 绿春县| 高青县| 长兴县| 黎平县| 天峨县| 米脂县| 新乡市| 巨鹿县| 洪泽县| 攀枝花市| 昌黎县| 鄂托克前旗| 内黄县| 武隆县| 清远市| 桃江县| 宿州市| 石首市| 东乌珠穆沁旗| 棋牌| 讷河市| 南城县| 友谊县| 山丹县| 黄冈市| 汶川县| 孟津县| 上林县| 长丰县| 丰都县| 贞丰县| 万全县| 新竹县| 郴州市| 阆中市| 乡城县| 乐安县|