- Applied Deep Learning and Computer Vision for Self/Driving Cars
- Sumit Ranjan;Dr. S. Senthamilarasu
- 445字
- 2021-04-09 23:13:03
The cost function of neural networks
We will now explore how can we evaluate the performance of a neural network by using the cost function. We will use it to measure how far we are from the expected value. We are going to use the following notation and variables:
Variable Y to represent the true value
Variable a to represent the neuron prediction
In terms of weight and biases, the formula is as follows:
We pass z, which is the input (X) times the weight (X) added to the bias (b), into the activation function of .
There are many types of cost functions, but we are just going to discuss two of them:
The quadratic cost function
The cross-entropy function
The first cost function we are going to discuss is the quadratic cost function, which is represented with the following formula:
In the preceding formula, we can see that when the error is high, which means the actual value (Y) is less than the predictive value (a), then the value of the cost function will be negative and we cannot use a negative value as cost. So, we are going to square the result, then the value of the cost function will be a positive value. But unfortunately, when we use the quadratic cost function, the way the formula works actually reduces the learning rate of the network.
Instead, we are going to use the cross-entropy function, which can be defined as follows:
This cost function allows faster learning because the larger the difference between y and a, the faster our neurons' learning rate. This means that if the network has a large difference between the predicted value and the actual value at the beginning of the model training process, then we can essentially move toward using a cost function because the larger the difference, the faster the neurons are going to learn.
There are two key components of how neural networks learn from features. First, there are neurons and their activation functions and cost functions, but we are still missing a key step: the actual learning process. So, we need to figure out how we can use the neurons and their measurement of error (the cost function) to correct our prediction or make the network learn. Up until now, we have tried to understand neurons and perceptrons, and then linked them to get a neural net. We also understand that cost functions are essentially measurements of errors. Now, we are going to fix the errors between the actual and predicted values using gradient descent and backpropagation.
- After Effects CC影視后期制作實戰從入門到精通
- DWR Java AJAX Applications
- Python數據分析實戰:從Excel輕松入門Pandas
- ABAQUS基礎入門與案例精通
- SolidWorks2014基礎實例教程
- Adobe創意大學After Effects產品專家認證標準教材(CS6修訂版)
- CryENGINE 3 Cookbook
- Adobe創意大學Photoshop產品專家認證標準教材(CS6修訂版)
- OpenCart 1.4 Template Design Cookbook
- Microsoft Azure: Enterprise Application Development
- Choosing an Open Source CMS: Beginner's Guide
- Premiere Pro基礎與實戰教程
- Configuring IPCop Firewalls: Closing Borders with Open Source
- Photoshop CC設計與應用任務教程
- Creo快速入門教程(Creo 8.0中文版)