- Applied Deep Learning and Computer Vision for Self/Driving Cars
- Sumit Ranjan;Dr. S. Senthamilarasu
- 461字
- 2021-04-09 23:13:03
Understanding activation functions
Activation functions are so important to neural networks as they introduce non-linearity to a network. Deep learning consists of multiple non-linear transformations, and activation functions are the tools for non-linear transformation. Hence, activation functions are applied before sending an input signal to the next layer of neural networks. Due to activation functions, a neural network has the power to learn complex features.
Deep learning has many activation functions:
The threshold function
The sigmoid function
The rectifier function
The hyperbolic tangent function
The cost function
In the next section, we will start with one of the most important activation functions, called the threshold activation function.
The threshold function
The threshold function can be seen in the following diagram:
On the x axis, we have the weighted sum of the input, and on the y axis, we have the threshold values from 0 to 1. The threshold function is very simple: if the value is less than 0, then the threshold will be 0 and if the value is more than 0, then the threshold will be 1. This works as a yes-or-no function.
The sigmoid function
The sigmoid function is a very interesting type of function; we can see it in the following diagram:
The sigmoid function is nothing but a logistic function. In this function, anything below 0 will be set to 0. This function is often used in the output layer, especially when you're trying to find the predictive probability.
The rectifier linear function
The Rectifier Linear (ReLU) function is one of the most popular functions in the field of ANNs. If the value is less than or equal to 0, then the value of x is set to 0, and then from there, it gradually progresses as the input value increases. We can observe this in the following diagram:
In the next section, we will learn about the hyperbolic tangent activation function.
The hyperbolic tangent activation function
Finally, we have another function, called the Hyperbolic Tangent Activation (tanh) function, which looks as follows:
The tanh function is very similar to the sigmoid function; the range of a tanh function is (-1,1). Tanh functions are also S-shaped, like sigmoid functions. The advantage of the tanh function is that a positive will be mapped as strongly positive, a negative will be mapped as strongly negative, and 0 will be mapped to 0, as shown in Fig 2.16.
In the next section of this chapter, we will learn about the cost function.
- 中文版CorelDRAW 2022基礎(chǔ)教程
- Flash CC動畫制作案例教程
- MATLAB 2015從入門到精通
- Drupal: Creating Blogs, Forums, Portals, and Community Websites
- Flash基礎(chǔ)與實戰(zhàn)教程
- Photoshop CS6實戰(zhàn)從入門到精通(超值版)
- 量化投資與FOF投資:以MATLAB+Python為工具
- PowerPoint 2016實戰(zhàn)從入門到精通(超值版)
- Photoshop CC移動UI設(shè)計實用教程
- UI功夫:PC和APP界面設(shè)計全流程圖解
- Microsoft Azure: Enterprise Application Development
- Configuring IPCop Firewalls: Closing Borders with Open Source
- AutoCAD 2019中文版實戰(zhàn)從入門到精通
- Maya Paint Effect 特效應(yīng)用手冊
- Photoshop CC 2015中文版實用教程(第7版)