Deep Learning Fundamentals

In this chapter, we will introduce deep learning(DL) and deep neural networks (DNNs), that is, neural networks with multiple hidden layers. You may wonder what the point of using more than one hidden layer is, given the universal approximation theorem. This is in no way a naive question, and for a long time neural networks were used in that way. Without going into too much detail, one reason is that approximating a complex function might require a huge number of neurons in the hidden layer, making it impractical to use. There is also another, more important, reason for using deep networks, which is not directly related to the number of hidden layers, but to the level of learning. A deep network does not simply learn to predict output Y given input X; it also understands basic features of the input. It's able to learn abstractions of features of input examples, to understand the basic characteristics of the examples, and to make predictions based on those characteristics. This is a level of abstraction that is missing in other basic machine learning(ML) algorithms and in shallow neural networks.

In this chapter, we will cover the following topics:

• Introduction to deep learning
• Fundamental deep learning concepts 
• Deep learning algorithms
• Applications of deep learning
• The reasons for deep learning's popularity
• Introducing popular open source libraries