Chapter 2. Neural Networks

In the previous chapter, we described several machine learning algorithms and we introduced different techniques to analyze data to make predictions. For example, we suggested how machines can use data of home selling prices to make predictions on the price for new houses. We described how large companies, such as Netflix, use machine learning techniques in order to suggest to users new movies they may like based on movies they have liked in the past, using a technique that is widely utilized in e-commerce by giants such as Amazon or Walmart. Most of these techniques, however, necessitate labeled data in order to make predictions on new data, and, in order to improve their performance, need humans to describe the data in terms of features that make sense.

Humans are able to quickly extrapolate patterns and infer rules without having the data cleaned and prepared for them. It would then be desirable if machines could learn to do the same. As we have discussed, Frank Rosenblatt invented the perceptron back in 1957, over 50 years ago. The perceptron is to modern deep neural nets what unicellular organisms are to complex multi-cellular lifeforms, and yet it is quite important to understand and become familiar with how an artificial neuron works to better understand and appreciate the complexity we can generate by grouping many neurons together on many different layers to create deep neural networks. Neural nets are an attempt to mimic the functioning of a human brain and its ability to abstract new rules through simple observations. Though we are still quite far from understanding how human brains organize and process information, we already have a good understanding of how single human neurons work. Artificial neural networks attempt to mimic the same functionality, trading chemical and electrical messaging for numerical values and functions. Much progress has been made in the last decade, after neural networks had become popular and then been forgotten at least twice before: such resurgence is due in part to having computers that are getting faster, the use of GPUs (Graphical Processing Units) versus the most traditional use of CPUs (Computing Processing Units), better algorithms and neural nets design, and increasingly large datasets, as we will see in this book.

In this chapter, we will formally introduce what neural networks are, we will thoroughly describe how a neuron works, and we will see how we can stack many layers to create and use deep feed-forward neural networks.