Chapter 2. Classifying Handwritten Digits with a Feedforward Network

The first chapter presented Theano as a compute engine, with its different functions and specificities. With this knowledge, we'll go through an example and introduce some of the main concepts of deep learning, building three neural networks and training them on the problem of handwritten digit classification.

Deep learning is a field of machine learning in which layers of modules are stacked on top of each of other: this chapter introduces a simple single-linear-layer model, then adds a second layer on top of it to create a multi-layer perceptron (MLP), and last uses multiple convolutional layers to create a Convolutional Neural Network (CNN).

In the meantime, this chapter recaps the basic machine learning concepts, such as overfitting, validation, and loss analysis, for those who are not familiar with data science:

• Small image classification
• Handwritten digit recognition challenge
• Layer design to build a neural network
• Design of a classical objective/loss function
• Back-propagation with stochastic gradient descent
• Training on a dataset with validation
• Convolutional neural networks
• Towards state-of-art results for digit classification