官术网_书友最值得收藏!

Understanding the scenario of overfitting

In some of the previous recipes, we have noticed that the training accuracy is ~100%, while test accuracy is ~98%, which is a case of overfitting on top of a training dataset. Let's gain an intuition of the delta between the training and the test accuracies.

To understand the phenomenon resulting in overfitting, let's contrast two scenarios where we compare the training and test accuracies along with a histogram of the weights:

  • Model is run for five epochs
  • Model is run for 100 epochs

The comparison-of-accuracy metric between training and test datasets between the two scenarios is as follows:

Once we plot the histogram of weights that are connecting the hidden layer to the output layer, we will notice that the 100-epochs scenario has a higher spread of weights when compared to the five-epochs scenario:

From the preceding pictures, you should note that the 100 epochs scenario had a higher dispersion of weight values when compared to the five-epochs scenario. This is because of the higher amount of opportunity that the model had to overfit on top of the training dataset when the model is run for 100-epochs, when compared to when the model is run for five epochs, as the number of weight updates in the 100-epochs scenario is higher than the number of weight updates in the five-epochs scenario.

A high value of weight (along with a difference in the training and test dataset) is a good indication of a potential over-fitting of the model and/or a potential opportunity to scale input/weights to increase the accuracy of the model.

Additionally, also note that a neural network can have hundreds of thousands of weights (and millions in certain architectures) that need to be adjusted, and thus, there is always a chance that one or the other weight can get updated to a very high number to fine-tune for one outlier row of the dataset.

主站蜘蛛池模板: 景宁| 黑龙江省| 宜川县| 万源市| 武义县| 冀州市| 枝江市| 成安县| 锦屏县| 宝清县| 姚安县| 郧西县| 宜宾县| 昆明市| 武宣县| 清水河县| 赤城县| 章丘市| 郓城县| 唐河县| 宜宾县| 福建省| 张北县| 莱州市| 武邑县| 永兴县| 确山县| 乌审旗| 嵩明县| 太仆寺旗| 阳城县| 阜康市| 房产| 平和县| 夏河县| 江油市| 惠东县| 腾冲县| 合水县| 西昌市| 罗山县|