官术网_书友最值得收藏!

Understanding the scenario of overfitting

In some of the previous recipes, we have noticed that the training accuracy is ~100%, while test accuracy is ~98%, which is a case of overfitting on top of a training dataset. Let's gain an intuition of the delta between the training and the test accuracies.

To understand the phenomenon resulting in overfitting, let's contrast two scenarios where we compare the training and test accuracies along with a histogram of the weights:

  • Model is run for five epochs
  • Model is run for 100 epochs

The comparison-of-accuracy metric between training and test datasets between the two scenarios is as follows:

Once we plot the histogram of weights that are connecting the hidden layer to the output layer, we will notice that the 100-epochs scenario has a higher spread of weights when compared to the five-epochs scenario:

From the preceding pictures, you should note that the 100 epochs scenario had a higher dispersion of weight values when compared to the five-epochs scenario. This is because of the higher amount of opportunity that the model had to overfit on top of the training dataset when the model is run for 100-epochs, when compared to when the model is run for five epochs, as the number of weight updates in the 100-epochs scenario is higher than the number of weight updates in the five-epochs scenario.

A high value of weight (along with a difference in the training and test dataset) is a good indication of a potential over-fitting of the model and/or a potential opportunity to scale input/weights to increase the accuracy of the model.

Additionally, also note that a neural network can have hundreds of thousands of weights (and millions in certain architectures) that need to be adjusted, and thus, there is always a chance that one or the other weight can get updated to a very high number to fine-tune for one outlier row of the dataset.

主站蜘蛛池模板: 鄱阳县| 龙井市| 铜山县| 福建省| 玉溪市| 永州市| 休宁县| 洛南县| 仲巴县| 弥渡县| 应城市| 嫩江县| 高台县| 南京市| 台东市| 攀枝花市| 辽源市| 灵寿县| 扬州市| 灵川县| 民乐县| 大田县| 眉山市| 东乡族自治县| 茌平县| 临沧市| 阿图什市| 广灵县| 广水市| 德令哈市| 林周县| 镇雄县| 青阳县| 石城县| 乃东县| 常宁市| 北流市| 东宁县| 娱乐| 平安县| 韶关市|