官术网_书友最值得收藏!

Overcoming overfitting using dropout

In the previous section of overcoming overfitting using regularization, we used L1/ L2 regularization as a means to avoid overfitting. In this section, we will use another tool that is helpful to achieve the same—dropout.

Dropout can be considered a way in which only a certain percentage of weights get updated, while the others do not get updated in a given iteration of weight updates. This way, we are in a position where not all weights get updated in a weight update process, thus avoiding certain weights to achieve a very high magnitude when compared to others:

model = Sequential()
model.add(Dense(1000, input_dim=784, activation='relu'))
model.add(Dropout(0.75))
model.add(Dense(10, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy'])
history = model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=100, batch_size=1024, verbose=1)

In the preceding code, we have given a dropout of 0.75; that is, randomly, 75% of weights do not get updated in a certain weight update iteration.

The preceding would result in the gap between the training and test accuracy being not as high as it is when the model was built without dropout in the previous scenario, where the spread of weights was higher.

Note the histogram of weights of the first layer now:

plt.hist(model.get_weights()[-2].flatten())

Note that in the preceding scenario, the frequency count of weights that are beyond 0.2 or -0.2 is less when compared to the 100-epochs scenario.

主站蜘蛛池模板: 大安市| 彭阳县| 昆山市| 普兰店市| 蒙自县| 鹤岗市| 五家渠市| 诸城市| 景泰县| 信宜市| 昌吉市| 密山市| 金寨县| 兴国县| 永福县| 宁武县| 偏关县| 岚皋县| 平远县| 越西县| 大埔县| 浦北县| 郸城县| 齐齐哈尔市| 广昌县| 苏尼特右旗| 乌拉特中旗| 岳普湖县| 盐边县| 棋牌| 彭泽县| 镇雄县| 海安县| 西乌| 大兴区| 蒙阴县| 九龙县| 安阳市| 玉山县| 香港 | 金坛市|