官术网_书友最值得收藏!

Overcoming overfitting using dropout

In the previous section of overcoming overfitting using regularization, we used L1/ L2 regularization as a means to avoid overfitting. In this section, we will use another tool that is helpful to achieve the same—dropout.

Dropout can be considered a way in which only a certain percentage of weights get updated, while the others do not get updated in a given iteration of weight updates. This way, we are in a position where not all weights get updated in a weight update process, thus avoiding certain weights to achieve a very high magnitude when compared to others:

model = Sequential()
model.add(Dense(1000, input_dim=784, activation='relu'))
model.add(Dropout(0.75))
model.add(Dense(10, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy'])
history = model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=100, batch_size=1024, verbose=1)

In the preceding code, we have given a dropout of 0.75; that is, randomly, 75% of weights do not get updated in a certain weight update iteration.

The preceding would result in the gap between the training and test accuracy being not as high as it is when the model was built without dropout in the previous scenario, where the spread of weights was higher.

Note the histogram of weights of the first layer now:

plt.hist(model.get_weights()[-2].flatten())

Note that in the preceding scenario, the frequency count of weights that are beyond 0.2 or -0.2 is less when compared to the 100-epochs scenario.

主站蜘蛛池模板: 隆子县| 二连浩特市| 翁牛特旗| 罗山县| 普兰县| 白银市| 密云县| 城口县| 呼玛县| 随州市| 通化市| 巴中市| 黄平县| 东宁县| 余姚市| 岚皋县| 缙云县| 梧州市| 青海省| 西乌| 连云港市| 禄丰县| 十堰市| 固原市| 拜城县| 德钦县| 佛冈县| 武陟县| 芷江| 延长县| 肥东县| 汝城县| 罗城| 黔江区| 乳山市| 衡东县| 武功县| 新密市| 沙田区| 喀喇| 文化|