官术网_书友最值得收藏!

Overcoming overfitting using dropout

In the previous section of overcoming overfitting using regularization, we used L1/ L2 regularization as a means to avoid overfitting. In this section, we will use another tool that is helpful to achieve the same—dropout.

Dropout can be considered a way in which only a certain percentage of weights get updated, while the others do not get updated in a given iteration of weight updates. This way, we are in a position where not all weights get updated in a weight update process, thus avoiding certain weights to achieve a very high magnitude when compared to others:

model = Sequential()
model.add(Dense(1000, input_dim=784, activation='relu'))
model.add(Dropout(0.75))
model.add(Dense(10, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy'])
history = model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=100, batch_size=1024, verbose=1)

In the preceding code, we have given a dropout of 0.75; that is, randomly, 75% of weights do not get updated in a certain weight update iteration.

The preceding would result in the gap between the training and test accuracy being not as high as it is when the model was built without dropout in the previous scenario, where the spread of weights was higher.

Note the histogram of weights of the first layer now:

plt.hist(model.get_weights()[-2].flatten())

Note that in the preceding scenario, the frequency count of weights that are beyond 0.2 or -0.2 is less when compared to the 100-epochs scenario.

主站蜘蛛池模板: 太保市| 阳山县| 阜康市| 金华市| 梁平县| 古丈县| 来宾市| 休宁县| 桐乡市| 新丰县| 肥西县| 平阴县| 永宁县| 南岸区| 双鸭山市| 连城县| 班玛县| 洛浦县| 清流县| 梅州市| 新巴尔虎左旗| 册亨县| 连云港市| 汉源县| 建昌县| 南雄市| 永仁县| 宿迁市| 石阡县| 弥渡县| 宁明县| 顺昌县| 安庆市| 平邑县| 古丈县| 宁海县| 三明市| 微山县| 孟州市| 泰宁县| 杨浦区|