官术网_书友最值得收藏!

Overcoming overfitting using dropout

In the previous section of overcoming overfitting using regularization, we used L1/ L2 regularization as a means to avoid overfitting. In this section, we will use another tool that is helpful to achieve the same—dropout.

Dropout can be considered a way in which only a certain percentage of weights get updated, while the others do not get updated in a given iteration of weight updates. This way, we are in a position where not all weights get updated in a weight update process, thus avoiding certain weights to achieve a very high magnitude when compared to others:

model = Sequential()
model.add(Dense(1000, input_dim=784, activation='relu'))
model.add(Dropout(0.75))
model.add(Dense(10, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy'])
history = model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=100, batch_size=1024, verbose=1)

In the preceding code, we have given a dropout of 0.75; that is, randomly, 75% of weights do not get updated in a certain weight update iteration.

The preceding would result in the gap between the training and test accuracy being not as high as it is when the model was built without dropout in the previous scenario, where the spread of weights was higher.

Note the histogram of weights of the first layer now:

plt.hist(model.get_weights()[-2].flatten())

Note that in the preceding scenario, the frequency count of weights that are beyond 0.2 or -0.2 is less when compared to the 100-epochs scenario.

主站蜘蛛池模板: 靖宇县| 临颍县| 泗阳县| 会泽县| 望江县| 房产| 梧州市| 静安区| 上犹县| 宽城| 团风县| 镇沅| 宁陵县| 县级市| 神木县| 唐山市| 麻阳| 襄垣县| 大安市| 塘沽区| 木兰县| 灵宝市| 延津县| 天台县| 铜鼓县| 长兴县| 呼玛县| 清苑县| 保德县| 石林| 丹凤县| 华蓥市| 罗江县| 龙泉市| 开原市| 久治县| 古丈县| 白山市| 高淳县| 海晏县| 平阳县|