官术网_书友最值得收藏!

How to do it

L1/L2 regularization is implemented in Keras, as follows:

model = Sequential()
model.add(Dense(1000,input_dim=784,activation='relu',kernel_regularizer=l2(0.1)))model.add(Dense(10, activation='softmax',kernel_regularizer=l2(0.1)))
model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy'])
history = model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=500, batch_size=1024, verbose=1)

Note that the preceding involves invoking an additional hyperparameter—kernel_regularizerand then specifying whether it is an L1/L2 regularization. Furthermore, we also specify the lambda value that gives the weight to regularization.

We notice that, post regularization, the training dataset accuracy does not happen to be at ~100%, while the test data accuracy is at 98%. The histogram of weights post-L2 regularization is visualized in the next graph.

The weights of connecting the hidden layer to the output layer are extracted as follows:

model.get_weights()[0].flatten()

Once the weights are extracted, they are plotted as follows:

plt.hist(model.get_weights()[0].flatten())

We notice that the majority of weights are now much closer to zero when compared to the previous scenario, thus presenting a case to avoid the overfitting issue. We would see a similar trend in the case of L1 regularization.

Notice that the weight values when regularization exists are much lower when compared to the weight values when regularization is performed.

Thus, the L1 and L2 regularizations help us to avoid the overfitting issue on top of the training dataset.

主站蜘蛛池模板: 湘阴县| 古丈县| 衡水市| 洞头县| 象山县| 温州市| 增城市| 亚东县| 贵州省| 梅州市| 栾川县| 开鲁县| 夏河县| 卢氏县| 澎湖县| 新安县| 龙里县| 高邑县| 舞钢市| 双辽市| 香河县| 弥渡县| 长泰县| 巴林右旗| 宜宾县| 济源市| 皋兰县| 长岭县| 搜索| 虞城县| 银川市| 湘潭市| 高密市| 板桥市| 甘洛县| 墨脱县| 都兰县| 亳州市| 卓资县| 丹棱县| 莱芜市|