官术网_书友最值得收藏!

How to do it

L1/L2 regularization is implemented in Keras, as follows:

model = Sequential()
model.add(Dense(1000,input_dim=784,activation='relu',kernel_regularizer=l2(0.1)))model.add(Dense(10, activation='softmax',kernel_regularizer=l2(0.1)))
model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy'])
history = model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=500, batch_size=1024, verbose=1)

Note that the preceding involves invoking an additional hyperparameter—kernel_regularizerand then specifying whether it is an L1/L2 regularization. Furthermore, we also specify the lambda value that gives the weight to regularization.

We notice that, post regularization, the training dataset accuracy does not happen to be at ~100%, while the test data accuracy is at 98%. The histogram of weights post-L2 regularization is visualized in the next graph.

The weights of connecting the hidden layer to the output layer are extracted as follows:

model.get_weights()[0].flatten()

Once the weights are extracted, they are plotted as follows:

plt.hist(model.get_weights()[0].flatten())

We notice that the majority of weights are now much closer to zero when compared to the previous scenario, thus presenting a case to avoid the overfitting issue. We would see a similar trend in the case of L1 regularization.

Notice that the weight values when regularization exists are much lower when compared to the weight values when regularization is performed.

Thus, the L1 and L2 regularizations help us to avoid the overfitting issue on top of the training dataset.

主站蜘蛛池模板: 大城县| 共和县| 松潘县| 易门县| 建昌县| 忻城县| 阜城县| 宜黄县| 皋兰县| 阜南县| 新疆| 茂名市| 乃东县| 铁力市| 镇坪县| 墨脱县| 调兵山市| 天全县| 大方县| 应城市| 阿克| 慈溪市| 攀枝花市| 兴文县| 泗水县| 大理市| 文化| 贺州市| 滕州市| 寿光市| 定西市| 奈曼旗| 民和| 五莲县| 普宁市| 上犹县| 石景山区| 三亚市| 革吉县| 巫山县| 砚山县|