官术网_书友最值得收藏!

The vanishing gradient problem

The vanishing gradient problem is one of the problems associated with the training of artificial neural networks when the neurons present in the early layers are not able to learn because the gradients that train the weights shrink down to zero. This happens due to the greater depth of neural network, along with activation functions with derivatives resulting in low value.

Try the following steps:

  1. Create one hidden layer neural network
  2. Add more hidden layers, one by one

We observe the gradient with regards to all the nodes, and find that the gradient values get relatively smaller when we move from the later layers to the early layers. This condition worsens with the further addition of layers. This shows that the early layer neurons are learning slowly compared to the later layer neurons. This condition is called the vanishing gradient problem.

主站蜘蛛池模板: 塘沽区| 齐齐哈尔市| 上栗县| 顺昌县| 古丈县| 灵寿县| 廉江市| 年辖:市辖区| 棋牌| 永定县| 当阳市| 登封市| 浦江县| 邳州市| 仲巴县| 光泽县| 汕尾市| 宁化县| 平谷区| 都匀市| 长兴县| 崇礼县| 广水市| 裕民县| 晋州市| 吉安市| 察隅县| 三江| 大兴区| 昌宁县| 罗甸县| 崇州市| 北海市| 龙口市| 二连浩特市| 视频| 高台县| 丰都县| 巍山| 澄城县| 中阳县|