官术网_书友最值得收藏!

The vanishing gradient problem

The vanishing gradient problem is one of the problems associated with the training of artificial neural networks when the neurons present in the early layers are not able to learn because the gradients that train the weights shrink down to zero. This happens due to the greater depth of neural network, along with activation functions with derivatives resulting in low value.

Try the following steps:

  1. Create one hidden layer neural network
  2. Add more hidden layers, one by one

We observe the gradient with regards to all the nodes, and find that the gradient values get relatively smaller when we move from the later layers to the early layers. This condition worsens with the further addition of layers. This shows that the early layer neurons are learning slowly compared to the later layer neurons. This condition is called the vanishing gradient problem.

主站蜘蛛池模板: 治县。| 扎鲁特旗| 青神县| 牟定县| 丰顺县| 张家港市| 平乐县| 红安县| 双柏县| 南充市| 海门市| 阳曲县| 兰溪市| 红河县| 陆良县| 呼伦贝尔市| 金昌市| 德格县| 垣曲县| 丰台区| 大港区| 苍南县| 隆化县| 曲周县| 云南省| 平和县| 扬州市| 双辽市| 来凤县| 忻州市| 成武县| 锦州市| 万源市| 平原县| 北票市| 平凉市| 凤凰县| 安化县| 镇康县| 吉安市| 甘南县|