官术网_书友最值得收藏!

Overcoming the limitations of deep learning

These two possible problems can be overcome by:

  • Minimizing the use of the sigmoid and tanh activation functions
  • Using a momentum-based stochastic gradient descent
  • Proper initialization of weights and biases, such as xavier initialization
  • Regularization (add regularization loss along with data loss and minimize that)
For more detail, along with mathematical representations of the vanishing and exploding gradient, you can read this article: Intelligent Signals : Unstable Deep Learning. Why and How to solve them ?
主站蜘蛛池模板: 兴化市| 防城港市| 泸西县| 扬中市| 泗洪县| 高淳县| 吴桥县| 积石山| 宜州市| 乐平市| 镇江市| 吉林市| 义马市| 泰州市| 团风县| 温宿县| 泾川县| 绥滨县| 海安县| 邯郸市| 子长县| 罗山县| 东丰县| 辽源市| 泸定县| 邢台县| 宣恩县| 昌乐县| 县级市| 抚州市| 山阴县| 万州区| 马龙县| 灵台县| 安福县| 老河口市| 巴里| 广饶县| 高密市| 沁源县| 象州县|