官术网_书友最值得收藏!

Experiment with hidden layers and hidden units

The most commonly used layers in general neural networks are fully-connected layers. In fully-connected layers, the units in two successive layers are all pairwise connected. However, the units within a layer don't share any connections. As stated before, the connections between the layers are also called trainable parameters. The weights of these connections are trained by the network. The more connections, the more parameters and the more complex patterns can be modeled. Most state-of-the-art models have 100+ million parameters. However, a deep neural network with many layers and units takes more time to train. Also, with extremely deep models the time to infer predictions takes significantly longer (which can be problematic in a real-time environment). In the following chapters, we will introduce other popular layer types that are specific to their network types. 

Picking the correct number of hidden layers and hidden units can be important. When using too few nodes, the model won't be able to pick up all the signals, resulting in a low accuracy and poor predictive performance (underfitting). Using too many nodes, the model will tend to overfit on the training data (see regularization for techniques to prevent overfitting) and won't be able to generalize well. Therefore, we always have to look at the performance on the validation data to find the right balance. In the next recipe, we will show an example of overfitting and output the number of trainable parameters.

Deep FNNs perform well if there is a lot of high dimensional training data available. For simple classification or regression tasks, often a single-layer neural network performs best.
主站蜘蛛池模板: 肥城市| 济宁市| 西青区| 深圳市| 泗洪县| 津市市| 泾阳县| 揭阳市| 太仆寺旗| 淅川县| 铁岭县| 潍坊市| 南丹县| 兴文县| 郓城县| 贵州省| 扬中市| 本溪市| 承德县| 清原| 镇安县| 太康县| 兴化市| 年辖:市辖区| 泰和县| 静安区| 汉阴县| 威远县| 保靖县| 黄石市| 治多县| 洪江市| 乐亭县| 永胜县| 桃园市| 阿合奇县| 克什克腾旗| 长葛市| 涟源市| 个旧市| 莱西市|