官术网_书友最值得收藏!

Experiment with hidden layers and hidden units

The most commonly used layers in general neural networks are fully-connected layers. In fully-connected layers, the units in two successive layers are all pairwise connected. However, the units within a layer don't share any connections. As stated before, the connections between the layers are also called trainable parameters. The weights of these connections are trained by the network. The more connections, the more parameters and the more complex patterns can be modeled. Most state-of-the-art models have 100+ million parameters. However, a deep neural network with many layers and units takes more time to train. Also, with extremely deep models the time to infer predictions takes significantly longer (which can be problematic in a real-time environment). In the following chapters, we will introduce other popular layer types that are specific to their network types. 

Picking the correct number of hidden layers and hidden units can be important. When using too few nodes, the model won't be able to pick up all the signals, resulting in a low accuracy and poor predictive performance (underfitting). Using too many nodes, the model will tend to overfit on the training data (see regularization for techniques to prevent overfitting) and won't be able to generalize well. Therefore, we always have to look at the performance on the validation data to find the right balance. In the next recipe, we will show an example of overfitting and output the number of trainable parameters.

Deep FNNs perform well if there is a lot of high dimensional training data available. For simple classification or regression tasks, often a single-layer neural network performs best.
主站蜘蛛池模板: 宣城市| 名山县| 天津市| 肃宁县| 定边县| 德保县| 黄冈市| 陵川县| 洛扎县| 临猗县| 弥勒县| 巢湖市| 华坪县| 紫云| 汝南县| 新乡县| 祁连县| 南京市| 龙门县| 石屏县| 西乡县| 昌江| 壶关县| 鄂伦春自治旗| 类乌齐县| 农安县| 通辽市| 张家港市| 鲁山县| 蛟河市| 香港 | 永登县| 清水县| 疏附县| 稻城县| 乐安县| 忻州市| 齐河县| 兴和县| 雷山县| 南安市|