batch_size, nb_epoch
Optimization algorithm: ['SGD', 'RMSprop', 'Adagrad', 'Adadelta', 'Adam', 'Adamax', 'Nadam']
Learning Rate & Momentum for SGD
Weight Init: ['uniform', 'lecun_uniform', 'normal', 'zero', 'glorot_normal', 'glorot_uniform', 'he_normal', 'he_uniform']
Activation Function:['softmax', 'softplus', 'softsign', 'relu', 'tanh', 'sigmoid', 'hard_sigmoid', 'linear']
Dropout Rate: -> Keras's regularization
the dropout rate for regularization in an effort to limit overfitting and improve the model’s ability to generalize.
[0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]
# of Neurons:[1, 5, 10, 15, 20, 25, 30]
Also, generally, a large enough single layer network can approximate any other neural network, at least in theory.
Optimization algorithm: ['SGD', 'RMSprop', 'Adagrad', 'Adadelta', 'Adam', 'Adamax', 'Nadam']
Learning Rate & Momentum for SGD
Weight Init: ['uniform', 'lecun_uniform', 'normal', 'zero', 'glorot_normal', 'glorot_uniform', 'he_normal', 'he_uniform']
Activation Function:['softmax', 'softplus', 'softsign', 'relu', 'tanh', 'sigmoid', 'hard_sigmoid', 'linear']
Dropout Rate: -> Keras's regularization
the dropout rate for regularization in an effort to limit overfitting and improve the model’s ability to generalize.
[0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]
# of Neurons:[1, 5, 10, 15, 20, 25, 30]
Also, generally, a large enough single layer network can approximate any other neural network, at least in theory.
- Reproducibility is a Problem. Although we set the seed for the random number generator in NumPy, the results are not 100% reproducible. There is more to reproducibility when grid searching wrapped Keras models than is presented in this post.
Start with Single Hidden Layer and add more layers
No comments:
Post a Comment