"Neural networks" is a term usually used to refer to feedforward neural networks. Deep Neural Networks are feedforward Neural Networks with many layers.
A Deep belief network is not the same as a Deep Neural Network.
As you have pointed out a deep belief network has undirected connections between some layers. This means that the topology of the DNN and DBN is different by definition.
The undirected layers in the DBN are called Restricted Boltzmann Machines. This layers can be trained using an unsupervised learning algorithm (Contrastive Divergence) that is very fast (Here's a link! with details).
Some more comments:
The solutions obtained with deeper neural networks correspond to solutions that perform worse than the solutions obtained for networks with 1 or 2 hidden layers. As the architecture gets deeper, it becomes more difficult to obtain good generalization using a Deep NN.
In 2006 Hinton discovered that much better results could be achieved in deeper architectures when each layer (RBM) is pre-trained with an unsupervised learning algorithm (Contrastive Divergence). Then the Network can be trained in a supervised way using backpropagation in order to "fine-tune" the weights.
No comments:
Post a Comment