TY - GEN
T1 - Accelerated learning for Restricted Boltzmann Machine with momentum term
AU - Zareba, Szymon
AU - Gonczarek, Adam
AU - Tomczak, Jakub M.
AU - Światek, Jerzy
PY - 2015/1/1
Y1 - 2015/1/1
N2 - Restricted Boltzmann Machines are generative models which can be used as standalone feature extractors, or as a parameter initialization for deeper models. Typically, these models are trained using Contrastive Divergence algorithm, an approximation of the stochastic gradient descent method. In this paper, we aim at speeding up the convergence of the learning procedure by applying the momentum method and the Nesterov's accelerated gradient technique. We evaluate these two techniques empirically using the image dataset MNIST.
AB - Restricted Boltzmann Machines are generative models which can be used as standalone feature extractors, or as a parameter initialization for deeper models. Typically, these models are trained using Contrastive Divergence algorithm, an approximation of the stochastic gradient descent method. In this paper, we aim at speeding up the convergence of the learning procedure by applying the momentum method and the Nesterov's accelerated gradient technique. We evaluate these two techniques empirically using the image dataset MNIST.
KW - Contrastive Divergence
KW - Deep learning
KW - Nesterov's momentum
KW - stochastic gradient descent
UR - http://www.scopus.com/inward/record.url?scp=84906535729&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84906535729&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-08422-0_28
DO - 10.1007/978-3-319-08422-0_28
M3 - Conference contribution
AN - SCOPUS:84906535729
SN - 9783319084213
T3 - Advances in Intelligent Systems and Computing
SP - 187
EP - 192
BT - Progress in Systems Engineering - Proceedings of the 23rd International Conference on Systems Engineering
PB - Springer Verlag
T2 - 23rd International Conference on Systems Engineering, ICSEng 2014
Y2 - 19 August 2014 through 21 August 2014
ER -