批量正则化DBN分类方法研究 postprint

Abstract: Aiming at the problem that the deep belief network (DBN) is susceptible to the training parameters燿uring the fine-tune process, this paper proposed a kind of batch normalization DBN classification method (BNDBN) . Firstly, this method used unsupervised learning to obtain high-level representation of raw data. Then through the introduction of scale transformation and translation transformation parameters, it processed the output characteristics of each layer by batch normalization. And it fed the post-processing characteristics into the nonlinear transformation activation layer. Finally, it trained and studied the parameters of the affine transformation and the original network by using the stochastic gradient descent method. The BNDBN method reduced the dependence of the gradient on the parameter size, which effectively resolved the problem of changing the value distribution of activation function caused by the change of network parameters and improves the training efficiency. To verify the effectiveness of the proposed method, it selected MNIST handwritten database and the USPS handwritten digital identification library for testing. Compared with the Dropout-DBN, DBN, ANN, SVM and KNN, the results show that the proposed method significantly improved the classification accuracy and had stronger feature extraction ability.