Publikation
Learning Class-relevant Features and Class-irrelevant Features via a Hybrid third-order RBM.
H. Luo; Rui-Min Shen; C. Niu; Carsten Ullrich
In: o.A. (Hrsg.). Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. International Conference on Artificial Intelligence and Statistics (AISTATS), Pages 47-478, 2011.
Zusammenfassung
Restricted Boltzmann Machines are com- monly used in unsupervised learning to ex- tract features from training data. Since these features are learned for regenerating train- ing data a classifier based on them has to be trained. If only a few of the learned features are discriminative other non-discriminative features will distract the classifier during the training process and thus waste com- puting resources for testing. In this paper, we present a hybrid third-order Restricted Boltzmann Machine in which class-relevant features (for recognizing) and class-irrelevant features (for generating only) are learned si- multaneously. As the classification task uses only the class-relevant features, the test it- self becomes very fast. We show that class- irrelevant features help class-relevant features to focus on the recognition task and intro- duce useful regularization effects to reduce the norms of class-relevant features. Thus there is no need to use weight-decay for the parameters of this model. Experiments on the MNIST, NORB and Caltech101 Silhou- ettes datasets show very promising results.