Publication
Learning in Compressed Space
Alexander Fabisch; Yohannes Kassahun; Hendrik Wöhrle; Frank Kirchner
In: Neural Networks, Vol. 42, Pages 83-93, Elsevier, 2013.
Abstract
We examine two methods which are used to deal with complex machine learning problems: compressed sensing and model compression. We discuss both methods in the context of feed-forward artificial neural networks andmdevelop the ackpropagation method in compressed parameter space. We further show that compressing the weights of a layer of a multilayer perceptron is equivalent to compressing the input of the layer. Based on this theoretical framework, we will use orthogonal functions and especially random projections for compression and perform experiments in supervised and reinforcement learning to demonstrate that the presented methods reduce training time significantly.