Publication
A Comprehensive Dataset for Evaluating Approaches of various Meta-Learning Tasks
Matthias Reif
In: J. Salvador Sánchez; Pedro Latorre Carmona (Hrsg.). Proceedings of the First International Conference on Pattern Recognition Applications and Methods. International Conference on Pattern Recognition Applications and Methods (ICPRAM-12), February 6-8, Vilamoura, Portugal, SciTePress, 2/2012.
Abstract
New approaches in pattern recognition are typically evaluated against standard datasets, e.g. from UCI or
StatLib. Using the same and publicly available datasets increases the comparability and reproducibility of
evaluations. In the field of meta-learning, the actual dataset for evaluation is created based on multiple other
datasets. Unfortunately, no comprehensive dataset for meta-learning is currently publicly available. In this
paper, we present a novel and publicly available dataset for meta-learning based on 83 datasets, six classi-
fication algorithms, and 49 meta-features. Different target variables like accuracy and training time of the
classifiers as well as parameter dependent measures are included as ground-truth information. Therefore, the
meta-dataset can be used for various meta-learning tasks, e.g. predicting the accuracy and training time of
classifiers or predicting the optimal parameter values. Using the presented meta-dataset, a convincing and
comparable evaluation of new meta-learning approaches is possible.