Meta-Hyperband: Hyperparameter optimization with meta-learning and Coarse-to-Fine
File Type:
PDFItem Type:
Conference PaperDate:
2020Access:
openAccessCitation:
Payrosangari S., Sadeghi A., Graux D., Lehmann J. (2020) Meta-hyperband: Hyperparameter Optimization with Meta-learning and Coarse-to-Fine. In: Analide C., Novais P., Camacho D., Yin H. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2020. IDEAL 2020. Lecture Notes in Computer Science, vol 12490. Springer, ChamDownload Item:
Abstract:
Hyperparameter optimization is one of the main pillars of machine learning algorithms. In this paper, we introduce Meta-Hyperband: a Hyperband based algorithm that improves the hyperparameter optimization by adding levels of exploitation. Unlike Hyperband method, which is a pure exploration bandit-based approach for hyperparameter optimization, our meta approach generates a trade-off between exploration and exploitation by combining the Hyperband method with meta-learning and Coarse-to-Fine modules. We analyze the performance of Meta-Hyperband on various datasets to tune the hyperparameters of CNN and SVM. The experiments indicate that in many cases Meta-Hyperband can discover hyperparameter configurations with higher quality than Hyperband, using similar amounts of resources. In particular, we discovered a CNN configuration for classifying CIFAR10 dataset which has a 3% higher performance than the configuration founded by Hyperband, and is also 0.3% more accurate than the best-reported configuration of the Bayesian optimization approach. Additionally, we release a publicly available pool of historically well-performed configurations on several datasets for CNN and SVM to ease the adoption of Meta-Hyperband.
Sponsor
Grant Number
Marie Curie
801522
Science Foundation Ireland (SFI)
13/RC/2106
Author's Homepage:
http://people.tcd.ie/grauxdDescription:
PUBLISHEDSponsor:
Marie CurieScience Foundation Ireland (SFI)
Type of material:
Conference PaperCollections
Availability:
Full text availableKeywords:
Hyperparameter optim, Meta-learning, Coarse-to-FineSubject (TCD):
MACHINE LEARNING , OPTIMISATIONDOI:
https://doi.org/10.1007/978-3-030-62365-4_32Metadata
Show full item recordLicences: