An Approach to Aggregating Ensembles of Lazy Learners that Supports Explanation

Abstract

Ensemble research has shown that the aggregated output of an ensemble of predictors can be more accurate than a single predictor. This is true also for lazy learning systems like Case-Based Reasoning (CBR) and k-Nearest- Neighbour. Aggregation is normally achieved by voting in classification tasks and by averaging in regression tasks. For CBR, this increased accuracy comes at the cost of interpretability however. If we consider the use of retrieved cases for explanation to be one of the advantages of CBR then this is lost in an ensemble. This is because a large number of cases will have been retrieved by the ensemble members. In this paper we present a new technique for aggregation that obtains excellent results and identifies a small number of cases for use in explanation. This new approach might be viewed as a transformation process whereby cases are transformed from their feature based representation to a representation based on the predictions of ensemble members. This new representation produces very accurate predictions and allows a small number of similar neighbours to be identified.