Nonlinear Rescaling of Acquisition Metric Values Based on Distribution Fitting

Abstract

In this paper, we consider active learning for unbalanced datasets. When class imbalance exists, active learning algorithms tend to acquire more samples from the majority class. We present a nonlinear rescaling mechanism to compensate for the effect of class imbalance. Experiments on unbalanced datasets with multiple types of class imbalance show that the proposed scheme yields noticeable performance gain when applied to existing active learning algorithms.

Publication
In ICML 2022 Workshop: Adaptive Experimental Design and Active Learning in the Real World