Abstract
In many machine learning applications, the labeling of datasets is done by human experts, which is usually time-consuming in cases of large data sets. This raises the need for methods to make optimal use of the human expert by selecting model instances for which the expert opinion is of most added value. This paper introduces the problem of active pairwise distance learning (APDL), where the goal is to actively learn the pairwise distances between all instances. Any distance function can be used, which means that APDL techniques can e.g., be used to determine likeness between faces or similarities between users for recommender systems. Starting with an unlabeled dataset, each round an expert determines the distance between one pair of instances. Thus, there is an important choice to make each round: ‘Which combination of instances is presented to the expert?’ The objective is to accurately predict all pairwise distances, while minimizing the usage of the expert. In this research, we establish upper and lower bound approximations (including an update rule) for the pairwise distances and evaluate many domain-independent query strategies. The observations from the experiments are therefore general, and the selection strategies are ideal candidates to function as baseline in future research. We show that using the criterion max degree consistently ranks amongst the best strategies. By using this criterion, the pairwise distances of a new dataset can be labeled much more efficiently.
Original language | English |
---|---|
Pages (from-to) | 24689-24708 |
Number of pages | 20 |
Journal | Applied Intelligence |
Volume | 53 |
Issue number | 21 |
Early online date | 28 Jul 2023 |
DOIs | |
Publication status | Published - Nov 2023 |
Bibliographical note
Funding Information:The authors wish thank the anonymous referees for their useful comments, which has led to a significant improvement of the readability and quality of the paper.
Publisher Copyright:
© 2023, The Author(s).
Funding
The authors wish thank the anonymous referees for their useful comments, which has led to a significant improvement of the readability and quality of the paper.
Keywords
- Active learning
- Human expert
- Labeling
- Optimal strategy
- Pairwise distance