Active pairwise distance learning for efficient labeling of large datasets by human experts

Joris Pries*, Sandjai Bhulai, Rob van der Mei

*Corresponding author for this work

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

In many machine learning applications, the labeling of datasets is done by human experts, which is usually time-consuming in cases of large data sets. This raises the need for methods to make optimal use of the human expert by selecting model instances for which the expert opinion is of most added value. This paper introduces the problem of active pairwise distance learning (APDL), where the goal is to actively learn the pairwise distances between all instances. Any distance function can be used, which means that APDL techniques can e.g., be used to determine likeness between faces or similarities between users for recommender systems. Starting with an unlabeled dataset, each round an expert determines the distance between one pair of instances. Thus, there is an important choice to make each round: ‘Which combination of instances is presented to the expert?’ The objective is to accurately predict all pairwise distances, while minimizing the usage of the expert. In this research, we establish upper and lower bound approximations (including an update rule) for the pairwise distances and evaluate many domain-independent query strategies. The observations from the experiments are therefore general, and the selection strategies are ideal candidates to function as baseline in future research. We show that using the criterion max degree consistently ranks amongst the best strategies. By using this criterion, the pairwise distances of a new dataset can be labeled much more efficiently.

Original languageEnglish
Pages (from-to)24689-24708
Number of pages20
JournalApplied Intelligence
Volume53
Issue number21
Early online date28 Jul 2023
DOIs
Publication statusPublished - Nov 2023

Bibliographical note

Funding Information:
The authors wish thank the anonymous referees for their useful comments, which has led to a significant improvement of the readability and quality of the paper.

Publisher Copyright:
© 2023, The Author(s).

Funding

The authors wish thank the anonymous referees for their useful comments, which has led to a significant improvement of the readability and quality of the paper.

Keywords

  • Active learning
  • Human expert
  • Labeling
  • Optimal strategy
  • Pairwise distance

Fingerprint

Dive into the research topics of 'Active pairwise distance learning for efficient labeling of large datasets by human experts'. Together they form a unique fingerprint.

Cite this