Abstract
The key arguments underlying a large and noisy set of opinions help understand the opinions quickly and accurately. Fully automated methods can extract arguments but (1) require large labeled datasets and (2) work well for known viewpoints, but not for novel points of view. We propose HyEnA, a hybrid (human + AI) method for extracting arguments from opinionated texts, combining the speed of automated processing with the understanding and reasoning capabilities of humans. We evaluate HyEnA on three feedback corpora. We find that, on the one hand, HyEnA achieves higher coverage and precision than a state-of-the-art automated method, when compared on a common set of diverse opinions, justifying the need for human insight. On the other hand, HyEnA requires less human effort and does not compromise quality compared to (fully manual) expert analysis, demonstrating the benefit of combining human and machine intelligence.
| Original language | English |
|---|---|
| Title of host publication | HHAI2022: Augmenting Human Intellect |
| Subtitle of host publication | Proceedings of the First International Conference on Hybrid Human-Artificial Intelligence |
| Editors | Stefan Schlobach, María Pérez-Ortiz, Myrthe Tielman |
| Publisher | IOS Press |
| Pages | 32-45 |
| Number of pages | 14 |
| ISBN (Electronic) | 9781643683096 |
| ISBN (Print) | 9781643683089 |
| DOIs | |
| Publication status | Published - 2022 |
Publication series
| Name | Frontiers in Artificial Intelligence and Applications |
|---|---|
| Publisher | IOS Press |
| Volume | 354 |
Funding
Acknowledgements This research was (partially) funded by the Hybrid Intelligence Center, a 10-year programme funded by the Dutch Ministry of Education, Culture, and Science through the Netherlands Organisation for Scientific Research.