Haptic Exploratory Behavior During Object Discrimination: A Novel Automatic Annotation Method

S.E.M. Jansen, W.M. Bergmann Tiest, A.M.L. Kappers

    Research output: Contribution to JournalArticleAcademicpeer-review

    137 Downloads (Pure)


    In order to acquire information concerning the geometry and material of handheld objects, people tend to execute stereotypical hand movement patterns called haptic Exploratory Procedures (EPs). Manual annotation of haptic exploration trials with these EPs is a laborious task that is affected by subjectivity, attentional lapses, and viewing angle limitations. In this paper we propose an automatic EP annotation method based on position and orientation data from motion tracking sensors placed on both hands and inside a stimulus. A set of kinematic variables is computed from these data and compared to sets of predefined criteria for each of four EPs. Whenever all criteria for a specific EP are met, it is assumed that that particular hand movement pattern was performed. This method is applied to data from an experiment where blindfolded participants haptically discriminated between objects differing in hardness, roughness, volume, and weight. In order to validate the method, its output is compared to manual annotation based on video recordings of the same trials. Although mean pairwise agreement is less between human-automatic pairs than between humanhuman pairs (55.7% vs 74.5%), the proposed method performs much better than random annotation (2.4%). Furthermore, each EP is linked to a specific object property for which it is optimal (e.g., Lateral Motion for roughness). We found that the percentage of trials where the expected EP was found does not differ between manual and automatic annotation. For now, this method cannot yet completely replace a manual annotation procedure. However, it could be used as a starting point that can be supplemented by manual annotation.
    Original languageEnglish
    Article numbere0117017
    JournalPLoS ONE
    Issue number2
    Publication statusPublished - 2015


    Dive into the research topics of 'Haptic Exploratory Behavior During Object Discrimination: A Novel Automatic Annotation Method'. Together they form a unique fingerprint.

    Cite this