A Task-based Comparison of Linguistic and Semantic Document Retrieval Methods in the Medical Domain

Text-based and semantics-based methods are both studied intensively as methods for document retrieval. In order to gain insight in the respective merits of these two approaches, we have performed a controlled experiment where we executed a real-life task using both textbased and semantics-based techniques. To maximise the lessons that we could draw about the two approaches, we have performed an experiment where we used the same task (searching papers from the scientific literature needed for updating a medical guideline), the same test-case (updating the 2004 Dutch national breast-cancer guideline), the same gold standard (the updated 2012 Dutch national breast-cancer guideline) and the same corpus (PubMed). We then performed this task using two different methods: retrieving papers based on keywords (text-based approach) and retrieving papers based on semantic annotations (semantics-based approach). Based on this experiment, we discuss the insights that we gained from this dual set of experiments.