Abstract
A Web archive usually contains multiple versions of documents crawled from the Web at different points in time. One possible way for users to access a Web archive is through full-text search systems. However, previous studies have shown that these systems can induce a bias, known as the retrievability bias, on the accessibility of documents in community-collected collections (such as TREC collections). This bias can be measured by analyzing the distribution of the retrievability scores for each document in a collection, quantifying the likelihood of a document’s retrieval. We investigate the suitability of retrievability scores in retrieval systems that consider every version of a document in a Web archive as an independent document. We show that the retrievability of documents can vary for different versions of the same document and that retrieval systems induce biases to different extents. We quantify this bias for a retrieval system which is adapted to handle multiple versions of the same document. The retrieval system indexes each version of a document independently, and we refine the search results using two techniques to aggregate similar versions. The first approach is to collapse similar versions of a document based on content similarity. The second approach is to collapse all versions of the same document based on their URLs. In both cases, we found that the degree of bias is related to the aggregation level of versions of the same document. Finally, we study the effect of bias across time using the retrievability measure. Specifically, we investigate whether the number of documents crawled in a particular year correlates with the number of documents in the search results from that year. Assuming queries are not inherently temporal in nature, the analysis is based on the timestamps of documents in the search results returned using the retrieval model for all queries. The results show a relation between the number of documents per year and the number of documents retrieved by the retrieval system from that year. We further investigated the relation between the queries’ timestamps and the documents’ timestamps. First, we split the queries into different time frames using a 1-year granularity. Then, we issued the queries against the retrieval system. The results show that temporal queries indeed retrieve more documents from the assumed time frame. Thus, the documents from the same time frame were preferred by the retrieval system over documents from other time frames.
Original language | English |
---|---|
Pages (from-to) | 57-75 |
Number of pages | 19 |
Journal | International Journal on Digital Libraries |
Volume | 19 |
Issue number | 1 |
Early online date | 14 Nov 2017 |
DOIs | |
Publication status | Published - 1 Mar 2018 |
Funding
Acknowledgements This research was supported by the Netherlands Organization for Scientific Research (WebART project, NWO CATCH #640.005.001), and the Dutch COMMIT/program (SEALINCMedia project). We would like to thank the National Library of the Netherlands for their support. Part of the analysis work was carried out on the Dutch national e-infrastructure with the support of the SURF Foundation. We are grateful for the input from our colleagues Jiyin He and Desmond Elliott. This research was supported by the Netherlands Organization for Scientific Research (WebART project, NWO CATCH #640.005.001), and the Dutch COMMIT/program (SEALINCMedia project). We would like to thank the National Library of the Netherlands for their support. Part of the analysis work was carried out on the Dutch national e-infrastructure with the support of the SURF Foundation. We are grateful for the input from our colleagues Jiyin He and Desmond Elliott.
Funders | Funder number |
---|---|
Dutch COMMIT/program | |
NWO CATCH | CATCH #640.005.001 |
Netherlands Organization for Scientific Research | |
SEALINCMedia | |
SURF Foundation | |
Nederlandse Organisatie voor Wetenschappelijk Onderzoek | 640.005.001 |
Keywords
- Evaluation
- Retrieval bias
- Web archive