Peer reviewing revisited: Assessing research with interlinked semantic comments

Cristina Iulia Bucur, Tobias Kuhn, Davide Ceolin

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review


Scientific publishing seems to be at a turning point. Its paradigm has stayed basically the same for 300 years but is now challenged by the increasing volume of articles that makes it very hard for scientists to stay up to date in their respective fields. In fact, many have pointed out serious flaws of current scientific publishing practices, including the lack of accuracy and efficiency of the reviewing process. To address some of these problems, we apply here the general principles of the Web and the Semantic Web to scientific publishing, focusing on the reviewing process. We want to determine if a fine-grained model of the scientific publishing workflow can help us make the reviewing processes better organized and more accurate, by ensuring that review comments are created with formal links and semantics from the start. Our contributions include a novel model called Linkflows that allows for such detailed and semantically rich representations of reviews and the reviewing processes. We evaluate our approach on a manually curated dataset from several recent Computer Science journals and conferences that come with open peer reviews. We gathered ground-truth data by contacting the original reviewers and asking them to categorize their own review comments according to our model. Comparing this ground truth to answers provided by model experts, peers, and automated techniques confirms that our approach of formally capturing the reviewers' intentions from the start prevents substantial discrepancies compared to when this information is later extracted from the plain-text comments. In general, our analysis shows that our model is well understood and easy to apply, and it revealed the semantic properties of such review comments.

Original languageEnglish
Title of host publicationK-CAP 2019
Subtitle of host publicationProceedings of the 10th International Conference on Knowledge Capture
PublisherAssociation for Computing Machinery, Inc
Number of pages9
ISBN (Electronic)9781450370080
Publication statusPublished - 23 Sept 2019
Event10th International Conference on Knowledge Capture, K-CAP 2019 - Marina Del Rey, United States
Duration: 19 Nov 201921 Nov 2019


Conference10th International Conference on Knowledge Capture, K-CAP 2019
Country/TerritoryUnited States
CityMarina Del Rey


  • Linked data
  • Peer reviewing
  • Scientific publishing
  • Semantic web


Dive into the research topics of 'Peer reviewing revisited: Assessing research with interlinked semantic comments'. Together they form a unique fingerprint.

Cite this