Benchmarking Flaws Undermine Security Research

Erik Van Der Kouwe, Gernot Heiser, Dennis Andriesse, Herbert Bos, Cristiano Giuffrida

Research output: Contribution to JournalArticleAcademicpeer-review


Benchmarking systems is difficult. Mistakes can compromise guarantees and threaten reproducibility and comparability. We conduct a study to show that benchmarking flaws are widespread in systems security defense papers, even at tier-1 venues. We aim to raise awareness and provide recommendations for safeguarding the scientific process in our community.

Original languageEnglish
Article number9001062
Pages (from-to)48-57
Number of pages10
JournalIEEE Security and Privacy
Issue number3
Early online date18 Feb 2020
Publication statusPublished - May 2020


Dive into the research topics of 'Benchmarking Flaws Undermine Security Research'. Together they form a unique fingerprint.

Cite this