TY - JOUR
T1 - Benchmarking Flaws Undermine Security Research
AU - Van Der Kouwe, Erik
AU - Heiser, Gernot
AU - Andriesse, Dennis
AU - Bos, Herbert
AU - Giuffrida, Cristiano
PY - 2020/5
Y1 - 2020/5
N2 - Benchmarking systems is difficult. Mistakes can compromise guarantees and threaten reproducibility and comparability. We conduct a study to show that benchmarking flaws are widespread in systems security defense papers, even at tier-1 venues. We aim to raise awareness and provide recommendations for safeguarding the scientific process in our community.
AB - Benchmarking systems is difficult. Mistakes can compromise guarantees and threaten reproducibility and comparability. We conduct a study to show that benchmarking flaws are widespread in systems security defense papers, even at tier-1 venues. We aim to raise awareness and provide recommendations for safeguarding the scientific process in our community.
UR - http://www.scopus.com/inward/record.url?scp=85079625721&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85079625721&partnerID=8YFLogxK
U2 - 10.1109/MSEC.2020.2969862
DO - 10.1109/MSEC.2020.2969862
M3 - Article
AN - SCOPUS:85079625721
SN - 1540-7993
VL - 18
SP - 48
EP - 57
JO - IEEE Security & Privacy
JF - IEEE Security & Privacy
IS - 3
M1 - 9001062
ER -