An empirical methodology to evaluate vulnerability discovery models

F. Massacci, V.H. Nguyen

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

© 2014 IEEE.Vulnerability discovery models (VDMs) operate on known vulnerability data to estimate the total number of vulnerabilities that will be reported after a software is released. VDMs have been proposed by industry and academia, but there has been no systematic independent evaluation by researchers who are not model proponents. Moreover, the traditional evaluation methodology has some issues that biased previous studies in the field. In this work we propose an empirical methodology that systematically evaluates the performance of VDMs along two dimensions (quality and predictability) and addresses all identified issues of the traditional methodology. We conduct an experiment to evaluate most existing VDMs on popular web browsers' vulnerability data. Our comparison shows that the results obtained by the proposed methodology are more informative than those by the traditional methodology. Among evaluated VDMs, the simplest linear model is the most appropriate choice in terms of both quality and predictability for the first 6-12 months since a release date. Otherwise, logistics-based models are better choices.
Original languageEnglish
Pages (from-to)1147-1162
JournalIEEE Transactions on Software Engineering
Volume40
Issue number12
DOIs
Publication statusPublished - 2014
Externally publishedYes

Fingerprint

Dive into the research topics of 'An empirical methodology to evaluate vulnerability discovery models'. Together they form a unique fingerprint.

Cite this