Measuring the accuracy of software vulnerability assessments: experiments with students and professionals

L. Allodi, M. Cremonini, Fabio Massacci, W. Shim

Research output: Contribution to JournalArticleAcademicpeer-review


Assessing the risks of software vulnerabilities is a key process of software development and security management. This assessment requires to consider multiple factors (technical features, operational environment, involved assets, status of the vulnerability lifecycle, etc.) and may depend from the assessor’s knowledge and skills. In this work, we tackle with an important part of this problem by measuring the accuracy of technical vulnerability assessments by assessors with different level and type of knowledge. We report an experiment to compare how accurately students with different technical education and security professionals are able to assess the severity of software vulnerabilities with the Common Vulnerability Scoring System (v3) industry methodology. Our results could be useful for increasing awareness about the intrinsic subtleties of vulnerability risk assessment and possibly better compliance with regulations. With respect to academic education, professional training and human resources selections our work suggests that measuring the effects of knowledge and expertise on the accuracy of software security assessments is feasible albeit not easy.
Original languageEnglish
Pages (from-to)1063-1094
Number of pages32
JournalEmpirical Software Engineering
Issue number2
Early online date20 Jan 2020
Publication statusPublished - Mar 2020


This research has been partially supported by the European Union’s 7th Framework Programme under grant agreement no 285223 (SECONOMICS), the H2020 Framework Programme under grant agreement no 830929 (CyberSec4Europe) and from the NWO through the SpySpot project (no.628.001.004).

FundersFunder number
Horizon 2020 Framework Programme830929
Nederlandse Organisatie voor Wetenschappelijk Onderzoek628.001.004
Seventh Framework Programme285223


    Dive into the research topics of 'Measuring the accuracy of software vulnerability assessments: experiments with students and professionals'. Together they form a unique fingerprint.

    Cite this