Identifying the implied: Findings from three differentiated replications on the use of security requirements templates

M. Riaz, J. King, J. Slankas, L. Williams, F. Massacci, C. Quesada-López, M. Jenkins

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

© 2016, Springer Science+Business Media New York.Identifying security requirements early on can lay the foundation for secure software development. Security requirements are often implied by existing functional requirements but are mostly left unspecified. The Security Discoverer (SD) process automatically identifies security implications of individual requirements sentences and suggests applicable security requirements templates. The objective of this research is to support requirements analysts in identifying security requirements by automating the suggestion of security requirements templates that are implied by existing functional requirements. We conducted a controlled experiment in a graduate-level security class at North Carolina State University (NCSU) to evaluate the SD process in eliciting implied security requirements in 2014. We have subsequently conducted three differentiated replications to evaluate the generalizability and applicability of the initial findings. The replications were conducted across three countries at the University of Trento, NCSU, and the University of Costa Rica. We evaluated the responses of the 205 total participants in terms of quality, coverage, relevance and efficiency. We also develop shared insights regarding the impact of context factors such as time, motivation and support, on the study outcomes and provide lessons learned in conducting the replications. Treatment group, using the SD process, performed significantly better than the control group (at p-value <0.05) in terms of the coverage of the identified security requirements and efficiency of the requirements elicitation process in two of the three replications, supporting the findings of the original study. Participants in the treatment group identified 84 % more security requirements in the oracle as compared to the control group on average. Overall, 80 % of the 111 participants in the treatment group were favorable towards the use of templates in identifying security requirements. Our qualitative findings indicate that participants may be able to differentiate between relevant and extraneous templates suggestions and be more inclined to fill in the templates with additional support. Security requirements templates capture the security knowledge of multiple experts and can support the security requirements elicitation process when automatically suggested, making the implied security requirements more evident. However, individual participants may still miss out on identifying a number of security requirements due to empirical constraints as well as potential limitations on knowledge and security expertise.
Original languageEnglish
Pages (from-to)2127-2178
JournalEmpirical Software Engineering
Volume22
Issue number4
DOIs
Publication statusPublished - 1 Aug 2017
Externally publishedYes

Funding

This work is partially supported by NSA Science of Security lablet. Fabio Massacci is partially supported by the SESAR Joint Undertaking WP-E EMFASE Project. Christian Quesada-López and Marcelo Jenkins are supported by University of Costa Rica Project No. 834-B5-A18, and Ministry of Science, Technology and Telecommunications (MICITT). Special thanks to Patrick Francis and Patrick Morrison with their help in developing the study oracle. We are thankful to the Realsearch group for their collaboration and helpful comments.

FundersFunder number
Ministry of Science, Technology and Telecommunications
NSA Science of Security lablet
Universidad de Costa Rica834-B5-A18
Ministerio de Ciencia Tecnología y Telecomunicaciones

    Fingerprint

    Dive into the research topics of 'Identifying the implied: Findings from three differentiated replications on the use of security requirements templates'. Together they form a unique fingerprint.

    Cite this