Scalable Approaches for Test Suite Reduction

Emilio Cruciani, Breno Miranda, Roberto Verdecchia, Antonia Bertolino

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

241 Downloads (Pure)

Abstract

Test suite reduction approaches aim at decreasing software regression testing costs by selecting a representative subset from large-size test suites. Most existing techniques are too expensive for handling modern massive systems and moreover depend on artifacts, such as code coverage metrics or specification models, that are not commonly available at large scale. We present a family of novel very efficient approaches for similarity based test suite reduction that apply algorithms borrowed from the big data domain together with smart heuristics for finding an evenly spread subset of test cases. The approaches are very general since they only use as input the test cases themselves (test source code or command line input). We evaluate four approaches in a version that selects a fixed budget B of test cases, and also in an adequate version that does the reduction guaranteeing some fixed coverage. The results show that the approaches yield a fault detection loss comparable to state-of-the-art techniques, while providing huge gains in terms of efficiency. When applied to a suite of more than 500K real world test cases, the most efficient of the four approaches could select B test cases (for varying B values) in less than 10 seconds.
Original languageEnglish
Title of host publicationProceedings - 2019 IEEE/ACM 41st International Conference on Software Engineering, ICSE 2019
PublisherIEEE Computer Society
Pages419-429
Number of pages11
ISBN (Electronic)9781728108698
DOIs
Publication statusPublished - 25 May 2019
Externally publishedYes
Event41th ACM/IEEE International Conference on Software Engineering (ICSE 2019) - Montréal, Canada
Duration: 25 May 201931 May 2019
https://2019.icse-conferences.org/info/icse-logo

Publication series

NameProceedings - International Conference on Software Engineering
Volume2019-May
ISSN (Print)0270-5257

Conference

Conference41th ACM/IEEE International Conference on Software Engineering (ICSE 2019)
Abbreviated titleICSE2019
Country/TerritoryCanada
CityMontréal
Period25/05/1931/05/19
Internet address

Funding

ACKNOWLEDGMENT This research has been partly funded by the H2020 European Project ElasTest in the H2020 Programme under GA No 731535. Breno Miranda wishes to thank the postdoctoral fellowship jointly sponsored by CAPES and FACEPE (APQ-0826-1.03/16; BCT-0204-1.03/17).

FundersFunder number
H2020 European Project ElasTest
H2020 Programme
Horizon 2020 Framework Programme731535
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
Fundação de Amparo à Ciência e Tecnologia do Estado de PernambucoAPQ-0826-1.03/16, BCT-0204-1.03/17

    Keywords

    • Clustering
    • Random projection
    • Similarity-based testing
    • Software testing
    • Test suite reduction

    Fingerprint

    Dive into the research topics of 'Scalable Approaches for Test Suite Reduction'. Together they form a unique fingerprint.

    Cite this