Abstract
Test suite reduction approaches aim at decreasing software regression testing costs by selecting a representative subset from large-size test suites. Most existing techniques are too expensive for handling modern massive systems and moreover depend on artifacts, such as code coverage metrics or specification models, that are not commonly available at large scale. We present a family of novel very efficient approaches for similarity based test suite reduction that apply algorithms borrowed from the big data domain together with smart heuristics for finding an evenly spread subset of test cases. The approaches are very general since they only use as input the test cases themselves (test source code or command line input). We evaluate four approaches in a version that selects a fixed budget B of test cases, and also in an adequate version that does the reduction guaranteeing some fixed coverage. The results show that the approaches yield a fault detection loss comparable to state-of-the-art techniques, while providing huge gains in terms of efficiency. When applied to a suite of more than 500K real world test cases, the most efficient of the four approaches could select B test cases (for varying B values) in less than 10 seconds.
Original language | English |
---|---|
Title of host publication | Proceedings - 2019 IEEE/ACM 41st International Conference on Software Engineering, ICSE 2019 |
Publisher | IEEE Computer Society |
Pages | 419-429 |
Number of pages | 11 |
ISBN (Electronic) | 9781728108698 |
DOIs | |
Publication status | Published - 25 May 2019 |
Externally published | Yes |
Event | 41th ACM/IEEE International Conference on Software Engineering (ICSE 2019) - Montréal, Canada Duration: 25 May 2019 → 31 May 2019 https://2019.icse-conferences.org/info/icse-logo |
Publication series
Name | Proceedings - International Conference on Software Engineering |
---|---|
Volume | 2019-May |
ISSN (Print) | 0270-5257 |
Conference
Conference | 41th ACM/IEEE International Conference on Software Engineering (ICSE 2019) |
---|---|
Abbreviated title | ICSE2019 |
Country/Territory | Canada |
City | Montréal |
Period | 25/05/19 → 31/05/19 |
Internet address |
Funding
ACKNOWLEDGMENT This research has been partly funded by the H2020 European Project ElasTest in the H2020 Programme under GA No 731535. Breno Miranda wishes to thank the postdoctoral fellowship jointly sponsored by CAPES and FACEPE (APQ-0826-1.03/16; BCT-0204-1.03/17).
Funders | Funder number |
---|---|
H2020 European Project ElasTest | |
H2020 Programme | |
Horizon 2020 Framework Programme | 731535 |
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior | |
Fundação de Amparo à Ciência e Tecnologia do Estado de Pernambuco | APQ-0826-1.03/16, BCT-0204-1.03/17 |
Keywords
- Clustering
- Random projection
- Similarity-based testing
- Software testing
- Test suite reduction