Automata Language Equivalence vs. Simulations for Model-based Mutant Equivalence: An Empirical Evaluation

Research output: Contribution in Book/Catalog/Report/Conference proceedingConference contribution

16 Downloads (Pure)

Abstract

Mutation analysis is a popular test assessment method. It relies on the mutation score, which indicates how many mutants are revealed by a test suite. Yet, there are mutants whose behaviour is equivalent to the original system, wasting analysis resources and preventing the satisfaction of the full (100%) mutation score. For finite behavioural models, the Equivalent Mutant Problem (EMP) can be addressed through language equivalence of non-deterministic finite automata, which is a well-studied, yet computationally expensive, problem in automata theory. In this paper, we report on our preliminary assessment of a state-of-the-art exact language equivalence tool to handle the EMP against 3 models of size up to 15,000 states on 1170 mutants. We introduce random and mutation-biased simulation heuristics as baselines for comparison. Results show that the exact approach is often more than ten times faster in the weak mutation scenario. For strong mutation, our biased simulations are faster for models larger than 300 states. They can be up to 1,000 times faster while limiting the error of misclassifying non-equivalent mutants as equivalent to 10% on average. We therefore conclude that the approaches can be combined for improved efficiency.
Original languageEnglish
Title of host publicationProceedings - 10th IEEE International Conference on Software Testing, Verification and Validation, ICST 2017
Place of PublicationTokyo, Japan
PublisherIEEE
Pages424-429
Number of pages6
ISBN (Electronic)9781509060313
DOIs
Publication statusPublished - 13 Mar 2017
Event10th IEEE International Conference on Software Testing, Verification and Validation (ICST 2017) - Tokyo, Japan
Duration: 13 Mar 201718 Mar 2017
Conference number: 10
http://aster.or.jp/conference/icst2017/

Publication series

NameICST '17
PublisherIEEE

Conference

Conference10th IEEE International Conference on Software Testing, Verification and Validation (ICST 2017)
Abbreviated titleICST 2017
CountryJapan
CityTokyo
Period13/03/1718/03/17
Internet address

Keywords

  • Model-Based Mutation Analysis
  • Automata Language Equivalence
  • Random Simulations
  • Model-based mutation analysis
  • Random simulations
  • Automata language equivalence

Fingerprint Dive into the research topics of 'Automata Language Equivalence vs. Simulations for Model-based Mutant Equivalence: An Empirical Evaluation'. Together they form a unique fingerprint.

Cite this