MUPPAAL: Efficient Elimination and Reduction of Useless Mutants in Real-Time Model-based Systems

To assess test quality, mutation testing (MT) creates mutants by injecting artificial faults into the system and evaluates the ability of tests to distinguish these mutants. Tests distinguishing more mutants have also been proven empirically to detect more real faults. MT has been applied to many domains. We focus on MT for timed safety-critical systems modelled as Timed Automata (TA). While powerful, MT usually yields equivalent and duplicate mutants, the former having the same behaviour as the original system and the latter other mutants. Such useless mutants bring no value, waste execution time and can be difficult to detect. We integrate useless mutant detection and removal strategies in our mutation framework MUPPAAL. MUPPAAL leverages existing equivalence-avoiding mutation operators and focuses on detecting mutant duplicates using a scalable bisimulation algorithm and a fast approximate one based on biased simulation. We also demonstrate how to design an operator that reduces the occurrence of mutant duplicates. We evaluate MUPPAAL on six systems, demonstrating that (1) mutant duplicates account for up to 32% of all generated mutants, (2) our bisimulation approach scales effectively with these systems and (3) biased simulations further enhance performance. Our heuristic is 10 times faster than bisimulation and limits the exploration to two times the number of exact duplicates compared to up to 10 times for the baseline.