A benchmark-based evaluation of search-based crash reproduction

Mozhan Soltani, Pouria Derakhshanfar, Xavier Devroey, Arie van Deursen

Résultats de recherche: Contribution à un journal/une revueArticle

Résumé

Crash reproduction approaches help developers during debugging by generating a test case that reproduces a given crash. Several solutions have been proposed to automate this task. However, the proposed solutions have been evaluated on a limited number of projects, making comparison difficult. In this paper, we enhance this line of research by proposing JCrashPack, an extensible benchmark for Java crash reproduction, together with ExRunner, a tool to simply and systematically run evaluations. JCrashPack contains 200 stack traces from various Java projects, including industrial open source ones, on which we run an extensive evaluation of EvoCrash, the state-of-the-art tool for search-based crash reproduction. EvoCrash successfully reproduced 43% of the crashes. Furthermore, we observed that reproducing NullPointerException, IllegalArgumentException, and IllegalStateException is relatively easier than reproducing ClassCastException, ArrayIndexOutOfBoundsException and StringIndexOutOfBoundsException. Our results include a detailed manual analysis of EvoCrash outputs, from which we derive 14 current challenges for crash reproduction, among which the generation of input data and the handling of abstract and anonymous classes are the most frequents. Finally, based on those challenges, we discuss future research directions for search-based crash reproduction for Java.

langue originaleAnglais
Pages (de - à)96-138
Nombre de pages43
journalEmpirical Software Engineering
Volume25
Numéro de publication1
Les DOIs
Etat de la publicationPublié - 1 janv. 2020
Modification externeOui

    Empreinte digitale

Contient cette citation