Java Unit Testing Tool Competition - Seventh Round

Fitsum Kifetew, Xavier Devroey, Urko Rueda

Research output: Contribution in Book/Catalog/Report/Conference proceedingConference contribution

Abstract

We report on the results of the seventh edition of the JUnit tool competition. This year, four tools were executed on a benchmark with (i) new classes, selected from real-world software projects, and (ii) challenging classes from the previous edition. We use Randoop and manual test suites from the projects as baselines. Given the interesting findings of last year, we analyzed the effectiveness of the combined test suites generated by all competing tools and compared; results are confronted with the manual test suites of the projects, as well as those generated by the competing tools. This paper describes our methodology and the results, highlight challenges faced during the contest.

Original languageEnglish
Title of host publicationProceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages15-20
Number of pages6
ISBN (Electronic)9781728122335
DOIs
Publication statusPublished - 1 May 2019
Externally publishedYes
Event12th IEEE/ACM International Workshop on Search-Based Software Testing, SBST 2019 - Montreal, Canada
Duration: 27 May 2019 → …

Publication series

NameProceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019

Conference

Conference12th IEEE/ACM International Workshop on Search-Based Software Testing, SBST 2019
CountryCanada
CityMontreal
Period27/05/19 → …

Fingerprint

Testing

Keywords

  • automation
  • benchmark
  • combined performance
  • Java
  • mutation testing
  • statistical analysis
  • tool competition
  • unit testing

Cite this

Kifetew, F., Devroey, X., & Rueda, U. (2019). Java Unit Testing Tool Competition - Seventh Round. In Proceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019 (pp. 15-20). [8812209] (Proceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SBST.2019.00014
Kifetew, Fitsum ; Devroey, Xavier ; Rueda, Urko. / Java Unit Testing Tool Competition - Seventh Round. Proceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 15-20 (Proceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019).
@inproceedings{3322ba1f8fa247edbe10e8d4ef7d72b2,
title = "Java Unit Testing Tool Competition - Seventh Round",
abstract = "We report on the results of the seventh edition of the JUnit tool competition. This year, four tools were executed on a benchmark with (i) new classes, selected from real-world software projects, and (ii) challenging classes from the previous edition. We use Randoop and manual test suites from the projects as baselines. Given the interesting findings of last year, we analyzed the effectiveness of the combined test suites generated by all competing tools and compared; results are confronted with the manual test suites of the projects, as well as those generated by the competing tools. This paper describes our methodology and the results, highlight challenges faced during the contest.",
keywords = "automation, benchmark, combined performance, Java, mutation testing, statistical analysis, tool competition, unit testing",
author = "Fitsum Kifetew and Xavier Devroey and Urko Rueda",
year = "2019",
month = "5",
day = "1",
doi = "10.1109/SBST.2019.00014",
language = "English",
series = "Proceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "15--20",
booktitle = "Proceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019",

}

Kifetew, F, Devroey, X & Rueda, U 2019, Java Unit Testing Tool Competition - Seventh Round. in Proceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019., 8812209, Proceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019, Institute of Electrical and Electronics Engineers Inc., pp. 15-20, 12th IEEE/ACM International Workshop on Search-Based Software Testing, SBST 2019, Montreal, Canada, 27/05/19. https://doi.org/10.1109/SBST.2019.00014

Java Unit Testing Tool Competition - Seventh Round. / Kifetew, Fitsum; Devroey, Xavier; Rueda, Urko.

Proceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 15-20 8812209 (Proceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019).

Research output: Contribution in Book/Catalog/Report/Conference proceedingConference contribution

TY - GEN

T1 - Java Unit Testing Tool Competition - Seventh Round

AU - Kifetew, Fitsum

AU - Devroey, Xavier

AU - Rueda, Urko

PY - 2019/5/1

Y1 - 2019/5/1

N2 - We report on the results of the seventh edition of the JUnit tool competition. This year, four tools were executed on a benchmark with (i) new classes, selected from real-world software projects, and (ii) challenging classes from the previous edition. We use Randoop and manual test suites from the projects as baselines. Given the interesting findings of last year, we analyzed the effectiveness of the combined test suites generated by all competing tools and compared; results are confronted with the manual test suites of the projects, as well as those generated by the competing tools. This paper describes our methodology and the results, highlight challenges faced during the contest.

AB - We report on the results of the seventh edition of the JUnit tool competition. This year, four tools were executed on a benchmark with (i) new classes, selected from real-world software projects, and (ii) challenging classes from the previous edition. We use Randoop and manual test suites from the projects as baselines. Given the interesting findings of last year, we analyzed the effectiveness of the combined test suites generated by all competing tools and compared; results are confronted with the manual test suites of the projects, as well as those generated by the competing tools. This paper describes our methodology and the results, highlight challenges faced during the contest.

KW - automation

KW - benchmark

KW - combined performance

KW - Java

KW - mutation testing

KW - statistical analysis

KW - tool competition

KW - unit testing

UR - http://www.scopus.com/inward/record.url?scp=85072556942&partnerID=8YFLogxK

U2 - 10.1109/SBST.2019.00014

DO - 10.1109/SBST.2019.00014

M3 - Conference contribution

AN - SCOPUS:85072556942

T3 - Proceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019

SP - 15

EP - 20

BT - Proceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Kifetew F, Devroey X, Rueda U. Java Unit Testing Tool Competition - Seventh Round. In Proceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 15-20. 8812209. (Proceedings - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing, SBST 2019). https://doi.org/10.1109/SBST.2019.00014