Java Unit Testing Tool Competition - Eighth Round

Xavier Devroey; Sebastiano Panichella; Alessio Gambi

doi:10.1145/3387940.3392265

Java Unit Testing Tool Competition - Eighth Round

Xavier Devroey, Sebastiano Panichella, Alessio Gambi

Research output: Contribution in Book/Catalog/Report/Conference proceeding › Conference contribution

Abstract

We report on the results of the eighth edition of the Java unit testing tool competition. This year, two tools, EvoSuite and Randoop, were executed on a benchmark with (i) new classes under test, selected from open-source software projects, and (ii) the set of classes from one project considered in the previous edition. We relied on an updated infrastructure for the execution of the different tools and the subsequent coverage and mutation analysis based on Docker containers. We considered two different time budgets for test case generation: one an three minutes. This paper describes our methodology and statistical analysis of the results, presents the results achieved by the contestant tools and highlights the challenges we faced during the competition.

Original language	English
Title of host publication	Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020
Publisher	ACM Press
Pages	545-548
Number of pages	4
ISBN (Electronic)	9781450379632
DOIs	https://doi.org/10.1145/3387940.3392265
Publication status	Published - 27 Jun 2020
Externally published	Yes
Event	42nd IEEE/ACM International Conference on Software Engineering Workshops, ICSEW 2020 - Seoul, Korea, Republic of Duration: 27 Jun 2020 → 19 Jul 2020

Publication series

Name	Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020

Conference

Conference	42nd IEEE/ACM International Conference on Software Engineering Workshops, ICSEW 2020
Country/Territory	Korea, Republic of
City	Seoul
Period	27/06/20 → 19/07/20

Keywords

benchmark
Java
JUnit
software testing
test case generation
tool competition
unit testing

Access to Document

10.1145/3387940.3392265

Cite this

Devroey, X., Panichella, S., & Gambi, A. (2020). Java Unit Testing Tool Competition - Eighth Round. In Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020 (pp. 545-548). (Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020). ACM Press. https://doi.org/10.1145/3387940.3392265

@inproceedings{751e34c797cf4726a5551b2cd7a55e91,

title = "Java Unit Testing Tool Competition - Eighth Round",

abstract = "We report on the results of the eighth edition of the Java unit testing tool competition. This year, two tools, EvoSuite and Randoop, were executed on a benchmark with (i) new classes under test, selected from open-source software projects, and (ii) the set of classes from one project considered in the previous edition. We relied on an updated infrastructure for the execution of the different tools and the subsequent coverage and mutation analysis based on Docker containers. We considered two different time budgets for test case generation: one an three minutes. This paper describes our methodology and statistical analysis of the results, presents the results achieved by the contestant tools and highlights the challenges we faced during the competition. ",

keywords = "benchmark, Java, JUnit, software testing, test case generation, tool competition, unit testing",

author = "Xavier Devroey and Sebastiano Panichella and Alessio Gambi",

year = "2020",

month = jun,

day = "27",

doi = "10.1145/3387940.3392265",

language = "English",

series = "Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020",

publisher = "ACM Press",

pages = "545--548",

booktitle = "Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020",

address = "United States",

note = "42nd IEEE/ACM International Conference on Software Engineering Workshops, ICSEW 2020 ; Conference date: 27-06-2020 Through 19-07-2020",

}

Devroey, X, Panichella, S & Gambi, A 2020, Java Unit Testing Tool Competition - Eighth Round. in Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020. Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020, ACM Press, pp. 545-548, 42nd IEEE/ACM International Conference on Software Engineering Workshops, ICSEW 2020, Seoul, Korea, Republic of, 27/06/20. https://doi.org/10.1145/3387940.3392265

Java Unit Testing Tool Competition - Eighth Round. / Devroey, Xavier; Panichella, Sebastiano; Gambi, Alessio.
Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020. ACM Press, 2020. p. 545-548 (Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020).

Research output: Contribution in Book/Catalog/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Java Unit Testing Tool Competition - Eighth Round

AU - Devroey, Xavier

AU - Panichella, Sebastiano

AU - Gambi, Alessio

PY - 2020/6/27

Y1 - 2020/6/27

N2 - We report on the results of the eighth edition of the Java unit testing tool competition. This year, two tools, EvoSuite and Randoop, were executed on a benchmark with (i) new classes under test, selected from open-source software projects, and (ii) the set of classes from one project considered in the previous edition. We relied on an updated infrastructure for the execution of the different tools and the subsequent coverage and mutation analysis based on Docker containers. We considered two different time budgets for test case generation: one an three minutes. This paper describes our methodology and statistical analysis of the results, presents the results achieved by the contestant tools and highlights the challenges we faced during the competition.

AB - We report on the results of the eighth edition of the Java unit testing tool competition. This year, two tools, EvoSuite and Randoop, were executed on a benchmark with (i) new classes under test, selected from open-source software projects, and (ii) the set of classes from one project considered in the previous edition. We relied on an updated infrastructure for the execution of the different tools and the subsequent coverage and mutation analysis based on Docker containers. We considered two different time budgets for test case generation: one an three minutes. This paper describes our methodology and statistical analysis of the results, presents the results achieved by the contestant tools and highlights the challenges we faced during the competition.

KW - benchmark

KW - Java

KW - JUnit

KW - software testing

KW - test case generation

KW - tool competition

KW - unit testing

UR - http://www.scopus.com/inward/record.url?scp=85093076613&partnerID=8YFLogxK

U2 - 10.1145/3387940.3392265

DO - 10.1145/3387940.3392265

M3 - Conference contribution

AN - SCOPUS:85093076613

T3 - Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020

SP - 545

EP - 548

BT - Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020

PB - ACM Press

T2 - 42nd IEEE/ACM International Conference on Software Engineering Workshops, ICSEW 2020

Y2 - 27 June 2020 through 19 July 2020

ER -

Java Unit Testing Tool Competition - Eighth Round

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

JUGE: An infrastructure for benchmarking Java unit test generators

8th JUnit Testing Tool Competition

Cite this

Java Unit Testing Tool Competition - Eighth Round

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Research output

JUGE: An infrastructure for benchmarking Java unit test generators

Activities

8th JUnit Testing Tool Competition

Cite this