Effective and efficient API misuse detection via exception propagation and search-based testing

Maria Kechagia, Xavier Devroey, Annibale Panichella, Georgios Gousios, Arie van Deursen

Résultats de recherche: Contribution dans un livre/un catalogue/un rapport/dans les actes d'une conférenceArticle dans les actes d'une conférence/un colloque

Résumé

Application Programming Interfaces (APIs) typically come with (implicit) usage constraints. The violations of these constraints (API misuses) can lead to software crashes. Even though there are several tools that can detect API misuses, most of them suffer from a very high rate of false positives. We introduce Catcher, a novel API misuse detection approach that combines static exception propagation analysis with automatic search-based test case generation to effectively and efficiently pinpoint crash-prone API misuses in client applications. We validate Catcher against 21 Java applications, targeting misuses of the Java platform’s API. Our results indicate that Catcher is able to generate test cases that uncover 243 (unique) API misuses that result in crashes. Our empirical evaluation shows that Catcher can detect a large number of misuses (77 cases) that would remain undetected by the traditional coverage-based test case generator EvoSuite. Additionally, on average, Catcher is eight times faster than EvoSuite in generating test cases for the identified misuses. Finally, we find that the majority of the exceptions triggered by Catcher are unexpected to developers, i.e., not only unhandled in the source code but also not listed in the documentation of the client applications.

langue originaleAnglais
titreISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis
rédacteurs en chefAnders Moller, Dongmei Zhang
Lieu de publicationNew York, New York, USA
EditeurACM Press
Pages192-203
Nombre de pages12
ISBN (Electronique)9781450362245
ISBN (imprimé)9781450362245
Les DOIs
étatPublié - 10 juil. 2019
Modification externeOui

Série de publications

NomISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis

Empreinte digitale

Application programming interfaces (API)
Testing

Citer ceci

Kechagia, M., Devroey, X., Panichella, A., Gousios, G., & van Deursen, A. (2019). Effective and efficient API misuse detection via exception propagation and search-based testing. Dans A. Moller, & D. Zhang (eds.), ISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (p. 192-203). (ISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis). New York, New York, USA: ACM Press. https://doi.org/10.1145/3293882.3330552
Kechagia, Maria ; Devroey, Xavier ; Panichella, Annibale ; Gousios, Georgios ; van Deursen, Arie. / Effective and efficient API misuse detection via exception propagation and search-based testing. ISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. Editeur / Anders Moller ; Dongmei Zhang. New York, New York, USA : ACM Press, 2019. p. 192-203 (ISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis).
@inproceedings{92cd6f4041764b4182cec76748eddcc2,
title = "Effective and efficient API misuse detection via exception propagation and search-based testing",
abstract = "Application Programming Interfaces (APIs) typically come with (implicit) usage constraints. The violations of these constraints (API misuses) can lead to software crashes. Even though there are several tools that can detect API misuses, most of them suffer from a very high rate of false positives. We introduce Catcher, a novel API misuse detection approach that combines static exception propagation analysis with automatic search-based test case generation to effectively and efficiently pinpoint crash-prone API misuses in client applications. We validate Catcher against 21 Java applications, targeting misuses of the Java platform’s API. Our results indicate that Catcher is able to generate test cases that uncover 243 (unique) API misuses that result in crashes. Our empirical evaluation shows that Catcher can detect a large number of misuses (77 cases) that would remain undetected by the traditional coverage-based test case generator EvoSuite. Additionally, on average, Catcher is eight times faster than EvoSuite in generating test cases for the identified misuses. Finally, we find that the majority of the exceptions triggered by Catcher are unexpected to developers, i.e., not only unhandled in the source code but also not listed in the documentation of the client applications.",
keywords = "API misuse, Search-based software testing, Software crash, Static exception propagation",
author = "Maria Kechagia and Xavier Devroey and Annibale Panichella and Georgios Gousios and {van Deursen}, Arie",
year = "2019",
month = "7",
day = "10",
doi = "10.1145/3293882.3330552",
language = "English",
isbn = "9781450362245",
series = "ISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis",
publisher = "ACM Press",
pages = "192--203",
editor = "Anders Moller and Dongmei Zhang",
booktitle = "ISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis",
address = "United States",

}

Kechagia, M, Devroey, X, Panichella, A, Gousios, G & van Deursen, A 2019, Effective and efficient API misuse detection via exception propagation and search-based testing. Dans A Moller & D Zhang (eds), ISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. ISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, ACM Press, New York, New York, USA, p. 192-203. https://doi.org/10.1145/3293882.3330552

Effective and efficient API misuse detection via exception propagation and search-based testing. / Kechagia, Maria; Devroey, Xavier; Panichella, Annibale; Gousios, Georgios; van Deursen, Arie.

ISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. Ed. / Anders Moller; Dongmei Zhang. New York, New York, USA : ACM Press, 2019. p. 192-203 (ISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis).

Résultats de recherche: Contribution dans un livre/un catalogue/un rapport/dans les actes d'une conférenceArticle dans les actes d'une conférence/un colloque

TY - GEN

T1 - Effective and efficient API misuse detection via exception propagation and search-based testing

AU - Kechagia, Maria

AU - Devroey, Xavier

AU - Panichella, Annibale

AU - Gousios, Georgios

AU - van Deursen, Arie

PY - 2019/7/10

Y1 - 2019/7/10

N2 - Application Programming Interfaces (APIs) typically come with (implicit) usage constraints. The violations of these constraints (API misuses) can lead to software crashes. Even though there are several tools that can detect API misuses, most of them suffer from a very high rate of false positives. We introduce Catcher, a novel API misuse detection approach that combines static exception propagation analysis with automatic search-based test case generation to effectively and efficiently pinpoint crash-prone API misuses in client applications. We validate Catcher against 21 Java applications, targeting misuses of the Java platform’s API. Our results indicate that Catcher is able to generate test cases that uncover 243 (unique) API misuses that result in crashes. Our empirical evaluation shows that Catcher can detect a large number of misuses (77 cases) that would remain undetected by the traditional coverage-based test case generator EvoSuite. Additionally, on average, Catcher is eight times faster than EvoSuite in generating test cases for the identified misuses. Finally, we find that the majority of the exceptions triggered by Catcher are unexpected to developers, i.e., not only unhandled in the source code but also not listed in the documentation of the client applications.

AB - Application Programming Interfaces (APIs) typically come with (implicit) usage constraints. The violations of these constraints (API misuses) can lead to software crashes. Even though there are several tools that can detect API misuses, most of them suffer from a very high rate of false positives. We introduce Catcher, a novel API misuse detection approach that combines static exception propagation analysis with automatic search-based test case generation to effectively and efficiently pinpoint crash-prone API misuses in client applications. We validate Catcher against 21 Java applications, targeting misuses of the Java platform’s API. Our results indicate that Catcher is able to generate test cases that uncover 243 (unique) API misuses that result in crashes. Our empirical evaluation shows that Catcher can detect a large number of misuses (77 cases) that would remain undetected by the traditional coverage-based test case generator EvoSuite. Additionally, on average, Catcher is eight times faster than EvoSuite in generating test cases for the identified misuses. Finally, we find that the majority of the exceptions triggered by Catcher are unexpected to developers, i.e., not only unhandled in the source code but also not listed in the documentation of the client applications.

KW - API misuse

KW - Search-based software testing

KW - Software crash

KW - Static exception propagation

UR - http://dl.acm.org/citation.cfm?doid=3293882.3330552

UR - http://www.scopus.com/inward/record.url?scp=85070649482&partnerID=8YFLogxK

U2 - 10.1145/3293882.3330552

DO - 10.1145/3293882.3330552

M3 - Conference contribution

SN - 9781450362245

T3 - ISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis

SP - 192

EP - 203

BT - ISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis

A2 - Moller, Anders

A2 - Zhang, Dongmei

PB - ACM Press

CY - New York, New York, USA

ER -

Kechagia M, Devroey X, Panichella A, Gousios G, van Deursen A. Effective and efficient API misuse detection via exception propagation and search-based testing. Dans Moller A, Zhang D, rédacteurs en chef, ISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. New York, New York, USA: ACM Press. 2019. p. 192-203. (ISSTA 2019 - Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis). https://doi.org/10.1145/3293882.3330552