Towards qality assurance of software product lines with adversarial configurations

Paul Temple, Mathieu Acher, Gilles Perrouin, Battista Biggio, Jean Marc Jézéquel, Fabio Roli

Résultats de recherche: Contribution dans un livre/un catalogue/un rapport/dans les actes d'une conférenceArticle dans les actes d'une conférence/un colloque

Résumé

Software product line (SPL) engineers put a lot of effort to ensure that, through the setting of a large number of possible configuration options, products are acceptable and well-tailored to customers' needs. Unfortunately, options and their mutual interactions create a huge configuration space which is intractable to exhaustively explore. Instead of testing all products, machine learning is increasingly employed to approximate the set of acceptable products out of a small training sample of configurations. Machine learning (ML) techniques can refine a software product line through learned constraints and a priori prevent non-acceptable products to be derived. In this paper, we use adversarial ML techniques to generate adversarial configurations fooling ML classifiers and pinpoint incorrect classifications of products (videos) derived from an industrial video generator. Our attacks yield (up to) a 100% misclassification rate and a drop in accuracy of 5%. We discuss the implications these results have on SPL quality assurance.

langue originaleAnglais
titreSPLC 2019 - 23rd International Systems and Software Product Line Conference
rédacteurs en chefThorsten Berger, Philippe Collet, Laurence Duchien, Thomas Fogdal, Patrick Heymans, Timo Kehrer, Jabier Martinez, Raul Mazo, Leticia Montalvillo, Camille Salinesi, Xhevahire Ternava, Thomas Thum, Tewfik Ziadi
EditeurACM Press
ISBN (Electronique)9781450371384
Les DOIs
étatPublié - 9 sept. 2019
Evénement23rd International Systems and Software Product Line Conference, SPLC 2019, co-located with the 13th European Conference on Software Architecture, ECSA 2019 - Paris, France
Durée: 9 sept. 201913 sept. 2019

Série de publications

NomACM International Conference Proceeding Series
VolumeA

Une conférence

Une conférence23rd International Systems and Software Product Line Conference, SPLC 2019, co-located with the 13th European Conference on Software Architecture, ECSA 2019
PaysFrance
La villeParis
période9/09/1913/09/19

Empreinte digitale

Learning systems
Quality assurance
Classifiers
Engineers
Testing

Citer ceci

Temple, P., Acher, M., Perrouin, G., Biggio, B., Jézéquel, J. M., & Roli, F. (2019). Towards qality assurance of software product lines with adversarial configurations. Dans T. Berger, P. Collet, L. Duchien, T. Fogdal, P. Heymans, T. Kehrer, J. Martinez, R. Mazo, L. Montalvillo, C. Salinesi, X. Ternava, T. Thum, ... T. Ziadi (eds.), SPLC 2019 - 23rd International Systems and Software Product Line Conference (ACM International Conference Proceeding Series; Vol A). ACM Press. https://doi.org/10.1145/3336294.3336309
Temple, Paul ; Acher, Mathieu ; Perrouin, Gilles ; Biggio, Battista ; Jézéquel, Jean Marc ; Roli, Fabio. / Towards qality assurance of software product lines with adversarial configurations. SPLC 2019 - 23rd International Systems and Software Product Line Conference. Editeur / Thorsten Berger ; Philippe Collet ; Laurence Duchien ; Thomas Fogdal ; Patrick Heymans ; Timo Kehrer ; Jabier Martinez ; Raul Mazo ; Leticia Montalvillo ; Camille Salinesi ; Xhevahire Ternava ; Thomas Thum ; Tewfik Ziadi. ACM Press, 2019. (ACM International Conference Proceeding Series).
@inproceedings{1b4dc6fad1ab4dbc8517466e2356a51c,
title = "Towards qality assurance of software product lines with adversarial configurations",
abstract = "Software product line (SPL) engineers put a lot of effort to ensure that, through the setting of a large number of possible configuration options, products are acceptable and well-tailored to customers' needs. Unfortunately, options and their mutual interactions create a huge configuration space which is intractable to exhaustively explore. Instead of testing all products, machine learning is increasingly employed to approximate the set of acceptable products out of a small training sample of configurations. Machine learning (ML) techniques can refine a software product line through learned constraints and a priori prevent non-acceptable products to be derived. In this paper, we use adversarial ML techniques to generate adversarial configurations fooling ML classifiers and pinpoint incorrect classifications of products (videos) derived from an industrial video generator. Our attacks yield (up to) a 100{\%} misclassification rate and a drop in accuracy of 5{\%}. We discuss the implications these results have on SPL quality assurance.",
keywords = "Machine learning, Quality assurance, Software product line, Software testing, Software variability",
author = "Paul Temple and Mathieu Acher and Gilles Perrouin and Battista Biggio and J{\'e}z{\'e}quel, {Jean Marc} and Fabio Roli",
year = "2019",
month = "9",
day = "9",
doi = "10.1145/3336294.3336309",
language = "English",
series = "ACM International Conference Proceeding Series",
publisher = "ACM Press",
editor = "Thorsten Berger and Philippe Collet and Laurence Duchien and Thomas Fogdal and Patrick Heymans and Timo Kehrer and Jabier Martinez and Raul Mazo and Leticia Montalvillo and Camille Salinesi and Xhevahire Ternava and Thomas Thum and Tewfik Ziadi",
booktitle = "SPLC 2019 - 23rd International Systems and Software Product Line Conference",
address = "United States",

}

Temple, P, Acher, M, Perrouin, G, Biggio, B, Jézéquel, JM & Roli, F 2019, Towards qality assurance of software product lines with adversarial configurations. Dans T Berger, P Collet, L Duchien, T Fogdal, P Heymans, T Kehrer, J Martinez, R Mazo, L Montalvillo, C Salinesi, X Ternava, T Thum & T Ziadi (eds), SPLC 2019 - 23rd International Systems and Software Product Line Conference. ACM International Conference Proceeding Series, VOL. A, ACM Press, 23rd International Systems and Software Product Line Conference, SPLC 2019, co-located with the 13th European Conference on Software Architecture, ECSA 2019, Paris, France, 9/09/19. https://doi.org/10.1145/3336294.3336309

Towards qality assurance of software product lines with adversarial configurations. / Temple, Paul; Acher, Mathieu; Perrouin, Gilles; Biggio, Battista; Jézéquel, Jean Marc; Roli, Fabio.

SPLC 2019 - 23rd International Systems and Software Product Line Conference. Ed. / Thorsten Berger; Philippe Collet; Laurence Duchien; Thomas Fogdal; Patrick Heymans; Timo Kehrer; Jabier Martinez; Raul Mazo; Leticia Montalvillo; Camille Salinesi; Xhevahire Ternava; Thomas Thum; Tewfik Ziadi. ACM Press, 2019. (ACM International Conference Proceeding Series; Vol A).

Résultats de recherche: Contribution dans un livre/un catalogue/un rapport/dans les actes d'une conférenceArticle dans les actes d'une conférence/un colloque

TY - GEN

T1 - Towards qality assurance of software product lines with adversarial configurations

AU - Temple, Paul

AU - Acher, Mathieu

AU - Perrouin, Gilles

AU - Biggio, Battista

AU - Jézéquel, Jean Marc

AU - Roli, Fabio

PY - 2019/9/9

Y1 - 2019/9/9

N2 - Software product line (SPL) engineers put a lot of effort to ensure that, through the setting of a large number of possible configuration options, products are acceptable and well-tailored to customers' needs. Unfortunately, options and their mutual interactions create a huge configuration space which is intractable to exhaustively explore. Instead of testing all products, machine learning is increasingly employed to approximate the set of acceptable products out of a small training sample of configurations. Machine learning (ML) techniques can refine a software product line through learned constraints and a priori prevent non-acceptable products to be derived. In this paper, we use adversarial ML techniques to generate adversarial configurations fooling ML classifiers and pinpoint incorrect classifications of products (videos) derived from an industrial video generator. Our attacks yield (up to) a 100% misclassification rate and a drop in accuracy of 5%. We discuss the implications these results have on SPL quality assurance.

AB - Software product line (SPL) engineers put a lot of effort to ensure that, through the setting of a large number of possible configuration options, products are acceptable and well-tailored to customers' needs. Unfortunately, options and their mutual interactions create a huge configuration space which is intractable to exhaustively explore. Instead of testing all products, machine learning is increasingly employed to approximate the set of acceptable products out of a small training sample of configurations. Machine learning (ML) techniques can refine a software product line through learned constraints and a priori prevent non-acceptable products to be derived. In this paper, we use adversarial ML techniques to generate adversarial configurations fooling ML classifiers and pinpoint incorrect classifications of products (videos) derived from an industrial video generator. Our attacks yield (up to) a 100% misclassification rate and a drop in accuracy of 5%. We discuss the implications these results have on SPL quality assurance.

KW - Machine learning

KW - Quality assurance

KW - Software product line

KW - Software testing

KW - Software variability

UR - http://www.scopus.com/inward/record.url?scp=85072892794&partnerID=8YFLogxK

U2 - 10.1145/3336294.3336309

DO - 10.1145/3336294.3336309

M3 - Conference contribution

AN - SCOPUS:85072892794

T3 - ACM International Conference Proceeding Series

BT - SPLC 2019 - 23rd International Systems and Software Product Line Conference

A2 - Berger, Thorsten

A2 - Collet, Philippe

A2 - Duchien, Laurence

A2 - Fogdal, Thomas

A2 - Heymans, Patrick

A2 - Kehrer, Timo

A2 - Martinez, Jabier

A2 - Mazo, Raul

A2 - Montalvillo, Leticia

A2 - Salinesi, Camille

A2 - Ternava, Xhevahire

A2 - Thum, Thomas

A2 - Ziadi, Tewfik

PB - ACM Press

ER -

Temple P, Acher M, Perrouin G, Biggio B, Jézéquel JM, Roli F. Towards qality assurance of software product lines with adversarial configurations. Dans Berger T, Collet P, Duchien L, Fogdal T, Heymans P, Kehrer T, Martinez J, Mazo R, Montalvillo L, Salinesi C, Ternava X, Thum T, Ziadi T, rédacteurs en chef, SPLC 2019 - 23rd International Systems and Software Product Line Conference. ACM Press. 2019. (ACM International Conference Proceeding Series). https://doi.org/10.1145/3336294.3336309