Towards Quality Assurance of Software Product Lines with Adversarial Configurations

Paul Temple, Mathieu Acher, Gilles Perrouin, Battista Biggio, Jean-Marc Jézéquel, Fabio Roli

Research output: Contribution in Book/Catalog/Report/Conference proceedingConference contribution

1 Downloads (Pure)

Abstract

Software product line (SPL) engineers put a lot of effort to ensure that, through the setting of a large number of possible configuration options, products are acceptable and well-tailored to customers' needs. Unfortunately, options and their mutual interactions create a huge configuration space which is intractable to exhaustively explore. Instead of testing all products, machine learning is increasingly employed to approximate the set of acceptable products out of a small training sample of configurations. Machine learning (ML) techniques can refine a software product line through learned constraints and a priori prevent non-acceptable products to be derived. In this paper, we use adversarial ML techniques to generate \textit{adversarial configurations} fooling ML classifiers and pinpoint incorrect classifications of products (videos) derived from an industrial video generator. Our attacks yield (up to) a 100% misclassification rate and a drop in accuracy of 5%. We discuss the implications these results have on SPL quality assurance.
Original languageEnglish
Title of host publication23rd International Systems and Software Product Line Conference
PublisherACM Press
Publication statusPublished - 2019
Event23rd Systems and Software Product Line Conference - Paris, Paris, France
Duration: 9 Sep 201913 Sep 2019
https://splc2019.net/

Conference

Conference23rd Systems and Software Product Line Conference
Abbreviated titleSPLC 2019
CountryFrance
CityParis
Period9/09/1913/09/19
Internet address

Fingerprint

Quality assurance
Learning systems
Classifiers
Engineers
Testing

Cite this

Temple, P., Acher, M., Perrouin, G., Biggio, B., Jézéquel, J-M., & Roli, F. (2019). Towards Quality Assurance of Software Product Lines with Adversarial Configurations. In 23rd International Systems and Software Product Line Conference ACM Press.
Temple, Paul ; Acher, Mathieu ; Perrouin, Gilles ; Biggio, Battista ; Jézéquel, Jean-Marc ; Roli, Fabio. / Towards Quality Assurance of Software Product Lines with Adversarial Configurations. 23rd International Systems and Software Product Line Conference. ACM Press, 2019.
@inproceedings{2f96d410ebed471680341428c93ed580,
title = "Towards Quality Assurance of Software Product Lines with Adversarial Configurations",
abstract = "Software product line (SPL) engineers put a lot of effort to ensure that, through the setting of a large number of possible configuration options, products are acceptable and well-tailored to customers' needs. Unfortunately, options and their mutual interactions create a huge configuration space which is intractable to exhaustively explore. Instead of testing all products, machine learning is increasingly employed to approximate the set of acceptable products out of a small training sample of configurations. Machine learning (ML) techniques can refine a software product line through learned constraints and a priori prevent non-acceptable products to be derived. In this paper, we use adversarial ML techniques to generate \textit{adversarial configurations} fooling ML classifiers and pinpoint incorrect classifications of products (videos) derived from an industrial video generator. Our attacks yield (up to) a 100{\%} misclassification rate and a drop in accuracy of 5{\%}. We discuss the implications these results have on SPL quality assurance.",
author = "Paul Temple and Mathieu Acher and Gilles Perrouin and Battista Biggio and Jean-Marc J{\'e}z{\'e}quel and Fabio Roli",
year = "2019",
language = "English",
booktitle = "23rd International Systems and Software Product Line Conference",
publisher = "ACM Press",
address = "United States",

}

Temple, P, Acher, M, Perrouin, G, Biggio, B, Jézéquel, J-M & Roli, F 2019, Towards Quality Assurance of Software Product Lines with Adversarial Configurations. in 23rd International Systems and Software Product Line Conference. ACM Press, 23rd Systems and Software Product Line Conference, Paris, France, 9/09/19.

Towards Quality Assurance of Software Product Lines with Adversarial Configurations. / Temple, Paul; Acher, Mathieu; Perrouin, Gilles; Biggio, Battista; Jézéquel, Jean-Marc; Roli, Fabio.

23rd International Systems and Software Product Line Conference. ACM Press, 2019.

Research output: Contribution in Book/Catalog/Report/Conference proceedingConference contribution

TY - GEN

T1 - Towards Quality Assurance of Software Product Lines with Adversarial Configurations

AU - Temple, Paul

AU - Acher, Mathieu

AU - Perrouin, Gilles

AU - Biggio, Battista

AU - Jézéquel, Jean-Marc

AU - Roli, Fabio

PY - 2019

Y1 - 2019

N2 - Software product line (SPL) engineers put a lot of effort to ensure that, through the setting of a large number of possible configuration options, products are acceptable and well-tailored to customers' needs. Unfortunately, options and their mutual interactions create a huge configuration space which is intractable to exhaustively explore. Instead of testing all products, machine learning is increasingly employed to approximate the set of acceptable products out of a small training sample of configurations. Machine learning (ML) techniques can refine a software product line through learned constraints and a priori prevent non-acceptable products to be derived. In this paper, we use adversarial ML techniques to generate \textit{adversarial configurations} fooling ML classifiers and pinpoint incorrect classifications of products (videos) derived from an industrial video generator. Our attacks yield (up to) a 100% misclassification rate and a drop in accuracy of 5%. We discuss the implications these results have on SPL quality assurance.

AB - Software product line (SPL) engineers put a lot of effort to ensure that, through the setting of a large number of possible configuration options, products are acceptable and well-tailored to customers' needs. Unfortunately, options and their mutual interactions create a huge configuration space which is intractable to exhaustively explore. Instead of testing all products, machine learning is increasingly employed to approximate the set of acceptable products out of a small training sample of configurations. Machine learning (ML) techniques can refine a software product line through learned constraints and a priori prevent non-acceptable products to be derived. In this paper, we use adversarial ML techniques to generate \textit{adversarial configurations} fooling ML classifiers and pinpoint incorrect classifications of products (videos) derived from an industrial video generator. Our attacks yield (up to) a 100% misclassification rate and a drop in accuracy of 5%. We discuss the implications these results have on SPL quality assurance.

M3 - Conference contribution

BT - 23rd International Systems and Software Product Line Conference

PB - ACM Press

ER -

Temple P, Acher M, Perrouin G, Biggio B, Jézéquel J-M, Roli F. Towards Quality Assurance of Software Product Lines with Adversarial Configurations. In 23rd International Systems and Software Product Line Conference. ACM Press. 2019