Statistical prioritization for software product line testing: an experience report

Research output: Contribution to journalArticle

90 Downloads (Pure)

Abstract

Software product lines (SPLs) are families of software systems sharing common assets and exhibiting variabilities specific to each product member of the family. Commonalities and variabilities are often represented as features organized in a feature model. Due to combinatorial explosion of the number of products induced by possible features combinations, exhaustive testing of SPLs is intractable. Therefore, sampling and prioritization techniques have been proposed to generate sorted lists of products based on coverage criteria or weights assigned to features. Solely based on the feature model, these techniques do not take into account behavioural usage of such products as a source of prioritization. In this paper, we assess the feasibility of integrating usage models into the testing process to derive statistical testing approaches for SPLs. Usage models are given as Markov chains, enabling prioritization of probable/rare behaviours. We used featured transition systems, compactly modelling variability and behaviour for SPLs, to determine which products are realizing prioritized behaviours. Statistical prioritization can achieve a significant reduction in the state space, and modelling efforts can be rewarded by better automation. In particular, we used MaTeLo, a statistical test cases generation suite developed at ALL4TEC. We assess feasibility criteria on two systems: Claroline, a configurable course management system, and Sferion™, an embedded system providing helicopter landing assistance.

Original languageEnglish
Pages (from-to)153-171
Number of pages19
JournalSoftware and Systems Modeling
Volume16
Issue number1
DOIs
Publication statusPublished - 1 Feb 2017

Fingerprint

Software Product Lines
Prioritization
Testing
Feature Model
Statistical tests
Landing
Helicopters
Embedded systems
Markov processes
Explosions
Transition Systems
Helicopter
Statistical test
Probable
Automation
System Modeling
Embedded Systems
Explosion
Software System
Sampling

Keywords

  • Prioritization
  • Software product line testing
  • Statistical testing

Cite this

@article{5191b711732e4a2bbedf24d8c7d0b53b,
title = "Statistical prioritization for software product line testing: an experience report",
abstract = "Software product lines (SPLs) are families of software systems sharing common assets and exhibiting variabilities specific to each product member of the family. Commonalities and variabilities are often represented as features organized in a feature model. Due to combinatorial explosion of the number of products induced by possible features combinations, exhaustive testing of SPLs is intractable. Therefore, sampling and prioritization techniques have been proposed to generate sorted lists of products based on coverage criteria or weights assigned to features. Solely based on the feature model, these techniques do not take into account behavioural usage of such products as a source of prioritization. In this paper, we assess the feasibility of integrating usage models into the testing process to derive statistical testing approaches for SPLs. Usage models are given as Markov chains, enabling prioritization of probable/rare behaviours. We used featured transition systems, compactly modelling variability and behaviour for SPLs, to determine which products are realizing prioritized behaviours. Statistical prioritization can achieve a significant reduction in the state space, and modelling efforts can be rewarded by better automation. In particular, we used MaTeLo, a statistical test cases generation suite developed at ALL4TEC. We assess feasibility criteria on two systems: Claroline, a configurable course management system, and Sferion™, an embedded system providing helicopter landing assistance.",
keywords = "Prioritization, Software product line testing, Statistical testing",
author = "Xavier Devroey and Gilles Perrouin and Maxime Cordy and Hamza Samih and Axel Legay and Schobbens, {Pierre Yves} and Patrick Heymans",
year = "2017",
month = "2",
day = "1",
doi = "10.1007/s10270-015-0479-8",
language = "English",
volume = "16",
pages = "153--171",
journal = "Software and Systems Modeling",
issn = "1619-1366",
publisher = "Springer Verlag",
number = "1",

}

TY - JOUR

T1 - Statistical prioritization for software product line testing: an experience report

AU - Devroey, Xavier

AU - Perrouin, Gilles

AU - Cordy, Maxime

AU - Samih, Hamza

AU - Legay, Axel

AU - Schobbens, Pierre Yves

AU - Heymans, Patrick

PY - 2017/2/1

Y1 - 2017/2/1

N2 - Software product lines (SPLs) are families of software systems sharing common assets and exhibiting variabilities specific to each product member of the family. Commonalities and variabilities are often represented as features organized in a feature model. Due to combinatorial explosion of the number of products induced by possible features combinations, exhaustive testing of SPLs is intractable. Therefore, sampling and prioritization techniques have been proposed to generate sorted lists of products based on coverage criteria or weights assigned to features. Solely based on the feature model, these techniques do not take into account behavioural usage of such products as a source of prioritization. In this paper, we assess the feasibility of integrating usage models into the testing process to derive statistical testing approaches for SPLs. Usage models are given as Markov chains, enabling prioritization of probable/rare behaviours. We used featured transition systems, compactly modelling variability and behaviour for SPLs, to determine which products are realizing prioritized behaviours. Statistical prioritization can achieve a significant reduction in the state space, and modelling efforts can be rewarded by better automation. In particular, we used MaTeLo, a statistical test cases generation suite developed at ALL4TEC. We assess feasibility criteria on two systems: Claroline, a configurable course management system, and Sferion™, an embedded system providing helicopter landing assistance.

AB - Software product lines (SPLs) are families of software systems sharing common assets and exhibiting variabilities specific to each product member of the family. Commonalities and variabilities are often represented as features organized in a feature model. Due to combinatorial explosion of the number of products induced by possible features combinations, exhaustive testing of SPLs is intractable. Therefore, sampling and prioritization techniques have been proposed to generate sorted lists of products based on coverage criteria or weights assigned to features. Solely based on the feature model, these techniques do not take into account behavioural usage of such products as a source of prioritization. In this paper, we assess the feasibility of integrating usage models into the testing process to derive statistical testing approaches for SPLs. Usage models are given as Markov chains, enabling prioritization of probable/rare behaviours. We used featured transition systems, compactly modelling variability and behaviour for SPLs, to determine which products are realizing prioritized behaviours. Statistical prioritization can achieve a significant reduction in the state space, and modelling efforts can be rewarded by better automation. In particular, we used MaTeLo, a statistical test cases generation suite developed at ALL4TEC. We assess feasibility criteria on two systems: Claroline, a configurable course management system, and Sferion™, an embedded system providing helicopter landing assistance.

KW - Prioritization

KW - Software product line testing

KW - Statistical testing

UR - http://www.scopus.com/inward/record.url?scp=84937933560&partnerID=8YFLogxK

U2 - 10.1007/s10270-015-0479-8

DO - 10.1007/s10270-015-0479-8

M3 - Article

VL - 16

SP - 153

EP - 171

JO - Software and Systems Modeling

JF - Software and Systems Modeling

SN - 1619-1366

IS - 1

ER -