Covering SPL Behaviour with Sampled Configurations: An Initial Assessment

Research output: Contribution in Book/Catalog/Report/Conference proceedingConference contribution

173 Downloads (Pure)

Abstract

Structural approaches to Software Product Lines (SPL) testing (such as pairwise testing) have gained momentum as they are able to scale to larger SPLs described as feature diagrams (FD). However, these methods are agnostic with respect to behaviour: the sampled configurations have thus no reason to satisfy any given behavioural criterion. In this paper, we investigate the behavioural coverage of two structural testing criteria: pairwise and similarity. To do so, we modelled four SPLs in terms of feature diagrams and associated featured transitions systems (FTSs). We then computed state, action and transition coverage for a set of generated configurations. Preliminary results indicate that for relatively small variability models with few cross-tree constraints, structural coverage-driven tools tend to cover large parts of behaviour with less than 8 configurations. Though structural coverage cannot be used directly as a replacement for behavioural driven SPL test generation, opportunities to mix structural and behavioural coverage for efficient and effective SPL testing do exist.
Original languageEnglish
Title of host publicationProceedings of the Ninth International Workshop on Variability Modelling of Software-intensive Systems
Place of PublicationHildesheim, Germany
PublisherACM Press
Pages59:59-59:66
Number of pages8
Volume21-23-January-2015
ISBN (Print)9781450332736
DOIs
Publication statusPublished - 2015
Event9th International Workshop on Variability Modelling of Software-intensive Systems (VaMoS '15) - Hildesheim, Germany
Duration: 21 Jan 201523 Jan 2015

Publication series

NameVaMoS '15
PublisherACM

Conference

Conference9th International Workshop on Variability Modelling of Software-intensive Systems (VaMoS '15)
CountryGermany
CityHildesheim
Period21/01/1523/01/15

Fingerprint

Testing
Momentum

Keywords

  • Featured transition system
  • SPL testing
  • Structural coverage

Cite this

Devroey, X., Perrouin, G., Legay, A., Schobbens, P-Y., & Heymans, P. (2015). Covering SPL Behaviour with Sampled Configurations: An Initial Assessment. In Proceedings of the Ninth International Workshop on Variability Modelling of Software-intensive Systems (Vol. 21-23-January-2015, pp. 59:59-59:66). (VaMoS '15). Hildesheim, Germany: ACM Press. https://doi.org/10.1145/2701319.2701325
Devroey, Xavier ; Perrouin, Gilles ; Legay, Axel ; Schobbens, Pierre-Yves ; Heymans, Patrick. / Covering SPL Behaviour with Sampled Configurations: An Initial Assessment. Proceedings of the Ninth International Workshop on Variability Modelling of Software-intensive Systems. Vol. 21-23-January-2015 Hildesheim, Germany : ACM Press, 2015. pp. 59:59-59:66 (VaMoS '15).
@inproceedings{883691fe68c248e69cf303772826f68c,
title = "Covering SPL Behaviour with Sampled Configurations: An Initial Assessment",
abstract = "Structural approaches to Software Product Lines (SPL) testing (such as pairwise testing) have gained momentum as they are able to scale to larger SPLs described as feature diagrams (FD). However, these methods are agnostic with respect to behaviour: the sampled configurations have thus no reason to satisfy any given behavioural criterion. In this paper, we investigate the behavioural coverage of two structural testing criteria: pairwise and similarity. To do so, we modelled four SPLs in terms of feature diagrams and associated featured transitions systems (FTSs). We then computed state, action and transition coverage for a set of generated configurations. Preliminary results indicate that for relatively small variability models with few cross-tree constraints, structural coverage-driven tools tend to cover large parts of behaviour with less than 8 configurations. Though structural coverage cannot be used directly as a replacement for behavioural driven SPL test generation, opportunities to mix structural and behavioural coverage for efficient and effective SPL testing do exist.",
keywords = "Featured transition system, SPL testing, Structural coverage",
author = "Xavier Devroey and Gilles Perrouin and Axel Legay and Pierre-Yves Schobbens and Patrick Heymans",
year = "2015",
doi = "10.1145/2701319.2701325",
language = "English",
isbn = "9781450332736",
volume = "21-23-January-2015",
series = "VaMoS '15",
publisher = "ACM Press",
pages = "59:59--59:66",
booktitle = "Proceedings of the Ninth International Workshop on Variability Modelling of Software-intensive Systems",
address = "United States",

}

Devroey, X, Perrouin, G, Legay, A, Schobbens, P-Y & Heymans, P 2015, Covering SPL Behaviour with Sampled Configurations: An Initial Assessment. in Proceedings of the Ninth International Workshop on Variability Modelling of Software-intensive Systems. vol. 21-23-January-2015, VaMoS '15, ACM Press, Hildesheim, Germany, pp. 59:59-59:66, 9th International Workshop on Variability Modelling of Software-intensive Systems (VaMoS '15), Hildesheim, Germany, 21/01/15. https://doi.org/10.1145/2701319.2701325

Covering SPL Behaviour with Sampled Configurations: An Initial Assessment. / Devroey, Xavier; Perrouin, Gilles; Legay, Axel; Schobbens, Pierre-Yves; Heymans, Patrick.

Proceedings of the Ninth International Workshop on Variability Modelling of Software-intensive Systems. Vol. 21-23-January-2015 Hildesheim, Germany : ACM Press, 2015. p. 59:59-59:66 (VaMoS '15).

Research output: Contribution in Book/Catalog/Report/Conference proceedingConference contribution

TY - GEN

T1 - Covering SPL Behaviour with Sampled Configurations: An Initial Assessment

AU - Devroey, Xavier

AU - Perrouin, Gilles

AU - Legay, Axel

AU - Schobbens, Pierre-Yves

AU - Heymans, Patrick

PY - 2015

Y1 - 2015

N2 - Structural approaches to Software Product Lines (SPL) testing (such as pairwise testing) have gained momentum as they are able to scale to larger SPLs described as feature diagrams (FD). However, these methods are agnostic with respect to behaviour: the sampled configurations have thus no reason to satisfy any given behavioural criterion. In this paper, we investigate the behavioural coverage of two structural testing criteria: pairwise and similarity. To do so, we modelled four SPLs in terms of feature diagrams and associated featured transitions systems (FTSs). We then computed state, action and transition coverage for a set of generated configurations. Preliminary results indicate that for relatively small variability models with few cross-tree constraints, structural coverage-driven tools tend to cover large parts of behaviour with less than 8 configurations. Though structural coverage cannot be used directly as a replacement for behavioural driven SPL test generation, opportunities to mix structural and behavioural coverage for efficient and effective SPL testing do exist.

AB - Structural approaches to Software Product Lines (SPL) testing (such as pairwise testing) have gained momentum as they are able to scale to larger SPLs described as feature diagrams (FD). However, these methods are agnostic with respect to behaviour: the sampled configurations have thus no reason to satisfy any given behavioural criterion. In this paper, we investigate the behavioural coverage of two structural testing criteria: pairwise and similarity. To do so, we modelled four SPLs in terms of feature diagrams and associated featured transitions systems (FTSs). We then computed state, action and transition coverage for a set of generated configurations. Preliminary results indicate that for relatively small variability models with few cross-tree constraints, structural coverage-driven tools tend to cover large parts of behaviour with less than 8 configurations. Though structural coverage cannot be used directly as a replacement for behavioural driven SPL test generation, opportunities to mix structural and behavioural coverage for efficient and effective SPL testing do exist.

KW - Featured transition system

KW - SPL testing

KW - Structural coverage

UR - http://www.scopus.com/inward/record.url?scp=84961237579&partnerID=8YFLogxK

U2 - 10.1145/2701319.2701325

DO - 10.1145/2701319.2701325

M3 - Conference contribution

SN - 9781450332736

VL - 21-23-January-2015

T3 - VaMoS '15

SP - 59:59-59:66

BT - Proceedings of the Ninth International Workshop on Variability Modelling of Software-intensive Systems

PB - ACM Press

CY - Hildesheim, Germany

ER -

Devroey X, Perrouin G, Legay A, Schobbens P-Y, Heymans P. Covering SPL Behaviour with Sampled Configurations: An Initial Assessment. In Proceedings of the Ninth International Workshop on Variability Modelling of Software-intensive Systems. Vol. 21-23-January-2015. Hildesheim, Germany: ACM Press. 2015. p. 59:59-59:66. (VaMoS '15). https://doi.org/10.1145/2701319.2701325