Towards Automated Testing for Simple Programming Exercises

Pierre Ortegat; Benoit Vanderose; Xavier Devroey

doi:10.1145/3548660.3561334

Towards Automated Testing for Simple Programming Exercises

Pierre Ortegat, Benoit Vanderose, Xavier Devroey

Research output: Contribution in Book/Catalog/Report/Conference proceeding › Conference contribution

35 Downloads (Pure)

Abstract

Automated feedback and grading platforms can require substantial effort when encoding new programming exercises for first-year students. Such exercises are usually simple but require defining several test cases to ensure their functional correctness. This paper describes our initial effort to leverage automated test case generation for simple programming exercises. We rely on grey-box fuzzing and random combinations of method calls to test the students' solutions and compare their execution to the results produced by a reference implementation. We implemented our approach in a pro- totype, called SimPyTest, openly available on GitHub. We discuss its usage and possible future extensions.

Original language	English
Title of host publication	EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022
Editors	Andreea Vescan, Camelia Serban, Julie Henry, Upsorn Praphamontripong
Place of Publication	Singapore
Publisher	ACM Press
Pages	33-36
Number of pages	4
ISBN (Electronic)	9781450394536
DOIs	https://doi.org/10.1145/3548660.3561334
Publication status	Published - 7 Nov 2022
Event	4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence - Singapore, Singapore Duration: 18 Nov 2022 → 18 Nov 2022 Conference number: 4 https://easeai.github.io

Publication series

Name	EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022

Workshop

Workshop	4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence
Abbreviated title	EASEAI '22
Country/Territory	Singapore
City	Singapore
Period	18/11/22 → 18/11/22
Internet address	https://easeai.github.io

Keywords

programming education
automated software testing
fuzzing

Access to Document

10.1145/3548660.3561334

EASEAI2022-authorversionAccepted author manuscript, 935 KB

Cite this

Ortegat, P., Vanderose, B., & Devroey, X. (2022). Towards Automated Testing for Simple Programming Exercises. In A. Vescan, C. Serban, J. Henry, & U. Praphamontripong (Eds.), EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022 (pp. 33-36). (EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022). ACM Press. https://doi.org/10.1145/3548660.3561334

Ortegat, Pierre ; Vanderose, Benoit ; Devroey, Xavier. / Towards Automated Testing for Simple Programming Exercises. EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022. editor / Andreea Vescan ; Camelia Serban ; Julie Henry ; Upsorn Praphamontripong. Singapore : ACM Press, 2022. pp. 33-36 (EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022).

@inproceedings{17fa5e58b90d444583c3aa275cb3a96d,

title = "Towards Automated Testing for Simple Programming Exercises",

abstract = "Automated feedback and grading platforms can require substantial effort when encoding new programming exercises for first-year students. Such exercises are usually simple but require defining several test cases to ensure their functional correctness. This paper describes our initial effort to leverage automated test case generation for simple programming exercises. We rely on grey-box fuzzing and random combinations of method calls to test the students' solutions and compare their execution to the results produced by a reference implementation. We implemented our approach in a pro- totype, called SimPyTest, openly available on GitHub. We discuss its usage and possible future extensions.",

keywords = "programming education, automated software testing, fuzzing",

author = "Pierre Ortegat and Benoit Vanderose and Xavier Devroey",

note = "Funding Information: This research was partially funded by the CyberExcellence (No. 2110186) project, funded by the Service Public de Wallonie (SPW Recherche). Publisher Copyright: {\textcopyright} 2022 ACM.; 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, EASEAI '22 ; Conference date: 18-11-2022 Through 18-11-2022",

year = "2022",

month = nov,

day = "7",

doi = "10.1145/3548660.3561334",

language = "English",

series = "EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022",

publisher = "ACM Press",

pages = "33--36",

editor = "Andreea Vescan and Camelia Serban and Julie Henry and Upsorn Praphamontripong",

booktitle = "EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022",

address = "United States",

url = "https://easeai.github.io",

}

Ortegat, P, Vanderose, B & Devroey, X 2022, Towards Automated Testing for Simple Programming Exercises. in A Vescan, C Serban, J Henry & U Praphamontripong (eds), EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022. EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022, ACM Press, Singapore, pp. 33-36, 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, Singapore, Singapore, 18/11/22. https://doi.org/10.1145/3548660.3561334

Towards Automated Testing for Simple Programming Exercises. / Ortegat, Pierre; Vanderose, Benoit ; Devroey, Xavier.
EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022. ed. / Andreea Vescan; Camelia Serban; Julie Henry; Upsorn Praphamontripong. Singapore: ACM Press, 2022. p. 33-36 (EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022).

Research output: Contribution in Book/Catalog/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Towards Automated Testing for Simple Programming Exercises

AU - Ortegat, Pierre

AU - Vanderose, Benoit

AU - Devroey, Xavier

N1 - Conference code: 4

PY - 2022/11/7

Y1 - 2022/11/7

N2 - Automated feedback and grading platforms can require substantial effort when encoding new programming exercises for first-year students. Such exercises are usually simple but require defining several test cases to ensure their functional correctness. This paper describes our initial effort to leverage automated test case generation for simple programming exercises. We rely on grey-box fuzzing and random combinations of method calls to test the students' solutions and compare their execution to the results produced by a reference implementation. We implemented our approach in a pro- totype, called SimPyTest, openly available on GitHub. We discuss its usage and possible future extensions.

AB - Automated feedback and grading platforms can require substantial effort when encoding new programming exercises for first-year students. Such exercises are usually simple but require defining several test cases to ensure their functional correctness. This paper describes our initial effort to leverage automated test case generation for simple programming exercises. We rely on grey-box fuzzing and random combinations of method calls to test the students' solutions and compare their execution to the results produced by a reference implementation. We implemented our approach in a pro- totype, called SimPyTest, openly available on GitHub. We discuss its usage and possible future extensions.

KW - programming education

KW - automated software testing

KW - fuzzing

UR - http://www.scopus.com/inward/record.url?scp=85142928749&partnerID=8YFLogxK

U2 - 10.1145/3548660.3561334

DO - 10.1145/3548660.3561334

M3 - Conference contribution

T3 - EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022

SP - 33

EP - 36

BT - EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022

A2 - Vescan, Andreea

A2 - Serban, Camelia

A2 - Henry, Julie

A2 - Praphamontripong, Upsorn

PB - ACM Press

CY - Singapore

T2 - 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence

Y2 - 18 November 2022 through 18 November 2022

ER -

Ortegat P, Vanderose B , Devroey X. Towards Automated Testing for Simple Programming Exercises. In Vescan A, Serban C, Henry J, Praphamontripong U, editors, EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022. Singapore: ACM Press. 2022. p. 33-36. (EASEAI 2022 - Proceedings of the 4th International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, co-located with ESEC/FSE 2022). doi: 10.1145/3548660.3561334

Towards Automated Testing for Simple Programming Exercises

Abstract

Publication series

Workshop

Keywords

Access to Document

Other files and links

Fingerprint

CYBEREXCELLENCE: The project of excellence in cyber security within the framework of the plan of the Walloon Region (CyberWal)

Cite this

Towards Automated Testing for Simple Programming Exercises

Abstract

Publication series

Workshop

Keywords

Access to Document

Other files and links

Fingerprint

Projects

CYBEREXCELLENCE: The project of excellence in cyber security within the framework of the plan of the Walloon Region (CyberWal)

Cite this