Developing an Assessment to Profile Students based on their Understanding of the Variable Programming Concept

Novice programmers often hold misconceptions, and so incorrect mental models, that hinder their progress. These mental models have to be identified, with the help of a concept inventory, for the teachers to become aware of them and to provide appropriate intervention in the future. An assessment using a concept inventory to profile students based on their understanding of selected programming concepts was developed. The variable concept was chosen as a first case study to ensure the assessment's validity and reliability. During two academic years, two groups of students enrolled in a university introductory programming course were submitted to a concept inventory at four different times according to a precise schedule time. Measuring when students understand the concept of variable assignment and move from using incorrect mental models to use a single correct mental model, made it possible to define nine profiles among the 219 first-year students (validity). For each identified profile, the frequency distributions of students stayed consistent from one academic year to another (reliability). Finally, the value of such an assessment was briefly discussed with five teachers.