Representing and Evaluating Strategies for Solving Parsons Puzzles
Parsons puzzles are popular for programming education. Identifying the strategies used by students solving Parsons puzzles is of interest because they can be used to determine to what extent students use the strategies typically associated with programming expertise, and to provide feedback and monitor the progress of students in a tutor. We propose solution sequence as an approximation of the student’s strategy for solving Parsons puzzles. It is scalable in terms of both the size of the puzzle and the number of students solving the puzzle. We propose BNF grammar to represent desirable puzzle-solving strategies associated with expert programmers. This representation is extensible and agnostic to the puzzle-solving strategies themselves. Finally, we propose a best match parser that matches a student’s solution sequence against the BNF grammar of a desirable strategy and quantifies the degree to which the student’s solution conforms to the desirable strategy. As a proof of concept, we used the parser to analyze the data collected by a Parsons puzzle tutor on if-else statements over five semesters and found a significant difference between C++ and Java puzzle-solvers in terms of their conformance to one desirable puzzle-solving strategy. Being able to tease out the effects of individual components of a strategy is one benefit of our approach: we found that relaxing shell-first constraint in the strategy resulted in significant improvement in the conformance of both C++ and Java students.
KeywordsParsons puzzle Puzzle-solving strategy Context free grammar Evaluation
Partial support for this work was provided by the National Science Foundation under grants DUE-1432190 and DUE-1502564. The author thanks Anthony Bucci for suggesting the use of BNF grammars and Alessio Gaspar, Paul Wiegand and Jennifer Albert for associated discussions.
- 2.Denny, P., Luxton-Reilly, A., Simon, B.: Evaluating a new exam question: Parsons problems. In: Proceedings of the Fourth International Workshop on Computing Education Research (ICER 2008), New York, NY, USA, pp. 113–124. ACM (2008)Google Scholar
- 3.Ericson, B.J., Margulieux, L.E., Rick, J.: Solving Parsons problems versus fixing and writing code. In: Proceedings of the 17th Koli Calling International Conference on Computing Education Research (Koli Calling 2017), New York, NY, USA, pp. 20–29. ACM (2017)Google Scholar
- 4.Ericson, B.J., Guzdial, M.J., Morrison, B.B.: Analysis of interactive features designed to enhance learning in an Ebook. In: Proceedings of the 11th annual International Conference on International Computing Education Research (ICER 2015), New York, NY, USA, pp. 169–178. ACM (2015)Google Scholar
- 5.Fabic, G., Mitrovic, A., Neshatian, K.: Towards a mobile Python tutor: understanding differences in strategies used by novices and experts. In: Micarelli, A. (ed.) ITS 2016. LNCS, vol. 9684, pp. 447–448. Springer, Heidelberg (2016)Google Scholar
- 6.Fix, V., Wiedenbeck, S., Scholtz, J.: Mental representations of programs by novices and experts. In: Proceedings of the INTERACT 1993 and CHI 1993 Conference on Human Factors in Computing Systems (CHI 1993), New York, NY, USA, pp. 74–79. ACM (1993)Google Scholar
- 7.Helminen, J., Ihantola, P., Karavirta, V., Malmi, L.: How do students solve parsons programming problems? An analysis of interaction traces. In: Proceedings of the Ninth Annual International Conference on International Computing Education Research (ICER 2012), New York, NY, USA, pp. 119–126. ACM (2012)Google Scholar
- 8.Ihantola, P., Karavirta, V.: Two-dimensional Parson’s puzzles: the concept, tools, and first observations. J. Inf. Technol. Educ.: Innov. Pract. 10, 1–14 (2011)Google Scholar
- 9.Kumar, A.N.: Epplets: a tool for solving parsons puzzles. In: Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE 2018), New York, NY, USA, pp. 527–532. ACM (2018)Google Scholar
- 10.Letovsky, S.: Cognitive processes in program comprehension. In: Soloway, E., Iyengar, S. (eds.) Empirical Studies of Programmers, pp. 58–79. Ablex, Norwood (1986)Google Scholar
- 11.Lopez, N., Whalley, J., Robbins, P., Lister, R.: Relationships between reading, tracing and writing skills in introductory programming. In: Proceedings of the 4th International Workshop on Computing Education Research (ICER 2008), New York, NY, USA, pp. 101–112. ACM (2008)Google Scholar
- 12.Nanja, M., Cook, C.R.: An analysis of the online debugging process. In: Olson, G.M., Sheppard, S., Soloway, E. (eds.) Empirical Studies of Programmers: Second Workshop, pp. 172–184. Ablex, Norwood (1987)Google Scholar
- 13.Parsons, D., Haden, P.: Parson’s programming puzzles: a fun and effective learning tool for first programming courses. In: Proceedings of the 8th Australasian Conference on Computing Education (ACE 2006), vol. 52, pp 157–163. Australian Computer Society, Inc. (2006)Google Scholar