Quizzes in Elementary-Level Visual Programming: Synthesis Methods and Pedagogical Utility
Ahana Ghosh
Max-Planck-Institut for Software System
01 Sep 2026, 1:00 pm - 2:00 pm
Saarbrücken building E1 5, room 029
SWS Student Defense Talks - Thesis Defense
Block-based visual programming initiatives such as Hour of Code by code.org and
Intro to Programming with Karel by CodeHS.com, have transformed introductory
computer science education by making programming more accessible to K-8
learners. Despite their accessibility, students often struggle with multi-step
reasoning and conceptual abstraction when solving open-ended tasks. Quizzes
(such as fill-in-the-gap exercises, and multiple-choice conceptual questions
based on code debugging and task design) offer interactive practice and
targeted feedback that can promote active learning and scaffold novice
programmers. ...
Block-based visual programming initiatives such as Hour of Code by code.org and
Intro to Programming with Karel by CodeHS.com, have transformed introductory
computer science education by making programming more accessible to K-8
learners. Despite their accessibility, students often struggle with multi-step
reasoning and conceptual abstraction when solving open-ended tasks. Quizzes
(such as fill-in-the-gap exercises, and multiple-choice conceptual questions
based on code debugging and task design) offer interactive practice and
targeted feedback that can promote active learning and scaffold novice
programmers. However, manually designing such quizzes is time-consuming and
difficult to scale. This thesis tackles these challenges by developing
automated synthesis techniques for programming tasks and quizzes, and evaluates
their pedagogical utility.
The first part of the thesis introduces algorithmic methods for synthesizing programming tasks and quizzes in block-based environments. Specifically, we develop methods for the following : (i) synthesizing conceptually similar and yet visually dissimilar write-code tasks; and (ii) synthesizing adaptive multiple-choice programming quizzes that address student-specific misconceptions; Each method leverages symbolic execution, sketch-based code mutation, and search-guided generation to ensure pedagogical utility, relevance, and technical correctness. Empirical evaluations conducted through controlled user studies demonstrate the efficacy of these approaches, showing that they not only support novice learners effectively but also outperform existing methods, including next-step code edit based feedback methods.
The second part of the thesis empirically evaluates the pedagogical utility of programming quizzes in these environments via user studies and classroom deployments with K-8 learners. Specifically, we examine: (i) the design, validation, and classification of quiz types using cognitive frameworks such as Bloom's Revised Taxonomy; (ii) the impact of embedding quizzes within programming curricula on post-learning outcomes; and (iii) the effectiveness of quiz-based feedback scaffolds with different quiz-types. Our findings show that quizzes designed using metacognitive strategies and adapted to learners’ attempts significantly enhance engagement and task performance. Moreover, we observe that richer and more diverse quiz types—when integrated into the curriculum—lead to improved post-learning outcomes, while simpler, less cognitively demanding quizzes may hinder post-learning performance.
Overall, this thesis contributes novel synthesis methods for programming quizzes and empirical evidence of their effectiveness in elementary-level programming education. These findings provide a foundation for scalable and adaptive support in elementary computing curricula.
Read more
The first part of the thesis introduces algorithmic methods for synthesizing programming tasks and quizzes in block-based environments. Specifically, we develop methods for the following : (i) synthesizing conceptually similar and yet visually dissimilar write-code tasks; and (ii) synthesizing adaptive multiple-choice programming quizzes that address student-specific misconceptions; Each method leverages symbolic execution, sketch-based code mutation, and search-guided generation to ensure pedagogical utility, relevance, and technical correctness. Empirical evaluations conducted through controlled user studies demonstrate the efficacy of these approaches, showing that they not only support novice learners effectively but also outperform existing methods, including next-step code edit based feedback methods.
The second part of the thesis empirically evaluates the pedagogical utility of programming quizzes in these environments via user studies and classroom deployments with K-8 learners. Specifically, we examine: (i) the design, validation, and classification of quiz types using cognitive frameworks such as Bloom's Revised Taxonomy; (ii) the impact of embedding quizzes within programming curricula on post-learning outcomes; and (iii) the effectiveness of quiz-based feedback scaffolds with different quiz-types. Our findings show that quizzes designed using metacognitive strategies and adapted to learners’ attempts significantly enhance engagement and task performance. Moreover, we observe that richer and more diverse quiz types—when integrated into the curriculum—lead to improved post-learning outcomes, while simpler, less cognitively demanding quizzes may hinder post-learning performance.
Overall, this thesis contributes novel synthesis methods for programming quizzes and empirical evidence of their effectiveness in elementary-level programming education. These findings provide a foundation for scalable and adaptive support in elementary computing curricula.