Publication
LLM-generated tips rival expert-created tips in helping students answer quantum-computing questions
Lars Krupp; Jonas Bley; Isacco Gobbi; Alexander Geng; Sabine Müller; Sungho Suh; Ali Moghiseh; Arcesio Castaneda Medina; Valeria Bartsch; Artur Widera; Herwig Ott; Paul Lukowicz; Jakob Karolus; Maximilian Kiefer-Emmanouilidis
In: EPJ Quantum Technology, Vol. 12, No. 1, Pages 0-0, Springer Nature, Berlin, Heidelberg, 2025.
Abstract
Alleviating high workloads for teachers is crucial for continuous high quality education. To evaluate if Large Language Models (LLMs) can alleviate this problem in the quantum computing domain, we conducted two complementary studies exploring the use of GPT-4 to automatically generate tips for students. (1) A between-subject survey in which students (N = 46) solved four multiple-choice quantum computing questions with either the help of expert-created or LLMgenerated tips. To correct for possible biases, we additionally introduced two deception conditions. (2) Experienced educators and students (N = 23) directly compared the LLM-generated and expert-created tips. Our results show that the LLM-generated tips were significantly more helpful and pointed better towards relevant concepts while also giving away more of the answers. Furthermore, we found that participants in the first study performed significantly better in answering the quantum computing questions when given tips labeled as LLM-generated, even if they were expert-created. This points towards a placebo effect induced by the participants’ biases for LLM-generated content. Ultimately, we contribute that LLM-generated tips can be used instead of expert tips to support teaching of quantum computing basics.
