Skip to main content Skip to main navigation

Publication

EdTec-ItemGen: Enhancing Retrieval-Augmented Item Generation Through Key Point Extraction

Alonso Palomino Garibay; David Buschhüter; Roland Roller; Niels Pinkwart; Benjamin Paaßen
In: Proceedings of the 3rd Workshop on Towards Knowledgeable Foundation Models (KnowFM). Workshop on Towards Knowledgeable Foundation Models (KnowFM), August 1, Vienna, Austria, Pages 14-25, Association for Computational Linguistics, 2025.

Abstract

A major bottleneck in exam construction involves designing test items (ie, questions) that accurately reflect key content from domain-aligned curricular materials. For instance, during formative assessments in vocational education and training (VET), exam designers must generate updated test items that assess student learning progress while covering the full breadth of topics in the curriculum. Large language models (LLMs) can partially support this process, but effective use requires careful prompting and task-specific understanding. We propose a new key point extraction method for retrieval-augmented item generation that enhances the process of generating test items with LLMs. We exhaustively evaluated our method using a TREC-RAG approach, finding that prompting LLMs with key content rather than directly using full curricular text passages significantly improves item quality regarding key information coverage by 8%. To demonstrate these findings, we release EdTec-ItemGen, a retrieval-augmented item generation demo tool to support item generation in education.