Digital Pens as Smart Objects in Multimodal Medical Application Frameworks

Markus Weber, Christian Husodo Schulz, Daniel Sonntag, Takumi Toyama

In: Proceedings of the Second Workshop on Interacting with Smart Objects, in conjunction with IUI’13. International Conference on Intelligent User Interfaces (IUI-13) March 19-22 Santa Monica CA United States ACM Press 2013.


In this paper, we present a novel mobile interaction system which combines a pen-based interface with a head-mounted display (HMD) for clinical radiology reports in the field of mammography. We consider a digital pen as an anthropocentric smart object, one that allows for a physical, tangible and embodied interaction to enhance data input in a mobile onbody HMD environment. Our system provides an intuitive way for a radiologist to write a structured report with a special pen on normal paper and receive real-time feedback using HMD technology. We will focus on the combination of new interaction possibilities with smart digital pens in this multimodal scenario due to a new real-time visualisation possibility.

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz