VRySmart: A Framework for Embedding Smart Devices in Virtual Reality

Akhmajon Makhsadov, Donald Degraen, André Zenner, Felix Kosmalla, Kamila Mushkina, Antonio Krüger

In: CHI Conference on Human Factors in Computing Systems Extended Abstracts. ACM International Conference on Human Factors in Computing Systems (CHI-2022) New York, NY, USA CHI EA '22 ISBN 9781450391566 Association for Computing Machinery 2022.


As immersive virtual experiences find their way into our living room entertainment, they are becoming part of our daily technological consumption. However, state-of-the-art virtual reality (VR) remains disconnected from other digital devices in our environment, such as smartphones or tablets. As context switches between acting in the virtual environment and resolving external notifications negatively influence immersion, we look towards integrating smart devices into virtual experiences. To this aim, we present the VRySmart framework. Through either optical marker tracking or simultaneous localization and mapping (SLAM), embedded smart devices can be used as VR controllers with different levels of integration while their content is incorporated into the virtual context to support the plausibility of the illusion. To investigate user impressions, we conducted a study (N = 10) where participants used a smartphone in four different virtual scenarios. Participants positively assessed smart device usage in VR. We conclude by framing implications for future work.

Weitere Links

makhsadov2022vrysmart.pdf (pdf, 7 MB )

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence