Publication
In Silico User Testing for Mid-Air Interactions with Deep Reinforcement Learning
Noshaba Cheema
Mastersthesis, Saarland University, 9/2019.
Abstract
User interface design for Virtual Reality and other embodied interaction contexts has to carefully consider ergonomics. A common problem is that mid-air interaction may cause excessive arm fatigue, known as the “Gorilla arm” effect. To predict and prevent such problems at a low cost, this thesis investigates user testing of mid-air interaction without real users, utilizing biomechanically simulated AI agents trained using deep Reinforcement Learning (RL). This is implemented in a pointing task and four experimental conditions, demonstrating that the simulated fatigue data matches ground truth human data. Additionally, two effort models are compared against each other: 1) instantaneous joint torques commonly used in computer animation and robotics, and 2) the recent Three Compartment Controller (3CC-r) model from biomechanical literature. 3CC-r yields movements that are both more efficient and natural, whereas with instantaneous joint torques, the RL agent can easily generate movements that are unnatural or only reach the targets slowly and inaccurately. This thesis demonstrates that deep RL combined with the 3CC-r provides a viable tool for predicting both interaction movements and user experience in silico, without users.