Move with the theremin: Body Posture and Gesture Recognition using the Theremin in Loose-Garment with Embedded Textile Cables as AntennasHymalai Bello; Bo Zhou; Sungho Suh; Sanchez Marin Luis Alfredo; Paul Lukowicz
In: Xianta Jiang (Hrsg.). Frontiers in Computer Science, Vol. 4 - Mobile and Ubiquitous Computing, Pages 1-15, Frontiers, 6/2022.
We present a novel intelligent garment design approach for body posture/gesture detection in the form of a loose-fitting blazer prototype, “the MoCaBlazer.” The design is realized by leveraging conductive textile antennas with the capacitive sensing modality, supported by an open-source electronic theremin system (OpenTheremin). The use of soft textile antennas as the sensing element allows flexible garment design and seamless tech-garment integration for the specific structure of different clothes. Our novel approach is evaluated through two experiments involving defined movements (20 arm/torso gestures and eight dance movements). In cross-validation, the classification model yields up to 97.18% average accuracy and 92% f1-score, respectively. We have also explored real-time inference enabled by a radio frequency identification (RFID) synchronization method, yielding an f1-score of 82%. Our approach opens a new paradigm for designing motion-aware smart garments with soft conductive textiles beyond traditional approaches that rely on tight-fitting flexible sensors or rigid motion sensor accessories.