Publication
Classifying Words in Natural Reading Tasks Based on EEG Activity to Improve Silent Speech BCI Training in a Transfer Approach
Maurice Rekrut; Andreas Fey; Johannes Ihl; Tobias Jungbluth; Matthias Nadig; Antonio Krüger
In: Proceedings of the IEEE International Conference on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering. IEEE International Conference on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE-2022), October 26-28, Rome, Italy, IEEE, 2022.
Abstract
Silent speech Brain-Computer Interfaces (BCIs) classify imagined words from brain activity. Those BCIs require a tremendous amount of training data collected during tedious sessions in which the user continuously silently repeats certain words. In order to overcome those cumbersome training sessions, this work proposes a transfer learning approach in which we try to classify words silently read embedded in a text during a natural reading task and transfer this classifier to silent speech data of the same words. We designed an instruction manual with 9 keywords repeatedly embedded in the text and conducted a study with 23 healthy participants who read the text while EEG and eye-tracking data were recorded simultaneously. We used the eye-tracking data to label the EEG data on a word level and trained classifiers based on this data to detect words silently read. Those classifiers were later transferred to data recorded during silently speaking the same keywords in a separate task. Our results show that it was possible to classify words silently read from EEG data with an average classification accuracy of 20.82\% and an individual best of 25.41% significantly above the chance level of 11\% for a 9 class classification problem. The attempt to transfer this classifier to silent speech data did not significantly exceed chance level. Due to our successful classification of silently read words during natural reading and the similarity of the two
concepts we still see a great potential in this transfer and elaborate on adjustments of the presented method to enable silent speech BCI training based on natural reading in the future.