Comparison of strategies to elicit motor imagery-related brain patterns in multimodal BCI settings

Tristan Venot (1, 2) , Arthur Desbois (1, 2) , Marie-Constance Corsi (1, 2) , Laurent Hugueville (2) , Ludovic Saint-Bauzel (3) , Fabrizio de Vico Fallani (1, 2)

1 ARAMIS - Algorithms, models and methods for images and signals of the human brain = Algorithmes, modèles et méthodes pour les images et les signaux du cerveau humain [ICM Paris]
2 SU - Sorbonne Université
3 ISIR - Institut des Systèmes Intelligents et de Robotique

Cognitive tasks such as motor imagery (MI) used in Brain Machine Interface (BMI) present many issues: they are demanding, often counter-intuitive, and complex to describe to the subject during the instruction. Engaging feedback related to brain activity are key to maintain the subject involved in the task. We build a framework where the subject controls a robotic arm both by gaze and brain activity in an enriched environment using eye tracking glasses and electro-encephalography (EEG). In our study, we tackle the important question of the preferable moment to perform the MI task in the context of the robotic arm control. To answer this question, we design a protocol where subjects are placed in front of the robotic arm and choose with gaze which object to seize. Then based on stimuli blended in an augmented table, the subjects perform MI or resting state tasks. The stimuli consist of a red (MI) or blue (Resting) dot circling the object to seize. At the end of a MI task, the hand should close. There are three strategies corresponding to three different moments when to perform the mental task, 1) After the robot's movement towards the object, 2) Before the robot's movement, 3) Meanwhile the robot's movement. The experimentation is split into a calibration and two control phases, in the calibration phase, the hand always close during MI task, and in the control phases, it relies on the subject's brain activity. We rely on power spectral density estimate using Burg Auto regressive method to differentiate between MI and resting state in the alpha and beta bands (8-35 Hz). Our method to compare the strategies relies on classification performance (LDA 2 classes) using sensitivity, and statistical differences between conditions (R-squared map). The early results on the first 10 subjects show significant differences between strategy 1 and 2 for offline classification analysis and a trend on the real performance scores in favor of the strategy one. We observed in all subjects' brain activity localized in the motor cortex at significant level with regards to resting state. This indicates that the framework placing the subjects at the center with high sense of agency reinforced by gaze control is giving good results and allows to be more certain that the subject is doing the right task. Taken together, our results indicate that investigating the moment when to perform MI in the framework is a relevant parameter. The strategy where the robot is at the object level when MI is performed seems so far to be the best strategy.

Domaines: Neurosciences; Automatique / Robotique

Tristan Venot, Arthur Desbois, Marie-Constance Corsi, Laurent Hugueville, Ludovic Saint-Bauzel, et al.. Comparison of strategies to elicit motor imagery-related brain patterns in multimodal BCI settings. SFN 2022 - Society for Neuroscience Annual Meeting 2022, Nov 2022, San Diego, United States. ⟨hal-03867137⟩