Real-Time Detection of Robot Failures by Monitoring Operator’s Brain Activity with EEG-based Brain-Computer Interface
Date
2024
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Tartu Ülikool
Abstract
The rapidly growing field of Brain-Computer Interfaces (BCIs) provides an innovative
method for human-robot interaction, enabling machines to be controlled directly via
brain signals. This study investigates the use of BCI technology to monitor the operator’s
brain activity in real-time and detect robot malfunctions through the identification of
Error-Related Potentials (ErrPs).
In this thesis, a real-time system was developed to monitor the brain activity of robot
operators continuously using an OpenBCI consumer-grade EEG device. Machine learning algorithms were implemented to analyze brain activity, specifically targeting ErrPs,
which indicate that the operator is noticing something unexpected. By recognizing
these brain signals, our system can identify potential robot malfunctions based on the
operator’s cognitive response and trigger an immediate halt to the operation. This system
integrates several key components: signal processing techniques such as resampling,
filtering, and normalization to prepare the EEG data for analysis; machine learning
classifiers to identify ErrPs associated with robot malfunctions; a robotic simulation that
generates realistic scenarios to elicit ErrPs in participants’ brain activity for safe system
testing; and real-time brain signal acquisition allowing immediate detection of ErrPs
and response to faults. All these components are packed into one coherent system that
detects robot malfunctions and triggers intervention.
The final system underwent live testing with 10 participants, demonstrating its capability
to detect ErrPs associated with robot malfunctions effectively. These tests showed that
the system achieves an average sensitivity of 0.53 while maintaining a specificity of 0.98,
suggesting that the system rarely reacts without a valid reason while being able to correctly detect more than half of the events of interest. For some users, sensitivity reached
as high as 0.7 or more. These findings demonstrate the potential of consumer-grade EEG
devices for practical applications of BCI-based robot fault detection.
Description
Keywords
Brain-Computer Interface (BCI), Electroencephalography (EEG), Error-Related Potential (ErrP), Robotics, Robot Operating System (ROS)