ABSTRACT
Severely disabled people, like completely paralyzed persons either with tetraplegia or similar disabilities who cannot use their arms and hands, are often considered as a user group of Brain Computer Interfaces (BCI). In order to achieve high acceptance of the BCI by this user group and their supporters, the BCI system has to be integrated into their support infrastructure.
Critical disadvantages of a BCI are the time consuming preparation of the user for the electroencephalography (EEG) measurements and the low information transfer rate of EEG based BCI. These disadvantages become apparent if a BCI is used to control complex devices. In this paper, a hybrid BCI is described that enables research for a Human Machine Interface (HMI) that is optimally adapted to requirements of the user and the tasks to be carried out.
The solution is based on the integration of a Steady-state visual evoked potential (SSVEP)-BCI, an Event-related (de)-synchronization (ERD/ERS)-BCI, an eye tracker, an environmental observation camera, and a new EEG head cap for wearing comfort and easy preparation. The design of the new fast multimodal BCI (called sBCI) system is described and first test results, obtained in experiments with six healthy subjects, are presented. The sBCI concept may also become useful for healthy people in cases where a “hands-free” handling of devices is necessary.
USE CASES AND GENERAL DESIGN DECISIONS FOR SBCI
The use cases are inspired by the support of users with tetraplegia in an ADL scenario. Simple devices like a fridge or microwave and more complex ones like an internet radio have to be operated. Typically, the user has to select a device and then issue commands to the selected device. For a fulfillment rate of the initialized tasks larger than 80%, the user has to support the automation system. The user can interrupt it if a problem in task execution arises and issue corrective commands. After the problem is solved, control is handed back to the automation system.
STATE OF ART FOR EEG-BASED BCI
The BCI system establishes a direct communication channel between the human brain and a control or communication device. BCIs detect the human intention from various electrophysiological signal components, such as steady-state visual evoked potentials (SSVEPs), P300 potentials and sensorimotor rhythms (SMR) and translate it into commands. The brain signals are recorded from the scalp using electroencephalography (EEG).
SYSTEM LAYOUT AND DESIGN DECISIONS
Figure 2. Layout of eye tracker and environmental camera to recognize gaze direction and objects to be controlled
Two cameras are for tracking of left and right eye, the remaining one is for monitoring the environment. Figure 2 shows the principal layout, with one of the eye cameras and the environmental camera of the tracking system DeLock USB CMOS Cameras 95,852, 1.3 Megapixel are used in all cases. The resolution of the DeLock cameras is set to 1280 × 1024 pixels for environmental images and to 640 × 480 pixels for eye tracking.
Figure 4. Marker layout used to distinguish between home devices, object with specific color and size as marker
The object may be recognized in the image of the environmental camera by using specific colors that are not immediately recognizable as marker, by using SIFT (Scale Invariant Feature Transform) or a comparable algorithm to recognize known objects or by specific markers, e.g., chosen from ARTool Kit. Figure 4 shows typical markers of the different categories. Based on the estimation of the gaze direction the object of interest is determined.
USER INTERFACE AND CONTROL METHOD
Figure 7. GUI for the internet radio
Figure 7 shows the graphical user interface (GUI) for the internet radio. It gives access to the main functionalities and informs the user about the currently selected position inside the hierarchical structure of the interface. sBCI user interface presently contains three external home devices: internet radio (Grundig Cosmopolit 7), fridge and microwave.
Figure 9. Hybrid ERD/ERS-SSVEP BCI with sequential processing. The ERD/ERS-BCI acts as a selector which activates the SSVEP system
The ERD/ERS acts in this mode as a selector which activates the SSVEP system. Figure 9 shows the principle operation. In the current study, sensorimotor rhythms related to the imagery of left hand, feet, and right hand movements are detected, thus providing three control commands.
SUBJECT AND DATA ACQUISITION AND RESULTS
A preliminary evaluation of the sBCI-SSVEP and a comparability test of the sBCI-ERD/ERS system is carried out in order to prove the feasibility of the concept and compare sBCI results with previous ones that use the same software packages. A total of six able-bodied subjects were recruited. All participants used the sBCI-SSVEP system. Five participants (subjects A–E, aged 30.8 ± 7.4; 3 female and 2 male) used the eye tracker system to select a target device.
DISCUSSION AND CONCLUSIONS
A multimodal, hybrid BCI is designed which combines an eye tracker, an SSVEP-BCI, and a multiclass ERD/ERS-based BCI and offers the possibility for a detailed study in which different comb inations of the system are researched and evaluated in relation to the disability of the user. For easy evaluation of this hybrid system, a multimodal sBCI-headset was designed. The proposed multimodal BCI system was used to control three devices that play an important role in future ADL application.
The benefits of using two BCI modalities include the possibility to activate the eye-tracker and SSVEP-based control only on demand, i.e., both can independently be turned off during inactive periods. Thus, the hybrid setup of the system minimizes the number of involuntary selections and increases the convenience of the whole interface. The multimodal sBCI-headset is a sensing system which integrates multi-channel EEG equipment, an eye tracking system and a visual stimulator for the SSVEP-BCI. During the designing phase of the headset, all effort has been made to optimize the long term wearing comfort, while maintaining the ergonomic and aesthetic appearance and also the quality of EEG-signals.
Despite the substantial investment of time and resources, it was not possible to successfully develop a one-size helmet that fits onto any adult’s head. All one-size prototypes of the sBCI-headset have failed the long term comfort tests. Consequently, the sBCI-headset is provided in three different sizes based on head circumference (Small: 56cm, Medium: 58cm, Large: 60cm) in order to fit the head to the majority of adult users. Small distances up to 5 mm between the hard case cap and the skin were easily bridged by the soft springs of the electrode holders.
Using such holders yield s a double benefit. The wearing comfort is increased and the electrode-skin coupling enhanced. A preliminary test with six able-bodied volunteers using the newly designed sBCI-headset was performed. Two fusion techniques were evaluated: Gaze-SSVEP and an ERD/ERS-SSVEP, called a physiological and pure interface, respectively. The performance measurements show that the sBCI system provides an effective environmental control method for all six subjects.
The two fusion techniques are compared with a limited data set only. The eye tracker as the selection device is in the set up obviously much faster than an ERD/ERS-BCI (3.9s vs. 20s) and achieves a high accuracy. However, the accuracy of the device selection is until now only tested with AR ToolKit markers. While there is a limited accuracy of the ERD/ERS-BCI the probability to select an undesired device with an ARToolKit marker was close to zero.
But usage of markers requires preparation of the users’ environment and limits the usage to such a prepared environment. To avoid markers, they will be replaced by object recognition based on SIFT features. That may decrease recognition speed and recognition accuracy especially if no constant illumination can be guaranteed. A statistically sound comparison of all features and possible combinations is in preparation.
Source: University Bremen
Authors: Tatsiana Malechka | Tobias Tetzel | Ulrich Krebs | Diana Feuser | Axel Graeser