Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Research efforts have improved Brain Neural Computer Interface (BNCI) technology in many ways and numerous applications have been prototyped. With the hope of restoring independence to the disabled individual, the focus of such applications [7] is on communication [6], environmental control  [1], as well as neuro-rehabilitation  [11]. Until recently, such BCI systems have been explored almost exclusively in laboratories primarily with developers and sympathetic, enthusiastic populations for example researchers. Home usage has been demonstrated, though only with on-going expert supervision. A significant advance on BNCI research and its implementation as a feasible assistive technology is therefore the migration and easy set up of BNCIs into people’s homes to provide new options for communication and control that increase independence, autonomy and reduce social exclusion. In this paper, a new BNCI system is presented which combines a wireless EEG recording unit, which sets new standards of lightness, autonomy, comfort and reliability, with easy-to-use software, tailored to people’s needs, within a platform of everyday applications, i.e. twitter and support services. The BackHome BNCI platform also moves beyond the individual to the space where they live bringing in a sensor-based telemonitoring and home support system. This work is part of the European BackHome project (GA no: 288566) and the overall system is installed at end-users’ facilities for testing. The results achieved by the end-users are presented.

Section 2 illustrates the system architecture of the developed system together with the description of its components. Section 3 presents the results coming from the system validation which are briefly discussed in Sect. 4.

2 System Architecture

Within the BackHome platform, the end user interacts and communicates with her/his environment through the Primary User Interface (PUI) of the User Station (see Fig. 1) with the electrodes as harvesting brain signals as the actuator. The User Station includes the BNCI component with the PUI providing including a user interface for visual stimulation and feedback. The AmI block, which is the central control unit interlinks all the services with the BNCI system and thereby provides control over these systems to the user.

Fig. 1.
figure 1

BackHome architecture overview

Full size image

The User Station is connected to a remote Therapist Station, as shown in Fig. 1. A web-based easy-to-use service, which allows a therapist from her base of work to deliver tele-assistance service or professional to access information stored in the cloud and gathered from users and sensors around them: users’ inputs, activities, selections and sensor data, as well as planning and remote coordination of the users BNCI based rehabilitation tasks. This service was developed based on request from target end users, i.e. people with acquired brain injury who have a desire for prolonged engagement with therapist post discharge from acute rehabilitation services following an acquired brain injury.

As Fig. 1 also shows, the system provides a number of services that can be grouped as follows: Smart Home control, cognitive stimulation, and Web access.

2.1 User Station

The BCI user interface is based upon the Screen Overlay Control Interface (SOCI) library  [4], which allows embedding the BCI stimuli on top of the native interface of any user application and communicates with the BCI hardware via a network connection.

Along with the integration of SOCI, the PUI has been designed following the recommendations expressed by target end users, according to a user-centred design approach. The number of lines, columns, the spaces in between and thus the display size of the individual icons are defined depending upon users’ capabilities and do not change depending the dimension of the matrix and number of icons displayed. Masks which are smaller are automatically centred by SOCI within the visible area. The Application ConTrol and Online Reconfiguration (ACTOR) protocol [3] allows the application to explicitly position its masks on the screen.

Figure 2 above shows the PUI, which is split into three sections. On top of the screen a history bar shows the last selections made. On the top right side the current time of the day is displayed along with the quality of the EEG signal recorded. The middle section displays the active P300 matrix for the currently selected service or service group, for example the Smart Home group for interaction with user’s environment and the multimedia player. The bottom row of the screen displays the menu which allows switching between the different services and service groups (i.e., Smart Home control, Web access, Cognitive Rehabilitation Games, Brain Painting, and Speller). When the user selects one of the icons on the screen, the system automatically activates the selected service group and displays the corresponding masks.

Fig. 2.
figure 2

Primary user interface showing the main screen of the smart home service

Full size image

The user interface for the caregiver (see Fig. 3) allows starting the system with just one-click, to create the classifier (in training mode) and to shutdown (Quit) the BCI system. Furthermore, when the system is started it automatically activates the check signals mode and starts the signal acquisition. This simplifies the mounting of the electrode cap as the signal quality display is visible during the whole procedure.

Fig. 3.
figure 3

Care giver interface. The interface has been designed to optimally guide the care giver through the setup process step by step (a) and provide access to help and support information (b) on every screen. Only those information and controls are shown which are necessary to accomplish the current step (c) or advance to the next one when finished or go back to the previous one.

Full size image

The Wireless EEG Recording System. Based upon the user feedback, a new biosignal acquisition system called g.NautilusFootnote 1 was developed. Its biosignal amplifier uses wireless technology to transmit the EEG signals with 24 bit resolution. The signal of each EEG channel is highly oversampled in order to keep the signal to noise ratio (SNR) high at the offered rates of 250 Hz and 500 Hz. Furthermore, it is capable of measure the electrode-skin impedance at each electrode position for both gel based and dry electrodes (see Fig. 4).

Fig. 4.
figure 4

g.Nautilus headset with gel based (a) and dry elecectrodes (b)

Full size image

A base station which is connected to the host system through USB is used to receive the recorded and digitized EEG signals. The biosignal amplification unit consists of the headset including the wireless biosignal amplifier electrodes, an EEG cap and the base station including a USB cable for connecting it to the host computer and a QI compatible wireless charging station. The 10 electrodes including reference channel and ground are pre-connected to the amplifier with their positions preconfigured for recording visual P300.

The user interface of the headset consists of the power switch and the status LED. Both are located on the top face of the head set. The electrodes are connected to two groups of monopolar amplifiers. The first group is connected to the ground electrode and the first 16 EEG channels and the second group is connected to the electrode positions 17-to 32 and the reference channel.

The Implemented Services. As mentioned earlier, the system provides a number of services grouped as follows: Smart Home control, cognitive stimulation, and Web access [5]. Smart Home control service is aimed at giving control over the environment, as well as over useful devices. The ultimate aim is to facilitate the user to exert control home devices (e.g., a light, a fan, a radio) as well as to interact with the XBMC multimedia player. Cognitive stimulation services allow users to improve their cognitive capabilities by performing cognitive rehabilitation tasks assigned by a therapist [12] or by using their creative skills through Brain Painting [9]. Web access services enable participation and inclusion by offering users the possibility to engage in social interaction through the Web, such as Web browsing, emailing and tweeting.

All those services, and in particular, those related to Web access, rely on a P300 spelling and control system. In the adopted speller, the highlighting happens with freely selectable images (famous faces) instead of just changing the colour of the background.

Sensor-based Telemonitoring and Home Support. Community based living, often alone with intermittent care creates possible scenarios of risks for all individuals. When cognitive changes are likely to have taken place it is crucial to understand what the risks may be and monitor these. To monitor users at home, we develop a sensor-based system [8] which is able to gather data and report on the stability and evolution of the user’s daily life activity [14]. The implemented system is able to monitor indoor activities by relying on a set of home automation sensors and outdoor activities by relying to MOVES.Footnote 2 As for indoor activities, we use presence sensors (i.e., Everspring SP103), to identify the room where the user is located (one sensor for each monitored room); a door sensor (i.e., Vision ZD 2012), to detect when the user enters or exits the premises; electrical power meters and switches, to control leisure activities (e.g., television and pc); pressure sensors (i.e., bed and seat sensors), to measure the time spent in bed and wheelchair. From a technological point of view, we use wireless z-wave sensors that send the retrieved data to a central unit located (based on Raspberry pi) at user’s home. That central unit collects all the retrieved data and sends it to the cloud where it is processed, mined, and analysed. As for outdoor activities, we are currently using the user’s smartphone as a sensor by relying to MOVES, an app for smartphones able to recognize physical activities (such as walking, running, and cycling) and movements by transportation.

Information gathered by the sensor-based system is also used to provide context-awareness by relying on ambient intelligence [2]. Moreover, data collected by the sensor-based system has been used to monitoring activities [10] as well as automatically assess quality of life of people [13].

2.2 Therapist Station

The Therapist StationFootnote 3 is a Web based application providing clinicians/therapists tools for user management, cognitive rehabilitation task management, quality-of-life assessment, statistics on BCI usage, as well as for communication between therapist and user. Therapists are able to interact with the users remotely in real time or asynchronously and monitor the use and outcomes of the cognitive rehabilitation tasks and quality-of-life assessment. A therapist can prescribe, schedule, telemonitor and personalize cognitive rehabilitation tasks and the assessment of the users quality-of-life through questionnaires and adopt those tasks to the users therapeutic range (i.e. motivating and supporting their progress), in order to help to attain beneficial therapeutic results.

As for the cognitive rehabilitation sessions, using the Therapist Station, healthcare professionals can remotely manage a caseload of people recently discharged from acute sector care. They can prescribe and review rehabilitation. Through the Therapist Station, game sessions can be configured, setting the type of games that the user will execute, their order within the session the difficulty level and game specific parameters. Once the session is scheduled, users will see their BCI matrix updated on the User Station the day the session is scheduled. Through that icon, the user will start the session. The user can then execute all the games contained in it in consecutive order. Upon completion of the game session execution on User Station, results are sent back to the Therapist Station for review. A notification indicating that the user has completed the session, will be posted on the dashboard of the concerned healthcare professionals involved. Healthcare professionals with the right credentials can browse user session results once they are received. The Therapist Station provides session results in a summary view for each person and an overview of completed sessions to map progress, which shows session parameters and statistics along the specific results.

The results and statistics about the quality of life coming from the telemonitoring and home support system are sent to the Therapist Station to inform the therapist about improvement and worsening of user’s quality of life (Fig. 5). Moreover, the therapist may directly ask the user to fill a questionnaire, either once or on a regular basis. Once scheduled, the user receives an update in the BCI matrix indicating that a questionnaire has to be filled. After the user has completed the questionnaire with the help of the caregiver its results are sent to the therapist who may revise them.

Fig. 5.
figure 5

An example of data coming from the sensors as shown to the therapist

Full size image

Finally, through the Therapist Station, therapists may consult a summary of activities performed at home by the user; e.g., visited rooms, sleeping hours and time elapsed at home. Moreover, also the BCI usage is monitored and high-level statistics provided. This information includes BCI session duration, setup time and training time as well as the number of selections, the average elapsed time per selection and a breakdown of the status of the session selections. Therapists have also the ability to browse the full list of selections executed by a user, such as context information as application running, selected value, grid size and selected position.

3 System Validation

The development and validation of BackHome has gone through three main iterative stages of design and work directly with target end users, i.e. people with acquired brain injury and therapists working in neurology. In preparation for the installation of the system at the end users homes for long term evaluation of half a year, a pre-evaluation phase of the services in the overall system was undertaken by a control group of fourteen healthy users (9 females, M = 28.1 years ± 8.6, range: 21-46) [5]. Subsequently a total of nine end-users, four users with muscle impairments (f, 80 years; f, 58 years; m, 42 years; m, 51 years) and five users (1 female, M = 37 years, ± 8.7) who are living with acquired brain injury (Post ABI M = 9.8 yrs, ± 3.7) operated the system. All of them completed the protocol on three different occasions.

Seven of the nine end users involved in the evaluation were able to gain control over the BCI and achieved satisfactory accuracies. Among the end-users with muscle impairments, one achieved scores in advance of >90 % accuracy for the spelling, games and twitter tasks. Another achieved lower accuracies scores averaging 75 %. Two of the initial four end-users undertaking the evaluation were unable to operate the BCI on one occasion. One end user had difficulty focusing on the relatively small symbols in the control matrices and the second had significant muscle spasticity that caused artifacts in the signal. The extended evaluation with end-users with acquired brain injury recorded an average accuracy score of 76 % across the four applications. The highest overall accuracy was achieved with the Speller (82.07 % ± 13.34) and the lowest with the camera task (64 % ± 22.8).

4 Discussion and Conclusion

The aim of the BackHome system presented in this paper is to support the autonomy and independence of people living at home with a disability. The purpose is to support interaction and communication with their physical environment and social networks, thereby regain independence and social inclusion. This has been achieved by integrating the BNCI system with a set of applications and services recommended by the users and their care givers while keeping the user interfaces for both simple and easy to use.

At time of writing, the overall system is installed in three end-users’ home in Belfast and the test phase will go on for about 4 months.