Keywords

1 Introduction

Parkinson’s Disease (PD) is a neurodegenerative disease primarily caused by the idiopathic degeneration of the dopaminergic neurons in the substantia nigra pars compacta. The core symptoms of PD are bradykinesia, tremor, rigor and postural instability. Besides motor symptoms, cognitive symptoms are also typical of PD [1], particularly executive dysfunction [2]. Once objectified, it is called Mild Cognitive Impairment in PD (PD-MCI) and presents itself as a subtle difficulty on complex functional tasks. The prevalence of PD-MCI is around 25% [3] and it is a risk factor for decline into PD Dementia (PDD). PDD excludes PD patients from certain treatments such as deep brain stimulation, and currently has no approved pharmacological intervention.

However, recent research shows cognitive training may improve or stabilize the cognitive skills of affected patients [4]. This research also indicates that transfer effects can be expected, that is, cognitive training may improve motor symptoms and vice versa [5]. An excellent way to combine cognitive training with motor rehabilitation is the use of exergames, i.e. games that require the player to perform physical movements. Another advantage of this approach is the sensory feedback provided by game controllers and sensors [6]. A recent systematic review [7] indicated the significant progress of exergames for PD patients in the recent years, also pointing to the requirements of future work, mainly the standardization of outcome evaluation and the inclusion of new sensors and control techniques to train unaddressed areas, such as fine motor skills.

For this purpose, the Leap Motion Sensor (LMS) seems to be a highly promising approach, since it provides an intuitive, natural interface while providing sensory feedback on motor skills. The LMS has already been shown to have potential in evaluating the motor performance of PD patients [8, 9], particularly for hand opening and closing exercises or finger tapping [10]. In fact, researchers have already begun developing LMS based exergames or LMS-digitalized version of traditional motor skill assessment methods such as the Fugl-Meyer test or the Box and Blocks test [11].

2 Methods

As a first step, we considered the diverse cognitive symptoms of PD and how they can be trained via LMS Exercises. Table 1 summarizes our observations in this regard.

Table 1. Cognitive Symptoms of PD and potential training in LMS scenarios
Full size table

In order to create our exergaming framework, we accessed the LMS data directly by using LeapC, a C-style Application Programming Interface (API) for the LMS. This API was connected to our application, developed under Kha, a Haxe-based open source multimedia framework. By using LeapC, it is possible to access the raw data of the LMS directly, as presented in Table 2. Besides palm and finger coordinates, the LMS is capable of measuring “pinch” and “grab” parameters, which determine the degree of closure of a pinching motion (that is, grabbing an object with the index finger and thumb) and grabbing motion (closing the fist) respectively. This provides a great advantage when developing scenarios for people with varied fine motor mobility, since the pinch and grab parameter thresholds can then be adjusted to adapt to users with limited mobility, as it occurs in PD.

Table 2. LMS and game extracted parameters
Full size table

The main advantage of the LMS, besides acquiring hand position with a sampling rate of 200 Hz, is precisely this grabbing and pinching motion detection, since it allows developers to adapt their environments to the varied motor skills of their target group [11]. For this purpose, we implemented to unitary parameters, one for pinching and one for grabbing, 1 being a completely closed pinch or grab. This permits us to demand specific movements for patients with determinate degrees of hand closure, which we can quickly change to ease playability or increase difficulty with time to further improve the motor skills of players.

Considering our observations, we decided to create two scenarios. Firstly, the Tower of Hanoi (ToH), also called tower of London (Fig. 1). The goal in this game is to move a set of n discs from the left to either the middle or right columns in the shortest possible time. It is only possible to place a disc over an empty tower or a larger disc, one by one, which limits the possible movements. The game can be adapted to operate with pinching motions, grabbing motions, or both. This task combines motor skills with problem solving and sequencing, two important cognition areas.

Fig. 1.
figure 1

Example images of the TOH game

Full size image

Secondly, we decided to include a Simon-based Memory Game (SMG), in which the player has to follow a visual and musical sequence by clicking on blocks. This allows the training of working memory and sequencing. For the SMG, the goal is to grab or pinch the blocks, following a randomized music and color sequence generated by the game, that is one element longer for every successfully completed sequence. The sequence can be limited to music or color exclusively and it may be requested from the player to repeat the sequence forwards or backwards. This interaction pattern was chosen purposely to mimic a hand opening/closing test. Figure 2 presents the SMG game.

Fig. 2.
figure 2

Example image of the SMG game

Full size image

For every game session, relevant game data as well as finger data are saved for future processing. This data is described in Table 2, and is saved both locally (as a .csv file) and remotely using a Rest API if desired. The data can be then automatically processed in Matlab and relevant parameters can be extracted, for example detected tremor (if any), variations in elapsed time or number of movements across different sessions, or maximum fist closing/pinching achieved across different sessions. On a future pilot test, we plan to link these game and sensor parameters to relevant PD assessment scores, such as UPDRS-III (motor examination) to create difficulty adaptation algorithms as well as a potential home monitoring scenario [12]. To evaluate the scenario, we firstly plan to analyze its feasibility, considering its technical, therapeutic and patient-centered outcomes. Followed by this is a cross-over evaluation, meaning the PD patient sample will be divided into two groups: Group 1 will first test a physically demanding but cognitively lenient version and then a cognitively demanding but physically easy one, while group 2 will test the same versions in the opposite order. A diagram of the study protocol is presented on Fig. 2. The goal of this phase is to determine whether the order in which the cognitive or motor task is presented plays a role (Fig. 3).

Fig. 3.
figure 3

Study protocol

Full size image