1 Introduction

Educational needs depend on children’s individual abilities, yet, most school programs are based on the assumption that every child learns in the same way, equally fast whereas being stimulated in a similar way. Banks and McCoy (2011) report that estimates for the number of children with special educational needs have increased dramatically in recent decades. It is estimated that 10-26 % of the children population have special educational needs (SEN). These estimates tend to vary, depending on qualification methods used in different countries, and may go as high as 35 %. This is a result of a policy redefining towards inclusive education, which broadens the definition of special educational needs. Due to this fact, to compare such data internationally, SEN are divided into three categories: A, B and C, where Category A refers to a disability from an organic impairment (Disability); B refers to intellectual, behavioral or other learning difficulties (Difficulties); and Category C refers to difficulties because of social disadvantage (Disadvantage). The data even more drawing attention to the apparent problem comes from the 2011th WHO (World Health Organization) report. Screening tests show up to 35 % percent of children in many countries are at risk of dysfunction impeding school activity. It is a well-known fact that many cognitive deficits, often diagnosed at school age, are caused by disrupted communication between the hemispheres of the brain, which occurs often in conjunction with laterality disorders. In such cases, polysensory integration training may be a method supporting early learning (Petrushin 2004). A computer vision system extended by audio may play an important role in polysensory integration and in conscious or unconscious information processing. However, the mainstream studies on computer-based training applications are focused either on the impact of video games on children’s mental health or on approaches to mental health care that integrate video game playing for children with autism or ADHD syndrome rather than with the state-of-the-art technology serving the needs of children with mental disabilities (Granic et al. 2013). It was shown by Hamilton (2013) that modern video games such as those available for game consoles can help some players overcome ADHD symptoms. There are also video games that involve both normally-sighted children and children with visual impairments. There are also many examples of sonification software applied to education supporting. Bonebright et al. (2001) designed three laboratory experiments exploring whether students can match auditory representations with the correct visual graphs, comprehend graphed data sets more effectively by adding sonified components, and be trained to use sonified graphs better with practice. Another report on the use of sonification software for educational purposes, especially for pupils with low linguistic and logical-mathematical capabilities, was delivered by Upson (2001). This involved training in Cartesian graphing over several sessions with sonification software and a sonification-enhanced curriculum. Educational potential of sonification exercises may be judged in terms of student motivation and cognitive development (Upson 2001). Rovithis et al. (2014) devised a role-playing scenario to facilitate the sonification of the relevant curriculum and to create an educational platform that combines an audio-based gaming environment with a musical instrument. Even though a common definition says (Kramer et al. 1999) that sonification is “the transformation of data relations into perceived relations in an acoustic signal for the purposes of facilitating communication or interpretation”, one may argue that such a supposition needs broadening. Though this basic definition encompasses assistive sonification technologies for children with disabilities, especially those who are blind and visually impaired, in education, the sonification may play a key role in enhancing or reinforcing haptic skills in response to audio associated with video games. Education of mentally handicapped pupils involves four stages: stimulating senses, sensory-kinesthetic integration, developing somatognosia and developing readiness to learn. In many studies one may find a statement that perceptual motor skills, both fine and gross, require the integration of sensory input (visual, auditory and kinesthetic) (Ayres 1972; Dore 2006). Ayres introduced the concept of sensory integration, i.e. most of incoming messages to the brain are received by the senses (Ayres 1972). It has been applied in therapy of various disabilities, including intellectual ones (Arendt and MacLean 1988). Assuming these facts, teaching methods in education should be developed in two dimensions: individualized teaching and training of cognitive capacity. The past decades, taking advantage of computer technology, developed many didactic computer exercises, collaborative learning (Lingnau et al. 2003), mixed-reality technology such as motion capture or up-to-date robots, games that progress both mental and manual skills in schoolchildren and students (Gider et al. 2012; Sugimoto 2011; Sung et al. 2011). The newer technology era concerns game technology that are created to assist students in other than school activities, such as for example learning to dance. Dance training systems are however designed for users with sufficient skills to perform the dance movements. Though not very difficult for intellectually agile students, such solutions may be undeniably too sophisticated for mentally handicapped pupils. In the paper, a multimedia system with specially designed computer-assisted exercises addressing pupils with special educational needs including moderate and severe mental retardation is presented. The system is aimed at therapy of dyslexia of types L (left hemisphere overdeveloped in relation to right hemisphere), P (perceptual), and M (mixed - both hemispheres underdeveloped), and at stimulating the development of kinesthetic-perceptual functions of mentally handicapped pupils. By definition dyslexia is characterized as difficulties with accurate and/or fluent word recognition, reading deficit, poor spelling and decoding abilities. Section 2 provides details of the system components and features and shows the way children interact with it. This Section describes how standard computer vision tools can be used to track the human body and to detect pupil’s basic behaviors. In Section 3 therapeutic exercises with emphasis on description of interaction with the child aimed at particular brain hemisphere stimulation are presented. The purpose of the proposed games are to activate the child’s response to the shown images and the audio associated with it. Section 4 shows the results of the use of the system for several weeks in a school for educable mentally handicapped pupils along with a thorough analysis of the results of carried out investigations. Conclusions drawn from this study, as well as further directions of development of the system are included in Section 5.

2 System description

2.1 System components

The system architecture comprises a personal computer with an audio-visual application installed, four monitors, a therapeutic mat, two USB cameras, four surround sound system speakers, a stand for the speakers and one of the cameras (Fig. 1).

Fig. 1
figure 1

Components of the multimedia polysensory integration system

Full size image

In a less hardware demanding version of the system four monitors are reduced to two monitors placed back to back (Fig. 2). Each exercise has its own interface proposed enhanced with audio. One of the monitors displays an exercise interface to the pupil and the other one displays modified exercise screens and controls to the therapist. In such a scenario a pupil moves on the therapeutic pad always in the direction of the therapist who sits in front of the monitor. The therapeutic area is limited by a stand construction with a square therapeutic mat positioned in its center on the floor. The therapeutic mat has nine square areas separated by straight lines. One of the cameras is placed on the floor in such a location that the pupil, when walking on the therapeutic mat, is always visible in the camera frame. The second camera is placed over the middle position of the therapeutic mat at such a height that the same requirement as for the floor camera is met. The speakers are hung in the corners of the stand construction.

Fig. 2
figure 2

View of the two monitor version of the system during therapy

Full size image

2.2 Interaction with a child

During the application initialization the pupil does not occupy the field of view of any of the cameras. Thus, initial video streams used for image processing during exercising can be retrieved. After initialization, before running the chosen exercise, a pupil is asked to occupy the middle area of the therapeutic mat. The interaction with the system lies in walking on the mat and thus choosing one of the square areas at a time and bouncing to confirm the choice. Occupying a particular area causes an associated image to be displayed and sound generated, according to the exercises described in the next Section. Bouncing produces an image display and also sound generation.

2.3 Video processing

The system operates on processed video streams, resulting from subtracting the initial video streams that do not contain the pupil from the video streams retrieved during exercising. The key elements are background subtraction and contour detection, which should be sufficiently robust to impose strict control of system parameters. This principle concerns both the floor and the overhead cameras. The subtraction method uses absolute difference. After subtraction, the video streams are converted from BGR color space to grey scale and binary thresholded. The result of this operations is presented in Fig. 3 (Czyzewski and Kostek 2012; Lech et al. 2014).

Fig. 3
figure 3

Processed video frames from ceiling (a) and floor (b) cameras

Full size image

The video processing threshold values can be changed using application control panels, separately for each camera. Thus a teacher/therapist can adjust the processing, for example to pupils clothing, raising the threshold when the contrast between the pupil and the therapeutic mat or walls is high. This results in an increase in the pupil’s position detection efficacy. After applying the binarization, the video streams are median filtered with a mask of a size equal to 9 by default. Detection of pupil silhouettes is performed using a contour detection algorithm implemented in OpenCV library (Bradski and Kaehler 2008). The algorithm is based on the method proposed by Reeb (1946) and further developed by Bajaj et al. (1997) and Carr et al. (2004).

Each object in the video streams is represented by a sequence of points connected by segments creating its shape. The algorithm, known as Teh-Chin chain approximation algorithm in L1 metric flavor (Chan and Chin 1992) has been used for contour approximation. The sequences are found using the method proposed by Suzuki and Abe (1985). The extracted objects are represented in the form of trees, in which the root is a sequence representing the outer contour and the children are sequences designating inner contours of successive depth levels. The implemented algorithm, developed originally by Suzuki and Abe (1985), bases on two principle identifiers denoted by NBD and LNBD. The NBD identifier stands for the sequential number of the newly-found border and LNBD represents the sequential number of the border recently encountered during the searching process. Thus, the border identified by LNBD is either the parent border of the newly-found border or a border which shares the common parent with the newly-found border. The searching process is performed in a scanning manner. The input image F = {f i j } is scanned with a given raster starting with NBD set to 1. LNBD is reset to 1 every time a new row of the image is to be scanned. The following steps of the algorithm are performed for each pixel such that f i j ≠0.

figure a

Additionally, amongst all the objects detected in the video streams using this method, objects that are marked as silhouettes must have a width, height and number of pixels greater than the thresholds set. Thus, possible noise which was not eliminated in the processing phase can be reduced. For a silhouette in the image captured by the floor camera, the initial y position of the feet is determined. This process is performed before running the chosen exercise when the middle area is occupied by the pupil. The position is identified by the bottom edge of the shoes. Relative to this value bouncing is detected. The bounce threshold, i.e. the distance between the feet y initial position and the position reflecting the highest point of the bounce, can be adjusted by the therapist. Thus for pupils bouncing high, the bounce detection efficacy can be increased. The bounce is recognized after the system detects that the feet have left the ground, exceeded the bounce threshold, and returned to their initial position. The position of a pupil on the therapeutic mat is determined by the center of gravity of the silhouette detected in the image captured by the overhead camera. The boundaries of each square of the therapeutic mat are determined manually by the therapist soon after the application is initialized. This operation is performed using the therapeutic mat calibrator (Fig. 4).

Fig. 4
figure 4

Therapeutic mat calibrator

Full size image

3 Therapeutic exercises

The system contains various therapeutic exercises for pupils with special educational needs including pupils with moderate and severe mental retardation. In particular, the exercises in the system provide therapy of dyslexia of types L, P and M, lateralization support, and support of general development. The system provides audio feedback channel to the graphics displayed on the computer screen, such as applause sound and animal sounds in therapeutic exercises described further on. Also, specially prepared sounds are generated from particular direction (to the particular ear), which provide stimulation of a particular part of the brain. The method implemented is based on Dirk Bakker’s neuropsychological concept of therapy of dyslexia (Bakker 1990; Bakker and Vinke 1985). There are two types of therapy proposed in this concept, i.e. via hemisphere specific stimulation (HSS) and via hemisphere-alluding stimulation (HAS). The HSS method is a direct stimulation in which the sound stimulus is given to the particular ear or visual stimulus is displayed in the particular field of vision, and via pyramidal tract in brain activates the opposite hemisphere at first. In the HAS method the stimulation is indirect and bases on preparing the stimulus in such a form that it activates mainly the chosen hemisphere. For example, although reading process as a form of linguistic communication mainly involves left hemisphere, text written in complex fonts stimulates right hemisphere as decoding strategies are employed at the same time. The HSS stimulation in the developed system is mainly used in the exercise for therapy of dyslexia of type P, described below.

3.1 Exercise for therapy of dyslexia of type P

Dyslexia of type P (Perceptual) is associated with sticking to the right hemisphere strategies while reading, after reaching the age of 8 when perceptual strategies should be smoothly exchanged by linguistic ones (left hemisphere strategies). According to Bakker’s theory, in therapy of dyslexia of type P the left hemisphere should be stimulated to activate linguistic strategies (Bakker 1990). In all exercises therapy begins with taking up a middle position of the therapeutic pad by a pupil. In this exercise sequences (bars) of various numbers of metronome ticks are generated from right side and thus activate the left hemisphere at first (Bakker 1990). The first sequence (e.g. four ticks) of the metronome ticks is being played and simultaneously a corresponding sequence of shapes (e.g. rectangles) is presented to a pupil on the screen (Fig. 5). One of the shapes differs from the others. The task of a child is to count the metronome ticks and analyze the displayed sequence, and bounce when the tick associated with the highlighted shape is played. The bounce should be made in synchronization with the metronome tick. After bouncing, the child gets graphical information about its precision (Fig. 5) and also gets the audio feedback.

Fig. 5
figure 5

The graphical user interface displayed to a pupil in the exercise for therapy of dyslexia of the type P

Full size image

The audio feedback is given from the corresponding side depending on whether a child performs a jump earlier than the particular metronome tick occurs or later. If a jump is performed too early, the audio feedback is given from the left side and activates the right hemisphere. In such a case, the early jump occurred because the left hemisphere, which is considered quick and inaccurate, had taken over the right hemisphere. Thus the right hemisphere needs to be stimulated. In the case of a late jump the situation is reversed. The characteristics of the sound vary depending on the accordance of the pupil’s reaction with the metronome beat. Four-degree scale has been utilized with time deviation thresholds and audio feedback characteristics as presented in Table 1.

Table 1 Levels of jump time synchronization
Full size table

The worse the accordance of a bounce with a metronome beat, the lower frequency of the feedback tone generated. The smoothness of the sinusoidal-shape waveform and harshness of the saw-shape waveform are intuitively associated with perfect and not ideal time synchronization by the pupil, respectively.

3.2 Exercise for therapy of dyslexia of type M (mixed)

In dyslexia of type M problems in processing information by both hemispheres can be distinguished leading to auditory-linguistic and spatial-visual disorders. Therefore, therapy should employ stimulation of both hemispheres. Another aim is strengthening the interhemispheric connections. The exercise for therapy of dyslexia of type M is similar to the one for therapy of dyslexia of type L in use of metronome beats and task of jumping in accordance with the rhythmic structures. The difference is that the task of the child is not to bounce but jump on the appropriate position of the therapeutic pad (changing a position). The position on which the jump should be performed is determined by recognizing a direction (speaker) from which the metronome sequence is being played. A jump is performed on a neighboring position in this direction. Thus, both HSS and HAS stimulation methods are combined as sound is randomly generated from the particular side and reaches one of the ears at first, and a task of determining direction involving parts of the right hemisphere responsible for spatial orientation is employed (Bakker 1990; Bates and Roe 2001). After performing a jump, the child gets, just like in the training for therapy of dyslexia of type P, graphical information about a jump precision (Fig. 6) and also gets the audio feedback. In addition, information about jump direction accordance is given in the form of wind rose colored in green for jump performed on the proper position and, contrarily, colored in red in the opposite case (Fig. 6).

Fig. 6
figure 6

The graphical user interface displayed to a pupil in the exercise for therapy of dyslexia of the type M (mixed)

Full size image

3.3 Exercise for therapy of dyslexia of type L

Dyslexia of type L is typical for pupils up to the age of 8 in whom the left hemisphere is overdeveloped in relation to the right hemisphere. Thus linguistic strategies are employed by pupils during reading when in fact perceptual ones should be used as graphical features of text are not well memorized yet. In therapy of this type of dyslexia the right hemisphere should be stimulated to recall perceptual reading strategies (Bakker 1990). The exercise begins with creating a perceptually complex letter mix using various font styles and colors and presenting it to the pupil on the monitor screen (Fig. 7). This type of stimulation is a direct implementation of the HAS method proposed by Bakker (1990). The letter mix contains syllables composing words naming animals. There is one word consisting of letters of the same font style and color contained in the letter mix. The first task of a child is to find this word in the letter mix. In the example given in Fig. 7 the proper word is: “DOLPHIN”. Other words than the one to be found are written using inconsistent font style or color thus the child knows that this is not the word of interest (eg. word “TIGER”).

Fig. 7
figure 7

Perceptually complex letter mix created using various font styles and colors

Full size image

With each position of the therapeutic pad there is one animal associated. One of the positions must be associated with the animal that is assigned to the word to be found in the letter mix. When a child changes the position walking on the floor, an animal assigned to the occupied position is displayed on the monitor screen. The task of the child is to find the position to which the animal associated with the appropriate word contained in the letter mix is assigned, by walking on the floor and memorizing associations between positions and displayed animals, like in a “Memory” game. A child signalizes that he/she has found the right position (animal) by a bounce. Audio feedback posing animal sound is given from the speaker on the left side of the child. If a bounce is performed at the correct position on the mat, a green label with the name of the animal is displayed and a new letter mix is created, a new set of animals is assigned to the positions of the therapeutic pad, and the training continues. If a bounce is performed at the position to which the animal not corresponding to the word found in the letter mix is assigned, a red label with the name of the animal is displayed and the set of position-animal associations stays the same until the child finds the correct position. Colorful varying font of the letter mix due to its perceptual complexity activates mainly the right hemisphere while searching for the word (Bakker 1990; Haan De 2001). This hemisphere is also activated by looking at images of animals and necessity of remembering where on the therapeutic pad is a place with the assigned particular animal as parts of the brain responsible for spatial orientation are employed (Bakker 1990; Bates and Roe 2001; Gazzaniga and Hustler 1999; Leckman et al. 2001). Playing animal sounds from speakers on the left side of the child constitutes the HSS stimulation method of the right hemisphere additionally (Bakker 1990).

3.4 Exercises for mentally handicapped pupils

The system contains 11 exercises/lessons, diversified in terms of the level of difficulty, which are simple for intellectually developed pupils but challenging for those that are mentally handicapped. The exercises involve various combinations of the task of searching for an image associated with a particular therapeutic mat area. By changing the position on the mat the pupil changes the displayed image and sound generated. The scheme of the exercises is fixed. A therapist/teacher can, however, adjust images and sounds to suit the educational needs, age or interests of a particular pupil. A description of the exercises is given below.

3.4.1 Exercise 1 - searching for the same picture

A pupil occupies the middle area of the therapeutic mat and looks at the screen. In the bottom area of the screen an image is displayed. This image is associated with one of the therapeutic mat areas and thus constitutes the pattern to be found by the pupil. Walking on the mat causes the image associated with the area occupied at the moment to be displayed. The task of the pupil is to watch the screen and react with a bounce when both images are identical. In such a case sound associated with the pattern is additionally played to stimulate the hearing attention. When the task is performed appropriately the pupil is awarded with a smiley and applause generated from the speakers. The exercise presented develops perception and visual memory (recognizing individuals, animals, objects) as well as forms an understanding of the term “the same”. A therapist additionally stimulates the pupil’s hearing attention and develops orientation in space and body schema by using verbal hints, such as “one step forward”.

3.4.2 Exercise 2 - associating the image with sound

The difference between this exercise and the previous one lies in the lack of image displayed above the pattern. Instead, a sound associated with each therapeutic mat square is played. The task of the pupil is to bounce when the sound reflects the pattern displayed. The exercise mainly develops audio perception and auditory memory (recognizing sounds made by animals, objects, etc.) A pupil is motivated to concentrate on auditory stimuli, isolate them, associate in memory the sound with its source and make a decision whether this is the correct sound.

3.4.3 Exercise 3 - finding a path to the image

In the bottom area of the screen an image pattern is displayed. Above it there are nine images associated with the therapeutic mat squares (Fig. 8). The pupil’s task is to find among these nine images the image that is the same as the pattern. Next, the pupil should plan a path to the image. In certain aspects the exercise is similar to the exercises already described. The exercise develops visual and audio perception as well as space orientation and designating directions from the body axis.

Fig. 8
figure 8

A view of the screen for a pupil conducting exercise 3

Full size image

3.4.4 Exercise 4 - following the sound

In this exercise the nine square scheme is displayed on the monitor screen with four corner squares painted in different colors, as presented in Fig. 9. At the bottom of the screen one of these four squares is presented to the pupil and constitutes the pattern. The task of the pupil is to find the second square in the same color as the pattern on the screen, transpose its localization onto the therapeutic pad, and move in the appropriate direction. The essential element of the exercise is the sound generated from one of the speakers that is positioned the nearest to the target square. The aim of this exercise is to develop or improve the ability to localize sound sources. Additionally the ability to differentiate colors and understanding the meaning of phrase “the same” is developed.

Fig. 9
figure 9

A view of the screen for a pupil conducting exercise 4

Full size image

4 Therapy results

The research on the effectiveness of therapy with the exercises for mentally handicapped pupils has been carried out for 8 weeks in Primary Special School No. 26 in Torun, in Poland. Eight pupils, aged 8-17, took part in therapy. The pupils were chosen based on their psychomotor skills, the degree of understanding tasks, readiness to perform the exercises and willingness to work with a therapist. Each skill had been trained with the system and next assessed using scale with three main levels distinguished (Table 2). Five groups of perceptual-motor functions have been distinguished with 12 skills in each group (Tables 3, 4, 5, 6 and 7). Half steps were used during the assessment. Therapeutic sessions were conducted regularly, once per week for each pupil. A single session lasted 45 minutes. After 8 weeks the progress was measured and registered on the development assessment scale.

Table 2 Scale used for skill progress assessment
Full size table
Table 3 Description of skills associated with employing visual perception in performing tasks
Full size table
Table 4 Description of skills associated with employing auditory perception in performing tasks
Full size table
Table 5 Description of skills associated with observed progress in motor skills necessary to perform the task
Full size table
Table 6 Description of skills associated with spatial orientation and understanding the directions
Full size table
Table 7 Description of skills associated with sense of cause and effect (attention concentration during exercising with the system)
Full size table

In Figs. 10, 11, 12, 13 and 14 an increase in progress and the number of pupils with observed progress in skills grouped by particular perceptual-motor function, after therapy with the system have been presented. Skills 11B and 12B have not been examined because the therapists had considered those tasks as too difficult to perform in the acoustical conditions in which the therapy sessions took place.

Fig. 10
figure 10

Increase in progress and the number of pupils with observed progress in skills associated with employing visual perception in performing tasks

Full size image
Fig. 11
figure 11

Increase in progress and the number of pupils with observed progress in skills associated with employing auditory perception in performing tasks

Full size image
Fig. 12
figure 12

Increase in progress and the number of pupils with observed progress in motor skills necessary to perform the task

Full size image
Fig. 13
figure 13

Increase in progress and the number of pupils with observed progress in skills associated with spatial orientation and understanding the directions

Full size image
Fig. 14
figure 14

Increase in progress and the number of pupils with observed progress in skills associated with sense of cause and effect (attention concentration during exercising with the system)

Full size image

In Fig. 15 the progress in skills which had been improved by pupils during therapy with the system has been presented.

Fig. 15
figure 15figure 15

Progress in improved skills achieved by pupils 1 - 8 (a - h) participating in the therapy exercises

Full size image

Analyzing the results, it can be observed that it was much easier to improve skills associated with auditory perception than the ones associated with visual perception. In particular, high progress was achieved in skill 4D among skills associated with spatial orientation and understanding the directions, and in skill 10E among skills associated with sense of cause and effect. The p values of the Friedman’s test for these skills equaled 0.0082<0.05 and 0.0253<0.05, respectively. The results of the Friedman’s test for both skills have been presented in Tables 8 and 9. Moreover, children found these interactive activities to be satisfying and motivating.

Table 8 Results of the Friedman’s test for progress in skill 4D
Full size table
Table 9 Results of the Friedman’s test for progress in skill 10E
Full size table

5 Conclusions

In this paper, a multimedia system aimed at listening and visual attention training that stimulate a particular brain hemisphere or both hemispheres of pupils with special educational needs, including moderate or severe mental retardation, has been presented. The system can facilitate individual educational-therapeutic programs designed for each pupil. Overall, pupils showed great enthusiasm for the proposed system and the ideas related to performing the exercises, thus they were amazingly cooperative. The high degree of motivation and interest in the interacting with multimedia equipment resulted in relatively fast progress. This progress was especially noticeable in two spheres: spatial orientation / directions understanding and sense of causation, i.e. concentration of attention and understanding of cause-and-effect relationships while interacting with the system. It is unlikely that the progress was due to the random factor as the therapists had never observed such improvements in skills when using traditional therapy. In none of the cases skill deterioration had occurred. One of the aspects that could be taken into account in the future version of the system is presentation speed, because as it was pointed out by Roginska (2012),the way in which auditory displays are presented can significantly influence the experience of listeners. Also, the transfer of skills developed during the therapy to everyday situations was noticed. For example, spontaneous utterances associated with spatial orientation and directions naming, like “Ann is sitting at the back of bus”, were formulated. The future development of the system may include an additional way of evaluating children’s performance based on the principles presented by Effenberg (2004), or may correlate more intensive-sport-like-movement with interactive sound as shown in Honer’s et al. study (Honer et al. 2011). The motor control can be enhanced by the associated sonification, and later judged whether it improves as well. It may also indicate whether a child feels fatigued, which can upset his/her motivation and learning abilities. Another direction of the system development is using music auditory displays (Kostek 2014) and a therapeutic mat which may bring an additional interest and motivation for the adolescent group of children, or enhance the system with wearable sensors (Grosshauser et al. 2012). Also, a training system based on the commercial video game consoles, which provide sensors and camera(s) to support interactive learning may be taken into consideration. However, the main problem with using sensors in case of children with disabilities may be related to two factors; the first one concerning attaching sensors to the child’s body while they typically dislike any of such an arrangement (as reported by their teachers), the second one associated with the usage of game consoles by children with mental disabilities that might be too difficult for them. However, this should be further explored, because some examples of such systems were proposed earlier for children with other types of disabilities.