Keywords

1 Introduction

There are various ways of remotely recording health-related data, facilitated by the development of new technologies [30]. However, the usage of these technological applications are not always backed by scientific evaluations and thus the safety of users can be at risk [20]. This is especially true if the users are members of a vulnerable group, such as chronically ill cancer patients. Further, it is hard to quantify the benefits of new technologies, especially when compared to analog methods of gathering data (e.g. through handwritten health diaries).

The project Interacct has the aim to foster communication between young leukaemia patients and their treating physicians using a mobile serious game. The project is conducted by the St. Anna Children’s Hospital ViennaFootnote 1, in cooperation with the University of Vienna (Research Group Entertainment Computing)Footnote 2 the University of Applied Arts ViennaFootnote 3 and T-Systems AustriaFootnote 4. Interacct includes a serious game for patients with a health reporting function, as well as a web-interface for physicians to evaluate and observe the submitted data.

The target group of project Interacct consists of children and adolescents who have been treated by an allogeneic hematopoietic stem cell transplantation (HSCT). This method is used to cure diseases like red-cell disorder or leukemia. After treatment, patients struggle with a variety of symptoms, risks, and sequelae [3, 26]. Although many symptoms occur independently of the cancer type and the treatment method [6, 31], no screening tool exists that can be used to query symptoms in a standardized way [8].

At the St. Anna Children’s Hospital, health monitoring of HSCT patients in childhood and adolescence is currently based on a paper diary. In this paper diary, the young patients document health-related changes on a daily basis. Once they visit the clinic, they hand over the diary to their aftercare physician. This procedure is accompanied by two major disadvantages. First, the adherence of patients regarding paper diaries is low [19, 27]. Second, the diaries are often filled later and not regularly, which is partly caused by the low adherence [16, 25]. These disadvantages may be prevented by real-time electronic symptom checking, for example via smartphone app [7, 18]. If information about symptoms is transmitted daily and directly to the physician, negative changes in health can be detected quickly without visiting the hospital. Therefore, patients can receive an immediate response after symptoms have been reported and can seek medical care or initiate coping strategies [14]. Due to the timestamps of electronic data, it is furthermore easy to arrange and evaluate data according to the date of entries [25]. Another advantage of mobile applications is that children and adolescents feel more comfortable with the electronic exchange of sensitive information [1, 2, 15].

Although a lot of health-related software is available, a review published in 2015 revealed only four pilot-tested smartphone applications for young cancer patients [28]. Often medical professionals are not included during the development of health-related software [14], but these four apps are theoretically based and evaluated scientifically. Currently, no study exists that has developed as well as evaluated such an app and compared its content directly with the traditional paper method. It is also unclear, whether a meaningful health profile can be conveyed via electronic data transmission without seeing the patient personally, due to the rare involvement of physicians in the development of those apps.

Therefore, this study aimed to develop and test a serious game for HSCT patients that provides information about the health condition of treated children and adolescents. Physicians evaluated the information content delivered through the serious game and compared it with the information obtained from the paper diary. The information content of the serious game was expected to be significantly higher than the one derived from the diary.

1.1 Related Work

Hochstenbach et al. tested the feasibility of a mobile and web-based self-management tool for outpatients with cancer pain. Patients (n = 11) and nurses (n = 3) used the tool for pain monitoring, medication monitoring, and educational sessions. Results show, that patients and nurses were positive about using the tool. The authors conclude, that the system demonstrates feasibility in everyday practice [11]. A similar system is \( {Pain Buddy}\), which is discussed by Fortier et al.: young cancer patients keep track of their pain during cancer treatment using tablet computers. The provided app uses an animated avatar and gamification components, as well as remote symptom monitoring. A pilot study has shown, that patients (n = 12) were highly satisfied [10]. In \( {Pain Squad}\), adolescent cancer patients keep track of their pain during therapy through an iPhone app. The app was found to be appealing to the patients, which resulted in high compliance rates [24]. A more specific approach was investigated by Rodgers et al. by implementing \( {EAT!}\), which should assist adolescents with self-management of eating-related issues during HSCT recovery. While patients (n = 16) initially embraced the app, the use decreased extensively over time. Authors suggest further development and studies [22]. Baggott et al. implemented an electronic diary for adolescent cancer patients. In a 3-week trial, patients (n = 10) showed high adherence (\(\ge \)90%) [2]. Wesley and Fizur reviewed several studies dealing with cancer treatment with app support. They conclude positively but state the demand for more empirical data and further research effort [29].

Outside of the field of cancer patient aftercare, Charlier et al. conducted a meta-analysis on the effect of serious games for improving knowledge and self-management in young people with chronic conditions. The authors conclude, that using serious games can significantly improve self-management as well as knowledge transfer [4]. Fishbein et al. give suggestions for the design and development of mobile applications to promote oral drug adherence and symptom management during chemotherapy [9]. Klaassen et al. have evaluated a coaching and gamification platform for the self-management of young diabetes patients. They give concise suggestions for the development of coaching and gamification platforms in medical practice [13]. Price et al. developed Painpad, a tangible device to self-log pain, and have shown increases in both frequency and compliance with pain logging [21]. They further note that data self-reported this way is more faithful than data reported to nurses.

2 Methods

2.1 Study Design and Statistical Methods

Patients and students between ages 7–18 participated in this study to compare the Interacct serious game app with a paper health diary. All participants were asked to use the diary as well as the serious game for 5 days (Monday-Friday) and to document given body function parameters (e.g.: tiredness, appetite, pain, etc.) as conscientiously as possible. Only participants who used both, paper diary and app, for at least one day were included. Due to this criterion, 10 students were excluded prior to the analyzes and the final sample consisted of 15 patients and 27 students. Due to organizational reasons, a completely balanced design (crossover between starting with the diary or the app) could not be realized and the diary was always used before the app.

The information content of the diary entries and the app data was checked by two independent, blinded physicians. The physicians used a scale ranging from 1 to 5 points to evaluate the information. One point reflects low information, whereas 5 points indicate a high information content. The interrater-reliability was evaluated by calculating the intraclass correlation coefficient (ICC), which was based on a two-way mixed model and the absolute agreement. Subsequently, a one-way repeated-measures ANOVA was used to check whether the information content of the app differs from the diary. Also, the between-subject-factor group was included to examine any differences between patients and students. T-tests for associated samples were conducted post-hoc to calculate contrasts between the groups.

After participation, the children were asked to answer a usability questionnaire regarding general user satisfaction as well as the suitability of the app design. User satisfaction was measured on a 5-point Likert scale, where 1 represented low values (“not at all”) and 5 high values (“extraordinary”). To evaluate the app design, participants could award 3 points for each app-function, with 3 points being the most positive rating. The median and the interquartile range (IQR) were calculated for each item. Additionally, participants had the opportunity to write down feedback to reveal additional ideas for the improvement of the app.

2.2 Technical Components

The technical components used in the study were a Unity 3DFootnote 5 app for the patients and an ASP.netFootnote 6 web-interface for the physicians. The persistence layer was implemented through the ASP.net-backend, accessing a Microsoft SQL database, running in a secure Windows Server 2016 environment. The connection between clients and server was established using Secure Socket Layer - SSL. Basic authentication and cross-site request forgery (CSRF) tokens were used to ensure data integrity and security.

2.3 Interacct Client

The Interacct client is a native smartphone application for Android and iOS, developed in Unity 3D. It was used as the main tool for the presented study. The main components of the client app are remote medical data entry as well as the game content itself. Completing the medical data entry survey, users were rewarded with virtual in-game currency, used to progress in the main game. Registration of new users was only possible through the administrative team of the project.

The following subchapters will describe the main components of project Interacct. A detailed project description, as well as design considerations, are covered by Kayali et al. in [12].

Game Design. The core game idea of Interacct is to collect avatars and complete procedurally generated levels. The avatar explores an island and needs to fight off hostile NPC (non-player characters) monsters to complete a level. Fights with the NPC monsters are easier for the player, once the avatar has reached a certain level or skill set. To upgrade an avatar, the player can spend science points, which are earned through the completion of health reports. At the end of each island, a boss monster is encountered and has a chance of dropping an egg shell, which eventually allowed the player to hatch a new avatar. The levels come in different graphical settings and with different NPC monsters to provide variety.

Health Reports/Remote Data Entry. Within the Interacct serious game, users can to report a set of health parameters. The parameters mostly follow a 0–3 scale and were adapted to match LOINCFootnote 7 codes, where possible. The parameters are listed and explained in Table 1. The health report data could be submitted several times per day by each user and was accumulated by the backend for each full day. For completing the report for the first time each day, the user was rewarded with science points, a virtual in-game currency, which could be used to extend the capabilities of the user’s avatar. Fig. 1 shows the health report UI with all health parameter categories. Fig. 2 shows the input for fluid intake amount and if the patient felt any pain during fluid intake.

Fig. 1.
figure 1

Interacct serious game client with the health report categories

Full size image
Fig. 2.
figure 2

Interacct serious game client showing input for fluid intake amount and fluid intake pain

Full size image
Table 1. Health parameters and their meaning, used by the Interacct serious game/health report tool
Full size table

3 Results

A high degree of interrater-reliability was found between the physicians. The average measure ICC was .867 (95% confidence interval [CI] from .733 to .932, F(41,41) = 8.530, p \(\le \) .001) for the paper diary and .912 (95% CI from .794 to .958, F(41,41) = 14.001, p \(\le \) .001) for the app. Over the entire sample, the physicians gave an average of 3.77 (± .91) points for the information content of the diary and 4.14 (±1.14) points for the app. The repeated measures ANOVA revealed a significant difference between information content of the diary compared to the app (F(1, 40) = 5.571, p = .023, \(\eta \) = .12). This means that the app provided significantly more information than the diary. The between subject factor group was not significant (F(1, 40) = .522, p = .474) and also no significant interaction effect was found (F(1, 40) = 1.807, p = .186).

Although there were no group-differences shown in the ANOVA, post-hoc t-tests revealed that the significant main effect was primarily driven by the patients (Fig. 3). Patients received an average of 3.73 (±.63) points for the paper diary and 4.43 (±1.07) points for the app. This difference was significant (t(1,14) = \(-2.941\), p = .011), while there was no significant difference in information content for students (t(1,26) = \(-.772\), p = .447). The information content of their diaries was rated with 3.78 (±1.05) points and the app with 3.98 (±1.17) points. The information content of the students’ diary and app was very similar to the information given by the patients in the diary. These results indicate that only the patients, who were probably the more intrinsically motivated group, documented the body function parameters conscientiously. It is quite possible that the students revealed rather superficial information.

This assumption is supported by considering the number of days participants used the diary and the app. Patients used the paper diary for an average of 5.67 (±1.23) days, which was more than the required 5 days. The app was used by this group for 4.67 (±2.53) days, but the difference to the diary was not significant (t(1,14) = 1.479, p = .161). The students used the diary as long as patients (5.89 ± 1.31 days), but there was a significant difference in the use of the app (t(1,26) = 6.138, p \(\le \).001), which they used only 3.41 (±1.69) days. As mentioned in the introduction, it is easy to fill the diary afterward, which is impossible with the app. Therefore, the probability is quite high that especially the students did not keep the diary up to date.

Furthermore, user satisfaction, as well as the evaluation of the app-design, were positive (Table 2). There was no evidence that the diary was preferred to the app.

Table 2. User satisfaction and evaluation of the app design
Full size table
Fig. 3.
figure 3

Main result: awarded points for paper diary and app compared between patients and students (error bars: 95% CI)

Full size image

4 Discussion

4.1 Main Findings

High usability of an app is not sufficient to motivate patients to enter data in the long term [23]. In addition to usability, illness experience, information technology infrastructure, emotional activation, degree of burden caused by the app and relevance of symptom monitoring are important factors for patient motivation [5]. Especially children and adolescents who are not interested in health-related symptom tracking and changes, such as the students participated in this study, are quickly demotivated if they must enter a large amount of data every day [17]. The fewer data users are asked to enter, the greater is the level of adherence in general [18].

5 Conclusion

In the presented study, the informative content of health data of a handwritten health diary was compared with health data submitted through a serious game. The serious game was implemented in the project Interacct, in which patients can submit health data through a mobile app on a daily basis to receive in-game rewards. Patients and students submitted 2 weeks of consecutive health reports, first with the Interacct serious game, eventually with the paper diary. Two independent, blinded physicians reviewed the health reports and rated the informative content.

The main hypothesis is, that the informative content of health data, which was submitted through a serious game, is higher than the informative content of handwritten health diaries. The statistical results confirm this hypothesis significantly. This is especially important for young and adolescent cancer patients, where the results were even more conclusive than in the control group of students.

Considering the positive side effects of the serious game (such as high engagement, compliance, and pleasure reporting health data through a serious game), the authors propose to conduct further studies and experiments regarding this topic.