Keywords

1 Introduction

Mobile application tutorials are the subject of a strong debate within the Human-Computer Interaction (HCI) community. On one hand, some researchers argue that mobile applications need to be intuitive (Lee et al. 2004), and the existence of a mobile application tutorial simply infers that the mobile application is not as usable and intuitive as it should be (Echessa 2014). On the other hand, other researchers suggest that learnability has always been important in any context, and that there are times when mobile application tutorials can be useful (Bedford 2014; Satia 2014). This is an important debate given the potential for a massive loss in revenue, as well as frustrated users, should mobile applications be uninstalled if they cannot be learnt quickly without mobile application tutorials being available.

Prior to mobile application tutorials, software and web application users had printed documentation and online help at their disposal. Yet, the availability of printed documentation and online help proved to be largely ineffective (Grayling 1998; Novick et al. 2006). Our research question was:

RQ1: Has the historical ineffectiveness of printed documentation and online help negatively influenced the perception of HCI experts in regard to mobile application tutorials?

In this work, the authors test this hypothesis. The results of this study could be important to the HCI community, including mobile application design teams, and others that wish to better understand the reasoning underpinning the mobile application tutorial debate, as well as other HCI researchers investigating this new, exciting, yet under-represented topic.

2 Related Work

The length of time it takes for a user to become capable in using any computer-based system or application has been a vital area of HCI for many years (Harrison 2013). Research suggests that the inability to become efficient in a system or application in a short period of time may affect the perceived usability of the system or application (Dumas et al. 1999; Ketola et al. 2001; Rogers et al. 2011). Further, given the distracted nature of mobile application users (Nilsson 2009; Oulasvirta et al. 2005), the ability to learn a mobile application quickly is even more important (Longaria 2004). To that end, a mobile application tutorial is one of the primary methods in assisting a user as they quickly learn how to use a mobile application (Clark 2010).

Yet, some researchers argue that mobile applications may not need tutorials, that they are simply blockers to using a mobile application (Echessa 2014). However, further research that has shown that users continue to expect some form of assistance for mobile applications (Bertini et al. 2006; Inostroza et al. 2012). Once a mobile application tutorial is available, a substantial proportion of first time users of a mobile application appear to interact with the mobile application tutorial either briefly or in-depth (Inbar et al. 2009; Tokárová et al. 2013). Well-designed mobile application tutorials, such as Intro Tours or Inline Cues (Hess 2010), will allow users to quickly scan content (Neil et al. 2014) and exit without the need to look at all pages within the tutorial (Higgins 2012). This allows the mobile application tutorial to be beneficial without being distracting nor a block to using a mobile application quickly.

3 Methodology

While each side debates their position for and against mobile application tutorials, few researchers consider if there is a reason that might better explain the difference in perspectives. Could that reason be the ineffectiveness of printed documentation and online help for desktop applications which may have influenced HCI experts, giving them a negative perception toward the usefulness of mobile application tutorials?

To explore this phenomena further, the authors examined data collected during a study which rated usability heuristics for mobile applications (Joyce et al. 2014). In particular, one of the heuristics concerned with mobile application tutorials. Survey participants were recruited through email, all of whom were authors of peer-reviewed HCI research papers. Of the 120 participants invited, 60 HCI experts hailing from 18 countries participated in the survey. 46 of the participants were HCI researchers, with the remainder primarily being HCI practitioners and HCI educators. During the survey, participants rated each heuristic using a five-point Likert Scale from 1 (Not Useful) to 5 (Very Useful) (Fig. 1), also offering free text comments. The quantitative and qualitative responses from the survey gave great insight into the perception of the usefulness of mobile application tutorials across the HCI community.

Fig. 1.
figure 1

Mobile application tutorial survey results (n = 60)

Full size image

4 Results

The results from the survey were thought-provoking in that the split within the HCI community for and against mobile application tutorials was evident. This was reflected by 26 (43 %) participants that considered a heuristic for mobile application tutorials as being useful or very useful. In contrast, 12 (20 %) participants felt that a heuristic for mobile application tutorials as being less useful. Interestingly, 22 (36 %) participants were neutral/undecided on the subject (Fig. 1).

With the split in perceptions toward mobile application tutorials clearly visible, the authors wished to explore these differences further. To that end, the authors examined how the length of time in an HCI role might have an impact, given that HCI experts with at least 10 years in their role would recall the ineffectiveness of desktop-based printed documentation and online help. Length of time in an HCI role was, therefore, classified into four groups, namely “Less than 5 years”, “5–10 years”, “11–15 years” and “More than 15 years”. Following this classification, the authors’ initial results showed that the Likert Scale modal values were found to decrease for groups with 5 or more years’ experience (Table 1). This suggested that length of time in an HCI role may have an effect the perception of the usefulness of mobile application tutorials.

Table 1. Mobile application tutorial survey responses likert scale modal values
Full size table

Further statistical analysis would determine if this was correct. As the data collected from a Likert scale is ordinal in nature, and as there are more than 3 independent samples to compare, the authors utilized the Kruskal-Wallis H test (Kruskal et al. 1952), with SPSS being used to analyze the data. A visual inspection of the resulting boxplot showed that the distribution of scores were similar for all groups (Fig. 2).

Fig. 2.
figure 2

Kruskal-Wallis H test boxplot results (n = 60)

Full size image

The results of the Kruskal-Wallis H test confirmed that the modal values did decrease. However, the results from the Kruskal-Wallis H test also indicated that the differences were not statistically significant, χ2(3) = 1.660, p = .646. Consequently, the authors’ hypothesis was rejected.

5 Conclusion

Given the prevalence of mobile applications today (Newark-French 2011; Murphy 2012; Rosenberg 2012), the decision to use a mobile application tutorial or not when designing a mobile application is an important one. In this paper, the authors investigated a potential reason for the difference in perceptions toward mobile application tutorials. This reason could have been that mobile application tutorials fill a role similar to that of ineffective traditional printed documentation and online help. Therefore, the authors explored if HCI experts with 10 or more years’ experience, who would have recalled traditional printed documentation and online help, had a negative impact toward the perception of usefulness of mobile application tutorials.

Following a survey of 60 HCI researchers, practitioners and educators in 18 countries, our research suggests that the number of years in an HCI role had no statistically significant effect on the perception of usefulness of mobile application tutorials.

This is an interesting result in that past experience with, and/or knowledge of, the ineffectiveness of printed documentation and online help seems to have not influenced HCI experts against their perception of the usefulness of mobile application tutorials. This result will be important to HCI researchers, practitioners, and educators as they consider the arguments for and against mobile application tutorials. Stakeholders on mobile application design teams, including Product Managers and Developers, that wish to better understand the reasoning behind the mobile application tutorial debate will also take an interest. As will researchers investigating the new and exciting topic of mobile application tutorials.

6 Areas for Further Research

While this research suggests that length of time in an HCI role is not a factor in the perception of usefulness of mobile application tutorials, the results confirm that a substantial proportion of HCI experts continue to consider mobile application tutorials as less than useful. This result, and the overarching debate, comes despite research which suggests that users expect some form of assistance in a mobile context.

Consequently, further research in this interesting, yet under-represented area within the Literature could help address the many questions that remain, including:

  • What factors underpin the perception that mobile application tutorials are less useful for a segment of the HCI community?

  • How effective are mobile application tutorials in assisting users as they learn to use mobile applications?

  • Which types of mobile application tutorials are more successful?

  • Why are these types of mobile application tutorials more successful?

  • Which types of mobile application tutorials are less successful?

  • Why are these types of mobile application tutorials less successful?

  • To what extent does application type, type of user, or context-of-use have an impact on the success or failure of a mobile application tutorial?

  • Which types of users utilize mobile application tutorials?

  • Can the most effective mobile application tutorials be further improved?

In an effort to address some of these questions, the authors plan to hold further cross-sectional and longitudinal studies of both HCI experts and non-HCI experts which includes observation of users, surveys, and interviews.