Keywords

1 Introduction

The mobile applications number grows each day, in February 2014 the two main application stores, Apple Store and Google Play, exceeded the mark of 1,100,000 applications available for download [1]. Along with the applications, number is the massive use of smartphones. According to a study done by Google [18], the number of smartphone users in Brazil rose from 14 % of the population in 2012 to 26 % in 2013. Given this scenario, obtaining a better understanding of how people use their mobile applications is fundamental to provide them a better user experience.

Although the usability is an important factor for the mobile devices, is not unusual to notice problems related to the usability of the applications only when they are already available in the stores. This is because the use of such software is dynamic. There are two approaches to observe the user interaction in mobile applications, lab tests and field tests. In lab tests, model, no or little influence from the external environment interferes in the test. Due to that fact, some authors argue that the usability results of mobile application studies performed in labs may not represent the application real use [22, 30]. On the other hand, the usability tests in the field observe user interaction with the application in real usage environments. However, to capture such interaction in this approach is not an easy task, to follow users in the field is an intrusive method and it may change the user behavior [8].

The mobile applications usage is relatively new when compared with the use of desktops and websites. Preliminary studies have not indicated what is the set of variables that experts can map in mobile applications to be able to acquire knowledge about the audience, context and interaction behavior. The present work had the goal to contribute to mobile devices HCI studies, considering the mobility condition through a variable systematic analysis, which can be considered for usability experience studies. As a result, this work catalogs the group of variables available in a mobile environment that help understands the application usage. The variables are the result of the review of 34 articles, and were collected in an Android application used by real users. Were collected 85,390 occurrences of device variables; 2,391,132 of interaction variables and 9,466,711 of context variables.

2 Usability and Mobility Condition

The traditional HCI evaluation methods are not applicable in an efficient way for the mobile applications, since they were projected for desktop computers which suffer little interference from the external environment [24]. Due to their nature, smartphones can be used in different daily activities, such as during exercising, commuting, during work time, in the traffic, etc. [5].

In literature, it is possible to find two main approaches for the usability studies of mobile applications: lab experiments and field studies. Laboratory experiments performed in controlled environments where the participants perform pre-defined task. Such experiments involve the observation of the participants performance while executing the tasks and the usability is evaluated during such interaction. In these cases, the control of the environment guarantees the only factors related to the experiment affect the interaction of the participant with the application. However, the usage of smartphones and applications takes place in chaotic environments and under the influence of many external factors, such as background noise, conversations, cars, people, etc. To simulate such kind of environment in a lab is a big challenge.

Tsiaousis [44] tried to replicate the main factors from environment distractions in mobile environments, however, it was not possible to predict all the situations in which an application can be used. The study did not consider a varied sample of devices, which also influence the usability of a smartphone. Taking into account a varied sample of devices in lab tests is a big challenge, since there are a great variety of devices available in the market.

Field studies consist in observation and interviews, and the usability is measured based on the execution of tasks in a real environment. While observing the users in their own environments it is possible to capture information, which affect the usage of the product, including interruptions and distractions. That is an investigation of the reality of the users and not of assumptions [4]. Field studies present benefits for the understanding of the interaction of the user with the system and how external factors affect such interaction. However, they present some problems [31]:

  • The participants have to deal with devices which they are not familiar, affecting the perception by the participants;

  • The recruiting of users is usually made locally, leading to a small number of user, which does not provide the studies with a heterogeneous sample and does not allow for the study of cultural differences;

To solve such problems McMillan [31] recommends studies in a large scale with applications available in the applications stores and participants using the devices, which they already have, and use daily, instead of using devices provided by the researcher. With the appearance of the applications stores in 2008, the researchers visualized a new way of reaching a large number of participants during their experiments. In this approach, applications are developed and published with the objective of measuring the behavior of the users and answering questions related to the study. Henze [16] developed an application to study the interaction of the users with sensitive touch screens. In the study, data of 91,731 players were collected. Analyzing the touches of the users during the game, it was possible to determine the margin of errors for different screen sizes and the position of the touches. Based on this, it was possible to develop a compensation function that reduced such margin of errors.

Through the large-scale studies, using applications stores it is possible to work in real usage environments, with users from different geographic regions and for a long period. Factors related to the usability, such as the screen size, orientation of the screen (vertical/horizontal), touches can be analyzed in different cell phone models and with a real sample of the application users [16, 35].

Such study intends to present a catalog of variables that can be collected in large-scale studies, capturing information in real time from the smartphone users. Such variables can be interconnected and help to answer important questions for the project interface decisions of the product or for the alignment of new strategies.

3 Identifying Process of Mobile Variables

One of the objectives of the present study is to identify, catalog and classify the variables, which are considered in studies about the user behavior in mobile applications. To meet such objective, the systematic review technique was used. The systematic review is a means of evaluation and interpretation of all the research studies available that are relevant for research topic of an area or phenomenon [23].

The systematic review was guided by the following research question: Which are the variables considered by the researchers to study the interaction between the users and smartphones in natural usage environments? The bibliographical research identified 1,225 potential articles, of which 34 were selected.

Besides the systematic review, it was necessary to develop three software modules. The client module is a component attached to the mobile device to collect data and send it to the server. Such module is responsible for capturing different usage contexts generating different paths for the study. The server module receives and stores the data collected in the mobile application. The data processing module executes a statistical analysis and applies the algorithms, which allow the determination of groups based on the level of similarity among the data.

In the review, it was possible to identify some of the variables considered during user behavior studies. Those can be classified into interaction, usage context and device variables. Variables of interaction are those that allow us to determine the user behavior while the user interacts with the application, for example, navigated screens, screen touches, amount of time spent in each screen, etc. The variables of context of use concern to the physical, social, temporal and technical environments [19] where the interaction took place, whereas date and time, geographic location and luminosity are examples of context variables. The device variables are those that represent the device characteristics with which the user interacted, for example, the operational system and the screen size, etc.

As a result from the systematic review, 95 variables presented in the usage behavior studies in mobile applications were identified. Among those variables, 58 (61 %) were classified as context variables, 29 as interaction variables (31 %) and 8 (8 %) as device variables. The device variables are described in Table 1, those of interaction on Table 2 and those of context on Table 3.

Table 1. Device variables
Full size table
Table 2. Interaction variables
Full size table
Table 3. Context variables
Full size table

The results of the research showed that only 7 studies used the application stores as a large-scale research mechanism.

4 Data Collection

The experiment was applied in a taxi booking application, which consists in two applications: one for the client that calls the taxi and another for the taxi driver, who is notified of the call. The applications are available in the Google Play and Apple Store applications stores and are used by real users. The application user base is made of 405,000 users, distributed in eight Brazilian cities, including people of different socio-economical profiles, ages and levels of familiarization with mobile devices. Another factor is the diversity of devices in which the application installed, which range from devices with few features and resources and small screens to devices with many features and resources and large screens.

The collection component for the mobile platform was initially developed for the Android platform. In this component, a service was developed that is initialized when the application is opened, which is responsible for collecting the variables coming from the device sensors and the variables that represent the device characteristics. To collect the interaction variables, it was necessary to intercept the touch events in each of the application screens.

Among the variables presented for this study, 54 variables were collected, of which 41 represent the usage context, six represent the interaction and seven the device. The collection period in the client application was of 30 days, being collected 85,390 occurrences of device variables; 2,391,132 of interaction variables and 9,466,711 of context variables. The application data of the taxi drivers was collected in a period of 25 days, being collected 76,557 occurrences of device variables; 1,891,092 of interaction variables and 75,080,231 of context variables.

5 Data Analysis

In the initial analysis only the passenger’s application data were considered, which correspond 20,829 the sessions of use. The application is used in 465 different devices, being the MotoG-XT1033 the most used one. For each session of use considering the user actions from the application opening until its closing. The analysis of the data collected reveals that on average each session lasts 2 min and 10 s, with an average of four screens viewed per session. To determine this information the variables date, session start time, session end time and number of navigated screens were considered. The analysis of the collected data indicate the following behaviors:

Which is the period of the day with more use of the application? Using the variables that mark the beginning and end of the session it was observed that the application is most used during the period of the night. Understanding the usage behavior based on time provides inputs for improvements in the interface design allowing tailor the application for nightly use. It’s also possible to determine in which periods of the day, new additional resources should be provided for the application infrastructure and what are the best time frames for application maintenance.

Where the user does uses the application? The geolocation mobility variable was used to indicate the regions with greater intensity of use in Brazil, highlighting the cities of Sao Paulo and Curitiba, as shown the map in the Fig. 1. Understand application usage by regions contributes to build users profile and can help on direct marketing campaigns. For the analyzed application, this information can be helpful to guide the growth of the taxi network.

Fig. 1.
figure 1

Application usage by geolocation

Full size image

Does the user use the application on the go? Which screens are used in motion? To answer these question the variables movement and navigated screen. To determine whether the user is in motion the device accelerometer was used. It has concluded that 48 % sessions were performed in motion. The screen which the user determinates the address for the taxi ride was the most used on the move. Understanding this type of factor helps to do optimizations in application design in order to facilitate use of the tasks that are used while the user is moving.

Can location accuracy cause some impact on the use? Indicate the pickup location when ordering a taxi is one of the most important tasks in the analyzed application. Currently the address is automatically detected by the application, allowing the user to change it if it is incorrect. It was decided to relate the address editing task with location accuracy, to understand if the low location accuracy would lead users to correct the address suggested by the application. The results showed that in 75 % of the time in which the user has corrected the suggested address, the location accuracy was below 60 meters. This information drives the interface improvements to offer mechanism to facilitate the address edition when location accuracy is below 60  m, before the user search for this functionality.

How often the users interrupt the application usage? To answer this question the application in the background variable was considered. It was found that in 85 % of sessions, users left the analyzed application, with an average frequency 6 times per session and keeping the background application on average 9 min and 12 s. All the screens in which this behavior happened were identified and what were the average time each screen remained in the background. This information leads to optimizations in the navigability flow of current screens and helps to identify in which screens the users loses interest or witch screens there is no need of interaction with the user.

6 Conclusions and Future Work

This work presented a study of variables that can be considered to understand the smartphone usage behavior through the collection in large scale in one application used by real users.

As a result it was possible to catalog a set of variables that were considered on user behavior studies in mobile applications and showed the application of some of this variables. To exemplify the use of some variables the data analysis was guided by the following questions: Which is the period of the day with more use of the application?; Where does the user uses the application?; Is the application used on the go by the user?; Which screens are used in motion?, Can location accuracy cause some impact on the use? and How often the users interrupt the application usage?

This research is part of an ongoing study, which aims to understand which are the variables that can be considered in studies of user behavior on mobile applications. The next steps will be: perform the correlation with more variables in order to determine which ones produces better results; identify personas through the collected variables; improve the data processing component to perform data normalization and analysis incrementally; study and apply pattern discovery techniques.