Keywords

1 Introduction

The mobile market in China has boomed recently. By the end of 2014, the total number of mobile internet users in China reached 0.56 billion, 85.8 % of internet users in all [1]. The applications in the Apple App Store and Google Play added up to 2.64 million by January 2015 [2]. Many practitioners in large internet enterprises in China have endeavored to adopt User-Centered Design (UCD) methodologies, emphasizing specialized methods of end-user research before the product is made. Methods typically used in mobile user experience research include lab-based usability studies, interviews, focus groups, lab experiments, log analysis, as well as observational and ethnographic research [3]. However, with the boom come challenges for application design and experience research, as designs of potentially complex applications need to be produced and updated at a fast pace. In many design programs, the above methods need to be combined to obtain solid results in a short period.

In this paper we will describe a typical practice using the UCD process for minimal cost to ensure user experience in fast iterative development, in which we combined several methods with the aim to overcome some of their limitations when used in isolation. We report key findings, and highlight the methodological insights we gained in our practice.

2 Baidu Map for Mobile (BMM)

The studies we are reporting investigated the user experience of Baidu Maps for Mobile (current version available for download at wuxian.baidu.com/map) [4]. Baidu (Nasdaq: BIDU) was a Chinese internet search service provider founded in 2000 by Robin Li, providing users with many channels to find and share information via its core web search engine and many more services [5]. BMM allows users to access the full set of maps (including street maps and satellite images) with traffic information available via Baidu Map [6], as well as Baidu’s local search index (i.e. businesses and services, landmarks, facilities, etc.). Additional features include speech navigation for driving and walking, on-the-road sharing and chatting, and favorites, among others. By June 2015, its market share in China exceeded 70 % [7].

The Location-Based Service User Experience (LBS UE) Group has been dedicated to building user interface standards and promoting consistent brand experience for BMM from 2014. As BMM is part of a rapidly growing class of mobile mapping, navigation and location-based service applications, its designing balances navigation tools and shopping services, and requires consideration of multiple use scenarios. It also calls for cross-cultural thinking to design for other Asian countries (e.g.: Japan, Korea, Singapore, and Thailand). By the end of 2015, the LBS UE Group is a team of over 50 based in both Beijing and Shenzhen, China, consisting experts in areas of user interface design, interaction design, and user experience research.

3 Case Study: An Innovation of Simpler Interface with More Flexibility

3.1 The Design

Before 2015, Baidu Map for Mobile (BMM) utilized the popular interaction architecture for map applications in China, with a search box at the top and other major features (i.e.: Local, Route, Car Navigation, and Mine) listed side by side at the bottom of the homepage. However, after local services were enriched and prioritized, the previous interaction design faced major challenges. The first challenge was to emphasize local services as a strategic focus, which was hard since too many buttons and tools attracted attention on the homepage. Second, each time a local service (e.g.: the Chauffeured Car service) needed traffic diversion, a new shortcut icon had to be added to the homepage, undermining the interaction standardization. Hence we decided to make improvements on the product design.

Figure 1 illustrates the old design and the design solution for test after several rounds of feature carding and brainstorming among the team. An outstanding achievement in the new design was reduction of numbers of buttons and colors applied. Only two round buttons were present at the bottom, the Local (marked 1 in Fig. 1) and the Navigation buttons (marked 2 in Fig. 1), to highlight the importance of the Local feature. The feature Mine was moved to the left of the Search Box, represented with an image icon (marked 3 in Fig. 1). Car Navigation was shrunk under Travel, as listed below the Route feature on a semi-screen layer triggered when the Travel button was clicked (marked 4 in Fig. 1). Other travel services were also listed on the layer, such as the chauffeured car services, and services for drivers like fueling and washing. They were listed vertically, instead of in a matrix, to allow more space to display promotional activities. Colors for the two buttons were selected in consistency with the Baidu logo, with red indicating life and consumption, and blue indicating tool and efficiency. The design was also inspired by the current trend to simplize mobile user interfaces, as evident in many applications including Google Maps for Mobile (e.g.: version 4.14.74645), but the process to earn executive votes proved to be challenging given the many features in BMM.

Fig. 1.
figure 1

Homepages of BMM version 8.5.0 (the old) and version 8.5.5 (the new for trial), with major changes marked in numbers.

Full size image

3.2 Study Method

We wanted to evaluate the overall user experience of the new design. A trial version (version 8.5.5, 8.5.0 as the official version available) was developed for the study. We then ran online A/B experiments to quantify its acceptance and learning cost in real-life contexts, and conducted an online survey and usability tests as well to understand user’s behaviors and opinions. The entire process lasted for half a month.

For the online A/B experiment, we sampled 1.5 million random Android called user identification numbers (CUIDs) as well as all users of Android version 8.4 of BMM, and sent upgrade tips of trial version 8.5.5 to them via in-app pop-up notice and app pushes over a 5-day period. After the number of daily active user (DAU) stabilized, we started the experiment and monitored online log data for 5 days.

For the survey, we posted an online questionnaire with a banner in a third-tier menu in trial version 8.5.5, asking participants to rate their attitudes to the trial version. For the usability tests, a total of 8 participants were recruited to try our trial version for typical tasks and interviewed after in our lab. The participants consisted of 5 male and 3 female. 5 participants were frequent users of BMM, while 2 used Amap for Mobile, another popular mobile map application in China, and 1 used both for daily navigation. 4 participants drove more while 4 took public transport more in daily life. All participants were used to searching local services via mobile map application.

3.3 Results

Online A/B Experiment.

The average DAU of the trial version was 54,224. Figure 2 shows the click-through rate (CTR) and search rate of each major feature on the two versions in percentage of DAU over the treatment period.

Fig. 2.
figure 2

CTR in percent of the search box, the local button and the travel button, as well as the search rate in percent of Point-of-Interest (POI) in both versions.

Full size image

On the trial version, the average CTR of the Search box was 54.06 %, much higher than the 43.61 % on the official version. The actual search rate was also higher on the trial version. This suggests that compared with the previous version, user tended to search more because of the new design.

The CTR of the Local button on the trial version averaged 12.45 %, while that on the official (control) version was 10.66 %. Difference between the two versions narrowed as the experiment went on, which indicates that simply enhancing the visual importance of the button had little effect on its actual usage.

CTR of the Travel button averaged 24.9 % on the trial version, while that on the control version averaged 27 % (CTR on the control version equaled CTR sum-up of the Route button and the Car Navigation button). On the trial version, CTR of Travel button didn’t differ much from that on the control version in the beginning, but slipped a little later. On the fifth day, the difference between the two versions reached 3.28 %. These results suggest that cutting out the Car Navigation button caused small but limited effect on the usage of the feature.

We went on to investigate the entire usages of Route Search and Car Navigation in the two versions. Figure 3 illustrates CTR of Route and Car Navigation on the homepage versus the actual search rates regarding the two features. While CTR of both Route and Car Navigation via the Travel button dropped, compared with the control version, the actual search rates increased instead. This suggests that although the new design altered users’ click flows– instead of using the two features directly on the homepage, slightly more users were led to type in the destination before searching route or using speech navigation through the box– it didn’t shrink their actual navigation demands.

Fig. 3.
figure 3

Rates in percent in both versions throughout the treatment period, including CTR of route Search via the travel button versus actual search rate of routes (above), and CTR of car navigation via the travel button versus actual use of car navigation (below).

Full size image

Survey and Usability Tests.

We collected 4331 valid questionnaires from our channel in the trial version. Among them, 78.3 % agreed or strongly agreed that they were supportive of the version update. When asked about their preference between the previous and the new version, 50.2 % participants favored the new one. These results suggest obvious preference for the new design among the test users. (Fig. 4)

Fig. 4.
figure 4

Participants’ attitude towards in trial version (left), and preference between the two versions (right). (Color figure online)

Full size image

Face-to-face tests and interviews were conducted at the same time to investigate the usability of the new design. We focused on the most distinct change in the new design, which was the integration of the Route and the Car Navigation buttons into the Travel button. All participants understood the “Travel” button and clicked it when they needed to search routes or start speech navigation. The extra steps that it required to use the two features were still within their acceptance. However, participants found it weird to see services such as fuel charging on the semi-screen layer triggered by clicking the Travel button. A few participants tried several times to find the Mine icon, and found it unlike a clickable button and inconvenient for one-hand operation. Generally speaking, the participants found the trial version “awesome” and felt “a thicker taste of design”.

3.4 Findings and Improvements

The main results from the study were summarized as follows:

  • Most users preferred the new design to the previous version.

  • Due to reduction of buttons, users tended to use the Search box more in the trial version v8.5.5 compared to the official one v8.5.0.

  • Instead of clicking the buttons on the homepage to search routes or use speech car navigation, more users search destinations first in the new design. This alternation in interactive flow, however, didn’t cause the actual search rates to fall down, which means that despite of button reduction, the trial version could still satisfy users’ navigation demands, only in a new way.

  • The alignment of fueling services with route search was not comprehensible.

  • The icon of Mine was not imagic enough for BMM’s users.

We moved on to improve the design based on the study results, and the final interfaces were shown in Fig. 5. These were also the final designs that we released in version 9.0.0. Major improvements included:

Fig. 5.
figure 5

Improved and released interface designs based on the study results

Full size image

  • The Mine button was changed into users’ personal avatar to be more conspicuous (marked 1 in Fig. 5).

  • The Local button was adjusted into a small bar to the bottom edge (marked 2 in Fig. 5), which could be swiped up into a semi-screen layer listing all services provided in BMM, including food, drink and entertainment, services for drivers like fuel charging, and so on (marked 4 in Fig. 5). This layer could be further pressed and dragged to cover the entire screen, showing the full spectrum of services. This design reduced the interaction steps needed to use local services, and left the right of the bar a fixed place to expose promotion activities on the homepage.

  • When clicked, the Travel button switched into a new page instead of a semi-screen layer, where users could flick horizontally to change the navigation as wanted at the top (e.g.: bus route, driving route, bicycle route, Chauffeured Car service, etc.) (marked 3 in Fig. 5). And the button was redesigned to incorporate the previous icon for Car Navigation.

  • The beginner’s guide was presented in semi-transparent layers whenever a new design was triggered in real-life use, so that users could better understand the brand new interface.

4 Discussion and Conclusion

The paper described an agile testing methodology, integrating online experimentation, survey and usability testing, to validate mobile designs in Chinese enterprises. An empirical case was presented of the methodology validating and improving the innovative design of Baidu Mobile Map (BMM). It is evident that this methodology produced quick and persuading results to push forward design innovations in the enterprise.

Our experience with the application has also resulted in a number of lessons learnt. These could be considered by organizations planning for similar design innovation.

  • Reduction of first-tire buttons doesn’t necessarily result in usage decline, as long as the alternative path is natural and convenient enough for application users. Thus it is strongly recommended to re-consider application interfaces and flows facing inflating features under changes in technology and business models.

  • In the mobile context, convenience is especially important given the usually short time span that users could spare for each application. Half-screen draggable layers could shorten interaction paths, and have been proved to be acceptable for users in our experience. As handset screens increase in size, which is a trend witnessed in Asia, this design will probably become more popular.

  • It is advisable to leave space for exposure of promotional activities in mobile application designing, especially when the application is facing an intensive competitive market or in the process of introducing new features.