1 Introduction

The rapid economic growth and significant market potential of China attract a growing number of global companies. Brejcha’s studies on cross-cultural User Experience (UX) design pointed out that Chinese users have different preferences for user interface from European users [1]. Companies aspiring to enter the Chinese market should understand the needs and behaviors of Chinese consumers. Even though many of these companies have mature UX teams, they have minimal data or experience conducting UX research in China. The UX community in the US still lacks industry best practices for international user research [2]. Ethnography studies are usually recommended as the first step for studying users in another country, however, many companies do not have the time or budget to carry out long-term qualitative studies.

Following the acquisition by a Chinese Consortium in 2016, Lexmark shifted its focus to the Chinese market. However, lacking previous research efforts, teams at Lexmark had little data about printer users in China. The marketing team, information development team and the design teams have all been asking questions about the preferences and behaviors of our Chinese consumers but answers were not available. The User Experience Design (UX) team at Lexmark decided to take on the lead to conduct user research. As a small team with limited resources, the UX team had to explore practical ways to initiate UX research on users in China.

In the effort to develop an overall understanding of printer user needs in China, our team planned two survey studies and a diary study. The survey studies were used to define target users and understand their printing needs. The diary study was designed to examine how Chinese users use printers in real life settings. The first round of survey focused on collecting data on printer usage, printing needs and preferences. The second survey aimed to follow up on observations and questions generated from the first survey that require further clarification. Diary study is an economical alternative to field studies [3], where researchers can study user behaviors in context without travelling to observe users. The diary study would help us complete our data collection by probing into users’ reasons and attitudes. We planned to carry out the two survey studies first, followed by the diary study. We would recruit participants for the diary study based on the user data drawn from the survey studies.

We completed the first survey study in December 2017, where we encountered and resolved many challenges. These challenges included finding the right services to reach out to the target population, developing a localized research toolkit, and overcoming logistics issues. In this paper, we shared our experiences in how to identify research tools suitable for UX research in China, and how to examine the feasibility of potential research tools. At the end of this paper, we proposed guidelines for conducting remote user research in China. We hope the learning from our research journey could provide help to UX teams in the US at the early stage of planning user studies on Chinese users.

2 Related Work

A few research papers discussed how to conduct international user research. The International Research Toolkit [4] from Facebook Research centered around field studies without discussing budget or time constraints that most UX teams in the industry must face. Biesterfeldt and Capra [5] offered a step-to-step guide for conducting international UX research. The authors proposed high-level approaches from each stage of the research projects, from planning the research, designing the research, conducting fieldwork, to data analysis and reporting. Recommendations from these studies were useful, but they gave little guidance on the feasibility of each step. When it comes to research planning, we have to stay pragmatic. Some ideas might sound great, such as Biesterfeldt and Capra’s suggestions on establishing a long-term relationship with local vendors, but would not work well as we were tasked to gather reliable user data within a limited time frame. We were at the early stage of our research journey with a small budget. In addition, our team did not have established methods to handle international payments or vendor relationship with research firms in China. These logistics details might not seem to be a part of UX research, but we had to resolve these issues before we could execute our research plans.

Existing studies on international UX research usually compared user groups from different countries. Walsh et al. [6] found online storyboards to be an effective way to help participants from different cultures understand the targeted use cases. But we wanted to focus on Chinese users only, without comparing user behaviors from different countries. To fill in the gap of practical guidance on remote user studies in China, this study solely focused on conducting early user research in China. Some of the challenges and roadblocks discussed in this paper were very China-specific. Our goal was to provide detailed recommendations and practical suggestions to help other research teams find resources and avoid pitfalls.

3 Methods

3.1 Our Team

Lexmark has a small in-house User Experience Design team led by veteran UX researchers. Our team has done user interviews and usability testing for the past 20 years. We have profound knowledge in UX design and research for the printer-related market. However, our team has minimal knowledge about the Chinese market that Lexmark is entering. Our bilingual team members who are fluent in both Chinese and English led the efforts in researching and comparing the available local research platforms, communicating with local agent for logistics, and handling the definition and translation of the research materials.

3.2 Research Goals and Plans

Lexmark sells printers to enterprise customers and individual consumers. It is imperative for Lexmark to know the specific needs for both kinds of users. Surveys are a cost-effective tool for collecting data from a large sample in a short amount of time [7]. Our first survey study is designed to help us understand user segments and their printing behaviors. We defined three types of printer users: (1) people who primarily use printers at home, (2) people who primarily use printers at work, and (3) people who primarily use printing services, like printing shops. We didn’t limit the type or brand of printers, because our interest was in the existing printing behaviors.

The first survey started with questions about participants’ printing needs: “How often do you use printers?” and “Where do you use printers?” Based on each user’s responses, we classified the user into one of the four categories (home users with one printer, home users with multiple printers, work users, and printing service users) and asked questions specific to the user category:

  • For home users, we asked about their printer (brand, type, display type), what they use the printer for, how they use the printer (“How do you send documents to your printer?”) and printer ownership related questions (“How much did your printer cost?”).

  • For work users, we asked about their usage of features related to enterprise printers, how much they print and what they use printers for.

  • For printing service users, we asked them about where they satisfy their printing needs (“Where do you usually go to perform printing jobs?”) and printer ownership (“Why don’t you have your own printer?”).

We asked demographic questions, such as age, location and smartphone brands to all of our participants.

For this survey, we planned to recruit 800 printer users in China with at least 100 respondents in each of the defined categories. We needed to find a survey platform with a large user base in China. Also, the survey platform should provide sampling services that allow us to limit participants to printer users.

The second survey and the diary study were defined but not carried out yet. The second survey was designed to follow up on observations from the first survey. We planned to collect at least 800 responses from the second survey. We used the diary study as a complementary tool to the survey studies. Diary studies are useful for collecting in-context data on users’ behaviors and thoughts over a period of time [9]. We were interested in conducting a one week-long diary study to study how users interact with printers and how they deal with troubleshooting if applicable. We planned to recruit 10 to 20 printer users in China and collect data through a cloud-based collaboration tool.

3.3 Research Preparation

In order to properly develop the survey questions, we examined qualitative data from Chinese forums, Q&A sites and news articles. We read printer-related threads on Chinese forums, such as ZOL.com, online customer reviews on Lexmark printers, and printer purchase and printer repairment related discussions on Q&A sites such as Zhihu.com. We also read news articles relevant to the printer and printing business. From forums and Q&A sites, we learned customers’ preference and pain points about printers, which provided important guidance for our survey study.

Translation is another key component for overseas research. Our bilingual team members decided to translate all research materials and data instead of hiring external translators. The benefit of having the same researcher as the translator is that the researcher knows the content and thus is less likely to have critical information lost in translation. We drafted our survey questions in English, so that all team members could review the questions and provide feedback. After finalizing the questions, our bilingual team member translated the survey into Chinese. Different from English, Chinese language has honorifics. We carefully selected the proper tone when phrasing the questions. We also chose terminologies with extra caution. For example, we were not sure how Chinese consumer would interpret the meaning of “打印机 (printer in Chinese)”. Would they think a 打印机 (printer) is a machine that only does printing or makes copies as well? We did guerilla interviews with seven Chinese users and asked them about how they would define printers, copiers and scanners. We received seven different answers from these users. Therefore, we added definitions of terminologies in the survey to avoid potential misinterpretation of the questions.

3.4 Localization of Research Toolkit

Our UX research team has a defined research toolkit for the US market, with a partner vendor for participants screening and recruiting. We typically use survey distribution platforms such as Survey Monkey to collect nation-wide user data. Communicating with vendors and participants is usually done through emails and phone calls. We also use Google Drive for document-sharing and collaboration.

Unfortunately, these commonly used tools and methods are not available for research in China. Our vendor does not have any international connections. In addition, Google Drive and Survey Monkey are not available in China. We considered three factors when developing our research toolkit: (1) How laws and regulations in China affect the availability of tools, (2) Habits and behaviors of Chinese users, as it is unlikely for them to adopt new tools just for this study, and (3) Cost effectiveness: the qualified tools should be free for the participants to use, and can also generate reliable output at a reasonable cost for us.

The Great Firewall and Real-Name Registration Law

We are aware that the “Great Firewall” in China has blocked a list of websites including some of the most popular products in the world such as Google, Facebook, WhatsApp and YouTube. We had to find the Chinese substitutes for the handy research toolkit we were used to.

During the exploration of different products in China, the first roadblock we encountered was the real-name registration law enforced in 2017 [8]. This law requires internet services providers and companies to verify users’ identities. Since 2010, China adopted the real-name registration law for phone numbers. All phone numbers have to be linked to users’ true identity. To comply with the real-name registration law, the general practice of internet services in China is to ask users to use a phone number during registration, with confirmation code sent back via text message to that number. International phone numbers are not always accepted. Our team did not have a mobile number to receive the confirmation code. We initially tried to set up a Google Voice virtual number and soon found out that Google Voice also requested a real mobile number for sign-up as well. We eventually discovered TracFone, which provided a low-cost SIM card with a flexible short-term (as short as one-month) mobile plan.

User Preferences

Although sometimes a tool or product is not blocked by the “Great Firewall”, the tool might not be a proper choice for the Chinese study. For example, Chinese have access to Skype, but Skype is not a popular communication tool in China. To best reach out to the Chinese users, we explored tools prevalent in China. Our criteria for research tool selection are: (1) make use of products that Chinese users are already using and familiar with. (2) If we need to ask users to adopt a new tool, the tool should be easy to set up and easy to learn.

WeChat as the Communication Tool

We selected WeChat as our primary communication tool. WeChat, the mega mobile app developed by the Chinese company Tencent, has 980 million active users [8]. WeChat is not only a social media app, but also the primarily mobile payment method in China. During our research, we also found that even businesses use WeChat for business inquiries and customer service.

Survey Platforms

To reach out to a large and diversified population, we decided to use a local survey platform in China. We looked into three popular survey platforms in China: Wen Juan Xing (which means “survey star” in Chinese), Wen Juan Wang (which means “survey website” in Chinese) and Tencent Wen Juan (which means “Tencent survey” in Chinese). We compared the three services based on user base, sampling service and, payment methods (as shown in Table 1). Since Wen Juan Xing was the only platform that allowed researchers to define screening criteria, which in our case are printer users, we selected Wen Juan Xing. Also, Wen Juan Xing was the only service that accepted international bank transactions.

Table 1. Comparison of survey platforms
Full size table

4 International Payment

Payment was the biggest roadblock in our research journey. Like many companies in the US, our team only has a corporate credit card for business purchases. Despite of the ubiquity of credit card usage in the US, credit cards are not a popular payment method in China. With the rapid growth of the smartphone market, mobile payments have started to dominate the payment market in China. Alipay and WeChat Pay, the two most prevalent mobile pay methods, allow Chinese users to transfer money to one another, buy groceries, pay bills, and purchase internet services. During our investigation, we found that all three of the survey platforms we investigated only accepted Alipay and WeChat Pay. Wen Juan Xing was the only service that also accepts international bank transactions.

The finance department at Lexmark did not have any knowledge regarding mobile payments, because it is not a business-to-business payment method commonly used in the US. Our team explained the cultural difference to the accounting department and received approval on using mobile pay. However, setting up and using WeChat Pay from US was unexpectedly difficult.

To enable WeChat Pay, users need to link their WeChat Wallet to a credit card, a debit card, or a Chinese bank account. For non-Chinese credit cards or debit cards, users have to make sure that the card supports international payment. A linked credit card account would allow the user to transfer money via “Red Packet”, but the user cannot withdraw from or deposit to the linked credit card. Therefore, although our corporate credit card was linked, we were still not able to pay China survey service via WeChat Pay.

We tried an alternative solution to refill our WeChat Wallet. There are online services that accept credit card payments from the US and transfer money to WeChat wallets with a service fee. For example, Kavip.com provides such service at a service charge of 7% of the transaction amount. However, Kavip required the user to upload a photo of the cardholder holding the card for identity verification. This unusual request prevented us from loading WeChat Wallet via Kavip’s service. The only methods remaining for the Wen Juan Xing payment were (1) international bank transaction, or (2) adding Wen Juan Xing as a vendor. Since we planned to use Wen Juan Xing for future surveys, we decided to add Wen Juan Xing as a vendor.

Successful Data Collection

We have collected 855 responses for our first survey through Wen Juan Xing. The data collection process took about two weeks. The cost of the sampling service was calculated based on sampling criteria and the number of questions. Researchers needed to have the survey questions ready and send the survey to Wen Juan Xing to get a quote. If the cost is deemed too high, the researchers can adjust the screening criteria to lower the quote. Wen Juan Xing also tells researchers the estimated time needed to collect data.

As part of the sampling service, Wen Juan Xing automatically filtered out invalid responses, such as repetitive responses from the same IP or responses that took an unusually short amount of time. Wen Juan Xing automatically collected data on users’ location (city and province), how the user found out about the survey, when the user filled out the survey, overall time each user spent on the survey, and their IP addresses. We were able to see the data collection progress in real time in the user portal.

We have also used Wen Juan Xing’s built-in analytics tool for data analysis. Wen Juan Xing’s analytics tool allowed us to create contingency tables, compare subsets of our data and generate charts in the user portal. The data analytics tool is great for dataset exploration. It is easy to learn and simple to use. However, this tool does not allow users to aggregate data from more than one questions. To gain further insights, researchers will need to download the dataset and use other statistics analysis tools, such as: Excel, Tableau or Python. Our team used Python Pandas to aggregate data and plot out charts.

5 Recommendations

We have identified the challenges we experienced during our research planning and execution, and developed workarounds to address these challenges. Although logistics are not typically part of research planning, small UX teams sometimes have to take care of logistics to pave the road for study implementation. Logistics problems, including getting approvals from different departments within the company, processing time for transactions, shipping time for purchases, can take time and delay research progress. We advise researchers to plan the study in advance, detail the potential roadblocks and solutions, and work on logistics and study planning at the same time. We summarized the following recommendations for researchers who are new to conducting user research in China.

  • Get a team mobile number. Having a mobile number is the essential first-step to have access to almost all Chinese internet products. We recommend researchers to get a burner phone and a short-term mobile plan in advance. If the research team decides to order anything online, the team should take shipping time into consideration.

  • Build long-term relationship with reliable local vendors. Local vendors can help researchers take care of recruitments and international payments. As we mentioned earlier, international payment was our biggest roadblock. Research teams should consider what kinds of help they need from the vendors and define clear criteria to review, compare and select vendors in China.

  • Research payment methods. Payment methods should be part of the selection criteria when choosing services. If the service provider limits payment methods to certain options, such as Alipay or WeChat Pay, the research team might have to consider other options. Meanwhile, understanding the company’s policies on available payment options is also important. The research team will have to educate other departments of the company on the importance of adapting to the payment customs of the target market. Planning for unusual resources needed to conduct user research in China can be helpful. For example, a Chinese debit card that allows researchers to use WeChat Pay. If necessary, the research team should spend time in advance to help stakeholders understand why certain resources are essential for conducting UX research in China.

  • Research the target population. We recommend that the US-based UX team conduct literature review on technology usage in China to ensure proper language and questions are used in the survey. It is also important to understand the popular services and products used in China, as well as the demographics of the target participants. The research team should also factor into time differences and holiday schedules in China during timeline planning.

6 Limitation and Future Studies

For our survey study, the Chinese survey platform Wen Juan Xing was our only channel for participants recruitment and data collection. We had little control over the screening process of participants or quality of survey responses we collected. We fully relied on Wen Juan Xing to filter out participants who failed to meet our criteria and eliminate invalid responses. In the end, we still saw a few responses with random strings filled in our open-ended questions. Since we outsourced the participants recruiting process to Wen Juan Xing, we would not be able to run follow-up studies with the same group of users.

Ideally, we would like to conduct additional studies to cross-check the results from this survey, as it was the first time we tried this approach. We could conduct user interviews and contextual inquiries to validate the current findings. We could also try to outsource the research work to a reputable vendor in China. However, as mentioned earlier, we had very limited time and budget to complete this research project. Running comparison studies would defeat the purpose of taking this journey in the first place. This paper aims to share the learning from our exploration of potential solutions of conducting cost-effective remote research in China. We hope to encourage more UX teams in the US to join this exploration without being intimidated by the time and cost international UX research typically requires.

As part of our research plan, we have scheduled to carry out our second survey and the diary study. The second survey would allow us to dive deeper into user preferences and behavior patterns. The results from diary study would help us understand the reported user preferences and pain points in contexts. These three studies are just a start. Our efforts on learning about Chinese printer users still have a long way to go. We hope that the learning from planning and executing these studies would streamline our remote research projects in China moving forward.