1 Introduction

According to the estimation of US Census Bureau, the age structure will change from 13 percent of the population aged 65 and older in 2010 to 19 percent in 2030 [1]. As our reliance on Internet has got stronger in nowadays world, the number of senior web users has also grown. In the future, the senior citizen population would potentially become the main demographic of website users. And thus, web designer will be driven by market to consider a user-centered design of website for senior citizen group.

User-centered design (UCD) is an approach and also a design philosophy to put users at the center of all design decisions during an iterative design process. The idea could be implemented by continuously probing the users’ needs and modifying the design based on users’ physical and psychological capabilities and recognized individual differences by their demographic. The tools in the design process include user research, prototyping, and user testing [2]. In this paper, user-centered design for senior web users will be called as Aging-Centered Website Design since the aging group would be the target users in this context.

For facilitating Aging-Centered Website Design, several kinds of user research and user testing have been done to explore the physical and psychological limitations of aging group. Psychological findings on aging and their abilities have shown, elder people vary a lot in their behavior by many reasons, such as vision and hearing impairment, decline in working memory, and learning effects [3,4,5,6]. Hawthorn has further pointed out that the studies of abilities of elder people should include controls of education, eyesight, medication and especially, training effects [6].

Nielsen has conducted usability studies with 75 senior web users whose age is above 65. He has found out significant differences between elder and younger webs users in quantitative user testing metrics such as task completion time, error and success rate. In over all, senior web users are slower and they may make more mistakes [7]. There are many other studies have shown the similar results [4, 8,9,10]. Except the findings of age difference in the performance of using websites, age difference has also shown in attitude toward using computers, and thus it affects user mental models [11]. The real challenge of Aging-Centered Website Design may not only be the age difference between elderly and younger adults, but also the technology gap in between elderly and other age groups.

Richard Hodes, the Director of the National Institute on Aging, indicated at SPRY conference that the technology gap affecting older adults and difficulties in using search engines and navigating the Web have left elders hesitant about using the Internet [5]. More and more studies have also found age and prior experience differences of using technologies in performance, user behavior, human computer interactions, and attitude toward computers [8, 9, 12,13,14].

However, seldom do the practitioners of user experience (UX) evaluation consider the prior experience differences and use it as a control in user testing of their Aging-Centered Design project [15,16,17,18]. In this paper, we proposed a scientific study of the user testing for an Aging-Centered Website Design project. The study is consisted of 2 research models: a 2 by 2 factorial experiment to test the effects and the interaction of age and computer skill level; and a structural equation model to describe the causal relationships between age, computer skill level, usability and perceived value. The study results would imply that using these two factors as controls in user testing could help designer better understand the user behavior and the user mental model of aging group.

The organization of this paper is as follows: Sect. 2 presents an Aging-Centered Website Design project which is used as the research object in the proposed study; Sect. 3 introduces the hypotheses and research models and Sect. 4 details the experimental design and experiment procedures, testing setting of the proposed study; Sect. 5 would be discussions and conclusion is in Sect. 6.

2 An Aging-Centered Website Design Project: Care and Share

The Care & Share project is a Aging-Centered Website Design project that was done in a Fall 2016 course, CGT512 in Purdue university for participating in 2017 HCI student design competition. The goal of this design project is to solve the social isolation problem for both senior citizens and international students. The design idea is to create a website to bring seniors and international students together by volunteer services and social events. The main functions of the website are: seniors post social events or volunteer service request on the website, and students can search the interested events to join.

The first author in this paper was responsible for planning and conducting user research, usability testing and user feedback sessions for this project. In this project, we found out that elderly have lower performance than young people with a lower efficiency and a higher error rate. However, the difference wasn’t shown with a statistical significance while there was a large in-between-group variance in elderly group. During the user testing process for the low-fidelity prototypes, an inconsistency of computer skill level difference between elder users was shown. Some elderly with more experience of using computer show strong confidence in performing tasks independently and some totally rely on the guidance of experimenter. This result implies aging may not be the only factor affects user’s behavior in ageing-centered design.

Another interesting finding is user’s subjective rating of website usability may be affected by their perception of the website value, especially for elder people. During our testing process, the elder testers found this website may bring value to their social life, so they may have tended to give a better subjective rating to the website usability.

3 Hypotheses and Research Model

Based on the literature review and previous findings from an aging-centered design project, we formed four hypotheses as shown below:

Hypothesis 1: There is an interaction between age and computer skill levels in website user testing results including task completion time (TCT), error rate (ER), subjective usability scale (SUS), perceived value (PV).

Hypothesis 2: Age affects website user testing results (including TCT, ER, SUS, PV).

Hypothesis 3: Computer skill levels affects website user testing results.

Hypothesis 4: Website usability testing results and consumer Perceived Value of the website would be related variables.

In this study, we proposed 2 research models to test above hypotheses. The first one was built by a 2 by 2 factorial experiment structure as shown in Fig. 1 and will be analyzed by DOE analysis method. In this model, hypothesis 1 will be tested by H1; hypothesis 2 will be tested by H2–H5; and hypothesis 3 will be tested by H6–H9.

Fig. 1.
figure 1

Path diagram for the proposed 22 factorial experiment.

Full size image

The second one is a causal relationship model as shown in Fig. 2, and it will be analyzed by Structure Equation Modeling method [19]. In this structural model, hypothesis 2 will be tested by H11–H12; hypothesis 3 is tested by H13–H14; and hypothesis 4 is tested by H15.

Fig. 2.
figure 2

Path diagram for the proposed causal relationship model.

Full size image

4 Methodology

4.1 Experimental Design (DOE)

We designed a 2 by 2 factorial experiment to test the main effects of age and computer skill level and their interaction to the website user testing results. Two factors and their levels are as shown in the Table 1.

Table 1. Experimental design of age and computer skill level effects.
Full size table

The higher level of age factor is defined as those senior citizens whose age is above 65, which follows the US census Bureau’s definition of senior citizens [1]. And the lower level of age factor is the group of college students whose age is around 18–22.

The criterion of categorizing higher and lower levels of the computer skill level factor is based on the definition of Organization for Economic Co-operation and Development (OECD). Computer skill level 1 people can do “Tasks that usually required the widely available or familiar technology application such as web browser and email software. There is little or no navigation required to access the information or commands required to solve the problem.” and “The tasks involve few steps and a minimal number of operators” [20]. The example of skill level one task is “Find all emails from John Smith”. According to the OECD research in 2016, there are 57% of adults whose computer skill level is above level 1, and 43% of adults whose computer skill level is at level 1 or below level 1.

4.2 Participants

40 participants will be recruited for the test. Twenty of them will be from the age group which is above 65. And the other 20 are college students. We will perform a pre-test designed by OECD definition for selecting the subjects from a retired community and Purdue university with different computer skill levels, which are defined by OECD research. For each age group, 10 participants will be above the skill level 1 and the rest will be level 1 and below level 1.

4.3 Research Object

This study will simulate the user testing during human-centered design process, so the research object will be the prototypes of the website in Care & Share project. The example of the website homepage prototype is as shown in the figure below (Fig. 3).

Fig. 3.
figure 3

Low-fidelity prototype of Care & Share website homepage.

Full size image

4.4 Experiment Procedures

The moderator will firstly introduce the project idea and the main features of the website, and then the test scenario will be introduced. The test scenario assumes users are using the website for the first time. They will first need to sign up and review consent procedures before performing any further tasks.

And then we will run the Cognitive Walkthrough [21] to collect testing metrics, including task completion time and error rate. The participants will be told to finish a task without knowing the detailed steps of the task. Participants will be asked to Think Aloud by Nielsen’s protocol [22] when carrying out the task, while the observer taking notes. The test will be paused after they complete a task to record the completion time, and the moderator will judge the success in finishing the subtask. Below are the two tasks the participants will perform during the test:

  1. 1.

    Sign up

    1. 1.1

      Click on login button

    2. 1.2

      Click on sign up via email

    3. 1.3

      Fill up the information and click on the sign up button

  2. 2.

    Post Events

    1. 2.1

      Login

    2. 2.2

      Click on Post Events

    3. 2.3

      Select the event type

    4. 2.4

      Select the event theme

    5. 2.5

      Fill up the information

4.5 UX Evaluation Questionnaire

The user experience (UX) will be evaluated by a subjective questionnaire, which has 9 questions in two main categories, usability and perceived value. Each of those 9 questions is with a 7 units scale from strongly disagree to strongly agree. Usability questions were adopted based on the system usability scale (SUS) questions which are designed by Brooke [23]. And for testing users’ subjective feeling of the website service, we choose 3 perceived value questions from the E-S-QUAL. E-S-QUAL is a multiple-item scale for assessing electronic service quality, designed by Parasuraman et al. [24]. E-S-QUAL tests user’s subjective feeling in many different perspectives, including usability and customer satisfaction [25,26,27,28]. However, since the research object is the website prototypes, not the final product, we just have selected a subset of questions from the specific factor to do the test (Table 2).

Table 2. The subjective questionnaire for UX Evaluation. SUS is the abbreviation of System Usability Scale, which is used to test the usability; PV is the abbreviation of Perceived Value.
Full size table

5 Discussion

Previous studies often used “prior computer experience” to indicate the familiarity. Czaja et al. firstly defined the previous computer experience by the self-rating from a 5 point-scale (0–6 months, 6 months–1 year, 1–3 years, 3–5 years, or more than 5 years) [8]. Kang and Yoon summed up several 5 point-scale questions to represent the background knowledge, including the previous experience and frequency of use [13]. However, there’s a gap between experience and familiarity. In the proposed study, we will directly measure the computer skill level with a standardized test by the definition of OECD, which could be a more objective measurement rather than self-report. Since we have also found in our previous user testing that it’s hard for some elder users to recall and precisely describe their computer experience.

In the proposed study, we will consider age and computer skill level as predictors and user testing results as response so we could systematically examine the effects, interactions and the causal relationships between each variable. In many previous studies, the main goal is to compare the age difference, so the prior computer experience knowledge is just a control in experiment or it will be examined by ANCOVA [8, 13, 14]. Although the effect of prior computer experience could be seen but it’s hard for us to examine the interactions between age and prior computer experience.

With the real application for the use testing of an aging-design project, we expect to see the following 2 results in our proposed study: (1) Computer skill level difference affects the performance and it is confounded with the age factor. (2) Users’ subjective perceived value of the website will affect users’ subjective rating of usability.

6 Conclusion and Future Work

A factorial experiment to investigate the effects of age and computer skill level to the website user testing results had been proposed in this paper. In the proposed study, two research models were built to test 4 hypotheses, including the factorial experiment and structural equation modeling. This study would validate the findings in previous studies that elder people vary by prior computer experience [6]. This study implies computer skill level should be controlled in the user testing of the aging-centered website design project. Additionally, after user testing, the difference of user mental models of elderly and college students about their perceptions of the website service quality can be learned, which may provide designer a better understanding of different user groups.