Keywords

1 Introduction

Given the varying nature of mobility impairments there is a need for technology that has the flexibility to adapt to the abilities of a particular user. Adaptable, customizable, and ability-based interfaces have been proposed to address the needs of all users [1,2,3,4,5,6,7,8,9,10]. Typically, when a user tries to use a new piece of technology, he or she must adapt to the interface. For those with physical impairments, this may not be ideal as most interfaces assume a high level of fine-motor skills. The major benefit of an adaptable interface is that the interface adapts to the needs of the user rather the user adapting to the interface. Ideally, the interface would be able to determine the strengths and weaknesses of the user, and with this knowledge in mind, create the most effective layout for the user.

One instance of the adaptable interface is the Hierarchical Adaptive Interface Layout (HAIL). The HAIL model consists of large interactive buttons along the edges of the screen and content that rests in the center of the window. This format allows for a larger target audience, including those with severe motor impairment. In his prior research, Magee created a proof of concept web browser [3] and a Twitter client using the specifications of the HAIL interface [11]. These interfaces were created in conjunction with the program Camera Mouse [12]. Camera Mouse is a customizable mouse substitution interface typically configured to track a user’s face has been shown to be a useful tool for users who cannot use traditional human-computer interfaces. Following Magee’s research and other work in the area, we present an interface that allows Camera Mouse users to read and send email with ease compared to standard email interfaces - HAIL Gmail. HAIL Gmail adheres to the specifications of HAIL and is a practical and efficient method of managing an inbox for those lacking precision of mouse movement. HAIL was introduced with proof of concepts, HAILBrowser and HAILTwitter, we intend HAIL Gmail to extend the functionality of the concept to email clients.

2 HAIL Gmail

HAIL Gmail is based upon Google’s original email client. Gmail is a commonly used email client throughout the world, with over one billion active users per month. Given its popularity and widespread adoption, Gmail seemed to be the best choice to adapt for use with HAIL; users would be able to easily access their existing Gmail accounts through the Google API.

In this prototype, we have only included the most essential functionalities of email: sending and reading emails. The more advanced features, such as adding attachments and inbox organization, are planned as future work of HAIL Gmail. Compared to Google’s default Gmail client, HAIL Gmail is much less cluttered and has fewer advanced functions. The interface is very simplistic, making it ideal for users whose abilities restrict the use the traditional Gmail interface. Camera Mouse users in particular often have difficulty with cluttered interfaces consisting of small, interactive elements. The dwell-time clicking functionality of Camera Mouse can cause unintentional selections, causing unwanted actions. Small interface elements are difficult to select due to the lack of precise pointer control, movement ability limitations, or involuntary movements.

Our HAIL Gmail prototype is divided into two different windows for reading and sending: the Inbox screen and the Compose screen. Users can switch between the two screens with a single click – allowing them to switch between reading and writing email tasks easily. Here, we describe the layout and features of the prototype application.

The Inbox screen (Fig. 1) consists of a table in the center of the window with the user’s inbox. Each email’s sender, subject, and timestamp are displayed, and clicking on the subject will open that mail’s contents in a new window (Fig. 2). Buttons on the edges of the screen allow the user to scroll through their inbox, refresh the page, and go to the Compose screen.

Fig. 1.
figure 1

Screenshot of the HAIL Gmail inbox.

Full size image
Fig. 2.
figure 2

Single-view message reading.

Full size image

The Compose Screen (Fig. 3) has three text fields in the center of the window for the recipient, subject, and message. When the user selects one of the text fields, an onscreen keyboard will appear at the bottom of the screen. The text fields for To and Subject also have dropdown menus of preset recipients and headings that the user may select (Fig. 4). These presets could be set by a caretaker for people that the user frequently contacts. Buttons are also on the edges of the screen, which allow the user to send an email when all three text fields are filled, clear the text fields, and go back to the Inbox Screen.

Fig. 3.
figure 3

Compose message screen. The text boxes can be configured with presets. When a text area is selected, an on-screen keyboard is displayed for the user to enter information into the fields.

Full size image
Fig. 4.
figure 4

Compose message screen with a text area selected. An on-screen keyboard appears and any presets for the field are available for selection.

Full size image

The HAIL interface has been tested with the program Camera Mouse but is also suited to be used with other mouse substitution interfaces. Camera Mouse uses the web camera on a computer to track parts of the user’s face. As the user moves, the mouse pointer on the computer window will move in the same direction. Dwelling the mouse pointer on a button for a preset time interval will click it [5]. The buttons relating to the main functionality are on the sides of the window, so that the edges of the window “catches” the mouse when the user tries to click a button. These buttons can be considered of “infinite width,” ideal for users with motor impairments. By contrast, the center of the window is “rest area.” There are minimal functions in the center of the window, so the user can rest their mouse and body for the Camera Mouse.

3 Experimental Evaluation

We conducted initial experiments with 7 participants without disabilities who had different levels of prior experience with the Camera Mouse. Although we were not able to perform experiments with participants with disabilities, our prior experience developing applications for Camera Mouse users provides an opportunity to evaluate the interface and identify potential problems that users with disabilities may encounter. The primary goal of these studies was to obtain a qualitative assessment of the functionality and the ease of use of HAIL Gmail.

Each participant’s experiment consisted of trials which mirrored the intended everyday use of an email client. Each experiment consisted of three different trials. In Trial A, participants were tasked with reading emails. Participants were tasked with sending emails using preset email addresses and subject headings in Trial B. In Trial C, participants sent emails using the onscreen keyboard to enter all information. The time elapsed to complete each trial was recorded for each subject. The subject was observed while performing the trials. If the subject made an error, such as a misclick, while using the interface, the error was noted. Total errors were tallied for each subject. In addition, after using the interface, the subjects were interviewed on their opinions on the HAIL Gmail and HAIL interfaces. Below is the procedure of the experiments and the blank data collection sheet.

3.1 Procedure

  1. 1.

    The purpose of the experiment and how to use the tools are explained to the participant. Participants are given practice time to familiarize themselves with the Camera Mouse interface by using a simple paint program and target acquisition game.

  2. 2.

    Camera Mouse is configured to track the subject’s nose. It is further configured for a clicking dwell-time of 2 s, a large clicking radius, and to have clicking sounds on.

  3. 3.

    Participants are asked to perform the tasks below. It is recorded whether they could complete the task or not, the time elapsed while completing the task, and the number of errors the subject made while performing the task.

    1. (a)

      Open an email starting from the main page

      1. (i)

        Find and open an email from David J. Malan titled feedback on CS50x Coding Contest sent on Fri, 5 Aug 2016 13:50:47 -0400 (Fig. 2)

      2. (ii)

        Read its contents

      3. (iii)

        Close the email

    2. (b)

      Send an email using preset email address and subject header

      1. (i)

        Starting from the main page, click Compose to go to the send screen

      2. (ii)

        Select the email address from the preset selections

      3. (iii)

        Select the subject header from the preset selctions

      4. (iv)

        Use the onscreen keyboard to type Experiment Test in the text area for message

      5. (v)

        Click the Send button to send the email.

    3. (c)

      Send an email using the onscreen keyboard

      1. (i)

        Starting from the main page, click Compose to go to the send screen

      2. (ii)

        Use the onscreen keyboard to enter

        1. (1)

          The email address prithudasgupta@gmail.com

        2. (2)

          The subject header Hi

        3. (3)

          The content of

          • Hi Prithu,

          • How are you

          • (The user’s first name)

        4. (4)

          Click the Send button to send the email

  4. 4.

    After performing the tests, the participants were asked for specific feedback on the interface.

  5. 5.

    Data was compiled from tests and feedback from participants (Fig. 5) to consider future improvements on HAIL Gmail and HAIL interfaces in general.

    Fig. 5.
    figure 5

    Test subject data sheet. The authors used this sheet to record quantitative and qualitative data and feedback about the participant’s performance and usage of the interface.

    Full size image

4 Data and Analysis

In our experiments, Trial A was for reading emails, Trial B was for sending emails using presets, and Trial C was for sending emails with the onscreen keyboard. As shown by the Fig. 6, reading emails was very quick, sending emails took a longer time. Trial A had a mean time elapsed of 13.56 s with a standard deviation of 3.98 s. As for composing emails, Trials B and C had a mean time elapsed of 88.25 and 204.40 s with a standard deviation of 15.13 and 40.10 s, respectively. Table 1 shows the performance of each participant on all three trials.

Fig. 6.
figure 6

Bar graph of the data presented in Table 1 illustrating relative performance of the three trials across all participants.

Full size image
Table 1. Quantitative data recorded for 7 participants performing three trials, along with per-trial means and standard deviations.
Full size table

Test subjects had trouble precisely operating the onscreen keyboard in conjunction with the Camera Mouse. Comments from subjects after using the interface included grouping characters that are used most often together closely on the onscreen keyboard, having different keyboard layouts that the user can scroll between and using auto suggestions for text. Also, the participants tested did not have any sort of physical disabilities. Prior experience with Camera Mouse users with disabilities has typically resulted in tasks taking longer compared to users without disabilities, however, individual abilities vary widely and can dramatically affect performance with a particular interface.

In Fig. 6, we can observe some trends. For example, Participant 5 had the fastest overall time in both Trial B and Trial C. Participant 3 had the overall slowest time on Trial C, but the second-fastest time on Trial B.

5 Conclusion

In its current state, HAIL Gmail only features reading and sending emails. We plan to implement other features, such as replying to and forwarding emails, searching for an email in the inbox, adding attachments, and sending to multiple recipients. We will deploy HAIL Gmail either as a web application or as a plugin in the near future for mass use.

As discussed in the original HAIL paper, we are also considering the idea of creating a HAIL collection of “apps.” An email client, web browser, and Twitter client have already been created. To add to this, a news-feed aggregator, Facebook interface, and media player application are currently planned. An online store would be created to host all of these applications, similar to the Apple “app” store.

On the whole, HAIL Gmail is a practical method of bringing email accessibility to those with severe motor impairments. Features such as the simple and uncluttered screen, mouse substitution integration, and color-coded, larger buttons allow for a much easier experience for users with disabilities.