Deconaby: Animations for Improving Understandability of Web Images

Abstract

There are many “how-to” pages on the Web. However, on such pages, descriptions and images are often not clearly linked. As a result, it is difficult for viewers to understand which parts of the text description refer to which pars of the images. This study aims to solve this problem. With the developed Deconaby system, users can add animations just by clicking on an image on a Web page and moving their mouse. Our system then generates a URL that consists of the how-to page’s original URL and a hash ID for sharing the animation-added version with others. In this manner, our system can improve the understandability of Web pages with little time and effort. We conducted an experiment to test the effectiveness of our system. The results showed that our system succeeded in increasing the understandability of difficult-to-understand pages. We also showed that a page written in an unfamiliar language can be understood with the help of appropriate Deconaby animations.

1 Introduction

Many “how-to” pages are posted on the Web. These pages describe, for example, how to install PC applications (see Fig. 1), access restaurants from their nearest station, fill out and submit application forms, and draw illustrations. When people want to know something, they usually use a search engine such as Google and Bing to obtain information from these how-to pages.

Fig. 1.

Example of a “how-to” page.

To make their how-to Web pages easier to understand, authors often use images such as screenshots, maps, photographs, and illustrations. However, the descriptions and the images in how-to pages are often not clearly linked. As a result, readers cannot easily understand what part of a text description refers to what part of an image (see Fig. 2). In addition, these how-to pages sometimes require expertise in fields such as programming, software development, and foreign languages. Japanese students sometimes give up trying to understand a page if it is written in English. Likewise, elderly people cannot easily understand a page written with many technical terms (see Fig. 3).

Fig. 2.

Example of an incomprehensible image on a web page due to several overlapping descriptions such as repeated use of the term “download.”

Fig. 3.

Example of how technical terms (highlighted by red rectangles) confuse elderly persons. (Color figure online)

On a different note, people cannot help but gaze at something that is moving. For example, presenters often use a laser pointer to direct the audience’s attention to a point of note on the screen when giving a presentation using PowerPoint, Keynote, and other software applications. In addition, creators of instructional videos about application installation, programming, and similar fields aid their viewers’ understanding by using the mouse cursor to indicate specific parts of content. Thus, if we add movement to how-to pages, we might improve those pages’ understandability.

Aiming to solve the problems described above, we propose a system called “Deconaby” that enables a user to add animations directly to images on Web pages and to easily share the pages with the added animations with others. A mouse-cursor, pointing finger, or walking person animation can be added to images. The user can improve others’ understanding of Web pages by using the proposed system with little time and effort.

By using Deconaby, a user can use original content without infringing on its copyright as well as create and browse originally animated pages tailored to their needs. We expect that Deconaby will enhance the understanding of users unfamiliar with Website operations such as clicks and swipes, and prevent users from suffering unnecessary stress and confusion.

First, we describe related works and propose the Deconaby system. Then, we describe its implementation and show its effectiveness by conducting an experimental test. Finally, we describe the application of our system and discuss future work.

2 Related Work

Methods for modifying the content of pages on the Web have been developed extensively. For example, the Web browser extension Decoby enables users to add effects to images and movies posted on the Web and share those effects with others [1]. By using Decoby, users can enjoy content in their preferred form. In addition, Nakamura et al. proposed a system that can aurally decorate movies on the Web [2]. This system not only enables users to make decorations of their choice for videos but also creates a new movie-viewing experience. These systems have the advantage of enhancing the user experience by modifying the content on the Web. However, they do not increase the understandability of Web content.

WATSS (Web annotation tool for surveillance scenarios) is a Web-based annotation tool that allows users to annotate video content taken by a surveillance camera [3]. Web Annotator enables users to annotate e-learning materials [4]. InkAnnotation enables users to use styluses to add handwritten annotations to teaching materials [5]. Another Web-based annotation tool enables users to emphasize parts of teaching materials by highlighting them [6]. These tools enable annotations to be made on actual notes used in e-learning. Virtual Notes can embed annotations in the text; that is, annotations are inserted as “sticky notes” with a title and make it easier for users to understand the text [7]. These studies propose appropriate methods for the purpose of annotation; our research differs from them because its purpose is supporting user understanding.

A publication system that enables communication such as chatting between users browsing the same Web page has been proposed [8]. It allows users to exchange opinions and information. Another proposed system shows annotations made by support staff on users’ screens in real time as a help system for Web applications [9]. However, these systems differ from Deconaby, which enables asynchronous sharing irrespective of timing between users. In addition, a graphic annotation platform that provides annotated lines and figures on screenshots of Web pages and stores them in a database for sharing has also been proposed [10]. These annotation systems allow text and graphic annotations. In contrast, Deconaby does not insert text or graphics; instead, it supports the user by adding animations of icons such as a mouse cursor and a pointing finger.

3 Deconaby

3.1 Proposed Method

One way to increase the understandability of an image on a Web page is to highlight a part of the image by drawing a red rectangle around that part or by pointing to it with a red arrow. However, preparing such image content requires much time and effort. In addition, if a Web creator wants to highlight several parts of an image in order to indicate the order in which a user should click several buttons or indicate the route from a station to a destination, the image content might become incomprehensible because of too many highlighted parts. In addition, average users cannot fix these incomprehensible images themselves.

To solve this problem, we propose a method that enables average users to add animation to image content on the Web as a decoration. This method emulates actions done by users with superimposing animations, such as a mouse cursor, pointing a finger and moving person, on the image content (see Fig. 4). These animations enhance the user’s understanding without modifying the original content of the how-to pages.

Fig. 4.

Overview of Deconaby. (Color figure online)

For example, when Web creators want to make a page describing how to install an application on a PC, they usually take a screenshot of the page distributing the application and embed the screenshot in their web page. Then, they explain in the text the parts of the image that can be clicked on. When a reader visits the Web page to find out how to install the application, they sometimes cannot easily understand which parts they should click. If guidance with an animation of, for example, a cursor clicking on the relevant part of the screenshot could be given directly, the user would not have to check the text to determine where to click.

As another example, when Web creators want to create a page explaining how to access a facility from a train station, they usually upload an image of an appropriate map and describe the location of the facility, the nearest station, routes of access, and so forth in the text. However, if the map is too complicated, readers who visit the page to determine how to access the facility might get lost when they actually go. If it were possible to present a suitable route to the user on such a map using an animation, users would be able to find their way easily without viewing the page for a long time.

An image showing how an animation is assigned on a Web page by using Deconaby is shown in Fig. 4. An additional canvas on the target image is created automatically, and an animation is drawn on the canvas. The animation can be added by moving the cursor and clicking on this canvas.

Deconaby allows users to easily share the animations assigned to images on Web pages with others. Specifically, it saves the animations that a user adds to a page and issues a new URL. The new URL consists of the original URL and a hash ID generated automatically for each decorated version. A user can share this decorated version only by sending this new URL to other users, who can access the animation-decorated Web page by using Deconaby and clicking this new URL.

3.2 Implementation

We implemented the Deconaby system as a Google Chrome browser extension using JavaScript and its libraries jQuery and P5.js¹, and we implemented its server system using PHP and MySQL for storing animation information and sharing animations assigned by users.

When a user uses Deconaby to visit a Web page, all images on the page are listed, except for small images such as buttons, icons, and advertisements, which are excluded because they are unsuitable for adding animations. Next, transparent additional layers (canvases) are generated by P5.js (as many as the number of elements of the acquired IMG tag of HTML) according to the size of each image. These layers are overlapped according to each image’s position on the page.

When a user wants to add a mouse-cursor animation to an image on the page, they simply click the part of the image that they want to animate. At that time, if the image is designated as a link by the A tag of HTML, the Web browser judges that the user wants to access the linked image. In the animation mode wherein users animations, our system disables its link, as described below.

When the user assigns an animation to the page, the animation information is sent to the recording server (in JSON format) so that it can be shared with other users. In the recording server, the received data is directly collected in a database. However, it is possible that one page may be decorated with animations from multiple users. Therefore, the page to which the animation is assigned is linked to the animation information by attaching an ID to the original URL. The animation ID is given at the end of the URL as a hash value (i.e. the value after the “#” of a URL). IDs are automatically generated so that data do not overlap. For example, when a user decorates a page by adding an animation to it, a URL (“http://example.com/example.html”) and an ID (12345) are generated, and a new URL (“http://example.com/example.html#decoID=12345”) where other users can see the animation-decorated page is provided.

When this URL is opened with Deconaby in a Web browser, the “decoID” is detected from the URL, and animation information is requested by Ajax with ID “12345” from the recording server. The recording server retrieves the animation information matching the ID from the database and sends the information to Deconaby, which reproduces the animation on the relevant image of the page. If the user does not have Deconaby installed, the Web browser ignores the values after the “#” of the URL and shows the original page.

3.3 Animation Types

When adding animation to various how-to-type pages, users might want to use several kinds of animations. For example, how-to pages about PCs and smartphones have different UIs for each application, so there is likely to be significant demand for explanations of the different interfaces. To meet that demand, appropriate mouse-cursor and finger animations were prepared for those pages. In addition, a footprint animation (which can be used to express a trajectory or order of movement that is difficult to explain verbally) was also prepared.

The mouse-cursor animation (see Fig. 5) supports explanations of PC operations and can signify a mouse click with a pulsing circle. Users can indicate the order of clicking by recording the points that they click by clicking multiple times. The finger animation (see Fig. 6) supports explanations of smartphone operations. In this mode, a finger image moves along the trajectory recorded by mouse clicks, and an animation illustrating a tap on the smartphone’s touchscreen can be assigned to the image.

Fig. 5.

Mouse-cursor animation.

Fig. 6.

Finger animation.

The footprint animation (see Fig. 7) explains routes on a map. In this mode, a person-like icon animates the trajectory recorded by mouse clicks while leaving a trail of footprints. When the animation is played, the person-like icon stops for a while at the clicked location, so the user can understand that they are being guided to that location.

Fig. 7.

Footprint animation.

By appropriately using these three types of animation, users can add their intended commentary to the image.

3.4 Operation

When a user installs Deconaby, a mode-change button is displayed at the lower right of their browser. Clicking this button switches between animation mode and decoration mode. Animation mode only shows the assigned animations. Decoration mode enables the users to add animations to certain images on a Web page. Deconaby’s default setting is animation mode. When users want to add an animation to an image on a web page, they turn on the decoration function by clicking the mode-change button.

In decoration mode, a decoration menu is displayed when a user moves their cursor over an image on the Web page (see Fig. 8). The user can change the animation type by clicking the appropriate button on the decoration menu. If the user clicks on the image, an animation is created and recorded by monitoring their mouse movement. During the recording, an afterimage of the locus of the mouse is displayed. When the image is clicked on, the animation starts recording at the clicked point. The user can indicate the order of the clicks by clicking two or more times without stopping recording of the animation. When the user clicks the stop button in the decoration menu, the recording stops. The trajectory on the image from the last clicked position to the stop-button click point is not recorded because the mouse movement after the last click is unnecessary information. If the user does not click on the image, it is judged that the user does not intend to decorate the image, so the locus is not recorded.

Fig. 8.

Decoration menu for a mouse-cursor (top) and animation recording (bottom). (Color figure online)

After assigning animations to all selected images on the page, the animations can be saved in the server by clicking the “Issue” button in the menu, and a page with an ID in the URL is opened in another tab of the browser. The user can use this URL to share the animation-added Web page with others.

3.5 Additional Functions

Deconaby is implemented on and distributed through our Web site (http://deconaby.club/). On the basis of feedback from users, the following two additional functions were added.

• ID list display: Usually, users cannot view decorated pages without knowing the animation ID. However, they sometimes want to view animations when they cannot understand the page. The ID list display allows users to browse animations added by other users by selecting a decoration ID from the list. Also, if another user adds an animation to the page that the user is viewing, the color of the menu changes to green. As a result, the user can know whether an animation has been added to that page.

• Request: If a user cannot understand a page, and the page has not been improved with Deconaby, the user can send a request asking others to improve the page. In this manner, it is possible to get many users to add animations to the image to increase its understandability.

4 Experiment

4.1 Pre-experimental Analysis

Deconaby was evaluated with an experiment using 14 university students (aged 20 to 23) as subjects. The experiment used 22 how-to Web pages, namely pages explaining the operations of PCs and smartphones or giving directions to a destination (10 pages for PC operation, 5 for smartphone operation, and 7 for route directions). Each subject was asked to use Deconaby to explain a page or improve its understandability for their families.

Examining the animated pages made by the subjects revealed that they tended to click on the same position on the page concerning how to operate applications or settings. In addition, most of the things described in the text on the page were reproduced with animation.

The subjects sometimes used animation to emphasize multiple positions in an image in the order of operation. Sometimes, when text explanations were omitted because the operation was basic, there were still decorations to supplement the instructions. In other words, the animations can be classified into two types: one type reproduces the content written in the text as animations; the other supplements insufficient explanations on the page.

The assigned animations sometimes reflected the users’ situations. For example, several operating systems and several versions of Windows OS are in use around the world. Accordingly, some applications can run on several different operating systems and versions. Pages that distribute these applications list several links to download for each operating system and each version. In our experiment, the subjects chose the indicated download link depending on the operating system and versions running on their families’ computers. As for when the subjects animated a page that gives directions from a station to a building, half of them used animation to indicate the shortest path, and the other half indicated the least crowded path.

Some animations added by the subjects encircled parts of an image, and these animations traced the designated part with the finger animation. These animations were added to emphasize certain parts of the image. In addition, some animations pointed to a station or destination with the finger animation, rather than the footprint animation, on the map. These explanatory animations were unexpected.

According to the feedback from the subjects, many subjects felt that using Deconaby made it easy to understand pages visually by animating them and that it was handy and easy to use. These impressions demonstrate that Deconaby has utility and usability. However, some subjects expressed the opinion that they would like to signify certain operations, such as double-clicking and dragging-and-dropping, as animations; animations for these operations are not implemented in Deconaby. It would indeed be useful if users could use such animations. However, other subjects expressed the opinion that Deconaby is handy because it is simply operated by mouse-clicking only. Therefore, when increasing the functionality of Deconaby, we must take care not to overcomplicate its operation.

As for other opinions on the functionality of Deconaby, some subjects felt that it would be helpful to be able to add animations to explain where to click on an entire Web page. Currently, Deconaby only targets images on Web pages for animation because many how-to pages contain difficult-to-understand images. If the entire page is targeted, Deconaby would need to consider the window size of the browser and the position of content depending on its size. However, we aim to develop Deconaby further so that it can be used to decorate an entire Web page and be more usable for giving general commentary with animations.

4.2 Analysis of Results of Evaluation Experiment

The extent to which Deconaby improved the understandability of how-to pages by assigning animations was experimentally tested as follows. One hundred pages were prepared. Among them, fifty were considered difficult to understand, so Deconaby was used to add animations to each of those 50 difficult-to-understand pages. The understandability was evaluated on five levels (−2 to +2) before and after the animations were added. A total of 50 evaluations were made by each subject, evaluating pages that were seen for the first time; specifically, each subject evaluated 25 unanimated pages and 25 animated pages.

The numbers of pages sorted into each how-to category used in the experiment are listed in Table 1. In the 50 pages examined, the average understandability score indicated increased easiness-to-understand for 32 pages and decreased easiness for 11 pages. The statistical significance of the difference was verified using a sign test (p < .01). In addition, the evaluation score of 14 of the pages was negative before decoration. This result means that the original pages were highly difficult to understand. Of these pages, the average score for 13 of the 14 increased (Table 2).

Table 1. Average understandability score of how-to pages classified by category.

Table 2. Average understandability score for each highly difficult-to-understand page.

As shown in Table 2, although there are seven pages in the smartphone category for which the understandability score increased, only one page was originally particularly difficult to understand. All of these pages were easy to understand by indicating the order of operations, and because there were many understandability values that increased, it was a case that demonstrated the advantage of using Deconaby to show order. Considering these categories, the mouse-cursor and finger animations are necessary, but the footprint animation needs to be further studied. The footprint animation is a mode that can indicate a locus that cannot be indicated with other modes. From now on, it will be necessary to further verify the usefulness of having an animation that shows tracks and to implement other optimal animation.

5 Application

Even if a page is written in a language that is unfamiliar to the viewer, it is possible to understand the page by assigning appropriate animations with Deconaby. Even if animation cannot facilitate perfect understanding, it is at least helpful in improving understanding. In addition, we believe that Deconaby is useful for enhancing e-learning content. Deconaby can potentially be used to support learning by animating, for example, the order in which characters are written and of flow charts. For example, many pages explain algorithms for problems such as searching a binary search tree, solving a shortest-path problem, and giving the best route to solve the Seven Bridges of Konigsberg problem. Deconaby could also be used to indicate the stroke order of Chinese characters and to show the best route to escape a labyrinth.

The current prototype of Deconaby only handles simple patterns with limited animations, namely, a mouse cursor, footprints, and fingers. It is thought that it is effective to show images of food ingredients on a page explaining how to cook a meal and to use an animation of a kitchen knife to show how to cut certain ingredients properly. Also, for pages explaining how to assemble things such as tents, plastic models, and origami, animation suitable for the content can be added. The addition of such animation to Deconaby will also be studied in the future.

Animation can be added to an image with an attached URL. We can use Gyazo, which shares screen captures with URLs, and users can add animation to the screen captures they take [11]. Gyazo has a GIF-shooting function. However, Gyazo’s GIF-shooting function has a time limit, so it cannot capture a long operation. It is thought that Deconaby has advantages not found in the GIF-shooting function.

With Deconaby, animation can be added only to still images, but the animation for moving images can be added with continuous animations such as those created by Decoby [1]. For example, it is possible to add commentary animation too, for instance, a person in a sports movie. One could also use animation to a dance movie to instruct viewers on how to move their foot or hand. One could also use an animation of a finger with a movie of someone playing music to instruct viewers on how to play.

6 Summary and Future Work

We proposed and implemented a system called Deconaby for assigning explanatory animations to the images of how-to pages to make them easier to understand visually and for sharing those animations with others. The effectiveness of Deconaby was tested by conducting an experiment.

Three animation modes—namely, a mouse cursor, fingers, and footprints—were implemented in Deconaby. However, they are not enough for users to improve all of the vast types of how-to pages on the Web. In future work, we plan to consider other types of animation. In the prototype system, the user has to install our extension to Google Chrome. Users who do not install Deconaby cannot see the animations. Accordingly, we are planning to create a Web service that allows users to see the animation without installing Deconaby.