Keywords

1 Introduction

Usability is a basic attribute in software quality. Its complex and evolving nature is hard to describe in a unique definition. Usability refers to ease of use and the way users can perform their tasks. User eXperience (UX) goes beyond the three generally accepted usability’s dimensions: effectiveness, efficiency and satisfaction. UX covers all aspects of someone’s interaction with a product, application, system and/or service. UX takes a broader view, looking at the individual’s entire interaction with the thing, as well as the thoughts, feelings and perceptions that result from that interaction [1]. As a psychological assessment tool, psychometrics could be helpful in UX studies as a complement to usability evaluation methods [2].

The Semiotic Engineering views the use of interactive software systems as a computer-mediated communication between designers and users, at interaction time [3]. Semiotic engineering proposes two methods to evaluate the communicability: (1) the semiotic inspection and (2) the communicability evaluation. The latter explores the reception in the meta communication and tries to identify through observation empirical evidence of the effects produced by the designer’s messages on the user as they appear during the interaction.

The paper explores how user testing (co-discovery), communicability evaluation, query techniques, and psychometrics (motivation scale) may complement each other when assessing UX. Empirical evidences are analyzed, using the World Digital Library (www.wdl.org) as a case study [4].

The paper is organized as follow: Sect. 2 explores the theoretical background. Section 3 presents the first experiment performed: co-discovery test, perception questionnaire and psychometric test. Section 4 presents the second experiment performed: the communicability test. Finally, the Sect. 5 shows conclusions and future work.

2 Theoretical Background

The ISO 9241-210 standard defines UX as a “person’s perceptions and responses resulting from the use and/or anticipated use of a product, system or service” [5]. On the other hand the current ISO 9241 definition of usability refers to “the extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [5].

Usability evaluation methods are usually classified as: (1) empirical usability testing, based on users’ participation [6], and (2) inspection methods, based on experts’ judgment [7]. Evaluating UX is more challenging and arguably overwhelming for newcomers [8]. Almost 90 UX evaluation methods are described at http://www.allaboutux.org/ [9].

The “User Experience White Paper” [10] highlights the multidisciplinary nature of UX, which has led to several definitions of (and perspectives on) UX, each approaching the concept from a different point of view: from a psychological to a business perspective, and from quality centric to value centric.

It is important to mention that the ISO 9241-210 standard considers that UX “includes all the users’ emotions, beliefs, preferences, perceptions, physical and psychological responses, behaviors and accomplishments that occur before, during and after use” [5]. Therefore, psychometrics as a psychological assessment tool that studies “the operations and procedures used to measure variability in behavior and to connect those measurements to psychological phenomena” [11] could be helpful in UX studies as well.

Motivation which can be understood as “the drive that produces goal-directed behavior” concerning the “initiation, direction, intensity, and persistence of behavior” [12] is a significant psychological concept in different life domains [13]. As a psychometric resource, the Multidimensional Work Motivation Scale (MWMS) seeks to provide information on the motivation of people with respect to their work [14].

As for the Semiotic Engineering, it views the use of interactive software systems as a computer-mediated communication between designers and users, at interaction time [3]. The system is therefore the designer’s deputy, the artifact that transmits designer’s intentions. Communicability is the attribute that defines the quality of the metacommunication (“communication about communication”). The semiotic engineering proposes two evaluation methods: (1) the semiotic inspection, and (2) the communicability evaluation method.

The communicability evaluation method analyzes the metacommunication. Evaluators observe of how a group of users interacts with a particular system identifying communicative breakdowns. Evaluators interpret the results and then prepare the semiotic profile and the meta-communicational message [3].

3 First Experiment: Co-discovery, Perception Questionnaire and Psychometric Test

The first experiment was carried out in the Usability Laboratory of the School of Informatics Engineering of the Pontificia Universidad Católica de Valparaíso, Chile, and was conducted by two experts from UX. Both have a Diploma in UX, but also one currently studies Psychology and has a degree in Architecture, and the other one has a Master Degree in Computer Science.

The World Digital Library (WDL) was evaluated, based on a set of predefined tasks. The participants explored the site in pairs, their comments and facial expressions were recorded with cameras and the screens of their computers were recorded. They were also observed by the evaluators through a polarized glass that allows one-way vision.

After the participants signed a confidentiality agreement, they were informed about the test conditions, about the website to evaluate, and about the different stages of the test.

The experiment consisted of 4 parts:

  • (1) A pre-experiment questionnaire designed to individually identify the user profile and previous experience visiting portals similar to the evaluated product;

  • (2) A co-discovery test in pairs, presenting to the participants a series of tasks to explore the site as a whole and comment out their opinions;

  • (3) A post-experiment perception questionnaire that sought to know the different perceptions of each user regarding the site and the tasks;

  • (4) A post-experiment psychometric test, to know the motivations of each participant to perform the requested tasks.

3.1 The Pre-experiment Questionnaire

In a first stage, each user had to complete a pre-experiment questionnaire to collect general information about their profile and experience in other Digital Libraries (DLs). Five questions were included regarding sex, age, level of education and information about previous visits in other DLs. There were 12 users, 9 men and 3 women, between 23 and 32 years old, all being graduate students in Computer Science. Only a quarter of the users (3) reported having experience in visiting DLs, but stated that they (almost) never visit these types of sites.

3.2 The Co-discovery Test

To perform this test belonging to the second stage, each pair of participants was provided with a list of predefined tasks to explore the site. In addition, they were asked to freely discuss and comment aloud their opinions regarding WDL, the tasks, and what they considered relevant.

The first task was to find certain items associated with historical events, using the “Timelines”. Despite the fact that 83.3% of users completed the task, only 50% did it within the pre-established time (5 min). Difficulties arose to orient themselves within the different menus and to execute in a correct and efficient way the sequence of steps required to carry out the tasks.

The second task required opening a digital article, after placing it on the “Interactive maps” of the portal. 50% of the participants managed to do it and within the period of time assigned (5 min). The users had problems locating the different countries in the interactive map since their names did not appear, as well as difficulties to open the article since this option was only visualized when positioning the cursor over the main image. Therefore, the participants looked for alternative ways of doing the task or were distracted by other WDL contents.

The third task requested to find within the classification by places, articles associated to different geographical zones of Chile. 83.3% of users completed the task and did it within the time limit period (4 min). The rest showed a lack of attention to the instructions, looking for articles not associated with the geographical areas required. There was a tendency to look for articles in more direct ways, through the site’s search engine.

The fourth task was to explore different types of articles, in order to play a movie. 100% of the participants could execute it within the time limit (4 min). Half of them used the portal search engine to find the articles more expeditiously.

The various tasks requested allowed to know certain difficulties presented by the experiment participants to orient themselves through WDL. Problems were highlighted in recognizing the navigation mode offered by certain sections and the functionality of different tools (for example regarding the “Timelines” articles) and to identify the content associated with different graphic symbols (for example for the geographical areas in “Interactive maps”). Due to this, there are functions offered by the site that users did not manage to use effectively and efficiently.

The users tended to repeat the sequences of steps requested, probably hoping to be successful on a new occasion in the face of the presented inconveniences. The use of alternative search routes also stands out, which increases towards the end of the experiment, evidencing perhaps the need for greater flexibility and immediacy in the use of WDL. The distraction in the participants with other site contents that where mentioned in the specified tasks, could eventually indicate a form of compensation for failing to execute those tasks or an authentic interest in the information offered by the portal.

There are no greater differences between the performances of users with previous experience in DLs than those who had not previously visited this type of site, except in the realization of the second task. This obtained the lowest performance (50% of achievement), even for users with previous experiences in DLs (66.7% of these did not manage to complete the task). This may suggest that WDL has significant usability problems compared to other similar websites, related to the lack of clarity in the functionality of some tools and the insufficient information associated with them.

3.3 The Post-experiment Perception Questionnaire

After completing the test, in a third stage, users had to respond individually to a post-experiment questionnaire based on the System Usability Scale (SUS) [15]. The aim was to identify the users’ perceptions about the tasks’ difficulty levels, the orientation in the site and the conformity and satisfaction with it. Five questions were used using a Likert scale of 5 points and 4 open questions.

Regarding the difficulty to complete the requested tasks, most of the participants indicated that they considered it easy to achieve (41.7%) or neutral (41.7%), while two users considered it difficult (8.3%) or very difficult (8.3%). Orientation within the portal was perceived as variable, with 41.7% feeling less oriented, 33.3% feeling neutral, and the rest feeling oriented (8.3%) or very oriented (16.7%). In relation to the degree of satisfaction with WDL, the majority found it satisfactory (41.7%) or neutral (41.7%), while 16.7% found it unsatisfactory. On the other hand, as to the information found on the site, the majority felt satisfied (58.3%), with one user (8.3%) who considered it very satisfactory, while 25% felt neutral and one user (8.3%) considered it unsatisfactory. Finally, most users express the intention to re-use the WDL, agreeing very much (25%), or agreeing (58.3%) while 25% of the users are neutral, and two users disagreed (8.3%) or strongly disagreed (8.3%).

Users with previous experience in DLs tended to perceive tasks with a lower degree of difficulty but with a varying degree of orientation within the site. This could suggest that although WDL navigation modes and interfaces are not necessarily easier and friendlier than in other similar portals, possibly familiarization with these allows developing greater intuition for the user regarding the portal’s use. On the other hand, participants who have previously visited DLs tended to show satisfaction and intention of future WDL use, indicating probably a genuine attraction to the portal and towards this type of tasks, in comparison with other similar sites.

It should be noted that despite the users’ overall perception seems neutral, with an average of 3.33 [2], analyzing each particular dimension allows to obtain a more precise understanding of the participant’s perceptions and their experience. This results’ description in conjunction with a more markedly quantitative reading allows us to observe that although task completion and orientation trough the website (with an average of 3) tend towards neutrality (with averages of 3.17 and 3 respectively) there is a majority who declare themselves satisfied with WDL (41.7%). There is also a tendency to express satisfaction about the information that WDL offers, and the intention of future use (with averages of 3.67 and 3.58 respectively). By complementing this type of reading, it can be assumed that despite the tasks’ difficulty and the lack of orientation in the portal, users can feel challenged, interested in exploring the site and satisfied with it.

As for the aspects that most pleased the users, they rescue the site’s content and the found information, its vastity and diversity, the graphic resources, the WDL organization and the user interface. These elements can explain the satisfaction, interest and intention of future use of the portal.

However, among the aspects that made it more difficult for the participants to navigate the site and that disliked, they themselves highlighted problems in identifying the navigation mode offered by some sections (for example in “Timelines”), in not being intuitive enough and assuming that the user has certain knowledge, and in some tools’ functionality and problems to visualize and locate elements (as in the case of “Interactive maps”). These elements allow understanding also the failures in the performance of the requested tasks and the complications to be able to feel oriented within WDL. In addition, the users expressed that there is a lack of clarity in the use of search filters, many of which have gone to the use of these as an alternative search route.

Faced with the mentioned aspects, the participants pointed out the need for a guide to use the site, as well as a greater hierarchy and order, greater clarity and simplicity in navigation, and of a change in the portal’s chromatic to be perceived as more attractive.

3.4 The Post-experiment Psychometric Test

The fourth part of the test consisted of a questionnaire about the motivation of each participant to perform the tasks, based on the Multidimensional Work Motivation Scale (MWMS) [14]. The scale was adapted to the academic context of the experiment and 19 questions were included using a Likert scale of 5 points. The questions were organized into 3 major categories, covering 6 dimensions. “Amotivation”, “intrinsic motivation” and “extrinsic motivation” were the 3 major categories, and “extrinsic motivation” was divided into 4 dimensions: “external social regulation”, “external material regulation”, “introjected regulation” and “identified regulation”. We analyzed preliminary findings in a previous study [2].

In the first category and dimension, the “amotivation” or absence of motivation, referring to a perceived waste of time, unworthy effort, and useless tasks, the scale was reverted, being 1 as “strong”, and 5 as “lack of” amotivation, in order to be able to compare it with the rest of the dimensions. The majority of the participants stated that they did not have amotivation to perform the requested tasks (41.7% disagreed and 41.7% strongly disagreed with the presence of amotivation), while 16.7% showed neutrality. These results are understandable and expected since they freely volunteered in the experiment.

The second category belongs to “extrinsic motivation” or motivation based on winning rewards and avoiding punishments and includes 4 dimensions according to the type of regulation. With respect to the second dimension of “external social regulation” concerning other’s approval, recognition, and criticism avoidance, this was denied by the majority (16.7% disagreed and 66.7% strongly disagreed) while 16.7% was neutral. In this way students reject as influencing factors the attitudes of others in their motivation. On the dimension of “external material regulation”, referring to avoiding decreasing grades, getting academic rewards, and gaining experience, the majority of users were neutral (66.7%), 25% avoided recognizing their influence (being in disagreement), while one user (8.3%) agreed. Although the students do not deny that there may be material factors such as gaining experience (the reason that obtained the highest scores), those are probably not such determining factors in completing the requested tasks.

On the other hand, for the fourth dimension of “introjected regulation”, related to demonstrate self-capability, feel proud of oneself, avoid dissatisfaction for not complying, half of the students were neutral (50%) while others confirmed their impact (8.3% agreed and 33.3% strongly agree), and one user denied it (8.3% disagreed). The fifth dimension of “identified regulation” regarding the importance, value and personal significance of putting effort into tasks was expressed by all the participants (41.7% agreed and 58.3% strongly agree). The obtained results could point out that users get involved in this type of activities with the same commitment, seriousness and motivation as they would in the case of interacting with a portal in which they need or wish to navigate in not only experimental contexts.

Finally, on the third category and sixth dimension, “intrinsic motivation”, associated with doing inherently entertaining, interesting and challenging tasks and incitement to learn, this was expressed by all users (50% agreed and 50% strongly agree). This would allow reflecting once more on the authentic interest of the participants in carrying out this type of tasks and on the site, as they state in the perception questionnaire.

With respect to the users with previous experience in visiting DLs in comparison with the novice users of this type of portals, it should be noted that with respect to amotivation they tended towards greater neutrality. This is not necessarily surprising since they may have become accustomed to navigations and explorations in these types of sites. In relation to the external regulation dimensions, unlike the general trend, they showed in disagreement, but not in the rest of the dimensions, of greater internal regulation. This suggests once again that the repetitive navigation they have done on DLs probably comes from a real interest in this type of portals or in this type of exploration tasks.

It can be observed that there is a tendency for an increase in the motivation that the participants affirm as regards factors of a more internal, personal and subjective nature, such as their personal appreciations (identified regulation) and their innate attraction (intrinsic motivation) towards what do they do. The averages of the scores for each dimension (4.28 for “lack of” amotivation, from 1.39 for external social regulation to 4.44 in identified regulation, and 4.40 for intrinsic motivation) [2], also reflects this. Despite that from a quantitative view users’ overall motivation seems neutral, with an average of 3.48 [2], this indicator does not turn out to be the most representative for an analysis around users’ motivation. The applied psychometric test pretends to glimpse the presence of different dimensions involved in motivation, but these have different relevance, impact and weighting for the same participant, with amotivation possibly being the dimension that best integrates these aspects. The breakdown of each of these dimensions from a qualitative perspective is what allows a deeper and more complete understanding of the user’s experience in relation to motivational aspects.

4 Second Experiment: Communicability Test

The second experiment was also performed in the Usability Laboratory of the School of Informatics Engineering of the Pontificia Universidad Católica de Valparaíso, Chile, a month after the first experiment. It was conducted by two UX experts. The same expert who studies Psychology participated, and the other is a PhD student in Informatics Engineering.

The WDL was once again evaluated, based on a set of predefined tasks. These tasks were different from those performed in the first experiment (Sect. 3). In the communicability test, the participants explored the website alone. Their facial expressions were recorded with cameras and the screens of their computers were recorded. They were also observed by the evaluators through a polarized glass that allows one-way vision.

The experiment involved 6 participants, all being graduate students in Computer Science. After the participants signed a confidentiality agreement, they were informed about the test conditions, about the website to evaluate, and about the different stages of the test.

The experiment consisted of 3 parts:

  • (1) A pre-experiment questionnaire designed to identify the user profile and previous experience visiting portals similar to the evaluated product.

  • (2) An individual communicability test, presenting to each participant a series of tasks to explore the website.

  • (3) A post-experiment perception questionnaire that sought to know the user perceptions regarding the website and the tasks.

4.1 The Pre-experiment Questionnaire

Each user had to complete a pre-experiment questionnaire to collect general information about their profile and experience in other DLs. The same five questions included in the previous experiment were asked, adding a new one regarding the user’s profession.

There were 6 participants, 4 men and 2 women, between 23 and 34 years old. 83.3% of the users (5) had already visited DLs before, but stated that they (almost) never visit these types of sites.

4.2 The Communicability Test

Each participant was provided with a list of predefined tasks to explore the website. The tasks were aimed to identify communicative breakdowns [3]. While the participants were accomplishing the tasks, the evaluators identified all signs of communicative breakdowns in the user’s interaction. They took notes of these during the test.

The communicability test included 3 tasks:

  1. 1.

    T1: “Search an article”. The first task was to search an article browsing the word “mathematics” in the main search engine of the website and applying different filters to select a specific language (“Spanish”), a specific place (“Europe”) and to visualize the results in the form of a gallery.

  2. 2.

    T2: “Read a book online”. The second task was to read the book “Atlas of the Physical and Political History of Chile”, accessing to the “Natural Sciences and Mathematics” section, and then to the “Animals” section. The user should visualize the book in full screen, go to page 26 and write down the result of the animal that was shown on the screen (“swan”).

  3. 3.

    T3: “Find a museum on the map”. The third task was to visualize and search for a specific museum using the website map. After accessing to the “Institution” section, the user should filter the results by “museums” and select the museum “Walters Art Museum”. The user should note in which country the museum is located and the number of related articles.

Table 1 shows the number of communicative breakdowns observed in the test for each user (U1–U6), considering all 3 tasks.

Table 1. Number of communicative breakdowns observed for each user.
Full size table

The communicative breakdowns with greater frequency were: “Where is it?” (21 communicative breakdowns) and “Looks fine to me” (12 communicative breakdowns). Users had difficulty finding the item (or information) they were looking for (“Where is it?”). In addition, users believed that they achieved their goal, however this did not happen. The user was not aware of the communicative breakdown (“Looks fine to me”).

The communicative breakdowns with medium frequency were: “Oops” (11 communicative breakdowns), “Where am I?” (8 communicative breakdowns), and “I can do otherwise” (8 communicative breakdowns). The users made an error and immediately realized it. The users went back a step (“Oops”). In addition, users took actions that would be appropriate in another context. That is, they selected the wrong paths to achieve the task (“Where am I?”). Finally, users were not fully aware of the ways of action offered by the system to perform a task. Users chose to do something different than is expected, but achieved the same effect. That is, users achieved their goals by a non-optimal path (“I can do otherwise”).

The communicative breakdown “What is this?” did not occur in any of the tasks (0 communicative breakdowns). This means that all the users understood the elements of the website.

Table 2 shows the number of communicative breakdowns observed for each task.

Table 2. Number of communicative breakdowns observed for each task.
Full size table

As shown in Table 2, the task with the most communicative breakdowns was Task 3: “Find a museum on the map”. The users could not properly filter the “Institutions” by “museum”. This is because the website displays an unintuitive search filter for the user. The filter controls use a confusing color, so the user does not know when the filter is applied or not (see Figs. 1 and 2). Due to the difficulty in filtering the search by museum, users could not select the “Walters Art Museum” on the map. The communicative breakdowns that occurred most in task 3 were: “Where is it?”, “What now?”, “Where am I?”, “Why doesn’t it?”, “What happened?”, “I can do otherwise”, and “Thanks, but no, thanks”.

Fig. 1.
figure 1

The search was performed by the institutions types: “Archive”, “Library” and “Other”. “Museum” is not selected. The users thought that the white color indicated that “Museum” was selected, when it was otherwise.

Full size image
Fig. 2.
figure 2

Correct filter applied. The search was performed by institution type: “Museum”. The gray color indicates that only “Museum” is selected. However, this was not clear to users.

Full size image

For Task 1: “Search an article”, the communicative breakdowns that occurred most were: “Oops!” and “Looks fine to me” (both with 6 communicative breakdowns). Some users did the search without filtering by language and/or place, but they quickly realized the error and immediately corrected it (“Oops!”). Some users accessed the wrong book (they followed a different path) so they found the wrong animal. The users believed that they achieved the goal, but it was otherwise (“Looks fine to me”).

For Task 2: “Read a book online”, the communicative breakdown that occurred most was: “Where is it?”, with 10 communicative breakdowns. Users could not find the option to read the book online. This was because the website did not have a clear button to access that option. The users looked for the option in the page by minutes, until they realized that by positioning the mouse for a few seconds on the book image, it could be viewed to read.

Based on the results obtained in the communicability test, the evaluators explained the designer’s message [3]. To do this, the evaluators assumed the first person in discourse and spoke for the designer by answering the following questions:

  • Who do I think are the users of the product of my design? They are users who access digital libraries very rarely but who have experience in the use of websites, so they know the meaning of the elements and symbols of a website.

  • What have I learned about these users’ wants and needs? I have learned that users understand the elements of the interface without problems, but that some features of the website are not intuitive and easy to use (filters), which makes it difficult to use and search for information.

  • Which do I think are these users’ preferences with respect to their wants and needs, and why? Users prefer an easy to use and navigate website, which is interactive and informative. Users prefer a website with intuitive filters that allow them to find what you want efficiently and quickly.

  • What system have I therefore designed for these users, and why? I have designed a website that allows the user to acquire knowledge about different cultures of the world. I have designed a confusing website for some users, because certain functionalities are unclear and do not help the user in their information search process.

  • What system have I therefore designed for these users, and how can or should they use it? I have designed a digital library that allows users to access, free of charge and at any time, a wide variety of material about different cultures of the world. The user can search information by categories, timelines and interactive maps. In addition the articles can be reviewed using different multimedia elements.

  • What is my design vision? My design vision is to distribute information to users about different cultures in different languages through the use of interactive elements, such as: audio, video, maps and online reading. My design vision is to present highlighted information on the home page of the website, allowing the user to search for content through a general search engine, categories, timelines and/or interactive maps.

Based on the semiotic profiling presented above, the evaluators identified the meta-communicational message using the template proposed by De Souza and Leitão [3]. The metacommunication template sums up what designers are communicating to the users through systems interfaces.

  • Here is my understanding of who you are, what I’ve learned you want or need to do, in which preferred ways, and why. I think you are a user with or without experience in digital libraries. I think you are interested in learning about different cultures from different countries of the world. I think you would like to see digital articles. For this reason I have designed for you an easy-to-use and interactive website, with different articles, videos, audios and images that allow you to access information as if you were in a physical library. I have also designed the website with a structure that allows you to search for information by categories, sections, timelines and maps.

  • This is the system that I have therefore designed for you, and this is the way you can or should use it. The system that I have designed for you is to obtain information of your interest. You can interact in different ways, using audio, video and/or interactive reading about articles from different cultures of the world. I have designed the website with different search methods, so you can access the content as best suits you. You can search for a particular concept; navigate through the different sections that I present to you; or interact with the world map to look for information.

  • In order to fulfill a range of purposes that falls within this vision. The objectives that are within my vision are to promote the exchange of cultural knowledge in a global way; and expand the amount and variety of cultural content on the Internet. To do this, I present the content in a visual way and with appropriate sizes, allowing you to browse and search for information of interest through different mechanisms.

4.3 The Post-experiment Perception Questionnaire

After completing the test, in a third stage, users had to respond individually to a post-experiment questionnaire based on the System Usability Scale (SUS) [15]. The questionnaire applied was the same as in the first experiment (Sect. 3).

Regarding the difficulty to complete the requested tasks, half of the participants indicated that they considered it neutral (50%), while two users considered it easy to achieve (33.3%) and one user considered it difficult (16.7%). Orientation within the portal was perceived as variable. Half of the participants felt less oriented (50%), two users felt neutral (33.3%) and one user felt oriented (16.7%). In relation to the degree of satisfaction with WDL, two users found it satisfactory (33.3%); two users found it neutral (33.3%) and two users found it unsatisfactory (33.3%). With respect to the information found on the site, the majority felt satisfied (66.7%), with one user (16.7%) who considered it very satisfactory, while 33.3% felt neutral. Finally, half of the participants express the intention to re-use the WDL (50%), while 33.3% of the users are neutral, and one user disagreed (16.7%).

All users stated that the most difficult to use of the site were the search filters. Five users declared that after using the filter by “Institutions type” on the map for a while, they realized how it worked and managed to find the museum. However, they all declared that the use of the filter is not intuitive.

The users positively highlighted the website design, the variety and large number of articles presented, and the good images quality.

On the other hand, users stated that the website should correct certain elements, such as improving the use of filters (both the search filters of articles as the filter by institutions on the map) and clearly show the buttons of certain actions (e.g. reading articles online).

5 Conclusions

Users expressed similar concerns during the co-discovery experiment and the post-experiment perception questionnaire. They would like WDL to offer an intuitive and “user-friendly” navigation. They would also expect intuitive and easy to use functionalities. These are perceived as main obstacles in accomplishing the tasks that the experiment required. They had direct impact on UX, and also generated communicative breakdowns during the communicability test.

Users tried to find alternative ways to accomplish tasks. They wanted flexibility; they expected WDL to adapt to their working style and preferences. Flexibility could improve UX.

Users showed interest in WDL’s content, and they hoped finding diverse information. The post-experiment psychometric test showed lack of amotivation, and more internal regulation (identified regulation, intrinsic motivation), than external regulation (extrinsic, material, and social). Results are consistent with the perception questionnaire’s outcomes. It seems that a satisfactory UX would be influenced by accomplishing goals intuitively, and finding interesting content, that may generate enthusiasm.

Better motivation could lead to higher commitment in accomplishing tasks, better performance, better efficacy and efficient use of the tools that WDL offers. It would probably generate more sincere and authentic opinions on users’ experience. This highlights the importance of psychological aspects on a positive UX, aspects that the interaction designer should always consider. The experiments that we made highlight, as expected, the importance and complementarity of quantitative and qualitative aspects. It is also relevant to break them down into each constituent aspect and angles, in order to understand users’ experience in a more profound and holistic way.

Human-computer interaction is in fact a user-computer-designer interaction, a place where designer’s intentions meet and intersect user’s goals, and the way that there are accomplished. UX is the results of users’ expectations, goals, beliefs, preferences, but also their emotions, perceptions, physical and psychological responses, behaviors and achievements, towards what the designer offers.

Successful user-computer-designer interaction has to conciliate user’s goals and designer’s intentions. As future work we intend to study modes to better conciliate both aspects, to collect and analyze more experimental data, and to address other psychological aspects.