1 Introduction

People in old days get information about historical sites through travel books, TV programs, or word of mouth. Nowadays, as Internet technologies advance, people obtain these information through social forums, blogs, or cultural websites. Thus, websites were developed that attempt to provide tourists with an overview of (1) the culture, the lifestyle of the people living in the geographical areas, the history of the people, their art, architecture, and religion(s), as well as (2) the services that engage tourists with the culture, such as location/parking information, events, schedule planning, [1]. However, the interfaces/information of the current cultural heritage websites do not necessarily allow tourists to get a good understanding of the culture and connect people to local places, objects, and events.

An effective and user-friendly cultural website should not only ensure the quality of delivered culture [2], but also propagate knowledge and provide learning sources and scientific research, etc. [3]. However, what current cultural heritage websites communicate are simply factual information (e.g., direction to the place/site, the history of the place and/or the architecture, and tickets booking, etc.), in other words, filling visitors with facts. The websites are not designed to help visitors come to an understanding of ‘hidden truths.’ For example, visitors might want to know, while visiting the heritage site, how to get immersed into the cultural aspects of the heritage, as well as how to explore the value of the heritage and how to relate this value to them in their real life.

In the literature, the quality issues of cultural websites had been studied extensively. Di Blas, Guermand, Orsini, and Paolini [4] developed a framework, called “MiLE,” to evaluate the quality and usability of museum websites. The framework broke the features and the contents of a museum website into three groups of constituents: (1) site presentation, (2) museum presentation, and (3) virtual museum. The framework divided tourists’ concerns into three types of tasks: (a) Practical (a tourist wants to gather useful information), (b) Operational (a tourist wants to do something) and (c) Cognitive (a tourist wishes to learn something). In Di Blas et al.’s framework, twelve criteria for inspecting an intended website were proposed: Efficiency, Authority, Currency, Consistency, Structure effectiveness, Accessibility, Completeness, Richness, Clarity, Conciseness, Multimediality, Multilinguisticity. In a similar vein, Davoli, Mazzoni, and Corradini [5] looked at the quality of cultural websites from the perspectives of usability and technical performance. In their study, six quantitative linguistic indices were used: Basic functionality, Advanced functionality, Usability, Accessibility, Efficiency, and Maintainability and Compliance.

In fact, according to Minerva Working Group [3], users’ perceptions of a cultural website are determined not only by the cultural content it offers, but also by the history of the institution it represents, by its mission, by its functional organization and by its internal and external relations. Caffo and Hagedorn-Saupe [2] shared the same viewpoint of Minerva Working Group [3]. They asserted that a good quality cultural website must be transparent, content-effective, updated at an appropriate level, accessible to all users, user-centred, responsive, multi-linguality, being interoperable within cultural networks, managed to respect legal issues such as IPR and privacy, and finally preserved for the long-term use. The above studies indicated that the quality of a cultural website should not be viewed simply on its usability-related attributes (e.g., esthetics, functionality, efficiency, effectiveness, etc.). Tourists’ cultural needs and engagements were not seriously considered for increasing the satisfaction and enjoyment of visiting the heritage sites [4].

The purpose of cultural heritages is mainly about delivering cultural values, as well as provoking thoughts and awareness of the significance of culture. Thus, this study aimed at investigating current cultural heritage websites, as well as identifying the missing features/information that could impact viewers’ perceptions and expectations of an intended culture. For the purpose of the evaluation, the quality goals for cultural websites [6] were used. The following research question guided the design of the experiment in this study: “What are the misalignments with the quality goals that made the websites unable to effectively deliver cultural values to viewers?” This study is important as it provides insight and contributes to cultural heritage web design.

2 Literature Review

2.1 The Definition and Importance of Cultural Entity

Different countries and races have their own unique culture. However, the formation of culture takes time, which makes it a fairly abstract subject [7]. Conversely, if cultural elements (language, symbol, religion, artifact, object, and social norm) are not adequately preserved, culture could disappear quickly. Thus, organizations/institutions/societies for the conservation of cultural and scientific heritage started to draw public attentions. These organizations/institutions/societies, according to Minerva Group 5 [6], are called cultural entities who conserve tangible and intangible cultural knowledge and value. They generate and propagate knowledge to influence the value and the way of thinking among people in real world [2].

Museums or historical heritages are cultural entities that are having greater impacts. In one hand, they contain rich cultural elements. On the other hand, their numbers of visitors are increasing across these years [4]. Take the number of visitors of the Louvre Museum for instance, the number has grown exponentially, averagely increasing two hundred thousand visitors per year [8].

As web applications have become a major channel that tourists used to understand cultural entities, improving the design of cultural web applications has become a vital issue [9, 10] for designers.

2.2 The Quality Framework for Cultural Web Applications

The quality of cultural websites is important to ensure cultural contents are delivered to users. However, quality is a broad, generic and a subjective concept. In reality, the most comprehensive framework to assess the quality of cultural web applications was proposed by the Minerva Working Group 5 [6].

Minerva is a network of European Union member states’ ministries (started from 2002) working together as a national representatives group in building an information society for all European citizens that reflects the wealth of European cultural creation and heritage. One of the main goals of the network was to support cultural institutions all over Europe as they were in charge of preserving and exploiting heritage and played an important role in delivering the best services to European citizens.

As high-quality websites allow European citizens to discover, to explore and to benefit from online material representing the diversity of European culture, the Minerva group defined and published twelve goals (satisfying both the goals of cultural institutions and their users) to promote the quality of cultural websites. The goals are in the following:

  1. (1)

    Presentation of the identity of the cultural entity: A high-quality website should demonstrate the constitutional elements that contributed to forming the unique features of a cultural entity.

  2. (2)

    Transparency on the activities of the cultural entity: A high-quality website should show activities (including programmes, projects, funding, procedures, realization phases, and/or results) that achieve the mission of a cultural entity.

  3. (3)

    Transparency on the mission of the cultural web application: A high-quality website should show an obvious purpose or mission by stating the subject of the website, the most important content on the site, and the organization responsible for maintaining the site, and optionally the target audience.

  4. (4)

    Efficiency in the sector networks: A high-quality website must collaborate and coordinate with other similar online cultural entities. Each site activates a section which, via links to parallel available resources (those with the same objectives), puts information, databases, and references into common use. The co-ordination is based on co-operative participation (i.e. between equals and aimed at achieving a common objective). This principle broadens the focus of quality beyond the individual website, by considering how it can interface with other cultural websites and with entities, such as cultural portals, which are higher and lower in the information hierarchy.

  5. (5)

    Presentation of standards and regulations of the sector: A high-quality website must have updated references to the basic regulations in the cultural sector and on the mission of the cultural entity, with the added value of an institutional picture of the cultural activity.

  6. (6)

    Spreading of cultural content: A high-quality website must spread cultural content for the purpose of promoting culture growth. The spreading must be managed to respect legal issues (such as intellectual property rights and privacy) and clearly state the terms and conditions on which the website and its contents may be used.

  7. (7)

    Support of cultural tourism: A high-quality website must provide services of information in tourism activities that aim to sustain the territorial values of the cultural heritage.

  8. (8)

    Offer of educational services: A high-quality website must exploit the cultural heritage by providing multimedia and interactive didactic support to demonstrate how and when the cultural heritage gained its significance in the history. Didactic web services (such as e-magazines for diffusion of news and comments) between similar or related sites can be created to encourage connections between cultural heritage and the territory.

  9. (9)

    Offer of services of scientific research: A high-quality website should provide researchers with consulting scientific documentation and the access to reports, library catalogues, archive inventories, and/or museum catalogues.

  10. (10)

    Offer of services to specialists in the sector: A high-quality website should offer services to specialists in their sectors of cultural and scientific heritages. The services can be supplied on demand or through various enrolments.

  11. (11)

    Offer of services of reservation and acquisition of goods: A high-quality website must be accessible to all users (despite of their disabilities), regarding navigation, content, interactive elements, digital resources, and acquisition of goods, etc.

  12. (12)

    Promotion of web communities in the sector: A high-quality website must establish strategies for reaching various user categories in the culture sector through interactive tools/media on the website. User comments and feedback are expected to be collected, analyzed, and monitored continuously to enhance the quality of the services and to promote the influence of the cultural entity in its web communities.

The above goals form the quality framework for assessing cultural web applications. We believed that this quality framework could be used to predict tourists’ engagements with an intended culture. In fact, similar framework could be easily found in the literature. Although they are not as comprehensive as the one Minerva proposed, they were proved to be valid [4, 5]. What drew the attention of this research was that the adoption rates of these cultural design framework had been low-evidenced by the fact that cultural websites did not effectively deliver the missions/values of cultural heritages, causing bad user experiences in terms of cultural engagement. Thus, the research questions that were not particularly answered were: “What is wrong with the design of the current cultural heritage websites?”, and “what caused tourists’ misperceptions of the missions/values of the cultural heritages?” The following explains our research method for answering these research questions.

3 Method

3.1 Participants

A heuristic evaluation [11, 12] was conducted with eight subject matter experts (each having cultural product/web design experiences) (SMEs) on the levels of cultural engagement of 10 world heritage websites. Our number of experts satisfied Nielsen’s [13] evaluation requirements that suggested that five experts were able to identify 80% of the problems of a website. The twelve quality goals for cultural websites (developed by the Minerva Working Group 5) were given to the experts. The rating score ranged from 0 to 5, 0 being not at all satisfied with the quality principle, and 5 being extremely satisfied with the quality principle. Each of the experts evaluated the websites individually without communications. Ratings and design recommendations were collected and analyzed by researchers of this study.

3.2 Materials

The website assessments were conducted using a personal laptop. The ten selected world cultural heritage websites (determined by the popularity and the number of visitors stated on the websites) included: (1) Tower of London (https://www.hrp.org.uk/tower-of-london/), (2) Taj Mahal (https://www.tajmahal.gov.in/), (3) Emperor Qin Shi Huang’s Mausoleum Site Museum (http://www.bmy.com.cn/2015new/index.htm), (4) Leaning Tower of Pisa (http://www.towerofpisa.org/), (5) Palace and Park of Versailles (http://en.chateauversailles.fr/discover/estate/palace), (6) Itsukushima Shinto Shrine (http://www.en.itsukushimajinja.jp/), (7) Acropolis of Athens (http://www.acropolisofathens.gr/aoa/), (8) Statue of Liberty (https://www.nps.gov/stli/index.htm), (9) The Great Wall (http://www.great-wallofchina.com/), and (10) Hagia Sophia Museum (https://ayasofyamuzesi.gov.tr/en). Each SME was given a standardized form for performing the website assessments. The form included Minerva’s goals for cultural websites and their definitions. Along with each principle was a space for entering the rating score, the explanations for the score, and the design comments.

3.3 Procedure

Before the evaluation started, the researcher explained the purpose of the experiment and obtained the consents from the participants. The researchers then gave the assessment forms to the participants. Participants were told to take their time and that they could ask any questions about the form and the experiment.

After being familiarizing with the content of the form, participants were shown the list of ten world cultural heritage websites and asked to perform the assessments. Participants were asked to use as much time as they want to look at, in every website, the design, the features, the functions, the information, the layout, etc. Participants were asked to rate how well the website design satisfied the requirements of each of the Minerva’s goals. Participants were also asked to provide the explanations for their ratings, the comments, and the design recommendations for the websites.

In the experiment, every participant performed his/her assessments individually without communications and discussions. No time restrictions were put on the website assessments.

4 Results

The aim of the study was to investigate current cultural heritage websites and to identify the missing features/information and bad designs that caused misunderstanding and misinterpretations of an intended culture. The descriptive statistics of the ratings of the eight SMEs were shown in Table 1. From Table 1, the mean ratings for the websites were very different, ranging from 1.11 to 4.05.

Table 1. Descriptive statistics for the ratings of the ten cultural heritage websites
Full size table

Table 1 also shows that fifty percent of the websites received ratings above the average (2.5) of the 6-point rating scale. The website that received the highest mean rating was “Tower of London”, with the rating of 4.05. The website that received the lowest mean rating was “Itsukushima Shinto Shrine” with the rating of 1.11.

Table 2 describes how well the websites engage tourists with culture from the viewpoint of each quality principle. From Table 2, we can see that only five goals were satisfied (with the mean ratings ≧ 2.5) by the design of the cultural heritage websites. Particularly, “Goal #1: Presentation of the identity of the cultural entity” was mostly addressed, with the rating of 3.8. “Goal #5: Presentation of standards and regulations of the sector,” “Goal #12: Promotion of web communities in the sector,” and “Goal #9: Offer of services of scientific research,” were least addressed, with the rating of only 1.73, 1.65, and 1.55, respectively.

Table 2. Descriptive statistics: how-well the cultural websites satisfied the quality goals
Full size table

5 Discussion and Design Recommendations

The descriptive statistics of the heuristic evaluation suggested that the design of the cultural heritage websites was diverse in engaging tourists with culture.

To explore how each cultural website addressed each of the quality goals, Table 3 was constructed. From Table 3, we can see that only one website addressed Goal #5, only two websites addressed Goal #12, only three websites addressed Goal #7, Goal #9, and Goal #11, and only four websites addressed Goal #10. These results explained the low mean rating scores of some cultural websites in Table 1. In fact, the SMEs commented in the experiment that these cultural heritage websites did not (1) present information on the standards and regulations of their intended culture; (2) provide good amounts of information services in tourism activities; (3) provide enough supports and services for scientific research and specialist activities; (4) offer tourists a convenient channel to purchase goods and reserve services; (5) aggressively promote culture to other web communities/user groups. These comments answered the research question regarding the websites’ misalignments with the quality goals proposed by the Minerva Working Group 5.

Table 3. The average ratings for the cultural heritage websites in satisfying the quality goals
Full size table

Table 3 also shows how well each website satisfied the overall quality framework. From Table 3, we can see that only three websites (W1: Towel of London, W5: Palace and Park of Versailles, and W8: Statue of Liberty) were able to satisfy over 60% of the goals (≧8 goals) in the quality framework. The website design of W6: Itsukushima Shinto Shrine only satisfied one of the goals (Goal #1) in the quality framework.

By looking at the highest rating scores of the goals and their corresponding websites, we summarized the design comments made by the SMEs in the following. These could be used as design strategies for enhancing cultural engagements with tourists.

  1. (1)

    With regards to Goal #1, cultural heritage websites could clearly state their identity using a short and concise sentence along with the iconic image of the heritage on the landing page. On the landing page, it is important as well to embed images and information related with key cultural DNAs (including symbols, behaviors, objects, norms, and values, etc.) [14] of intended culture, allowing visitors virtually interacting with the heritage in the first place.

  2. (2)

    With regards to Goal #2, cultural heritage websites could present events and activities either all on the landing page or with a clearly identifiable link/tab on the landing page. Information about the events and activities could use differing media (e.g., text, image, or animation) to engage with viewer attention.

  3. (3)

    With regards to Goal #3, cultural heritage websites should clearly state their missions, responsibilities, and objectives using either clear/concrete/concise sentences or a numbered list of items. These information should be placed either in the landing page or with a clearly identifiable link/tab on the landing page.

  4. (4)

    With regards to Goal #4, cultural heritage websites could use an easily identifiable link/tab showing that they maintain good relationships and actively collaborate with other (nearby/related) cultural entities in preserving and creating shared cultural values.

  5. (5)

    With regards to Goal #5, cultural heritage websites should show updated and direct links/references to the standards/regulations related with cultural preservation.

  6. (6)

    With regards to Goal #6, the information shown in cultural heritage websites should be written in simple and easily-understood language and respect legal issues (such as intellectual property rights and privacy) and clearly state the terms and conditions on which the website and its contents may be used.

  7. (7)

    With regards to Goal #7, cultural heritage websites should provide touring services/information and/or links to other local cultural spots that tourists can benefit when/after visiting the cultural heritage.

  8. (8)

    With regards to Goal #8, cultural heritage websites should provide the didactic services (e.g., showing historical events chronically) that engage tourists with intended culture. The services should be tailored to satisfy differing customer needs (e.g., the elderly/people with disabilities, etc.) and be delivered with interactive multimedia/games.

  9. (9)

    With regards to Goal #9 and #10, cultural heritage websites should show the services (in details) that they provide particularly to researchers/specialists in the cultural sector. Some digital archival records such as books/magazines/reports could be made online available to a vast community of researchers.

  10. (10)

    With regards to Goal #11, cultural heritage websites could place links that can be easily seen in the landing page to guide visitors to the shopping/ticket/space and service booking pages. These pages should present information (e.g., tickets, rooms, gifts, books/publications, or collections, etc.) in a nice order and with high quality images and item descriptions. Payments should be made flexibly and easily.

  11. (11)

    With regards to Goal #12, cultural heritage websites could leverage the power of social media (e.g., using Facebook, Twitter, or Instagram applications and call-to-action buttons, etc.) to promote the explicit/implicit cultural values to differing social groups locally or internationally and to obtain feedback from their visitors for enhancing the quality of the provided services.

6 Conclusions

The purpose of the study was to assess well-known world cultural heritage websites and to identify the missing features/information that could impact viewers’ cultural engagement. Heuristic evaluation was conducted with eight SMEs using the Minerva’s quality framework for cultural web applications. Results of the study showed that the mean ratings for the ten cultural websites were diverse (ranging from 1.1 to 4.05), which suggested that current cultural heritage websites could not consistently and effectively arouse viewers’ cultural engagement.

Our evaluation also found that only five out of ten websites received ratings above 2.5 (the average of the 6-point rating scale); only three websites (W1: Towel of London, W5: Palace and Park of Versailles, and W8: Statue of Liberty) were able to satisfy over 60% of the goals (≧8 goals) in the quality framework.

The SMEs commented that, in general, cultural heritage websites did not (1) provide information on the standards and regulations of intended culture; (2) provide sufficient information on services in tourism activities; (3) provide information on the services for scientific research and specialist activities; (4) offer tourists an easily identifiable channel to purchase goods and reserve services; (5) put efforts in promoting culture to other community users.

This study also made twelve practical design recommendations for cultural heritage websites. It was our hope that the outcomes provide insight in the field of designing cultural heritage web. This study is limited by the number of evaluated cultural heritage websites and by the use of SMEs for identifying design problems. To enhance the validity of the research outcomes, more cultural websites should be included and surveys should be distributed to real end users for the evaluation.