Skip to main content
SpringerLink
Log in

Hierarchical Representation of Website Evaluation Model Using Survey and Perceptual Based Criteria

  • Conference paper
  • First Online:
Information Technology for Management. Ongoing Research and Development (ISM 2017, AITM 2017)

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 311))

Abstract

The high availability of e-commerce websites which deliver similar services and products, as well as the harsh rivalry between competitors, increased the importance of systematic evaluation of the e-commerce websites’ quality, usability and user experience. Multiple methodologies for performing the evaluation are available, however, they are based mainly on survey data. In our previous research, we introduced perceptual measurements from eye tracker (ET) to the set of evaluation criteria. In this paper, we present an approach based on AHP (Analytic Hierarchy Process) to allow a thorough analysis of the complex structure of criteria and its impact on the final evaluation. Additionally, we combine the AHP outputs with the COMET (Characteristic Objects METhod) technique to build a fuzzy rule base that provides a stable model of the entire domain of evaluation criteria. The results of the conducted empirical verification of the proposed approach are presented and discussed. The main research findings show that the rankings obtained with the presented approach are very stable and the probability of a rank reversal phenomenon is low.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Kemp, S.: Digital in 2017 global overview, January 2017

    Google Scholar 

  2. Kemp, S.: Digital in 2016, January 2016

    Google Scholar 

  3. Ecommerce News Europe: ecommerce in Europe, September 2016

    Google Scholar 

  4. Lindner, M.: Online sales will reach $523 billion by 2020 in the US, January 2016

    Google Scholar 

  5. Paul, R.: Just how big is the ecommerce market? you’ll never guess!, June 2015

    Google Scholar 

  6. Strzelecki, A., Furmankiewicz, M., Ziuziański, P.: The use of management dashboard in monitoring the efficiency of the internet advertising campaigns illustrated on the example of Google analytics. Studia Ekonomiczne 296, 136–150 (2016)

    Google Scholar 

  7. Grigera, J., Garrido, A., Panach, J.I., Distante, D., Rossi, G.: Assessing refactorings for usability in e-commerce applications. Empir. Softw. Eng. 21(3), 1224–1271 (2016)

    Article  Google Scholar 

  8. Sohaib, O., Kang, K.: Assessing web content accessibility of e-commerce websites for people with disabilities (2016)

    Google Scholar 

  9. Olsson, M.: Build a Profitable Online Business: The No-Nonsense Guide, 1st edn. Apress, Berkely (2013)

    Google Scholar 

  10. Kruger, R., Gelderblom, H., Beukes, W.: The value of comparative usability and UX evaluation for e-commerce organisations. In: CONF-IRM 2016 Proceedings, January 2016

    Google Scholar 

  11. Barnes, S.J., Vidgen, R.: The equal approach to the assessment of e-commerce quality: a longitudinal study of Internet Bookstores. In: Web Engineering, p. 161 (2005)

    Google Scholar 

  12. Webb, H.W., Webb, L.A.: SiteQual: an integrated measure of web site quality. J. Enterp. Inf. Manag. 17(6), 430–440 (2004)

    Article  Google Scholar 

  13. Parasuraman, A., Zeithaml, V.A., Malhotra, A.: ES-QUAL: a multiple-item scale for assessing electronic service quality. J. Ser. Res. 7(3), 213–233 (2005)

    Article  Google Scholar 

  14. Ahn, T., Ryu, S., Han, I.: The impact of the online and offline features on the user acceptance of internet shopping malls. Electron. Commer. Res. Appl. 3(4), 405–420 (2005)

    Article  Google Scholar 

  15. Barnes, S.J., Vidgen, R.: Measuring web site quality improvements: a case study of the forum on strategic management knowledge exchange. Ind. Manag. Data Syst. 103(5), 297–309 (2003)

    Article  Google Scholar 

  16. Yang, Z., Cai, S., Zhou, Z., Zhou, N.: Development and validation of an instrument to measure user perceived service quality of information presenting web portals. Inf. Manag. 42(4), 575–589 (2005)

    Article  Google Scholar 

  17. Barnes, S., Vidgen, R.: WebQual: an exploration of website quality. In: ECIS 2000 Proceedings, p. 74 (2000)

    Google Scholar 

  18. Kabir, G., Hasin, M.: Comparative analysis of topsis and fuzzy topsis for the evaluation of travel website service quality. Int. J. Qual. Res. 6(3), 169–185 (2012)

    Google Scholar 

  19. Brans, J.P., Mareschal, B.: Promethee methods. In: Figueira, J., Greco, S., Ehrogott, M. (eds.) Multiple Criteria Decision Analysis: State of the Art Surveys. ISOR, vol. 78, pp. 163–186. Springer, New York (2005). https://doi.org/10.1007/0-387-23081-5_5

    Chapter  Google Scholar 

  20. Sun, C.C., Lin, G.T.: Using fuzzy TOPSIS method for evaluating the competitive advantages of shopping websites. Expert Syst. Appl. 36(9), 11764–11771 (2009)

    Article  Google Scholar 

  21. Bilsel, R.U., Büyüközkan, G., Ruan, D.: A fuzzy preference-ranking model for a quality evaluation of hospital web sites. Int. J. Intell. Syst. 21(11), 1181–1197 (2006)

    Article  MATH  Google Scholar 

  22. Wątróbski, J., Ziemba, P., Jankowski, J., Wolski, W.: PEQUAL-e-commerce websites quality evaluation methodology. In: 2016 Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 1317–1327. IEEE (2016)

    Google Scholar 

  23. Chen, L., Pu, P.: Eye-tracking study of user behavior in recommender interfaces. In: De Bra, P., Kobsa, A., Chin, D. (eds.) UMAP 2010. LNCS, vol. 6075, pp. 375–380. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13470-8_35

    Chapter  Google Scholar 

  24. Chang Lee, K., Wook Chae, S.: Exploring the effect of the human brand on consumers’ decision quality in online shopping: an eye-tracking approach. Online Inf. Rev. 37(1), 83–100 (2013)

    Article  Google Scholar 

  25. Cowen, L., Ball, L.J., Delin, J.: An eye movement analysis of web page usability. In: Faulkner, X., Finlay, J., Détienne, F. (eds.) People and Computers XVI-Memorable Yet Invisible, pp. 317–335. Springer, London (2002). https://doi.org/10.1007/978-1-4471-0105-5_19

    Chapter  Google Scholar 

  26. Bojko, A.: Eye Tracking the User Experience. Rosenfeld Media, Brooklyn (2013)

    Google Scholar 

  27. Ziemba, P., Wątróbski, J., Karczmarczyk, A., Jankowski, J., Wolski, W.: Integrated approach to e-commerce websites evaluation with the use of surveys and eye tracking based experiments, pp. 1019–1030, September 2017

    Google Scholar 

  28. Wątróbski, J., Jankowski, J., Karczmarczyk, A., Ziemba, P.: Integration of eye-tracking based studies into e-commerce websites evaluation process with eQual and TOPSIS methods. In: Wrycza, S., Maślankowski, J. (eds.) SIGSAND/PLAIS 2017. LNBIP, vol. 300, pp. 56–80. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66996-0_5

    Chapter  Google Scholar 

  29. Saaty, T.L.: What is the analytic hierarchy process? In: Mitra, G., Greenberg, H.J., Lootsma, F.A., Rijkaert, M.J., Zimmermann, H.J. (eds.) Mathematical Models for Decision Support. NATO ASI Series, vol. 48, pp. 109–121. Springer, Heidelberg (1988). https://doi.org/10.1007/978-3-642-83555-1_5

    Chapter  Google Scholar 

  30. McSpadden, K.: You now have a shorter attention span than a goldfish. Time Online Mag. Accessed 7 May 2015 (2016)

    Google Scholar 

  31. Weatherhead, R.: Say it quick, say it well-the attention span of a modern internet consumer. The Guardian Online 19 (2012). https://www.theguardian.com/media-network/media-network-blog/2012/mar/19/attention-span-internet-consumer

  32. Jankowski, J., Kazienko, P., Wątróbski, J., Lewandowska, A., Ziemba, P., Zioło, M.: Fuzzy multi-objective modeling of effectiveness and user experience in online advertising. Expert Syst. Appl. 65, 315–331 (2016)

    Article  Google Scholar 

  33. Nielsen, J.: Usability Engineering. Elsevier, Amsterdam (1994)

    MATH  Google Scholar 

  34. International Organization for Standardization: ISO 9241–11: Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs): Part 11: Guidance on Usability (1998)

    Google Scholar 

  35. ISO/IEC 25010: 2011. Systems and software engineering-Systems and software Quality Requirements and Evaluation (SQuaRE)-System and software quality models (2011)

    Google Scholar 

  36. Hassenzahl, M.: User experience (UX): towards an experiential perspective on product quality. In: Proceedings of the 20th Conference on l’Interaction Homme-Machine, pp. 11–15. ACM (2008)

    Google Scholar 

  37. Jankowski, J., Wątróbski, J., Kolomvatsos, K., Kazienko, P.: Fuzzy modeling of user behaviors and virtual goods purchases in social networking platforms. JUCS - J. Univers. Comput. Sci. 22(3), 416–437 (2016)

    MathSciNet  Google Scholar 

  38. Fernandez, A., Insfran, E., Abrahão, S.: Usability evaluation methods for the web: a systematic mapping study. Inf. Softw. Technol. 53(8), 789–817 (2011)

    Article  Google Scholar 

  39. Boulding, W., Kalra, A., Staelin, R., Zeithaml, V.A.: A dynamic process model of service quality: from expectations to behavioral intentions. J. Mark. Res. 30(1), 7 (1993)

    Article  Google Scholar 

  40. Barnes, S.J., Vidgen, R.: An evaluation of cyber-bookshops: the webqual method. Int. J. Electron. Commer. 6(1), 11–30 (2001)

    Article  Google Scholar 

  41. Webb, H., Webb, L.: Business to consumer electronic commerce website quality: integrating information and service dimensions. In: AMCIS 2001 Proceedings, p. 111 (2001)

    Google Scholar 

  42. La Porte, T.M., Demchak, C.C., Friis, C.: Webbing governance: global trends across national-level public agencies. Commun. ACM 44(1), 63–67 (2001)

    Article  Google Scholar 

  43. Elling, S., Lentz, L., de Jong, M.: Website evaluation questionnaire: development of a research-based tool for evaluating informational websites. In: Wimmer, M.A., Scholl, J., Grönlund, Å. (eds.) EGOV 2007. LNCS, vol. 4656, pp. 293–304. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74444-3_25

    Chapter  Google Scholar 

  44. Ping Zhang, G.M.: User expectations and rankings of quality factors in different web site domains. Int. J. Electron. Commer. 6(2), 9–33 (2001)

    Article  Google Scholar 

  45. Roth, S.P., Tuch, A.N., Mekler, E.D., Bargas-Avila, J.A., Opwis, K.: Location matters, especially for non-salient features-an eye-tracking study on the effects of web object placement on different types of websites. Int. J. Hum.-Comput. Stud. 71(3), 228–235 (2013)

    Article  Google Scholar 

  46. Hu, L., Zhang, W., Xu, Q.: The determinants of online payment method choice: insight from an eye-tracking study. In: WHICEB, p. 80 (2013)

    Google Scholar 

  47. Menon, R.V., Sigurdsson, V., Larsen, N.M., Fagerstrøm, A., Foxall, G.R.: Consumer attention to price in social commerce: eye tracking patterns in retail clothing. J. Bus. Res. 69(11), 5008–5013 (2016)

    Article  Google Scholar 

  48. Hernández-Méndez, J., Muñoz-Leiva, F.: What type of online advertising is most effective for etourism 2.0? an eye tracking study based on the characteristics of tourists. Comput. Hum. Behav. 50, 618–625 (2015)

    Article  Google Scholar 

  49. Wang, Q., Yang, S., Liu, M., Cao, Z., Ma, Q.: An eye-tracking study of website complexity from cognitive load perspective. Decis. Support Syst. 62, 1–10 (2014)

    Article  Google Scholar 

  50. Khodambashi, S., Gilstad, H., Nytrø, Ø.: Usability evaluation of clinical guidelines on the web using eye-tracker. Stud. Health Technol. Inform. 228, 95 (2016)

    Google Scholar 

  51. Štrach, P., Slivkin, N.: Adaptation needed: eye-tracking study of cross-cultural differences in perception of B2B websites (2017)

    Google Scholar 

  52. Yuan, X., Guo, M., Ren, F., Peng, F.: Usability analysis of online bank login interface based on eye tracking experiment. Sens. Transducers 165(2), 203 (2014)

    Google Scholar 

  53. Albayrak, D., Cagiltay, K.: Analyzing Turkish e-government websites by eye tracking. In: 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 2013 Eighth International Conference on Software Process and Product Measurement (IWSM-MENSURA), pp. 225–230. IEEE (2013)

    Google Scholar 

  54. Pan, B., Zhang, L.: An eyetracking study on online hotel decision making: the effects of images and umber of options (2016)

    Google Scholar 

  55. Kim, S., Stoel, L.: Dimensional hierarchy of retail website quality. Inf. Manag. 41(5), 619–633 (2004)

    Article  Google Scholar 

  56. Chmielarz, W., Zborowski, M.: Comparative analysis of electronic banking websites in selected banks in Poland in 2014. In: 2015 Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 1499–1504. IEEE (2015)

    Google Scholar 

  57. Chmielarz, W.: Evaluation of selected mobile applications stores from the user’s perspective. Online J. Appl. Knowl. Manag. 3(1), 21–36 (2015)

    Google Scholar 

  58. Chmielarz, W.: Methods of comparative analysis of electronic bankings’ websites. case of Poland. In: Chroust, G., Kotsis, G., Risak, V., Rozsenich, N., Zinterhof, P. (eds.) 1-st CEE Symposium on Business Informatics, Osterreichische Computer Gesellschaft, Vienna, pp. 73–84. Citeseer (2009)

    Google Scholar 

  59. Zenebe, A., Zhou, L., Norcio, A.F.: User preferences discovery using fuzzy models. Fuzzy Sets Syst. 161(23), 3044–3063 (2010)

    Article  MathSciNet  Google Scholar 

  60. Del Vasto-Terrientes, L., Valls, A., Slowinski, R., Zielniewicz, P.: ELECTRE-III-H: an outranking-based decision aiding method for hierarchically structured criteria. Expert Syst. Appl. 42(11), 4910–4926 (2015)

    Article  Google Scholar 

  61. Kaya, T.: Multi-attribute evaluation of website quality in e-business using an integrated fuzzy ahptopsis methodology. Int. J. Comput. Intell. Syst. 3(3), 301–314 (2010)

    Article  MathSciNet  Google Scholar 

  62. Huang, J., Jiang, X., Tang, Q.: An e-commerce performance assessment model: its development and an initial test on e-commerce applications in the retail sector of china. Inf. Manag. 46(2), 100–108 (2009)

    Article  Google Scholar 

  63. Guitouni, A., Martel, J.M., Vincke, P., North, P.: A framework to choose a discrete multicriterion aggregation procedure. Defence Research Establishment Valcatier (DREV) (1998)

    Google Scholar 

  64. Konys, A., Wątróbski, J., Różewski, P.: Approach to practical ontology design for supporting COTS component selection processes. In: Selamat, A., Nguyen, N.T., Haron, H. (eds.) ACIIDS 2013. LNCS (LNAI), vol. 7803, pp. 245–255. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36543-0_26

    Chapter  Google Scholar 

  65. Chmielarz, W., Zborowski, M.: Aspects of mobility in e-marketing from the perspective of a customer. In: 2016 Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 1329–1333. IEEE (2016)

    Google Scholar 

  66. Chmielarz, W., Zborowski, M.: Comparative analysis of electronic banking websites in Poland in 2014 and 2015. In: Ziemba, E. (ed.) Information Technology for Management. LNBIP, vol. 243, pp. 147–161. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30528-8_9

    Chapter  Google Scholar 

  67. Chmielarz, W., Zborowski, M.: Comparative analysis of e-banking services in Poland in 2016. In: Wrycza, S., Maślankowski, J. (eds.) SIGSAND/PLAIS 2017. LNBIP, vol. 300, pp. 43–55. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66996-0_4

    Chapter  Google Scholar 

  68. Wątróbski, J., Jankowski, J., Ziemba, P.: Multistage performance modelling in digital marketing management. Econ. Sociol. 9(2), 101 (2016)

    Article  Google Scholar 

  69. Wątróbski, J., Ziemba, P., Jankowski, J., Zioło, M.: Green energy for a green city—a multi-perspective model approach. Sustainability 8(8), 702 (2016)

    Article  Google Scholar 

  70. Ziemba, P., Wątróbski, J., Zioło, M., Karczmarczyk, A.: Using the PROSA method in offshore wind farm location problems. Energies 10(11), 1755 (2017)

    Article  Google Scholar 

  71. Verly, C., Smet, Y.D.: Some results about rank reversal instances in the PROMETHEE methods. Int. J. Multicriteria Decis. Mak. 3(4), 325 (2013)

    Article  Google Scholar 

  72. Sałabun, W.: The characteristic objects method: a new distance-based approach to multicriteria decision-making problems: the comet: a new distance-based approach to MCDM problems. J. Multi-Criteria Decis. Anal. 22(1–2), 37–50 (2015)

    Article  Google Scholar 

  73. Saaty, T.L.: How to make a decision: the analytic hierarchy process. Eur. J. Oper. Res. 48(1), 9–26 (1990)

    Article  MATH  Google Scholar 

  74. Saaty, T.L.: Decision making with the analytic hierarchy process. Int. J. Ser. Sci. 1(1), 83–98 (2008)

    Google Scholar 

  75. Macharis, C., Springael, J., De Brucker, K., Verbeke, A.: PROMETHEE and AHP: the design of operational synergies in multicriteria analysis.: strengthening PROMETHEE with ideas of AHP. Eur. J. Oper. Res. 153(2), 307–317 (2004)

    Article  MATH  Google Scholar 

  76. Sałabun, W., Piegat, A.: Comparative analysis of MCDM methods for the assessment of mortality in patients with acute coronary syndrome. Artif. Intell. Rev. 48(4), 1–15 (2017)

    Google Scholar 

  77. Faizi, S., Sałabun, W., Rashid, T., Wątróbski, J., Zafar, S.: Group decision-making for hesitant fuzzy sets based on characteristic objects method. Symmetry 9(8), 136 (2017)

    Article  MathSciNet  Google Scholar 

  78. Bashir, Z., Wątróbski, J., Rashid, T., Sałabun, W., Ali, J.: Intuitionistic-fuzzy goals in zero-sum multi criteria matrix games. Symmetry 9(8), 158 (2017)

    Article  MathSciNet  Google Scholar 

  79. Faizi, S., Rashid, T., Sałabun, W., Zafar, S., Wątróbski, J.: Decision making with uncertainty using hesitant fuzzy sets. Int. J. Fuzzy Syst. 3(16), 1–11 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Szczecin, ul. Mickiewicza 64, 71-101, Szczecin, Poland

    Jarosław Wątróbski & Waldemar Wolski

  2. Department of Computer Science, West Pomeranian University of Technology in Szczecin, ul. Żołnierska 49, 71-210, Szczecin, Poland

    Artur Karczmarczyk & Jarosław Jankowski

  3. Department of Technology, The Jacob of Paradies University, ul. Teatralna 25, 66-400, Gorzów Wielkopolski, Poland

    Paweł Ziemba

Authors
  1. Jarosław Wątróbski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Artur Karczmarczyk
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jarosław Jankowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Paweł Ziemba
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Waldemar Wolski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jarosław Wątróbski .

Editor information

Editors and Affiliations

  1. University of Economics in Katowice, Katowice, Poland

    Ewa Ziemba

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wątróbski, J., Karczmarczyk, A., Jankowski, J., Ziemba, P., Wolski, W. (2018). Hierarchical Representation of Website Evaluation Model Using Survey and Perceptual Based Criteria. In: Ziemba, E. (eds) Information Technology for Management. Ongoing Research and Development. ISM AITM 2017 2017. Lecture Notes in Business Information Processing, vol 311. Springer, Cham. https://doi.org/10.1007/978-3-319-77721-4_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-77721-4_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-77720-7

  • Online ISBN: 978-3-319-77721-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation