Journal of Intelligent Information Systems

, Volume 46, Issue 2, pp 313–346 | Cite as

A hierarchical multi-criteria sorting approach for recommender systems

  • Luis Del Vasto-TerrientesEmail author
  • Aida Valls
  • Piotr Zielniewicz
  • Joan Borràs


Classification problems refer to the assignment of alternatives to predefined categories. In this work we focus on ordered classification, called sorting, in which the predefined categories indicate several degrees of interest or suitability of alternatives for a certain user. The assignment of alternatives is based on multiple conflicting criteria. This multi-criteria sorting approach is specially interesting for recommender systems aimed at finding the most suitable alternatives for each user. First, we study the ELECTRE-TRI-B sorting method, which follows the outranking approach based on comparing the evaluations of alternatives with the profile limits separating the categories. The complexity of some recommenders systems requires the extension of the classical ELECTRE-TRI-B method to manage a taxonomical organization of the set of criteria. In this paper we consider a set of criteria in the form of a hierarchy. The intermediate criteria in such a hierarchy correspond to different aspects of the recommendation procedure, such as content, context or cost. At each of these criteria, a sorting problem must be solved. Therefore, we propose extending ELECTRE-TRI-B to handle assignments of alternatives on several levels of the hierarchy. A hierarchical procedure for sorting is proposed, called ELECTRE-TRI-B-H. Secondly, the paper explains the integration of ELECTRE-TRI-B-H into a recommender system of touristic activities related to wine, called GoEno-Tur. This system is developed for the region of Tarragona, Catalonia (Spain), which is a well-recognized area of wine and cava production.


Recommender systems Classification Sorting Hierarchy of criteria Decision support systems 



This project has been funded by the Spanish research project SHADE (TIN-2012-34369: Semantic and Hierarchical Attributes in Decision Making). Luis Del Vasto-Terrientes is supported by a FI predoctoral grant from Generalitat de Catalunya (2014 FI_B200023). Joan Borràs is affiliated to the Science and Technology Park for Tourism and Leisure (PCT) in Vila-Seca, Catalonia (Spain). The authors have no conflicts of interest to declare.


  1. Adomavicius, G., Manouselis, N., & Kwon, Y. (2011). Multi-Criteria Recommender Systems. In Ricci, F., Rokach, L., Shapira, B, & Kantor, P. B. (Eds.), Recommender Systems Handbook, (pp. 769–803). US: Springer.CrossRefGoogle Scholar
  2. Arondel, C., & Girardin, P. (2000). Sorting cropping systems on the basis of their impact on groundwater quality. European Journal of Operational Research, 127, 467–482.CrossRefzbMATHGoogle Scholar
  3. Bana e Costa, C., & Vansnick, J. (1999). The MACBETH Approach: Basic Ideas, Software, and an Application. In Meskens, N., & Roubens, M. (Eds.), Advances in Decision Analysis, Mathematical Modelling: Theory and Applications, vol 4, (pp. 131–157). Netherlands: Springer.CrossRefGoogle Scholar
  4. Borràs, J., Moreno, A., Valls, A., Ferré, M., Ciurana, E., Salvat, J., Russo, A., & Anton-Clavé, S. (2012a). Uso de técnicas de inteligencia artificial para hacer recomendaciones enoturística personalizadas en la provincia de tarragona. In IX Congreso Nacional de Turismo y Tecnologías de la Información y las Comunicaciones (TURITEC) (pp. 217–230).Google Scholar
  5. Borràs, J., Valls, A., Moreno, A., & Isern, D. (2012b). Ontology-based management of uncertain preferences in user profiles. In 14th International Conference on Information Processing and Management of Uncertainty in Knowledge-based Systems (IPMU) (pp. 127–136).Google Scholar
  6. Borràs, J., Moreno, A., & Valls, A. (2014). Intelligent tourism recommender systems: A survey. Expert Systems with Applications, 41, 7370–7389.CrossRefGoogle Scholar
  7. Brito, A., de Almeida, A., & Mota, C. (2010). A multicriteria model for risk sorting of natural gas pipelines based on ELECTRE TRI integrating Utility Theory. European Journal of Operational Research, 200, 812–821.CrossRefzbMATHGoogle Scholar
  8. Cailloux, O., Meyer, P., & Mousseau, V. (2012). Eliciting Electre Tri category limits for a group of decision makers. European Journal of Operational Research, 223, 133–140.MathSciNetCrossRefzbMATHGoogle Scholar
  9. Cailloux, O., Mayag, B., Meyer, P., & Mousseau, V. (2013). Operational tools to build a multicriteria territorial risk scale with multiple stakeholders. Reliability Engineering & System Safety, 120, 88–97.CrossRefGoogle Scholar
  10. Chen, S., Liu, J., Wang, H., Xu, Y., & Augusto, J. (2014). A linguistic multi-criteria decision making approach based on logical reasoning. Information Sciences, 258, 266–276.MathSciNetCrossRefzbMATHGoogle Scholar
  11. Cloquell-Ballester, V., Monterde-Díaz, R., Cloquell-Ballester, V., & Santamarina-Siurana, M. (2007). Systematic comparative and sensitivity analyses of additive and outranking techniques for supporting impact significance assessments. Environmental Impact Assessment Review, 27, 62–83.CrossRefGoogle Scholar
  12. Corrente, S., Greco, S., Kadzinski, M., & Slowinski, R. (2013). Robust ordinal regression in preference learning and ranking. Machine Learning, 93(2–3), 381–422.MathSciNetCrossRefzbMATHGoogle Scholar
  13. De Gemmis, M., Iaquinta, L., Lops, P., Musto, C., Narducci, F., & Semeraro, G. (2009). Preference learning in recommender systems. In Preference Learning (PL-09) ECML/PKDD-09 Workshop.Google Scholar
  14. Deuk Hee, P., Hyea Kyeong, K., Il Young, C., & Jae Kyeong, K. (2012). A literature review and classification of recommender systems research. Expert Systems with Applications, 39(11), 10,059–10,072.CrossRefGoogle Scholar
  15. Doumpos, M., & Grigoroundis, E. (2013). Multicriteria Decision Aid and Artificial Intelligence. Wiley.Google Scholar
  16. Doumpos, M., & Zopounidis, C. (2011). Preference disaggregation and statistical learning for multicriteria decision support: A review. European Journal of Operational Research, 209(3), 203–214.MathSciNetCrossRefzbMATHGoogle Scholar
  17. Ehrgott, M., Figueira, J., & Greco, S. (2010). Trends in Multiple Criteria Decision Analysis. Berlin: Springer.CrossRefzbMATHGoogle Scholar
  18. Fenza, G., Fischetti, E., Fumo, D., & Loia, V. (2011). A hybrid context aware system for tourist guidance based on collaborative filtering. In Fuzzy Systems (FUZZ), 2011 IEEE International Conference (pp. 131–138).Google Scholar
  19. Figueira, J., & Roy, B. (2002). Determining the weights of criteria in the electre type methods with a revised simos’ procedure. European Journal of Operational Research, 139(2), 317–326.MathSciNetCrossRefzbMATHGoogle Scholar
  20. Gavalas, D., Konstantopoulos, C., Mastakas, K., & Pantziou, G. (2014). Mobile recommender systems in tourism. Journal of Network and Computer Applications, 39, 319–333.CrossRefGoogle Scholar
  21. Grabisch, M., Kojadinovic, I., & Meyer, P. (2008). A review of methods for capacity identification in Choquet integral based multi-attribute utility theory: Applications of the Kappalab R package. European Journal of Operational Research, 186(2), 766–785.MathSciNetCrossRefzbMATHGoogle Scholar
  22. Hdioud, F., Frikh B., & Ouhbi, B. (2013). Multi-criteria recommender systems based on multi-attribute decision making. In Proceedings of International Conference on Information Integration and Web-based Applications & Services, ACM, New York, NY, USA, IIWAS ’13 (pp. 203:203–203:210).Google Scholar
  23. Huang, S. (2011). Designing utility-based recommender systems for e-commerce: Evaluation of preference-elicitation methods. Electronic Commerce Research and Applications, 10, 398–407.CrossRefGoogle Scholar
  24. Lakiotaki, K., Matsatsinis, N., & Tsoukias, A. (2011). Multicriteria User Modeling in Recommender Systems. IEEE Intelligent Systems, 26(2), 64–76.CrossRefGoogle Scholar
  25. Law, R., Qi, S., & Buhalis, D. (2010). Progress in tourism management: A review of website evaluation in tourism research. Tourism Management, 31(3), 297–313.CrossRefGoogle Scholar
  26. Li, D., Liu, C., & Gan, W. (2009). A new cognitive model: Cloud model. International Journal of Intelligent Systems, 24, 357–375.CrossRefzbMATHGoogle Scholar
  27. Liu, L., Mehandjiev, N., & Xu, D. (2011). Multi-criteria Service Recommendation Based on User Criteria Preferences. In Proceedings of the Fifth ACM Conference on Recommender Systems, ACM, New York, NY, USA, RecSys ’11 (pp. 77–84).Google Scholar
  28. Lops, P., De Gemmis, M., Semeraro, G., Musto, C., & Narducci, F. (2013). Content-based and collaborative techniques for tag recommendation: An empirical evaluation. Journal of Intelligent Information Systems, 40, 41–61.CrossRefGoogle Scholar
  29. Mandl, M., Felfernig, A., Teppan, E., & Schubert, M. (2011). Consumer decision making in knowledge-based recommendation. Journal of Intelligent Information Systems, 37, 1–22.CrossRefGoogle Scholar
  30. Manjeevan, S., Chee Peng, L., Wei Shiung, L., Einly, L., & Chu Kiong, L. (2015). Classification of electrocardiogram and auscultatory blood pressure signals using machine learning models. Expert Systems with Applications, 42(7), 3643–3652.CrossRefGoogle Scholar
  31. Matsatsinis, N., Doumpos, M., & Zopounidis, C. (1997). Knowledge acquisition and representation for expert systems in the field of financial analysis. Expert Systems with Applications, 12, 247–262.CrossRefGoogle Scholar
  32. Mikeli, A., Sotiros, D., Apostolou, D., & Despotis, D. (2013). A multi-criteria recommender system incorporating intensity of preferences. In Information, Intelligence, Systems and Applications (IISA), 2013 Fourth International Conference on (pp. 1–6).Google Scholar
  33. Moreno, A., Valls, A., Isern, D., Marin, L., & Borràs, J. (2013). SigTur/E-Destination: Ontology-based personalized recommendation of Tourism and Leisure Activities. Engineering Applications of Artificial Intelligence, 26, 633–651.CrossRefGoogle Scholar
  34. Moreno, A., Valls, A., Martínez, S., Vicient, C., Marin, L., & Mata, F. (2015). Personalised recommendations based on novel semantic similarity and clustering procedures. AI Commun, 28(1), 127–142.MathSciNetGoogle Scholar
  35. Mousseau, V., Slowinski, R., & Zielniewicz, P. (2000). A user-oriented implementation of the ELECTRE-TRI method integrating preference elicitation support. Computers and Operations Research, 27, 757–777.Google Scholar
  36. Mustajoki, J. (2012). Effects of imprecise weighting in hierarchical preference programming. European Journal of Operational Research, 218, 193–201.MathSciNetCrossRefzbMATHGoogle Scholar
  37. Perny, P., & Pomerol, J. (1999). Use of artificial intelligence in mcdm. In Gal, T., Stewart, T., & Hanne, T. (Eds.) Multicriteria Decision Making: Advances in MCMD Models, Algorithms, Theory, and Applications, vol (pp. 1–43).Google Scholar
  38. Qiang, W.J., Peng, L., Yu Z.H., & Hong, C.X. (2014). Method of multi-criteria group decision-making based on cloud aggregation operators with linguistic information. Information Sciences, 1, 177–191.Google Scholar
  39. Roy, B. (1996). Multicriteria Methodology for Decision Analysis. Dordrecht: Kluwer Academic Publishers.CrossRefGoogle Scholar
  40. Saaty, R. (1987). The analytic hierarchy process-what it is and how it is used. Mathematical Modelling, 9(3-5), 161–176.MathSciNetCrossRefzbMATHGoogle Scholar
  41. Sánchez-Lozano, J., Henggeler Antunes, C., García-Cascales, M., & Dias, L. (2014). GIS-based photovoltaic solar farms site selection using ELECTRE-TRI: Evaluating the case for Torre Pacheco, Murcia, Southeast of Spain. Renewable Energy, 66, 478–494.CrossRefGoogle Scholar
  42. Silva, S., Alçada-Almeida, L., & Dias, L. (2014). Biogas plants site selection integrating Multicriteria Decision Aid methods and GIS techniques: A case study in a Portuguese region. Biomass and Bioenergy, 71, 58–68.CrossRefGoogle Scholar
  43. Siskos, Y., Grigoroudis, E., & Matsatsinis, N. (2005). Uta methods. Multiple Criteria Decision Analysis: State of the Art Surveys, International Series in Operations Research & Management Science, vol 78, (pp. 297–334). New York: Springer.Google Scholar
  44. Sobrie, O., Mousseau, V., & Pirlot, M. (2013). Learning a Majority Rule Model from Large Sets of Assignment Examples. In Perny, P., Pirlot, M., & Tsoukiàs, A. (Eds.), Algorithmic Decision Theory, Lecture Notes in Computer Science, vol 8176, (pp. 336–350). Berlin Heidelberg: Springer.Google Scholar
  45. Xidonas, P., Mavrotas, G., & Psarras, J. (2009). A multicriteria methodology for equity selection using financial analysis. Computers and Operations Research, 36, 3187–3203.CrossRefzbMATHGoogle Scholar
  46. Yu, W. (1992). Electre tri : Aspects mthodologiques et manuel dutilisation. Document du LAMSADE, No.74, Universit Paris-Dauphine.Google Scholar
  47. Zopounidis, C., & Doumpos, M. (2002). Multicriteria classification and sorting methods: A literature review. European Journal of Operational Research, 138, 229–246.MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Luis Del Vasto-Terrientes
    • 1
    Email author
  • Aida Valls
    • 1
  • Piotr Zielniewicz
    • 2
  • Joan Borràs
    • 3
  1. 1.Departament d’Enginyeria Informàtica i MatemàtiquesUniversitat Rovira i VirgiliTarragonaSpain
  2. 2.Institute of Computing SciencePoznan University of TechnologyPoznanPoland
  3. 3.Parc Científic Tecnològic de Turisme i OciVila-secaSpain

Personalised recommendations