Application of a hybrid Delphi and aggregation–disaggregation procedure for group decision-making

  • Andrej BregarEmail author
Original Article


A hybrid procedure for group multiple criteria decision analysis is introduced that consolidates a moderated Delphi process with an autonomous aggregation–disaggregation mechanism. The research is based on the assumption that both consolidated approaches can lead to synergistic effects when properly combined. The paper justifies the hybrid procedure by comparing the three approaches, and by assessing it with a preliminary case study that is based on several factors of a universal framework for the evaluation of group decision-making methods and systems. These factors address convergence of the decision-making process, conflict resolution and cognitive complexity. The paper aims to define the decision-making process completely and thoroughly, fully operationalize all steps of the hybrid procedure algorithmically and methodologically and provide an application of the procedure. The application shows that the process converges and imposes acceptable cognitive load on the decision-makers, which implies that the hybrid procedure is able to perform efficiently.


Multiple criteria decision analysis Group decision-making processes Delphi procedures Aggregationdisaggregation of preferences 

Mathematics Subject Classification

91B06 decision theory 



  1. Ben Khélifa S, Martel JM (2001) A distance-based collective weak ordering. Group Decis Negot 10(4):317–329. CrossRefGoogle Scholar
  2. Beuthe M, Scannella G (2001) Comparative analysis of UTA multicriteria methods. Eur J Oper Res 130(2):246–262. CrossRefGoogle Scholar
  3. Bolger F, Wright G (2011) Improving the Delphi process: lessons from social psychological research. Technol Forecast Soc Chang 78(9):1500–1513. CrossRefGoogle Scholar
  4. Bregar A (2013) Convergence of autonomous group decision-making procedures: application to ranking and sorting. In: Zadnik Stirn L et al. (eds) Proceedings of the 12th International symposium on operations research in Slovenia. Slovenian Society Informatika, Ljubljana, pp 189–194Google Scholar
  5. Bregar A (2014) Towards a framework for the measurement and reduction of user-perceivable complexity of group decision-making methods. Int J Decision Support Syst Technol 6(2):21–45. CrossRefGoogle Scholar
  6. Bregar A, Györkös J, Juric MB (2008) Interactive aggregation/disaggregation dichotomic sorting procedure for group decision analysis based on the threshold model. Informatica 19(2):161–190Google Scholar
  7. Bregar A, Györkös J, Juric MB (2009) Robustness and visualization of decision models. Informatica 33(3):385–395Google Scholar
  8. Cai FL, Liao X, Wang KL (2012) An interactive sorting approach based on the assignment examples of multiple decision-makers with different priorities. Ann Oper Res 197(1):87–108. CrossRefGoogle Scholar
  9. Cook WD, Kress M, Seiford LM (1997) A general framework for distance-based consensus in ordinal ranking models. Eur J Oper Res 96(2):392–397. CrossRefGoogle Scholar
  10. Dalkey NC (1967) Delphi. The RAND corporation, Santa Monica. Accessed 21 Dec 2017
  11. Damart S, Dias LC, Mousseau V (2007) Supporting groups in sorting decisions: methodology and use of a multi-criteria aggregation/disaggregation DSS. Decis Support Syst 43(4):1464–1475. CrossRefGoogle Scholar
  12. Dias L, Clímaco J (2000) ELECTRE TRI for groups with imprecise information on parameter values. Group Decis Negot 9(5):355–377. CrossRefGoogle Scholar
  13. Dias LC, Mousseau V, Figueira J, Clímaco J (2002) An aggregation/disaggregation approach to obtain robust conclusions with ELECTRE TRI. Eur J Oper Res 138(2):332–348. CrossRefGoogle Scholar
  14. Doumpos M, Zopounidis C (2004) Developing sorting models using preference disaggregation analysis: an experimental investigation. Eur J Oper Res 154(3):585–598. CrossRefGoogle Scholar
  15. Govindan K, Jepsen MB (2016) ELECTRE: a comprehensive literature review on methodologies and applications. Eur J Oper Res 250(1):1–29. CrossRefGoogle Scholar
  16. Guo H, Wang X, Wang L, Chen D (2016) Delphi method for estimating membership function of uncertain set. J Uncertain Anal Appl 4(3):1–17. CrossRefGoogle Scholar
  17. Helmer-Hirschberg O (1967) Analysis of the future: the Delphi method. The RAND corporation, Santa Monica. Accessed 21 Dec 2017
  18. Hodgkin J, Belton V, Koulouri A (2005) Supporting the intelligent MCDA user: a case study in multi-person multi-criteria decision support. Eur J Oper Res 160(1):172–189. CrossRefGoogle Scholar
  19. Hsu CC, Sandford BA (2007) The Delphi technique: making sense of consensus. Practical Assessment Research & Evaluation 12(10): 1–8.
  20. Ishikawa A, Amagasa M, Shiga T, Tomizawa G, Tatsuta R, Mieno H (1993) The max-min Delphi method and fuzzy Delphi method via fuzzy integration. Fuzzy Sets Syst 55(3):241–253. CrossRefGoogle Scholar
  21. Jacquet-Lagrèze E, Siskos Y (2001) Preference disaggregation: 20 years of MCDA experience. Eur J Oper Res 130(2):233–245. CrossRefGoogle Scholar
  22. Karabasevic D, Stanujkic D, Urosevic S, Popovic G, Maksimovic M (2017) An approach to criteria weights determination by integrating the DELPHI and the adapted SWARA methods. Manag J Sustain Bus Manag Solut Emerg Econ 22(3):15–25.
  23. Keeney RL, Raiffa H (1976) Decisions with multiple objectives: preferences and value tradeoffs. Willey, New YorkGoogle Scholar
  24. Lee G, Jun KS, Chung ES (2013) Integrated multi-criteria flood vulnerability approach using fuzzy TOPSIS and Delphi technique. Nat Hazards Earth Syst Sci 13:1293–1312. CrossRefGoogle Scholar
  25. Linstone HA, Turoff M (2002) The Delphi method: techniques and applications. New Jersey Institute of Technology, NewarkGoogle Scholar
  26. Matsatsinis NF, Samaras AP (2001) MCDA and preference disaggregation in group decision support systems. Eur J Oper Res 130(2):414–429. CrossRefGoogle Scholar
  27. Matsatsinis NF, Grigoroudis E, Samaras AP (2005) Aggregation and disaggregation of preferences for collective decision-making. Group Decision Negot 14(3):217–232. CrossRefGoogle Scholar
  28. Mousseau V, Dias L (2003) Valued outranking relations in ELECTRE providing manageable disaggregation procedures. Eur J Oper Res 156(2):467–482. CrossRefGoogle Scholar
  29. Mousseau V, Slowinski R, Zielniewicz P (2000) A user-oriented implementation of ELECTRE TRI method integrating preference elicitation support. Comput Oper Res 27(7–8):757–777. CrossRefGoogle Scholar
  30. Ngo The A, Mousseau V (2002) Using assignment examples to infer category limits for the ELECTRE TRI method. J Multi Criteria Decision Anal 11(1):29–43. CrossRefGoogle Scholar
  31. Okoli C, Pawlowski SD (2004) The Delphi method as a research tool: an example, design considerations and applications. Inf Manag 42(1):15–29. CrossRefGoogle Scholar
  32. Rowe G, Wright G (2001) Expert opinions in forecasting: the role of the Delphi technique. In: Armstrong JS (ed) Principles of forecasting. International Series in operations research & management science, vol 30. Springer, Boston, pp 125–144Google Scholar
  33. Roy B (1996) Multicriteria methodology for decision aiding. Kluwer Academic Publishers, DordrechtCrossRefGoogle Scholar
  34. Sarang P, Jennings F, Juric MB, Loganathan R (2007) SOA approach to integration: XML, web services, ESB, and BPEL in real-world SOA projects. Packt Publishing, BirminghamGoogle Scholar
  35. Siskos Y, Spyridakos A (1999) Intelligent multicriteria decision support: overview and perspectives. Eur J Oper Res 113(2):236–246. CrossRefGoogle Scholar
  36. Siskos Y, Grigoroudis E, Matsatsinis NF (2016) UTA methods. In: Greco S, Ehrgott M, Figueira J (eds) Multiple criteria decision analysis. International series in operations research & management science, vol 233. Springer, New York, pp 315–362Google Scholar
  37. Spyridakos A, Yannacopoulos D (2015) Incorporating collective functions to multicriteria disaggregation–aggregation approaches for small group decision making. Ann Oper Res 227(1):119–136. CrossRefGoogle Scholar
  38. Turban E, Aronson JE, Liang TP (2004) Decision support systems and intelligent systems, 7th edn. Prentice-Hall, Upper Saddle RiverGoogle Scholar
  39. Turoff M, Hiltz SR (1996) Computer based Delphi processes. In: Adler M, Ziglio E (eds) Gazing into the Oracle: the Delphi method and its application to social policy and public health. Kingsley Publishers, London, pp 56–85Google Scholar
  40. Tzeng GH, Huang CY (2012) Combined DEMATEL technique with hybrid MCDM methods for creating the aspired intelligent global manufacturing & logistics systems. Ann Oper Res 197(1):159–190. CrossRefGoogle Scholar
  41. Yousuf MI (2007) Using experts’ opinions through Delphi technique. Pract Assess Res Eval 12(4):1–8.
  42. Zopounidis C, Doumpos M (2002) Multicriteria classification and sorting methods: a literature review. Eur J Oper Res 138(2):229–246. CrossRefGoogle Scholar

Copyright information

© EURO - The Association of European Operational Research Societies and Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Informatika d.d.MariborSlovenia

Personalised recommendations