MeasApplInt - a novel intelligence metric for choosing the computing systems able to solve real-life problems with a high intelligence

  • László Barna IantovicsEmail author
  • László Kovács
  • Corina Rotar


Intelligent agent-based systems are applied for many real-life difficult problem-solving tasks in domains like transport and healthcare. In the case of many classes of real-life difficult problems, it is important to make an efficient selection of the computing systems that are able to solve the problems very intelligently. The selection of the appropriate computing systems should be based on an intelligence metric that is able to measure the systems intelligence for real-life problem solving. In this paper, we propose a novel universal metric called MeasApplInt able to measure and compare the real-life problem solving machine intelligence of cooperative multiagent systems (CMASs). Based on their measured intelligence levels, two studied CMASs can be classified to the same or to different classes of intelligence. MeasApplInt is compared with a recent state-of-the-art metric called MetrIntPair. The comparison was based on the same principle of difficult problem-solving intelligence and the same pairwise/matched problem-solving intelligence evaluations. Our analysis shows that the main advantage of MeasApplInt versus the compared metric, is its robustness. For evaluation purposes, we performed an illustrative case study considering two CMASs composed of simple reactive agents providing problem-solving intelligence at the systems’ level. The two CMASs have been designed for solving an NP-hard problem with many applications in the standard, modified and generalized formulation. The conclusion of the case study, using the MeasApplInt metric, is that the studied CMASs have the same real-life problems solving intelligence level. An additional experimental evaluation of the proposed metric is attached as an Appendix.


Applied machine intelligence Computational-hard real-life problem Cooperative multiagent system Intelligent system Machine intelligence Machine intelligence measure Real-life problem-solving intelligence 



This work has been funded by the CHIST-ERA programme supported by the Future and Emerging Technologies (FET) programme of the European Union through the ERA-NET funding scheme under the grant agreements, title SOON - Social Network of Machines.

Supplementary material

10489_2019_1440_MOESM1_ESM.tex (17 kb)
(TEX 17.0 KB)


  1. 1.
    Anthon A, Jannett TC (2007) Measuring machine intelligence of an agent-based distributed sensor network system. In: Elleithy K (ed) Advances and innovations in systems, computing sciences and software engineering. Springer, pp 531–535Google Scholar
  2. 2.
    Arif M, Illahi M, Karim A, Shamshirband S, Alam KA, Farid S, Iqbal S, Buang Z, Balas VE (2015) An architecture of agent-based multi-layer interactive e-learning and e-testing platform. Qual Quant 49 (6):2435–2458Google Scholar
  3. 3.
    Arik S, Iantovics LB, Szilagyi SM (2017) OutIntSys - a novel method for the detection of the most intelligent cooperative multiagent systems. In: Liu D et al (eds) 24th International conference on neural information processing, Guangzhou, China, November 14-18. Neural Information Processing, LNCS, 10637:31–40Google Scholar
  4. 4.
    Barnett V, Lewis T (1994) Outliers in statistical data, 3rd edn. Wiley, New YorkzbMATHGoogle Scholar
  5. 5.
    Beni G, Wang J (1993) Swarm intelligence in cellular robotic systems. In: Dario P, Sandini G, Aebischer P (eds) Robots and biological systems: towards a new bionics? NATO ASI Series (Series F: Computer and Systems Sciences), vol 102. Springer, Berlin, pp 703–712Google Scholar
  6. 6.
    Bejar II, Whalen SJ (2003) Methods and systems for presentation and evaluation of constructed responses assessed by human evaluators, US Patent 6,526,258Google Scholar
  7. 7.
    Besold T, Hernandez-Orallo J, Schmid U (2015) Can machine intelligence be measured in the same way as human intelligence? Kunstl Intell 29(3):291–297Google Scholar
  8. 8.
    Boctor FF, Laporte G, Renaud J (2003) Heuristics for the traveling purchaser problem. Comput Oper Res 30:491–504zbMATHGoogle Scholar
  9. 9.
    Bonett DG, Wright TA (2000) Sample size requirements for estimating Pearson, Kendall and Spearman correlations. Psychometrika 65:23–28zbMATHGoogle Scholar
  10. 10.
    Box FJ (1987) Guinness, gosset, fisher, and small samples. Stat Sci 2(1):45–52MathSciNetzbMATHGoogle Scholar
  11. 11.
    Brady SG, Fisher BL, Schultz TR, Ward PS (2014) The rise of army ants and their relatives: diversification of specialized predatory doryline ants. BMC Evol Biol 14:2–14Google Scholar
  12. 12.
    Bullnheimer B, Hartl RF, Strauss C (1999) A new rank based version of the ant system. A computational study. CEJOR 7(1):25–38MathSciNetzbMATHGoogle Scholar
  13. 13.
    Chakraborty UK, Konar D, Roy S, Choudhury S (2016) Intelligent fuzzy spelling evaluator for e-Learning systems. Educ Inf Technol 21(1):171–184Google Scholar
  14. 14.
    Chakravarti IM, Laha RG, Roy J (1967) Handbook of methods of applied statistics, vol I. Wiley, New York, pp 392–394Google Scholar
  15. 15.
    Chliaoutakis A, Chalkiadakis G (2016) Agent-based modeling of ancient societies and their organization structure. Auton Agent Multi-Agent Syst 30(6):1072–1116Google Scholar
  16. 16.
    Coelho CGC, Abreu CG, Ramos RM, Mendes AHD, Teodoro G, Ralha CG (2016) MASE-BDI: Agent-based simulator for environmental land change with efficient and parallel auto-tuning. Appl Intell 45(3):904–922Google Scholar
  17. 17.
    Chmait N, Dowe DL, Green DG, Li YF, Insa-Cabrera J (2015) Measuring universal intelligence in agent-based systems using the anytime intelligence test. Technical Report, Monash University, Report Num, 2015/279Google Scholar
  18. 18.
    Chouhan SS, Niyogi R (2017) MAPJA: multi-agent planning with joint actions. Appl Intell 47(4):1044–1058Google Scholar
  19. 19.
    Colom R, Karama S, Jung RE, Haier RJ (2010) Human intelligence and brain networks. Dialogues Clin Neurosci 12(4):489–501Google Scholar
  20. 20.
    Colorni A, Dorigo M, Maniezzo V (1991) Distributed optimization by ant colonies. In: Actes de la première conférence européenne sur la vie artificielle. Paris, France, Elsevier Publishing, 134–142Google Scholar
  21. 21.
    Conley W (1988) Travelling salesman problem solved with simulation techniques. Int J Syst Sci 19(10):2115–2122MathSciNetzbMATHGoogle Scholar
  22. 22.
    Conley W (1989) Two truck travelling salesman simulation. Int J Syst Sci 20(12):2495–2514zbMATHGoogle Scholar
  23. 23.
    Conley W (1990) Multi-stage Monte Carlo optimization applied to a large travelling salesman problem. Int J Syst Sci 21(3):547–566zbMATHGoogle Scholar
  24. 24.
    Conover WJ (1973) On methods of handling ties in the wilcoxon signed-rank test. J Am Stat Assoc 68 (344):985–988MathSciNetzbMATHGoogle Scholar
  25. 25.
    Cordon O, Herrera F, de Viana IF, Moreno L (2000) A new ACO model integrating evolutionary computation concepts: The Best-Worst ant system. In: Proceedings of ANTS’2000. From ant colonies to artificial ants: second international workshop on ant algorithms, Brussels, Belgium, September 7–9, 22–29Google Scholar
  26. 26.
    Cordon O, de Viana IF, Herrera F (2002) Analysis of the best-worst ant system and its variants on the QAP. In: Dorigo M, Di Caro G, Sampels M (eds) Ant algorithms, vol 2463. Springer, LNCS, Berlin, Heidelberg, pp 228–234Google Scholar
  27. 27.
    Crisan GC, Pintea CM, Palade V (2017) Emergency management using geographic information systems: application to the first Romanian traveling salesman problem instance. Knowl Inf Syst 50(1):265–285Google Scholar
  28. 28.
    Dantzig GB, Ramser JH (1959) The truck dispatching problem. Manag Sci 6:80–91MathSciNetzbMATHGoogle Scholar
  29. 29.
    Dantzig G, Fulkerson D, Johnson S (1954) Solution of a large scale traveling salesman problem. Oper Res 2:393–410MathSciNetGoogle Scholar
  30. 30.
    David HA, Gunnink JL (1997) The paired t test under artificial pairing. Am Stat 51(1):9–12MathSciNetGoogle Scholar
  31. 31.
    Dorigo M (1992) Optimization, learning and natural algorithms. PhD thesis, Politecnico di Milano, ItalyGoogle Scholar
  32. 32.
    Dorigo M, Gambardella LM (1997) Ant colony system: a cooperative learning approach to the traveling salesman problem. IEEE Trans Evol Comput 1(1):53–66Google Scholar
  33. 33.
    Dowe DL, Hernández-Orallo J (2014) How universal can an intelligence test be? Adapt Behavior Animals Animats Softw Agents Robots Adapt Syst Arch 22(1):51–69Google Scholar
  34. 34.
    Everitt B (1998) The cambridge dictionary of statistics Cambridge. Cambridge University Press, New YorkzbMATHGoogle Scholar
  35. 35.
    Fay MP, Proschan MA (2010) Wilcoxon–mann–whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Stat Surveys 4:1–39MathSciNetzbMATHGoogle Scholar
  36. 36.
    Ferrucci D, Levas A, Bagchi S, Gondek D, Mueller ET (2013) Watson: beyond jeopardy! Artif Intell 199–200:93–105Google Scholar
  37. 37.
    Franklin D, Abrao A (2000) Measuring software agent’s intelligence. In: Proceedings of international conference: advances in infrastructure for electronical business science and education on the internet. L’Aquila, ItalyGoogle Scholar
  38. 38.
    Galton F (1886) Regression towards mediocrity in hereditary stature. J Anthropol Inst G B Irel 15:246–263Google Scholar
  39. 39.
    Grotschel M, Padberg MW (1978) On the symmetric travelling salesman problem: theory and computation. In: Henn R, Korte B, Oettli W (eds) Optimization and operations research. Lecture notes in economics and mathematical systems. vol 157, Springer, Berlin, pp 105–115Google Scholar
  40. 40.
    Hernandez-Orallo J, Dowe DL (2010) Measuring universal intelligence: towards an anytime intelligence test. Artif Intell 174(8):1508–1539MathSciNetGoogle Scholar
  41. 41.
    Hernández-Orallo J, Dowe DL, Hernández-Lloreda MV (2014) Universal psychometrics: measuring cognitive abilities in the machine kingdom. Cogn Syst Res 27:50–74Google Scholar
  42. 42.
    Hibbard B (2011) Measuring agent intelligence via hierarchies of environments. Artificial General Intelligence, Lecture Notes in Computer Science 6830:303–308Google Scholar
  43. 43.
    Hsieh FS (2017) A hybrid and scalable multi-agent approach for patient scheduling based on Petri net models. Appl Intell 7(4):1068–1086MathSciNetGoogle Scholar
  44. 44.
    Iantovics LB, Emmert-Streib F, Arik S (2017) Metrintmeas a novel metric for measuring the intelligence of a swarm of cooperating agents. Cogn Syst Res 45:17–29Google Scholar
  45. 45.
    Iantovics LB, Rotar C, Niazi AN (2018) Metrintpair-a novel accurate metric for the comparison of two cooperative multiagent systems intelligence based on paired intelligence measurements. Int J Intell Syst 33(3):463–486Google Scholar
  46. 46.
    Iantovics LB, Zamfirescu CB (2013) ERMS: an evolutionary reorganizing multiagent system, innovative computing. Inf Control 9(3):1171–1188Google Scholar
  47. 47.
    Iqbal S, Altaf W, Aslam M, Mahmood W, Khan MUG (2016) Application of intelligent agents in health-care: review. Artif Intell Rev 46(1):83–112Google Scholar
  48. 48.
    Johnson BR, Borowiec ML, Chiu JC, Lee EK, Atallah J, Ward PS (2013) Phylogenomics resolves evolutionary relationships among ants, bees, and wasps. Curr Biol 23(20):1–5Google Scholar
  49. 49.
    Jussila J, Vuori V, Okkonen J, Helander N (2017) Reliability and perceived value of sentiment analysis for twitter data. In: Kavoura A, Sakas D, Tomaras P (eds) Strategic innovative marketing. Springer proceedings in business and economics. Springer, Cham, pp 43–48Google Scholar
  50. 50.
    Kafali O, Yolum P (2016) PISAGOR: a proactive software agent for monitoring interactions. Knowl Inf Syst 47(1):215–239Google Scholar
  51. 51.
    Kwon H, Pack DJ (2012) A robust mobile target localization method for cooperative unmanned aerial vehicles using sensor fusion quality. J Intell Robot Syst 65(1):479–493Google Scholar
  52. 52.
    Leeuwen JV (ed) (1998) Handbook of theoretical computer science, vol A. Algorithms and complexity. Elsevier, AmsterdamGoogle Scholar
  53. 53.
    Lowry R Concepts & applications of inferential statistics.
  54. 54.
    Mann PS (1995) Introductory statistics, 2nd edn. Wiley, New YorkzbMATHGoogle Scholar
  55. 55.
    Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat 18(1):50–60MathSciNetzbMATHGoogle Scholar
  56. 56.
    Marusteri M, Bacarea V (2010) Comparing groups for statistical differences: how to choose the right statistical test? Biochemia Medica 20(1):15–32Google Scholar
  57. 57.
    Merkle D, Middendorf M (2005) On solving permutation scheduling problems with ant colony optimization. Int J Syst Sci 36(5):255–266MathSciNetzbMATHGoogle Scholar
  58. 58.
    Munteanu C, Rosa A (2004) Gray-scale image enhancement as an automatic process driven by evolution. IEEE Trans Syst Man Cybern B Cybern 34(2):1292–1298Google Scholar
  59. 59.
    Myers JL, Well AD (2003) Research design and statistical analysis, 2nd edn. Lawrence Erlbaum, Mahwah, p 508Google Scholar
  60. 60.
    Neisser U, Boodoo G, Bouchard TJ, Boykin AW, Brody N, Ceci SJ, Halpern DF, Loehlin JC, Perloff R, Sternberg RJ, Urbina S (1996) Intelligence: knowns and unknowns. Am Psychol 51(2):77–101Google Scholar
  61. 61.
    Newborn M (1997) Kasparov vs deep blue: computer chess comes of age. Springer, New YorkGoogle Scholar
  62. 62.
    Niazi M, Hussain A (2011) Agent-based computing from multi-agent systems to agent-based models: a visual survey. Scientometrics 89(2):479–499Google Scholar
  63. 63.
    Nick TG (2007) Descriptive statistics. Topics in biostatistics. Methods Mol Biol 404:33–52Google Scholar
  64. 64.
    Ouaarab A, Ahiod B, Yang XS (2014) Discrete cuckoo search algorithm for the travelling salesman problem. Neural Comput Appl 24:1659Google Scholar
  65. 65.
    Pearson K (1895) Notes on regression and inheritance in the case of two parents. Proc R Soc Lond 58:240–242Google Scholar
  66. 66.
    Pholdee N, Bureerat S (2016) Hybrid real-code ant colony optimisation for constrained mechanical design. Int J Syst Sci 47(2):474–491zbMATHGoogle Scholar
  67. 67.
    Prakasam A, Savarimuthu N (2016) Metaheuristic algorithms and probabilistic behaviour: a comprehensive analysis of ant colony optimization and its variants. Artif Intell Rev 45(1):97–130Google Scholar
  68. 68.
    Pratt JW (1959) Remarks on zeros and ties in the Wilcoxon signed rank procedures. J Am Stat Assoc 54 (287):655–667MathSciNetzbMATHGoogle Scholar
  69. 69.
    Pratt JW, Gibbons JD (1981) Concepts of nonparametric theory. Springer, New YorkzbMATHGoogle Scholar
  70. 70.
    Rosing MT (1999) 13C-depleted carbon microparticles in >3700-Ma sea-floor sedimentary rocks from West Greenland. Science 283(5402):674–676Google Scholar
  71. 71.
    Rouse WB, Sandra H (1983) Rouse analysis and classification of human error. IEEE Trans Syst Man Cybern SMC-13(4):539—549Google Scholar
  72. 72.
    Runkler TA (2005) Ant colony optimization of clustering models. Int J Int Syst 20:1233–1251zbMATHGoogle Scholar
  73. 73.
    Schreiner K (2000) Measuring IS: toward a US standard. IEEE Intell Syst Their Appl 15(5):19–21Google Scholar
  74. 74.
    Sanghi P, Dowe DL (2003) A computer program capable of passing I.Q. tests. In: Slezak PP (ed) Proceedings of the joint international conference on cognitive science, 4th ICCS international conference on cognitive science and 7th ASCS Australasian society for cognitive science (ICCS/ASCS 2003). Sydney, NSW, Australia, pp 570–575Google Scholar
  75. 75.
    Sharkey AJC (2006) Robots, insects and swarm intelligence. Artif Intell Rev 26(4):255–268Google Scholar
  76. 76.
    Saska M, Vonasek V, Krajnik T, Preucil L (2014) Coordination and navigation of heterogeneous MAV–UGV formations localized by a ‘hawk-eye’-like approach under a model predictive control scheme. Int J Robot Res 33(10):1393–1412Google Scholar
  77. 77.
    Shapiro SS, Wilk MB (1965) An analysis of variance test for normality. Biometrika 52(3-4):591–611MathSciNetzbMATHGoogle Scholar
  78. 78.
    Sharpanskykh A, Haest R (2016) An agent-based model to study compliance with safety regulations at an airline ground service organization. Appl Intell 45(3):881–903Google Scholar
  79. 79.
    Siegel S (1956) Non-parametric statistics for the behavioral sciences. McGraw-Hill, New York, pp 75–83Google Scholar
  80. 80.
    Siorpaes K, Simperl E (2010) Human intelligence in the process of semantic content creation. World Wide Web 13(1-2):33–59Google Scholar
  81. 81.
    Song ZC, Ge YZ, Duan H, Qiu XG (2016) Agent-based simulation systems for emergency management. Int J Autom Comput 13(2):89–98Google Scholar
  82. 82.
    Stigler SM (1989) Francis galton’s account of the invention of correlation. Stat Sci 4(2):73–79MathSciNetzbMATHGoogle Scholar
  83. 83.
    Stutzle T, Hoos HH (1997) The MAX-MIN ant system and local search for the traveling salesman problem. In: Proceedings ICEC97. IEEE Press, Piscataway, pp 309–314Google Scholar
  84. 84.
    Stützle T, Hoos HH (2000) MAX MIN ant system. Futur Gener Comput Syst 16:889–914zbMATHGoogle Scholar
  85. 85.
    Tokody D, Mezei IJ, Schuster G (2017) An overview of autonomous intelligent vehicle systems. In: Jármai K, Bolló B (eds) Vehicle and automotive engineering. Lecture notes in mechanical engineering, vol PartF12. Springer, pp 287–307Google Scholar
  86. 86.
    Turing AM (1950) Computing machinery and intelligence. Mind 59:433–460MathSciNetGoogle Scholar
  87. 87.
    Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83Google Scholar
  88. 88.
    Winklerova Z (2013) Maturity of the particle swarm as a metric for measuring the collective intelligence of the swarm. Advances in Swarm Intelligence, LNCS 7928:40–54Google Scholar
  89. 89.
    Won ZB, Do CB, Jeong YK, Han S (2002) Machine intelligence quotient: its measurements and applications. Fuzzy Sets Syst 127(1):3–16MathSciNetzbMATHGoogle Scholar
  90. 90.
    Zarandi MHF, Hadavandi E, Turksen IB (2012) A hybrid fuzzy intelligent agent-based system for stock price prediction. Int J Intell Syst 27(11):947–969Google Scholar
  91. 91.
    Zhang Y, Wang H, Zhang Y, Chen Y (2011) Best-worst ant system. In: Proceedings of the 3rd international conference on advanced computer control (ICACC), pp 392–395Google Scholar
  92. 92.
    Yager RR (1997) Intelligent agents for World Wide Web advertising decisions. Int J Intell Syst 12(5):379–390Google Scholar
  93. 93.
    Yang K, Galis A, Guo X, Liu D (2003) rule-driven mobile intelligent agents for real-time configuration of IP networks, knowledge-based intelligent information and engineering systems. Lect Notes Comput Sci 2773:921–928Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • László Barna Iantovics
    • 1
    Email author
  • László Kovács
    • 2
  • Corina Rotar
    • 3
  1. 1.Informatics DepartmentUniversity of Medicine, Pharmacy, Sciences and Technology of Targu MuresTargu MuresRomania
  2. 2.Information Technology DepartmentUniversity of MiskolcMiskolcHungary
  3. 3.Computer Science Department1 Decembrie 1918 UniversityAlba IuliaRomania

Personalised recommendations