A Technique for Evaluation of Interactive Evolutionary Systems

  • M. Shackelford
  • D. W. Corne


Very large scale scheduling and planning tasks cannot be effectively addressed by fully automated schedule optimisation systems, since many key factors which govern ‘fitness’ in such cases are unformalisable. This raises the question of an interactive (or collaborative) approach, where fitness is assigned by the expert user. Though well-researched in the domains of interactively evolved art and music, this method is as yet rarely used in logistics. This paper concerns a difficulty shared by all interactive evolutionary systems (IESs), but especially those used for logistics or design problems. The difficulty is that objective evaluation of IESs is severely hampered by the need for expert humans in the loop. This makes it effectively impossible to, for example, determine with statistical confidence any ranking among a decent number of configurations for the parameters and strategy choices. We make headway into this difficulty with an Automated Tester (AT) for such systems. The AT replaces the human in experiments, and has parameters controlling its decision-making accuracy (modelling human error) and a built-in notion of a target solution which may typically be at odds with the solution which is optimal in terms of formalisable fitness. Using the AT, plausible evaluations of alternative designs for the IES can be done, allowing for (and examining the effects of) different levels of user error. We describe such an AT for evaluating an IES for very large scale planning.


Error Range Automate Test Target Profile Formalisable Fitness Project Planner 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Woodgate, H.S. (1964) Planning by Network, Business Publications Ltd.Google Scholar
  2. 2.
    Shackelford, M., Corne, D. (2001) Collaborative Evolutionary Multi-Project Resource Scheduling, in Proceedings of the 2001 Congress on Evolutionary Computation, IEEE Press, pp. 1131–1138.Google Scholar
  3. 3.
    Hartmann, S. (1999) Project Scheduling under Limited Resources, Springer-VerlagGoogle Scholar
  4. 4.
    Tsang, E., Voudouris, C. (1995) Fast Local Search and Guided Local Search and their Application to British Telecom’s Workforce Scheduling Problem, Operations Research Letters, 20, pp119–127.CrossRefGoogle Scholar
  5. 5.
    Louis, S., Xu, Z. (1996) Genetic Algorithms for Open-Shop Scheduling and Re-Scheduling, In M. E. Cohen and D. L. Hudson, editors, Proc. of the ISCA 11th Int’l Conf. on Computers and their Applications, pages 99–102. ISCA.Google Scholar
  6. 6.
    Bruns, R. (1997) Evolutionary Computation Applications: Scheduling, Ch 1.5, Handbook of Evolutionary Computation, OUP.Google Scholar
  7. 7.
    Corne, D.W., Ross, P. (1997) Practical Issues and Recent Advances in Job-and Open-Shop Scheduling, in D. Dasgupta and Z. Michalewicz (eds), Evolutionary Algorithms in Engineering Applications, Springer-Verlag, pp. 531–546.Google Scholar
  8. 8.
    Montana, D. (1998) Introduction to the Special Issue: Evolutionary Algorithms for Scheduling, Evolutionary Computation, 6(1)Google Scholar
  9. 9.
    Bierwirth, C., Mattfield, D.C. (1999) Production Scheduling and Rescheduling with Genetic Algorithms. Evolutionary Computation, 7(1), pp.1–17.CrossRefGoogle Scholar
  10. 10.
    Husbands, P. (1999) Genetic Algorithms for Scheduling, AISB Quarterly, 89 Google Scholar
  11. 11.
    Dawkins, R. (1988) The Blind Watchmaker, Penguin.Google Scholar
  12. 12.
    Sims, K. (1991) Artificial Evolution for Computer Graphics, Computer Graphics, 25(4), pp. 319–328.MathSciNetCrossRefGoogle Scholar
  13. 13.
    Smith, J.R. (1991) Designing Biomorphs with an Interactive Genetic Algorithm, Proc. 4th ICGA, Morgan Kaufmann.Google Scholar
  14. 14.
    Graf, J., Banzhaf, W. (1995) Interactive Evolution of Images, Proc. of Int’l Conf on Evolutionary Programming, San Diego.Google Scholar
  15. 15.
    Aoki, K., Takagi, H. (1997) 3-D CG Lighting with an Interactive GA, 1 st Int’l Conf. On Conventional and Knowledge-based Intelligent Elec. Sys. (KES’ 97).Google Scholar
  16. 16.
    Banzhaf, W. (1997) Interactive Evolution, Handbook of Evolutionary Computing, C2.10, OUP.Google Scholar
  17. 17.
    Johnston, V., Caldwell, C. (1997) Tracking a Criminal Suspect through Face Space with a Genetic Algorithm, Ch 8.3, Handbook of Evolutionary Computation, OUP.Google Scholar
  18. 18.
    Louis, S., Tang, R. (1999) Interactive Genetic Algorithms for the Travelling Salesman Problem, In Proc. of the Genetic and Evolutionary Computation Conference, GECCO’99, Morgan Kaufmann.Google Scholar
  19. 19.
    Oliver, A., Regragui, O., Monmarche, N., Venturini, G. (2002) Genetic and Interactive Optimization of Web Sites, In Proc. Of 11 st Int’l WWW Conf, Hawaii.Google Scholar
  20. 20.
    Inoue T., Furuhashi T., Fujii M. et al., 1999, Development of Nurse Scheduling Support System using Interactive Evolutionary Algorithms. Proceedings IEEE International Conference on Systems, Man and Cybernetics (SMC’99); pp 533–537Google Scholar
  21. 21.
    Parmee IC, Cvetkovic D, Watson AH, Bonham C, (2000) Multi-objective Satisfaction within an Interactive Evolutionary Design Environment. Evolutionary Computation, 8(2); 197–222.CrossRefGoogle Scholar
  22. 22.
    Parmee IC, Cvetkovic D, Bonham C, Packham I, (2001) Introducing Prototype Interactive Evolutionary Systems for Ill-defined Design Environments. Journal of Advances in Engineering Software, Elsevier, 32(6);pp 429–441.CrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag London 2004

Authors and Affiliations

  • M. Shackelford
    • 1
  • D. W. Corne
    • 2
  1. 1.School of Systems EngineeringUniversity of ReadingReading
  2. 2.School of EngineeringComputer Science and Mathematics University of ExeterExeter

Personalised recommendations