Skip to main content
SpringerLink
Log in

Multi-Objective Neural Network Optimization for Visual Object Detection

  • Chapter
Multi-Objective Machine Learning

Part of the book series: Studies in Computational Intelligence ((SCI,volume 16))

Abstract

In real-time computer vision, there is a need for classifiers that detect patterns fast and reliably. We apply multi-objective optimization (MOO) to the design of feed-forward neural networks for real-world object recognition tasks, where computational complexity and accuracy define partially conflicting objectives. Evolutionary structure optimization and pruning are compared for the adaptation of the network topology. In addition, the results of MOO are contrasted to those of a single-objective evolutionary algorithm. As a part of the evolutionary algorithm, the automatic adaptation of operator probabilities in MOO is described.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Hussein A. Abbass. Speeding up backpropagation using multiobjective evolutionary algorithms. Neural Computation, 15(11):2705–2726, 2003.

    Article  MATH  Google Scholar 

  2. C Bahlmann, Y Zhu, V Ramesh, M Pellkofer, and T Koehler. A system for traffic sign detection, tracking, and recognition using color, shape and motion information. In Proceedings of the IEEE Symposium on Intelligent Vehicles, pages 255–260, 2005.

    Google Scholar 

  3. P. L. Bartlett. The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 44(2):525–536, 1998.

    Article  MATH  MathSciNet  Google Scholar 

  4. Stefan Bleuler, Marco Laumanns, Lothar Thiele, and Eckart Zitzler. Pisa – A platform and programming language independent interface for search algorithms. In Carlos M. Fonseca, Peter J. Fleming, Eckart Zitzler, Kalyanmoy Deb, and Lothar Thiele, editors, Evolutionary Multi-Criterion Optimization (EMO 2003), volume 2632 of LNCS, pages 494 – 508. Springer-Verlag, 2003.

    Google Scholar 

  5. H. Braun. Neurale Netze: Optimierung durch Lernen und Evolution. Springer- Verlag, 1997.

    Google Scholar 

  6. T Bücher, C Curio, H Edelbrunner, C Igel, D Kastrup, I Leefken, G Lorenz, A Steinhage, and W von Seelen. Image processing and behaviour planning for intelligent vehicles. IEEE Transactions on Industrial electronics, 90(1):62–75, 2003.

    Article  Google Scholar 

  7. Rich Caruana, Steve Lawrence, and C. Lee Giles. Overfitting in neural networks: Backpropagation, Conjugate Gradient, and Early Stopping. In Advances in Neural Information Processing Systems, volume 13, pages 402–408. MIT Press, 2001.

    Google Scholar 

  8. MM Chun and JM Wolfe. Visual attention. In EB Goldstein, editor, Blackwell's Handbook of Perception, chapter 9, pages 272–310. Oxford, UK: Blackwell, 2001.

    Google Scholar 

  9. C. A. Coello Coello, D. A. Van Veldhuizen, and G. B. Lamont. Evolutionary Algorithms for Solving Multi-objective Problems. Kluwer Academic Publishers, New York, 2002.

    MATH  Google Scholar 

  10. Lawrence Davis. Adapting operator probabilities in genetic algorithms. In J. David Schaffer, editor, Proceedings of the Third International Conference on Genetic Algorithms, ICGA'89, pages 61–69. Morgan Kaufmann, 1989.

    Google Scholar 

  11. Kalyanmoy Deb. Multi-Objective Optimization using Evolutionary Algorithms. John Wiley & Sons, 2001.

    Google Scholar 

  12. Kalyanmoy Deb, Samir Agrawal, Amrit Pratap, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation, 6(2):182–197, 2002.

    Article  Google Scholar 

  13. E.D. Dickmanns and B.D. Mysliwetz. Recursive 3-D road and relative ego-state recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2):199–213, 1992.

    Article  Google Scholar 

  14. Agoston Endre Eiben, Robert Hinterding, and Zbigniew Michalewicz. Parameter control in evolutionary algorithms. IEEE Transactions on Evolutionary Computation, 3(2):124–141, 1999.

    Article  Google Scholar 

  15. D Ferster and K.D. Miller. Neural mechanisms of orientation selectivity in the visual cortex. Annual Review of Neuroscience, 23:441–471, 2000.

    Article  Google Scholar 

  16. J. E. Fieldsend and S. Singh. Pareto evolutionary neural networks. IEEE Transactions on Neural Networks, 16(2):338–354, 2005.

    Article  Google Scholar 

  17. David B. Fogel. Evolutionary Computation: Toward a New Philosophy of Machine Intelligence. IEEE Press, Piscataway, NJ, USA, 1995.

    Google Scholar 

  18. C. M. Fonseca, J. D. Knowles, L. Thiele, and E. Zitzler. A tutorial on the performance assessment of stochastic multiobjective optimizers. Presented at the Third International Conference on Evolutionary Multi-Criterion Optimization (EMO 2005), 2005.

    Google Scholar 

  19. W Freeman and E Adelson. The design and use of steerable filters. IEEE Transactions on Pattern analysis and machine intelligence, 13(9):891–906, 1991.

    Article  Google Scholar 

  20. N. Garcia-Pedrajas, C. Hervas-Martinez, and J. Munos-Perez. Multi-objective cooperative coevolution of artificial neural networks (multi-objective cooperative networks). Neural Networks, 15:1259–1278, 2002.

    Article  Google Scholar 

  21. A Gepperth, J Edelbrunner, and T Bücher. Real-time detection and classification of cars in video sequences. In Proceedings of the IEEE Symposium on Intelligent Vehicles, pages 625–631, 2005.

    Google Scholar 

  22. A. Gepperth and S. Roth. Applications of multi-objective structure optimization. In M. Verleysen, editor, 5th European Symposium on Artificial Neural Networks (ESANN 2005), pages 279–284. d-side Publications, 2005.

    Google Scholar 

  23. J. Gonzalez, I. Rojas, J. Ortega, H. Pomares, J. Fernandez, and A. Diaz. Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation. IEEE Transactions on Neural Networks, 14(6):1478–1495, November 2003.

    Article  Google Scholar 

  24. Eric Hjelmas and Boon Kee Low. Face detection: A survey. Computer Vision and Image Understanding, 83:236–274, 2001.

    Article  MATH  Google Scholar 

  25. H. Martin Hunke. Locating and tracking of human faces with neural networks. Master's thesis, University of Karlsruhe, Germany, 1994.

    Google Scholar 

  26. Michael Hüsken, Michael Brauckmann, Stefan Gehlen, Kazunori Okada, and Christoph von der Malsburg. Evaluation of implicit 3D modeling for pose invariant face recognition. In A. K. Jain and N. K. Ratha, editors, Defense and Security Symposium 2004: Biometric Technology for Human Identification, volume 5404 of Proceedings of SPIE. The International Society for Optical Engineering, 2004.

    Google Scholar 

  27. C. Igel and B. Sendhoff Synergies between evolutionary and neural computation. In M. Verleysen, editor, 13th European Symposium on Artificial Neural Networks (ESANN 2005), pages 241–252. d-side Publications, 2005.

    Google Scholar 

  28. Christian Igel and Michael Hüsken. Empirical evaluation of the improved Rprop learning algorithm. Neurocomputing, 50(C):105–123, 2003.

    Article  MATH  Google Scholar 

  29. Christian Igel and Martin Kreutz. Operator adaptation in evolutionary computation and its application to structure optimization of neural networks. Neurocomputing, 55(1–2):347–361, 2003.

    Article  Google Scholar 

  30. Christian Igel, Stefan Wiegand, and Frauke Friedrichs. Evolutionary optimization of neural systems: The use of self-adptation. In M. G. de Bruin, D. H. Mache, and J. Szabados, editors, Trends and Applications in Constructive Approximation, number 151 in International Series of Numerical Mathematics, pages 103–123. Birkhäuser Verlag, 2005.

    Google Scholar 

  31. Yaochu Jin, Tatsuya Okabe, and Bernhard Sendhoff Neural network regularization and ensembling using multi-objective evolutionary algorithms. In Proceedings of the Congress on Evolutionary Computation (CEC 2004), pages 1–8. IEEE Press, 2004.

    Google Scholar 

  32. J Kaszubiak, M Tornow, RW Kuhn, B Michaelis, and C Knoeppel. Real-time vehicle and lane detection with embedded hardware. In Proceedings of the IEEE Symposium on Intelligent Vehicles, pages 619–624, 2005.

    Google Scholar 

  33. J. D. Knowles and D. W. Corne. On metrics for comparing non-dominated sets. In Congress on Evolutionary Computation Conference (CEC 2002), pages 711–716. IEEE Press, 2002.

    Google Scholar 

  34. M. Lades, J. C. Vorbrüggen, J. Buhmann, J. Lange, C. von der Malsburg, R. P. Würtz, and W. Konen. Distortion invariant object recognition in the dynamic link architecture. IEEE Transactions on Computers, 42:301–311, 1993.

    Article  Google Scholar 

  35. S. Nolfi. Evolution and learning in neural networks. In M. A. Arbib, editor, The Handbook of Brain Theory and Neural Networks, pages 415–418. MIT Press, 2nd edition, 2002.

    Google Scholar 

  36. M Oren, C Papageorgiou, P Sinha, T Osuna, and T Poggio. Pedestrian detection using wavelet templates. In Proc. Computer Vision and Pattern Recognition, Puerto Rico, pages pp. 193–199, 1997.

    Google Scholar 

  37. Lutz Prechelt. Early stopping – but when? In Genevieve B. Orr and Klaus- Robert Müller, editors, Neural Networks: Tricks of the Trade, volume 1524 of LNCS, chapter 2, pages 57–69. Springer-Verlag, 1999.

    Google Scholar 

  38. Russell D. Reed and Robert J. Marks II. Neural Smithing. MIT Press, 1999.

    Google Scholar 

  39. Martin Riedmiller. Advanced supervised learning in multi-layer perceptrons – From backpropagation to adaptive learning algorithms. Computer Standards and Interfaces, 16(5):265–278, 1994.

    Article  Google Scholar 

  40. Henry A. Rowley, Shumeet Baluja, and Takeo Kanade. Neural network-based face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(1):23–38, 1998.

    Article  Google Scholar 

  41. A Shashua, Y Gdalyahu, and G Hayun. Pedestrian detection for driving assistance systems: Single-frame classification and system level performance. In Proceedings of the IEEE Symposium on Intelligent Vehicles, pages 1–6, 2004.

    Google Scholar 

  42. James Edward Smith and Terence C. Fogarty. Operator and parameter adaptation in genetic algorithms. Soft Computing, 1(2):81–87, 1997.

    Article  Google Scholar 

  43. Achim Stahlberger and Martin Riedmiller. Fast network pruning and feature extraction by using the unit-OBS algorithm. In Michael C. Mozer, Michael I. Jordan, and Thomas Petsche, editors, Advances in Neural Information Processing Systems, volume 9, pages 655–661. The MIT Press, 1997.

    Google Scholar 

  44. M Szarvas, A Yoshizawa, M Yamamoto, and J Ogata. Pedestrian detection using convolutional neural networks. In Proceedings of the IEEE Symposium on Intelligent Vehicles, pages 224–229, 2005.

    Google Scholar 

  45. Viisage Technology AG. http://www.viisage.com.

    Google Scholar 

  46. L. Darrell Whitley. The GENITOR algorithm and selection pressure: Why rank-based allocation of reproductive trials is best. In James David Schaffer, editor, Proceedings of the Third International Conference on Genetic Algorithms, ICGA'89, pages 116–121. Morgan Kaufmann, 1989.

    Google Scholar 

  47. S. Wiegand, C. Igel, and U. Handmann. Evolutionary multi-objective optimisation of neural networks for face detection. International Journal of Computational Intelligence and Applications, 4(3):237–253, 2004.

    Article  Google Scholar 

  48. S. Wiegand, C. Igel, and U. Handmann. Evolutionary optimization of neural networks for face detection. In M. Verleysen, editor, 12th European Symposium on Artificial Neural Networks (ESANN 2004), pages 139–144. Evere, Belgium: d-side publications, 2004.

    Google Scholar 

  49. C Wöhler and J. K. Anlauf. Real-time object recognition on image sequences with the adaptable time delay neural network algorithm - applications for autonomous vehicles. Image and Vision Computing, 19(9–10):593–618, 2001.

    Article  Google Scholar 

  50. J. M. Wolfe. Visual search. In H. D. Pashler, editor, Attention. London UK: University College London Press, 1998.

    Google Scholar 

  51. M.-H. Yang, D. J. Kriegman, and N. Ahuja. Detecting faces in images: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(1):34–58, 2002.

    Article  Google Scholar 

  52. X. Yao. Evolving artificial neural networks. Proceedings of the IEEE, 87(9):1423–1447, 1999.

    Article  Google Scholar 

  53. G. P. Zhang. Neural Networks for Classification: A Survey. IEEE Transactions on System, Man, and Cybernetics – Part C, 30(4):451–462, 2000.

    Article  Google Scholar 

  54. W. Zhao, R. Chellappa, P. Phillips, and A. Rosenfeld. Face recognition: A literature survey. ACM Computing Surveys (CSUR), 35(4):399–458, 2003.

    Article  Google Scholar 

  55. Eckart Zitzler and Lothar Thiele. Multiobjective optimization using evolutionary algorithms — a comparative case study. In Agoston E. Eiben, Thomas Bäck, Marc Schoenauer, and Hans-Paul Schwefel, editors, Fifth International Conference on Parallel Problem Solving from Nature (PPSN V), pages 292–301. Springer-Verlag, 1998.

    Google Scholar 

  56. Eckart Zitzler, Lothar Thiele, Marco Laumanns, Carlos M. Fonseca, and Viviane Grunert da Fonseca. Performance assessment of multiobjective optimizers: An analysis and review. IEEE Transactions on Evolutionary Computation, 7(2):117–132, 2003.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Ruhr-Universität Bochum Institut für Neuroinformatik, Bochum, 44780, Germany

    Stefan Roth, Alexander Gepperth & Christian Igel

Authors
  1. Stefan Roth
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexander Gepperth
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian Igel
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Honda Research Institute Europe GmbH, Carl-Legien-Str. 30, Offenbach, 63073, Germany

    Yaochu Jin Dr.

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer

About this chapter

Cite this chapter

Roth, S., Gepperth, A., Igel, C. (2006). Multi-Objective Neural Network Optimization for Visual Object Detection. In: Jin, Y. (eds) Multi-Objective Machine Learning. Studies in Computational Intelligence, vol 16. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-33019-4_27

Download citation

  • DOI: https://doi.org/10.1007/3-540-33019-4_27

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-30676-4

  • Online ISBN: 978-3-540-33019-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation