Design of Experiments for Performance Evaluation and Parameter Tuning of a Road Image Processing Chain

  • Yves LucasEmail author
  • Antonio Domingues
  • Driss Driouchi
  • Sylvie Treuillet
Open Access
Research Article
Part of the following topical collections:
  1. Performance Evaluation in Image Processing


Tuning a complete image processing chain (IPC) is not a straightforward task. The first problem to overcome is the evaluation of the whole process. Until now researchers have focused on the evaluation of single algorithms based on a small number of test images and ad hoc tuning independent of input data. In this paper, we explain how the design of experiments applied on a large image database enables statistical modeling for IPC significant parameter identification. The second problem is then considered: how can we find the relevant tuning and continuously adapt image processing to input data? After the tuning of the IPC on a typical subset of the image database using numerical optimization, we develop an adaptive IPC based on a neural network working on input image descriptors. By testing this approach on an IPC dedicated-to-road obstacle detection, we demonstrate that this experimental methodology and software architecture can ensure continuous efficiency. The reason is simple: the IPC is globally optimized, from a large number of real images and with adaptive processing of input data.


Input Image Parameter Tuning Image Database Software Architecture Numerical Optimization 


  1. 1.
    Haralick RM: Performance characterization protocol in computer vision. Proceedings of the ARPA Image Understanding Workshop, November 1994, Monterey, Calif, USA I: 667–673.Google Scholar
  2. 2.
    Courtney P, Thacker N, Clark A: Algorithmic modeling for performance evaluation. Proceedings of the ECCV Workshop on Performance Characteristics of Vision Algorithms, April 1996, Cambridge, UK 13.Google Scholar
  3. 3.
    Forstner W: 10 pros and cons against performance characterization of vision algorithms. Proceedings of the ECCV Workshop on Performance Characteristics of Vision Algorithms, April 1996, Cambridge, UKGoogle Scholar
  4. 4.
    Bowyer KW, Phillips PJ: Empirical Evaluation Techniques in Computer Vision. Wiley-IEEE Computer Society Press, Los Alamitos, Calif, USA; 1998.zbMATHGoogle Scholar
  5. 5.
    Meer P, Matei B, Cho K: Input guided performance evaluation. Theoretical Foundations of Computer Vision (TFCV '98), March 1998, Dagstuhl, Germany 115–124.Google Scholar
  6. 6.
    Phillips IT, Chhabra AK: Empirical performance evaluation of graphics recognition systems. IEEE Transactions on Pattern Analysis and Machine Intelligence 1999, 21(9):849–870. 10.1109/34.790427CrossRefGoogle Scholar
  7. 7.
    Blanc-Talon J, Ropert V: Evaluation des chaînes de traitement d'images. Revue Scientifique et Technique de la Défense 2000, (46):29–38.Google Scholar
  8. 8.
    Philipp-Foliguet S: Evaluation de la segmentation. ETIS, Cergy-Pontoise, France; 2001.zbMATHGoogle Scholar
  9. 9.
    Sebe N, Tian Q, Loupias E, Lew MS, Huang TS: Evaluation of salient point techniques. Proceedings of International Conference on Image and Video Retrieval (CIVR '02), July 2002, London, UK 2383: 367–377.CrossRefGoogle Scholar
  10. 10.
    Rosin PL, Ioannidis E: Evaluation of global image thresholding for change detection. Pattern Recognition Letters 2003, 24(14):2345–2356. 10.1016/S0167-8655(03)00060-6CrossRefGoogle Scholar
  11. 11.
    Yitzhaky Y, Peli E: A method for objective edge detection evaluation and detector parameter selection. IEEE Transactions on Pattern Analysis and Machine Intelligence 2003, 25(8):1027–1033. 10.1109/TPAMI.2003.1217608CrossRefGoogle Scholar
  12. 12.
    Ropert V: Proposition d'une architecture de contrôle pour un système de vision, Thèse de l'Université René Descartes (Paris 6). , Paris, France; Décembre 2001.Google Scholar
  13. 13.
    Levner I, Bulitko V: Machine learning for adaptive image interpretation. Proceedings of the 16th Innovative Applications of Artificial Intelligence Conference (IAAI '04), July 2004, San Jose, Calif, USA 870–876.Google Scholar
  14. 14.
    Schimmerling P, Sisson J-C, Zaïdi A: Pratique des Plans d'Expériences. Lavoisier Tec & Doc, Paris, France; 1998.zbMATHGoogle Scholar
  15. 15.
    Treuillet S: Analyse de l'influence des paramètres d'une chaîne de traitements d'images par un plan d'expériences. 19e colloque GRETSI sur le traitement du signal et des images (GRETSI '03), September 2003, Paris, FranceGoogle Scholar
  16. 16.
    Treuillet S, Driouchi D, Ribereau P:Ajustement des paramètres d'une chaîne de traitement d'images par un plan d'expériences fractionnaireOpen image in new window. Traitement du Signal 2004, 21(2):141–155.Google Scholar
  17. 17.
    Wright MH: The nelder-mead simplex method: recent theory and practice. Proceedings of the 16th International Symposium on Mathematical Programming (ISMP '97), August 1997, Lausanne, SwitzerlandGoogle Scholar
  18. 18.
    Domingues A, Lucas Y, Baudrier D, Marché P: Détection et suivi d'objets en temps réel par un système embarqué multi capteurs. Proceedings of the 18th Symposium GRETSI on Signal and Image Processing (GRETSI '01), Septembre 2001, Toulouse, FranceGoogle Scholar
  19. 19.
    Domingues A: Système embarqué multicapteurs pour la détection d'obstacles routiers—Développement du prototype et réglage automatique de la chaîne de traitement d'images, Thèse de l'Université d'Orléans. , Orléans, France; Juillet 2004.Google Scholar
  20. 20.
    Lucas Y, Domingues A, Boubal M, Marché P: Système de vision embarqué pour la détection d'obstacles routiers. Techniques de l'Ingénieur—Recherche & Innovation 2005, 9. IN-24Google Scholar
  21. 21.
    Lamaty P: Opérateurs de niveau intermédiaire pour le traitement temps réel des images, Thèse de Doctorat. Thèse de l'Université de Cergy-Pontoise, Cergy-Pontoise, France; 2000.Google Scholar
  22. 22.
    Lucas Y, Domingues A, Driouchi D, Marché P: Modeling, evaluation and control of a road image processing chain. Proceedings of the 14th Scandinavian Conference on Image Analysis (SCIA '05), June 2005, Joensuu, Finland 3540: 1076–1085.Google Scholar
  23. 23.
    Fries A, Hunter WG:Minimum aberrationOpen image in new window designs. Technometrics 1980, 22(4):601–608. 10.2307/1268198MathSciNetzbMATHGoogle Scholar
  24. 24.
    Rechtschaffner RL: Saturated fractions of 2n and 3n factorial designs. Technometrics 1967, 9: 569–575. 10.2307/1266195MathSciNetGoogle Scholar
  25. 25.
    Cheng H-D, Sun Y: A hierarchical approach to color image segmentation using homogeneity. IEEE Transactions on Image Processing 2000, 9(12):2071–2082. 10.1109/83.887975CrossRefGoogle Scholar

Copyright information

© Lucas et al. 2006

Authors and Affiliations

  • Yves Lucas
    • 1
    Email author
  • Antonio Domingues
    • 2
  • Driss Driouchi
    • 3
  • Sylvie Treuillet
    • 4
  1. 1.Laboratoire Vision et Robotique, IUT Mesures PhysiquesUniversité d'OrléansBourgesFrance
  2. 2.Laboratoire Vision et RobotiqueBourgesFrance
  3. 3.Laboratoire de Statistiques Théoriques et AppliquéesUniversité Pierre & Marie CurieParisFrance
  4. 4.Laboratoire Vision et RobotiqueOrleansFrance

Personalised recommendations