Multi-objective Genetic Programming for Visual Analytics

  • Ilknur Icke
  • Andrew Rosenberg
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6621)


Visual analytics is a human-machine collaboration to data modeling where extraction of the most informative features plays an important role. Although feature extraction is a multi-objective task, the traditional algorithms either only consider one objective or aggregate the objectives into one scalar criterion to optimize. In this paper, we propose a Pareto-based multi-objective approach to feature extraction for visual analytics applied to data classification problems. We identify classifiability, visual interpretability and semantic interpretability as the three equally important objectives for feature extraction in classification problems and define various measures to quantify these objectives. Our results on a number of benchmark datasets show consistent improvement compared to three standard dimensionality reduction techniques. We also argue that exploration of the multiple Pareto-optimal models provide more insight about the classification problem as opposed to a single optimal solution.


Genetic Programming Kernel Principal Component Analysis Projection Pursuit Multiple Discriminant Analysis Genetic Programming Expression 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
  3. 3.
  4. 4.
    Bellman, R.: Adaptive Control Processes. Princeton University Press, Princeton (1961)CrossRefzbMATHGoogle Scholar
  5. 5.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley, Chichester (2000)zbMATHGoogle Scholar
  6. 6.
    Estèbanez, C., Aler, R., Valls, J.M.: A method based on genetic programming for improving the quality of datasets in classification problems. International Journal of Computer Science and Applications 4, 69–80 (2007)Google Scholar
  7. 7.
    Luke, S., et al.: Ecj: A java-based evolutionary computation research system.,
  8. 8.
    Garcia-Osorio, C., Fyfe, C.: Comparing exploratory projection pursuit artificial neural networks. In: 3rd International Workshop on Practical Applications of Agents and Multiagent Systems, IWPAAMS 2004, pp. 201–210 (2004)Google Scholar
  9. 9.
    Guo, H., Nandi, A.K.: Breast cancer diagnosis using genetic programming generated feature. Pattern Recognition 39(5), 980–987 (2006)CrossRefGoogle Scholar
  10. 10.
    Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L. (eds.): Feature Extraction, Foundations and Applications. STUD FUZZ. Physica-Verlag, Springer, Heidelberg (2006)Google Scholar
  11. 11.
    Icke, I., Rosenberg, A.: Multi-objective genetic programming projection pursuit for exploratory data modeling. In: 5th Annual Machine Learning Symposium, New York Academy of Sciences (2010)Google Scholar
  12. 12.
    Icke, I., Rosenberg, A.: Symbolic regression for dimensionality reduction. In: Late Breaking Abstracts, GECCO 2010 (2010)Google Scholar
  13. 13.
    Jin, Y., Sendhoff, B.: Pareto-based multiobjective machine learning: An overview and case studies. IEEE Transactions on Systems, Man, and Cybernetics, Part C 38(3), 397–415 (2008)CrossRefGoogle Scholar
  14. 14.
    Krawiec, K.: Genetic programming-based construction of features for machine learning and knowledge discovery tasks. Genetic Programming and Evolvable Machines 3(4), 329–343 (2002)CrossRefzbMATHGoogle Scholar
  15. 15.
    Lee, E.-K., Cook, D., Klinke, S., Lumley, T.: Projection pursuit for exploratory supervised classification. Journal of Computational and Graphical Statistics 14(4), 831–846 (2005)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Muharram, M., Smith, G.D.: Evolutionary constructive induction. IEEE Trans. on Knowl. and Data Eng. 17(11), 1518–1528 (2005)CrossRefGoogle Scholar
  17. 17.
    Otero, F.E.B., Silva, M.M.S., Freitas, A.A., Nievola, J.C.: Genetic programming for attribute construction in data mining. In: Ryan, C., Soule, T., Keijzer, M., Tsang, E.P.K., Poli, R., Costa, E. (eds.) EuroGP 2003. LNCS, vol. 2610, pp. 384–393. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  18. 18.
    Sips, M., Neubert, B., Lewis, J.P., Hanrahan, P.: Selecting good views of high-dimensional data using class consistency. In: Hege, H.-C., Hotz, I., Munzner, T. (eds.) EuroVis 2009: Eurographics/ IEEE-VGTC Symposium on Visualization 2009. Computer Graphics Forum, vol. 28, pp. 831–838. Blackwell, Berlin (2009)Google Scholar
  19. 19.
    Tatu, A., Bak, P., Bertini, E., Keim, D.A., Schneidewind, J.: Visual quality metrics and human perception: an initial study on 2d projections of large multidimensional data. In: AVI-Proceedings of AVI Working Conference on Advanced Visual Interfaces, pp. 49–56 (2010)Google Scholar
  20. 20.
    Vladislavleva, E., Smits, G., den Hertog, D.: Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Trans. Evolutionary Computation 13(2), 333–349 (2009)CrossRefGoogle Scholar
  21. 21.
    Zhang, Y., Rockett, P.: A generic multi-dimensional feature extraction method using multiobjective genetic programming. Evolutionary Computation 17(1), 89–115 (2009)CrossRefGoogle Scholar
  22. 22.
    Zitzler, E., Laumanns, M., Thiele, L.: Spea2: Improving the strength pareto evolutionary algorithm. Technical report, ETH Zentrum, Gloriastrasse 35, CH-8092 Zurich, Switzerland (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Ilknur Icke
    • 1
  • Andrew Rosenberg
    • 1
    • 2
  1. 1.The Graduate CenterThe City University of New YorkNew YorkUSA
  2. 2.Queens CollegeThe City University of New YorkFlushingUSA

Personalised recommendations