Skip to main content
SpringerLink
Log in

A Discrete Particle Swarm for Multi-objective Problems in Polynomial Neural Networks used for Classification: A Data Mining Perspective

  • Chapter
Swarm Intelligence for Multi-objective Problems in Data Mining

Part of the book series: Studies in Computational Intelligence ((SCI,volume 242))

Abstract

Approximating decision boundaries of large datasets to classify an unknown sample has been recognized by many researchers within the data mining community as a very promising research topic. The application of polynomial neural networks (PNNs) for the approximation of decision boundaries can be considered as a multiple criteria problem rather than as one involving a single criterion. Classification accuracy and architectural complexity can be thought of as two different conflicting objectives when using PNNs for classification tasks. Using these two metrics as the objectives for finding decision boundaries, this chapter adopts a Discrete Pareto Particle Swarm Optimization (DPPSO) method. DPPSO guides the evolution of the swarm by using the two aforementioned objectives: classification accuracy and architectural complexity. The effectiveness of this method is shown on real life datasets having non-linear class boundaries. Empirical results indicate that the performance of the proposed method is encouraging.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abbass, H.: An evolutionary artificial neural networks approach to breast cancer diagnosis. Artificial Intelligence in Medicine 25(3), 265–281 (2002)

    Article  Google Scholar 

  2. Abido, M.A.: Two-level of non-dominated solutions approach to multi-objective particle swarm optimization. In: 2007 Genetic and Evolutionary Computation Conference (GECCO 2007), vol. 1, pp. 726–733. ACM Press, New York (2007)

    Chapter  Google Scholar 

  3. Aksyonova, T.I., Volkovich, V.V., Tetko, I.V.: Robust polynomial neural networks in quantitative-structure activity relationship studies. SAMS 43(10), 1331–1339 (2003)

    Google Scholar 

  4. Banks, A.: A review of particle swarm optimization. Part II: hybridization, combinatorial, multi-criteria and constrained optimization, and indicative applications. Natural Computing 7, 109–124 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  5. Blake, C.L., Merz, C.J.: UCI repository of machine learning databases, http://www.ics.uci.edu/~mlearn/MLRepository

  6. Bozdogan, H.: Statistical data mining and knowledge discovery. Chapman and Hall/CRC Press (2003)

    Google Scholar 

  7. Branke, J., Mostaghim, S.: About selecting the personal best in multi-objective particle swarm optimization. In: Runarsson, T.P., Beyer, H.-G., Burke, E.K., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds.) PPSN 2006. LNCS, vol. 4193, pp. 523–532. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  8. Carvalho, A.B.D., Pozo, A.: Non-ordered data mining rules through multi-objective particle swarm optimization: dealing with numeric and discrete attributes. In: Proceedings of the 8th International Conference on Hybrid Intelligent Systems, pp. 495–500. IEEE Press, Los Alamitos (2008)

    Chapter  Google Scholar 

  9. Clerc, M., Kennedy, J.: The particle swarm–explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation 6(1), 58–73 (2002)

    Article  Google Scholar 

  10. Coello, C.A.C., Salazar Lechuga, M.: MOPSO: A proposal for multiple objective particle swarm optimization. Proceedings of IEEE Congress on Evolutionary Computation 2, 1051–1056 (2002)

    Google Scholar 

  11. Coello Coello, C.A., Lamont, G.B., Van Veldhuizen, D.A.: Evolutionary algorithms for solving multi-objective problems, 2nd edn. Springer, New York (2007)

    MATH  Google Scholar 

  12. Correa, E.S., Freitas, A.A., Johnson, C.G.: A new discrete particle swarm algorithm applied to attribute selection in a bioinformatics data set. In: Proceedings of the 2006 Genetic and Evolutionary Computation Conference (GECCO 2006), Seattle, Washington, USA, pp. 35–42 (2006)

    Google Scholar 

  13. Deb, K.: Multi-objective optimization using evolutionary algorithms. Wiley Interscience Series in Systems and Optimization. John Wiley and Sons, Chichester (2001)

    MATH  Google Scholar 

  14. Deb, K., Goldberg, D.E.: An investigation of niche and species formation in genetic function optimization. In: Proceedings of the 3rd International Conference on Genetic Algorithms, pp. 42–50 (1989)

    Google Scholar 

  15. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA II. IEEE Transactions on Evolutionary Computation 6(2), 182–197 (2002)

    Article  Google Scholar 

  16. Eberhart, R., Kennedy, J.: A new optimizer using particles swarm theory. In: Proceedings of the Sixth International Symposium on Micro Machine and Human Science, pp. 39–43. IEEE Service Center, Piscataway (1995)

    Chapter  Google Scholar 

  17. Engelbrecht, A.P.: Fundamentals of computational swarm intelligence. Wiley, Chichester (2005)

    Google Scholar 

  18. Everson, R.M., Fieldsend, J.E., Singh, S.: Full elite sets for multi-objective optimization. In: Parmee, I.C. (ed.) Adaptive Computing in Design and Manufacture V, pp. 343–354 (2002)

    Google Scholar 

  19. Fayyad, U., Piatetsky-Shapiro, G., Smyth, P.: From data mining to knowledge discovery in databases. AI Magazine 17(3), 37–54 (1996)

    Google Scholar 

  20. Feng, L., Liu, Z., Ma, C.: Outlier identification and justification using multi-objective PSO based clustering algorithm in power system. In: Proceedings of 5th IEEE International Conference on Industrial Informatics, vol. 1, pp. 365–369 (2007)

    Google Scholar 

  21. Fieldsend, J.E., Everson, R.M., Singh, S.: Using unconstrained elite archives for multi-objective optimization. IEEE Transactions on Evolutionary Computation 7(3), 305–323 (2003)

    Article  Google Scholar 

  22. Fieldsend, J.E., Singh, S.: A multi-objective algorithm based upon particle swarm optimization, an efficient data structure and turbulence. In: Proceedings of the Workshop on Computational Intelligence, Birmingham, UK, pp. 37–44 (2002)

    Google Scholar 

  23. Fogel, D.B.: Evolutionary computation: Toward a new philosophy of machine intelligence. IEEE Press, Piscataway (2006)

    Google Scholar 

  24. Goldberg, D.E.: Genetic algorithms in search, optimization, and machine learning. Addison-Wesley Publishing Company, Reading (1989)

    MATH  Google Scholar 

  25. Goldberg, D.E., Richardson, J.: Genetic algorithms with sharing for multi-modal function optimization. In: Proceedings of 2nd International Conference on Genetic Algorithms, pp. 41–49 (1987)

    Google Scholar 

  26. Haykin, S.: Neural Networks. Macmillan, New York (1994)

    MATH  Google Scholar 

  27. Hilderman, R.J.: Knowledge discovery and measures of interest, 1st edn. Springer, Heidelberg (2001)

    MATH  Google Scholar 

  28. Holland, J.H.: Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor (1975)

    Google Scholar 

  29. Hu, X., Eberhart, R.: Multi-objective optimization using dynamic neighborhood particle swarm optimization. In: Proceedings of the 2002 Congress on Evolutionary Computation, vol. 2, pp. 1677–1681. IEEE Press, Los Alamitos (2002)

    Google Scholar 

  30. Ishida, C.Y., de Carvalho, A.B., Pozo, A.T.R., Goldbarg, E.F.G., Goldbarg, M.C.: Exploring multi-objective PSO and GRASP-PR for rule induction. In: van Hemert, J., Cotta, C. (eds.) EvoCOP 2008. LNCS, vol. 4972, pp. 73–84. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  31. Ivakhnenko, A.G., Ivakhnenko, G.A.: The review of problems solvable by algorithms of the group method of data handling (GMDH). Pattern Recognition and Image Analysis 5(4), 527–535 (1995)

    Google Scholar 

  32. Ivakhnenko, A.G.: The group method of data handling-a rival of the method of stochastic appriximation. Soviet Automatic Control c/c of Avtomatika 1(3), 43–55 (1968)

    Google Scholar 

  33. Ivakhnenko, A.G.: Polynomial theory of complex systems. IEEE Transactions on Systems, Man, and Cybernetics SMC-1(4), 1–13 (1971)

    MathSciNet  Google Scholar 

  34. Jiang, M.: Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm. Information processing Letters 102, 8–16 (2007)

    Article  MathSciNet  Google Scholar 

  35. Kennedy, J., Eberhart, R.C.: Swarm intelligence. Morgan Kaufmann Publishers, Inc, San Francisco (2001)

    Google Scholar 

  36. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the 1995 IEEE International Conference on Neural Networks, pp. 1942–1948. IEEE Press, Los Alamitos (1995)

    Chapter  Google Scholar 

  37. Kennedy, J., Eberhart, R.C.: A discrete binary version of the particle swarm algorithm. In: Proceedings of the 1997 Conference on Systems, Man, and Cybernetics, pp. 4104–4109 (1997)

    Google Scholar 

  38. Khaneswar, M.A., Teshnehlab, M., Shoorehdeli, M.A.: A novel binary particle swarm optimization. In: Proceedings of 2007 Mediterranean conference on Control and Automation, Athens-Greece, pp. 1–6. IEEE Press, Los Alamitos (2007)

    Google Scholar 

  39. Knowles, J.D., Corne, D.W.: Approximating the non-dominated front using the Pareto archived evolution strategy. Evolutionary Computation 8(2), 149–172 (2000)

    Article  Google Scholar 

  40. Kriegel, H.P., Borgwardt, K.M., Kröger, P., Pryakhin, A., Schubert, M., Zimek, A.: Future trends in data mining. Data and Knowledge Discovery 15(1), 87–97 (2007)

    Article  MathSciNet  Google Scholar 

  41. Laumanns, M., Thiele, L., Deb, K., Zitzler, E.: Combining convergence and diversity in evolutionary multi-objective optimization. Evolutionary Computation 10(3), 263–282 (2002)

    Article  Google Scholar 

  42. Lee, Y., Van Roy, B., Reed, C.D., Lippmann, R.P., Kennedy, R.L.: Solving data mining problems through pattern recognition. Prentice-Hall, Englewood Cliffs (1997)

    Google Scholar 

  43. Li, S.T., Chen, C.C., Li, J.W.: A multi-objective particle swarm optimization algorithm for rule discovery. In: Proceedings of 3rd International Conference on International Information Hiding and Multi-media Signal Processing (IIH-MSP 2007), pp. 597–600. IEEE Press, Los Alamitos (2007)

    Google Scholar 

  44. Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs. Springer, Heidelberg (1999)

    Google Scholar 

  45. Miettinen, K.: Nonlinear Multiobjective Optimization. Kluwer Academic Publishers, Boston (1999)

    MATH  Google Scholar 

  46. Misra, B.B., Dehuri, S., Dash, P.K., Panda, G.: A reduced and comprehensible polynomial neural network for classification. Pattern Recognition Letters 29(12), 1705–1712 (2008)

    Article  Google Scholar 

  47. Mostaghim, S., Teich, J.: Strategies for finding good local guides in multi-objective particle swarm optimization (MOPSO). In: Proceedings of the 2003 IEEE Swarm Intelligence Symposium, pp. 26–33 (2003)

    Google Scholar 

  48. Mostaghim, S., Teich, J.: The role of ε-dominance in multi-objective particle swarm optimization methods. In: Proceedings of the 2003 Congress on Evolutionary Computation, vol. 3, pp. 1764–1771. IEEE Press, Los Alamitos (2003)

    Chapter  Google Scholar 

  49. Noghin, V.D.: A logical justification of the Edgeworth-Pareto principle. Comp. Mathematics and Math. Physics 42, 915–920 (2002)

    Google Scholar 

  50. Noghin, V.D.: An axiomatization of the generalized Edgeworth-Pareto principle in terms of choice functions. Mathematical Social Sciences 52(2), 210–216 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  51. Oh, S.K., Pedrycz, W., Park, B.J.: Polynomial neural networks architecture: analysis and design. Computers and Electrical Engineering 29, 703–725 (2003)

    Article  Google Scholar 

  52. Piatetsky-Shapiro, G.: Data mining and knowledge discovery 1996 to 2005: overcoming the hype and moving from university to business and analytics. Data Mining and Knowledge Discovery 15(1), 99–105 (2007)

    Article  MathSciNet  Google Scholar 

  53. Reyes-Sierra, M., Coello, C.A.C.: Multi-objective particle swarm optimizers: A Survey of the state-of-the-art. International Journal of Computational Intelligence Research 2(3), 287–308 (2006)

    MathSciNet  Google Scholar 

  54. Shi, Y., Eberhart, R.: Parameter selection in particle swarm optimization. In: Porto, V.W., Waagen, D. (eds.) EP 1998. LNCS, vol. 1447, pp. 591–600. Springer, Heidelberg (1998)

    Chapter  Google Scholar 

  55. Shi, Y., Eberhart, R.: Empirical study of particle swarm optimization. In: Proceedings of the Congress on Evolutionary Computation (CEC 1999), pp. 1945–1950. IEEE Press, Los Alamitos (1999)

    Chapter  Google Scholar 

  56. Stacey, A., Jancic, M., Grundy, I.: Particle swarm optimization with mutation. In: Proceedings of the Congress on Evolutionary Computation, pp. 1425–1430. IEEE Press, Los Alamitos (2003)

    Chapter  Google Scholar 

  57. Tetko, I.V., Aksenova, T.I., Volkovich, V.V., Kashena, T.N., Filipov, D.V., Welsh, W.J., Livingstone, D.J., Villa, A.E.P.: Polynomial neural network for linear and non-linear model selection in quantitative structure activity relationship studies on the interent. SAR QSAR Environ Res. 11, 263–280 (2000)

    Article  Google Scholar 

  58. de Almeida Prado G. Torácio, A., Pozo, A.T.R.: Multiple Objective Particle Swarm for Classification-Rule Discovery. In: Proceedings of IEEE Congress on Evolutionary Computation (CEC 2007), pp. 684–691 (2007)

    Google Scholar 

  59. Trelea, I.C.: The particle swarm optimization algorithm: convergence analysis and parameter selection. Information Processing Letters 85, 317–325 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  60. Tsai, C.Y., Yeh, S.W.: A multiple objective particle swarm optimization approach for inventory classification. International Journal Production Economics 114, 656–666 (2008)

    Article  Google Scholar 

  61. Witten, I.H., Frank, E.: Data mining: practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)

    MATH  Google Scholar 

  62. Yao, X.: Evolving artificial neural networks. Proceedings of the IEEE 87, 1423–1447 (1999)

    Article  Google Scholar 

  63. Zhang, G.P.: Neural networks for classification: A survey. IEEE Transactions on Systems, Man and Cybernetics, Part C: Applications and Reviews 30(4), 451–462 (2000)

    Article  Google Scholar 

  64. Zhen, Y.L., Ma, L.H., Qian, J.X.: On the convergence analysis and parameter selection in particle swarm optimization. In: Proceedings of the 2nd International Conference on Machine Learning and Cybernetics, pp. 1802–1807. IEEE Computer Society Press, Los Alamitos (2003)

    Google Scholar 

  65. Zitzler, E., Thiele, L.: Multi-objective evolutionary algorithms: acomparative case study and the strength Pareto approach. IEEE Transactions on Evolutionary Computation 3(4), 257–271 (1999)

    Article  Google Scholar 

  66. Zribi, M., Ghorbel, F.: An unsupervised and non-parametric Bayesian classifier. Pattern Recognition Letters 24(1-3), 97–112 (2003)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Soft Computing Laboratory, Department of Computer Science, Yonsei University, 134 Shinchon-dong, Seodaemun-gu, Seoul, 120-749, South Korea

    Satchidananda Dehuri & Sung-Bae Cho

  2. CINVESTAV-IPN, Depto. de Computación (Evolutionary Computation Group), Av. IPN No. 2508, Col. San Pedro Zacatenco, Mexico, D.F., 07360, Mexico

    Carlos A. Coello Coello

  3. Machine Intelligence Unit, Indian Statistical Institute, 203 B. T. Road, Kolkata, 700 108, India

    Ashish Ghosh

Authors
  1. Satchidananda Dehuri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos A. Coello Coello
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sung-Bae Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ashish Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Departamento de Computacion, CINVESTAV-IPN, Av. IPN No. 2508, Col. San Pedro Zacatenco, Mexico, D.F., 07360

    Carlos Artemio Coello Coello

  2. Department of information and Communication Technology, FakirMohan University, VyasaVihar, 756 019, Balasore, Orissa, India

    Satchidananda Dehuri

  3. Department of Computer Science and Engineering, Jadavpur University, Raja S.C.Mallik Road, 700 032, Kolkata, India

    Susmita Ghosh

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Dehuri, S., Coello, C.A.C., Cho, SB., Ghosh, A. (2009). A Discrete Particle Swarm for Multi-objective Problems in Polynomial Neural Networks used for Classification: A Data Mining Perspective. In: Coello, C.A.C., Dehuri, S., Ghosh, S. (eds) Swarm Intelligence for Multi-objective Problems in Data Mining. Studies in Computational Intelligence, vol 242. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03625-5_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-03625-5_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-03624-8

  • Online ISBN: 978-3-642-03625-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation