Skip to main content
SpringerLink
Log in
Book cover

Multidimensional Particle Swarm Optimization for Machine Learning and Pattern Recognition pp 45–82Cite as

Particle Swarm Optimization

  • Chapter
  • First Online:

Part of the book series: Adaptation, Learning, and Optimization ((ALO,volume 15))

Abstract

The behavior of a single organism in a swarm is often insignificant, but their collective and social behavior is of paramount importance. The particle swarm optimization (PSO) was introduced by Kennedy and Eberhart [1] in 1995 as a population-based stochastic search and optimization process. It is originated from the computer simulation of the individuals (particles or living organisms) in a bird flock or fish school [2], which basically show a natural behavior when they search for some target (e.g., food). The goal is, therefore, to converge to the global optima of some multidimensional and possibly nonlinear function or system. Henceforth, PSO follows the same path of other evolutionary algorithms (EAs), [3] such as genetic algorithm (GA) [4], genetic programming (GP) [5], evolution strategies (ES) [6], and evolutionary programming (EP) [7]. Recall that the common point of all is that EAs are in population-based nature and they can avoid being trapped in a local optimum. Thus they can find the optimum solutions; however, this is never guaranteed.

We converse as we live by repeating, by combining and recombining a few elements over and over again just as nature does when of elementary particles it builds a world.

William Gass

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. J. Kennedy, R. Eberhart, Particle swarm optimization. In Proceedings of IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948, Perth, Australia, 1995

    Google Scholar 

  2. J.J. van Zyl, Unsupervised classification of scattering mechanisms using radar polarimetry data. IEEE Trans. Geosci. Remote Sens. 27, 36–45 (1989)

    Article  Google Scholar 

  3. A. Antoniou, W.-S. Lu, Practical Optimization, Algorithms and Engineering Applications (Springer, USA, 2007)

    MATH  Google Scholar 

  4. D. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning (Addison-Wesley, Reading MA, 1989), pp. 1–25

    MATH  Google Scholar 

  5. S. Kirkpatrick, C.D. Gelatt, M.P. Vecchi, Optimization by simulated annealing. Science 220, 671–680 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  6. T. Back, F. Kursawe, Evolutionary Algorithms for Fuzzy Logic: A Brief Overview. In Fuzzy Logic and Soft Computing (World Scientific, Singapore, 1995), pp. 3–10

    Book  Google Scholar 

  7. U.M. Fayyad, G.P. Shapire, P. Smyth, R. Uthurusamy, Advances in Knowledge Discovery and Data Mining (MIT Press, Cambridge, MA, 1996)

    Google Scholar 

  8. Y. Shi, R.C. Eberhart, A modified particle swarm optimizer. In Proceedings of the IEEE Congress on Evolutionary Computation, pp. 69–73

    Google Scholar 

  9. R. Eberhart, P. Simpson, R. Dobbins, Computational Intelligence. PC Tools (Academic Press Inc., Boston, MA, USA, 1996)

    Google Scholar 

  10. M. Clerc, J. Kennedy, The particle swarm-explosion, stability and convergence in a multidimensional complex space. IEEE Trans. Evol. Comput. 6(2), 58–73 (2002)

    Article  Google Scholar 

  11. L.-Y. Chuang, H.W. Chang, C.J. Tu, C.H. Yang, Improved binary PSO for feature selection using gene expression data. Comput. Biol. Chem. 32(1), 29–38 (2008)

    Article  MATH  Google Scholar 

  12. K. Yasuda, A. Ide, N. Iwasaki, “Adaptive Particle Swarm Optimization. Proc. IEEE Int. Conf. Sys. Man Cybern. 2, 1554–1559 (2003)

    Google Scholar 

  13. W.-J. Zhang, Y. Liu, M. Clerc, An adaptive PSO algorithm for reactive power optimization. Adv. Power Syst. Control Oper. Manage. (APSCOM) 1, 302–307 (2003). (Hong Kong)

    Google Scholar 

  14. Y. Shi, R.C. Eberhart, Fuzzy adaptive particle swarm optimization. In Proceedings of the IEEE Congress on Evolutionary Computation. IEEE Press, vol. 1, pp. 101–106, 2001

    Google Scholar 

  15. P.J. Angeline, Using selection to improve particle swarm optimization. In Proceedings of the IEEE Congress on Evolutionary Computation, pp. 84–89. IEEE Press, 1998

    Google Scholar 

  16. R.G. Reynolds, B. Peng, J. Brewster, Cultural swarms II: Virtual algorithm emergence. In Proceedings of IEEE Congress on Evolutionary Computation 2003 (CEC 2003), pp. 1972–1979, Australia, 2003

    Google Scholar 

  17. H. Higashi, H. Iba, Particle swarm optimization with Gaussian mutation. In Proceedings of the IEEE Swarm Intelligence Symposium, pp. 72–79, 2003

    Google Scholar 

  18. K. Ersahin, B. Scheuchl, I. Cumming, Incorporating texture information into polarimetric radar classification using neural networks. In Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, pp. 560–563, Anchorage, USA, Sep 2004

    Google Scholar 

  19. F. Van den Bergh, A.P. Engelbrecht, A new locally convergent particle swarm optimizer. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, pp. 96–101, 2002

    Google Scholar 

  20. W. McCulloch, W. Pitts, A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 7, 115–133 (1943)

    Article  MathSciNet  Google Scholar 

  21. M. Riedmiller, H. Braun, A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In Proceedings of the IEEE International Conference on Neural Networks, pp. 586–591, 1993

    Google Scholar 

  22. G.-J. Qi, X.-S. Hua, Y. Rui, J. Tang, H.-J. Zhang, Image classification with kernelized spatial-context. IEEE Trans on Multimedia 12(4), 278–287 (2010). doi:10.1109/TMM.2010.2046270

    Article  Google Scholar 

  23. M. Løvberg, T.K. Rasmussen, T. Krin, Hybrid particle swarm optimiser with breeding and sub-populations. In Proceedings of GECCO2001—Genetic and Evolutionary Computation Conference, p. 409, CA, USA. July 7–11, 2001

    Google Scholar 

  24. Y. Lin, B. Bhanu, Evolutionary feature synthesis for object recognition. IEEE Trans. Man Cybern. C 35(2), 156–171 (2005)

    Article  Google Scholar 

  25. M. Løvberg, T. Krink, Extending particle swarm optimizers with self-organized criticality. Proc. IEEE Congr. Evol. Comput. 2, 1588–1593 (2002)

    Google Scholar 

  26. T.M. Blackwell, J. Branke, Multi-swarm optimization in dynamic environments. Appl. Evol. Comput. 3005, 489–500 (2004). (Springer)

    Article  Google Scholar 

  27. T.M. Blackwell, J. Branke, Multiswarms, exclusion, and anti-convergence in dynamic environments. IEEE Trans. Evol. Comput. 10/4, 51–58 (2004)

    Google Scholar 

  28. T.M. Blackwell, Particle swarm optimization in dynamic environments. Evol. Comput. Dyn. Uncertain Environ. Stud. Comput. Intell. 51, 29–49 (2007). (Springer)

    Article  Google Scholar 

  29. R. Mendes, Population topologies and their influence in particle swarm performance. PhD thesis, Universidade do Minho, 2004

    Google Scholar 

  30. J. Kennedy, Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance. In Proceedings of the 1999 Congress on Evolutionary Computation, vol. 3. doi:10.1109/CEC.1999.785509, 1999

  31. H. Frigui, R. Krishnapuram, Clustering by competitive agglomeration. Pattern Recogn. 30, 1109–1119 (1997)

    Article  Google Scholar 

  32. A.K. Jain, M.N. Murthy, P.J. Flynn, Data clustering: A review. ACM Computing Reviews, Nov 1999

    Google Scholar 

  33. C.P. Tan, K.S. Lim, H.T. Ewe, Image processing in polarimetric SAR images using a hybrid entropy decomposition and maximum likelihood (EDML). In Proceedings of International Symposium on Image and Signal Processing and Analysis (ISPA), pp. 418–422, Sep 2007

    Google Scholar 

  34. B. Bhanu, J. Yu, X. Tan, Y.Lin, Feature synthesis using genetic programming for face expression recognition. Genetic and Evolutionary Computation (GECCO 2004), Lecture Notes in Computer Science, vol. 3103, pp. 896–907, 2004

    Google Scholar 

  35. G. Hammerly, Learning structure and concepts in data through data clustering. PhD thesis, June 26, 2003

    Google Scholar 

  36. B. Zhang, M. Hsu, K-harmonic means—a data clustering algorithm. Hewlett-Packard Labs Technical Report HPL-1999-124, 1999

    Google Scholar 

  37. G. Hammerly, C. Elkan, Alternatives to the k-means algorithm that find better clusterings. In Proceedings of the 11th ACM CIKM, pp. 600–607, 2002

    Google Scholar 

  38. N.R. Pal, J. Biswas, Cluster validation using graph theoretic concepts. Pattern Recogn. 30(6), 847–857 (1997)

    Article  Google Scholar 

  39. T. Ojala, M. Pietikainen, D. Harwood, A comparative study of texture measures with classification based on feature distributions. Pattern Recogn. 29, 51–59 (1996)

    Article  Google Scholar 

  40. T.G. Dietterich, G. Bakiri, Solving multiclass learning problems via error-correcting output codes. J. Artif. Intell. Res. 2, 263–286 (1995)

    MATH  Google Scholar 

  41. M. Halkidi, Y. Batistakis, M. Vazirgiannis, On cluster validation techniques. J. Intell. Inf. Syst. 17(2, 3), 107–145 (2001)

    Article  MATH  Google Scholar 

  42. T.N. Tran, R. Wehrens, D.H. Hoekman, L.M.C. Buydens, Initialization of Markov random field clustering of large remote sensing images. IEEE Trans. Geosci. Remote Sens. 43(8), 1912–1919 (2005)

    Article  Google Scholar 

  43. S.R. Cloude, E. Pottier, An entropy based classification scheme for land applications of polarimetric SAR. IEEE Trans. Geosci. Remote Sens. 35, 68–78 (1997)

    Article  Google Scholar 

  44. M. Halkidi, M. Vazirgiannis, Clustering validity assessment: finding the optimal partitioning of a dataset. In Proceedings of First IEEE International Conference on Data Mining (ICDM’01), pp. 187–194, 2001

    Google Scholar 

  45. M.G. Omran, A. Salman, A.P. Engelbrecht, Dynamic clustering using particle swarm optimization with application in image segmentation. Patt. Anal. Appl. 8, 332–344 (2006)

    Article  MathSciNet  Google Scholar 

  46. A. Abraham, S. Das, S. Roy, Swarm intelligence algorithms for data clustering. In Soft Computing for Knowledge Discovery and Data Mining Book, Part IV, pp. 279–313, Oct 25, 2007

    Google Scholar 

  47. S. Haykin, Neural Networks: a Comprehensive Foundation (Prentice hall, USA, 1998). June

    Google Scholar 

  48. S. Pittner, S.V. Kamarthi, Feature extraction from wavelet coefficients for pattern recognition tasks. IEEE Trans. Pattern Anal. Machine Intell. 21, 83–88 (1999)

    Article  Google Scholar 

  49. J.A. Nelder, R. Mead, A simplex method for function minimization. Comput. J. 7, 308–313 (1965)

    Article  MATH  Google Scholar 

  50. M. Carvalho, T.B. Ludermir, Particle swarm optimization of neural network architectures and weights. In Proceedings of the 7th International Conference on Hybrid intelligent Systems, pp. 336–339, Washington DC, 17–19 Sep 2007

    Google Scholar 

  51. M. Meissner, M. Schmuker, G. Schneider, Optimized particle swarm optimization (OPSO) and its application to artificial neural network training. BMC Bioinf 7, 125 (2006)

    Article  Google Scholar 

  52. Z. Ye, C.-C. Lu, Wavelet-based unsupervised SAR image segmentation using hidden markov tree models. In Proceedings of the 16th International Conference on Pattern Recognition (ICPR’02), vol. 2, pp. 20729, 2002

    Google Scholar 

  53. C. Zhang, H. Shao, An ANN’s evolved by a new evolutionary system and its application. In Proceedings of the 39th IEEE Conference on Decision and Control, vol. 4, pp. 3562–3563, 2000

    Google Scholar 

  54. H. Robbins, S. Monro, A stochastic approximation method. Ann. Math. Stat. 22, 400–407 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  55. J.F. Scott, The Scientific Work of René Descartes, 1987

    Google Scholar 

  56. L. Prechelt, Proben1—A set of neural network benchmark problems and benchmark rules. Technical Report 21/94, Fakultät für Informatik, Universität Karlsruhe, Germany, September, 1994

    Google Scholar 

  57. S. Amari, N. Murata, K.R. Muller, M. Finke, H.H. Yang, Asymptotic statistical theory of overtraining and cross-validation. IEEE Trans. Neural Networks 8(5), 985–996 (1997)

    Article  Google Scholar 

  58. T.L. Ainsworth, J.P. Kelly, J.-S. Lee, Classification comparisons between dual-pol, compact polarimetric and quad-pol SAR imagery. ISPRS J. Photogram. Remote Sens. 64, 464–471 (2009)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Signal Processing, Tampere University of Technology, 33101, Tampere, Finland

    Serkan Kiranyaz & Moncef Gabbouj

  2. Department of Electrical and Electronics Engineering, Izmir University of Economics, Sakarya Cd. No 156, 35330, Balcova, Izmir, Turkey

    Turker Ince

Authors
  1. Serkan Kiranyaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Turker Ince
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Moncef Gabbouj
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Serkan Kiranyaz .

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Kiranyaz, S., Ince, T., Gabbouj, M. (2014). Particle Swarm Optimization. In: Multidimensional Particle Swarm Optimization for Machine Learning and Pattern Recognition. Adaptation, Learning, and Optimization, vol 15. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37846-1_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-37846-1_3

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-37845-4

  • Online ISBN: 978-3-642-37846-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation