Skip to main content

EvoloPy-FS: An Open-Source Nature-Inspired Optimization Framework in Python for Feature Selection

  • Chapter
  • First Online:
Evolutionary Machine Learning Techniques

Part of the book series: Algorithms for Intelligent Systems ((AIS))

Abstract

Feature selection is a necessary critical stage in data mining process. There is always an arm race to build frameworks and libraries that ease and automate this process. In this chapter, an EvoloPy-FS framework is proposed, which is a Python open-source optimization framework that includes several well-regarded swarm intelligence (SI) algorithms. It is geared toward feature selection optimization problems. It is an easy to use, reusable, and adaptable framework. The objective of developing EvoloPy-FS is providing a feature selection engine to help researchers even those with less knowledge in SI in solving their problems and visualizing rapid results with a less programming effort. That is why the orientation of this work was to build an open-source, white-box framework, where algorithms and data structures are being explicit, transparent, and publicly available. EvoloPy-FS comes to continue our path for building an integrated optimization environment, which was started by the original EvoloPy for global optimization problems, then EvoloPy-NN for training multilayer perception neural network, and finally the new EvoloPy-FS for feature selection optimization. EvoloPy-FS is freely hosted on (www.evo-ml.com) with a helpful documentation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3(Mar):1157–1182

    Google Scholar 

  2. Huan L, Hiroshi M (eds) (2007) Computational methods of feature selection. CRC Press

    Google Scholar 

  3. Zhao, Z, Liu H (2007) Spectral feature selection for supervised and unsupervised learning. In: Proceedings of the 24th international conference on Machine learning. ACM

    Google Scholar 

  4. Huan L, Yu L (2005) Toward integrating feature selection algorithms for classification and clustering. IEEE Trans Knowl Data Eng 17(4):491–502

    Article  Google Scholar 

  5. Manoranjan D, Huan L (1997) Feature selection for classification. Intell Data Anal 1(3):131–156

    Article  Google Scholar 

  6. Hou C et al (2014) Joint embedding learning and sparse regression: a framework for unsupervised feature selection. IEEE Trans Cybern 44(6):793–804

    Article  Google Scholar 

  7. Celeux G et al (2011) A framework for feature selection in clustering. J Am Stat Assoc 105:713–726. J Am Stat Assoc 106(493)

    Google Scholar 

  8. Zhao Z et al (2010) Advancing feature selection research. ASU Featur Sel Repos 2010:1–28

    Google Scholar 

  9. Li J et al (2017) Feature selection: a data perspective. ACM Computing Surveys (CSUR) 50(6):94

    Article  Google Scholar 

  10. Ramrez-Gallego S et al (2018) An information theory-based feature selection framework for big data under apache spark. IEEE Trans Syst, Man, Cybern: Syst 48(9):1441–1453

    Article  Google Scholar 

  11. Verénica B-C, Noelia S-M, Amparo A-B (2015) Recent advances and emerging challenges of feature selection in the context of big data. Knowl-Based Syst 86:33–45

    Article  Google Scholar 

  12. Liu, H, Motoda H (2012) Feature selection for knowledge discovery and data mining vol 454. Springer Science and Business Media

    Google Scholar 

  13. Kohavi R, John GH (1997) Wrappers for feature subset selection. Artif Intell 97(1–2):273–324

    Article  MATH  Google Scholar 

  14. Abe S (2010) Feature selection and extraction. Support vector machines for pattern classification. Springer, London, pp 331–341

    Book  Google Scholar 

  15. Molina LC, Belanche L, Nebot À (2002) Feature selection algorithms: a survey and experimental evaluation. Data mining, 2002. ICDM 2003. 2002 IEEE international conference on. Proceedings. IEEE

    Google Scholar 

  16. Yong L, Feng T, Zhiyong Z (2015) Feature selection based on dependency margin. IEEE Trans Cybern 45(6):1209–1221

    Article  Google Scholar 

  17. Ensan F, Bagheri E, Gašević D (2012) Evolutionary search-based test generation for software product line feature models. In: International conference on advanced information systems engineering. Springer, Berlin, Heidelberg

    Google Scholar 

  18. Yusta SC (2009) Different metaheuristic strategies to solve the feature selection problem. Pattern Recognit Lett 30(5):525–534

    Article  Google Scholar 

  19. Yang X-S (2013) Metaheuristic optimization: nature-inspired algorithms and applications.In: Artificial intelligence, evolutionary computing and metaheuristics. Springer, Berlin, Heidelberg, pp 405–420

    Google Scholar 

  20. Holland JH (1992) Genetic algorithms. Sci Am 267(1):66–73

    Article  Google Scholar 

  21. Koza JR (1992) Genetic programming II, automatic discovery of reusable subprograms. MIT Press, Cambridge, MA

    Google Scholar 

  22. Kennedy J (2006) Swarm intelligence. Handbook of nature-inspired and innovative computing. Springer, Boston, MA, pp 187–219

    Book  Google Scholar 

  23. Eberhart R, Kennedy J (2011) ’A new optimizer using particle swarm theory. Micro Machine and Human Science, 1995. MHS’95. In: Proceedings of the sixth international symposium on. IEEE, 1995

    Google Scholar 

  24. Dorigo M, Birattari M (2011) Ant colony optimization. Encyclopedia of machine learning. Springer, Boston, MA, pp 36–39

    Google Scholar 

  25. Xin-She Y, Suash D (2009) Cuckoo search via Lévy flights. In: World Congress on nature and biologically inspired computing (2009) NaBIC 2009. IEEE, p 2009

    Google Scholar 

  26. Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69(2014):46–61

    Article  Google Scholar 

  27. Mirjalili S, Mirjalili SM, Hatamlou A (2016) Multi-verse optimizer: a nature-inspired algorithm for global optimization. Neural Comput Appl 27(2):495–513

    Article  Google Scholar 

  28. Mirjalili S (2015) Moth-flame optimization algorithm: a novel nature-inspired heuristic paradigm. Knowl-Based Syst 89:228–249

    Article  Google Scholar 

  29. Mirjalili S, Andrew L (2016) The whale optimization algorithm. Adv Eng Softw 95:51–67

    Article  Google Scholar 

  30. Xin-She Y (2010) A new metaheuristic bat-inspired algorithm. Nature inspired cooperative strategies for optimization (NICSO) Springer vol. 2010. Berlin, Heidelberg, pp 65–74

    Google Scholar 

  31. Xin-She Y (2010) Firefly algorithm, Levy flights and global optimization. Research and development in intelligent systems XXVI. Springer, London, pp 209–218

    Google Scholar 

  32. Mafarja M, Aljarah I, Faris H, Hammouri AI, Al-Zoubi AM, Mirjalili S (2019) Binary grasshopper optimisation algorithm approaches for feature selection problems. Expert Syst Appl 117:267–286

    Article  Google Scholar 

  33. Ahmed S, Mafarja M, Faris H, Aljarah I (2018) Feature selection using salp swarm algorithm with chaos. In: Proceedings of the 2nd international conference on intelligent systems, metaheuristics, and swarm intelligence ACM, pp 65–69

    Google Scholar 

  34. Faris H, Al-Zoubi AM, Heidari AA, Aljarah I, Mafarja M, Hassonah MA, Fujita H (2019) An intelligent system for spam detection and identification of the most relevant features based on evolutionary random weight networks. Inf Fusion 48:67–83

    Article  Google Scholar 

  35. Mafarja M, Aljarah I, Heidari AA, Faris H, Fournier-Viger P, Li X, Mirjalili S (2018) Binary dragonfly optimization for feature selection using time-varying transfer functions. Knowl-Based Syst 161:185–204

    Article  Google Scholar 

  36. Aljarah I, Mafarja M, Heidari AA, Faris H, Zhang Y, Mirjalili S (2018) Asynchronous accelerating multi-leader salp chains for feature selection. Appl Soft Comput 71:964–979

    Article  Google Scholar 

  37. Mafarja M, Aljarah I, Heidari AA, Hammouri AI, Faris H, Al-Zoubi AM, Mirjalili S (2018) Evolutionary population dynamics and grasshopper optimization approaches for feature selection problems. Knowl-Based Syst 145:25–45

    Article  Google Scholar 

  38. Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):67–82

    Article  Google Scholar 

  39. Wall M (1996) GAlib: A C++ library of genetic algorithm components. Mech Eng Dep Mass Inst Technol 87:54

    Google Scholar 

  40. Keijzer M et al (2001) Evolving objects: A general purpose evolutionary computation library. in: International conference on artificial evolution (Evolution Artificielle). Springer, Berlin, Heidelberg

    Google Scholar 

  41. Emmerich M, Hosenberg R (2001) TEA-a C++ library for the design of evolutionary algorithms. Universitätsbibliothek Dortmund

    Google Scholar 

  42. Harder R (2001) OpenTS: an open source java tabu search framework. INFORMS Annual Meeting, Miami

    Google Scholar 

  43. Bleuler S et al (2003) PISA-a platform and programming language independent interface for search algorithms. iN: International conference on evolutionary multi-criterion optimization. Springer, Berlin, Heidelberg

    Google Scholar 

  44. Cahon S, Melab N, Talbi E-G (2004) Paradiseo: a framework for the reusable design of parallel and distributed metaheuristics. J Heuristics 10(3):357–380

    Article  MATH  Google Scholar 

  45. Wagner S, Affenzeller M (2005) Heuristiclab: a generic and extensible optimization environment. Adaptive and natural computing algorithms. Springer, Vienna, pp 538–541

    Chapter  Google Scholar 

  46. Streichert F, Ulmer H (2005) JavaEvA-a java framework for evolutionary algorithms. In: Center for Bioinformatics Tübingen, University of Tübingen, Technical Report WSI-2005-06

    Google Scholar 

  47. Li Y, Yu S-M (2006) A unified optimization framework for real world problems. Lect Ser Comput Comput Sci 7:816–819

    Google Scholar 

  48. Pohlheim H (2007) Geatbx: genetic and evolutionary algorithm toolbox for use with matlab. H. Pohlheim, Berlin http://www.geatbx.com

  49. Pampara, G, Engelbrecht AP, Cloete T (2008) Cilib: a collaborative framework for computational intelligence algorithms-part I. In: IEEE International joint conference on neural networks, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence). IEEE

    Google Scholar 

  50. Ventura S et al (2008) JCLEC: a Java framework for evolutionary computation. Soft Comput 12(4):381–392

    Article  MathSciNet  Google Scholar 

  51. Perone CS (2009) Pyevolve: a Python open-source framework for genetic algorithms. Acm Sigevolution 4(1):12–20

    Article  Google Scholar 

  52. Kronfeld M, Planatscher H, Zell A (2010) The EvA2 optimization framework. In: International Conference on Learning and Intelligent Optimization. Springer, Berlin, Heidelberg

    Google Scholar 

  53. Durillo JJ, Nebro AJ (2011) JMetal: a Java framework for multi-objective optimization. Adv Eng Softw 42(10):760–771

    Article  Google Scholar 

  54. Weppenaar DVI, Vermaak HJ (2011) Solving planning problems with drools planner a tutorial. Interim: Interdiscip J 10(1):91–109

    Google Scholar 

  55. Lukasiewycz M et al (2011) Opt4J: a modular framework for meta-heuristic optimization. In: Proceedings of the 13th annual conference on Genetic and evolutionary computation. ACM

    Google Scholar 

  56. Fortin F-A et al (2012) DEAP: evolutionary algorithms made easy. J Mach Learn Res 13(Jul):2171–2175

    Google Scholar 

  57. Izzo, D (2012) Pygmo and pykep: open source tools for massively parallel optimization in astrodynamics (the case of interplanetary trajectory optimization). In: Proceedings of the fifth international conference on astrodynamics tools and techniques, ICATT

    Google Scholar 

  58. Luke, S (2017) ’ECJ then and now. In: Proceedings of the genetic and evolutionary computation conference companion. ACM

    Google Scholar 

  59. Tian Y et al (2017) PlatEMO: a MATLAB platform for evolutionary multi-objective optimization [educational forum]. IEEE Comput Intell Mag 12(4):73–87

    Article  Google Scholar 

  60. Kohavi R et al (1994) MLC++: a machine learning library in C++. In: Proceedings sixth international conference on tools with artificial intelligence. TAI 94. IEEE

    Google Scholar 

  61. Witten IH et al (1999) Weka: practical machine learning tools and techniques with Java implementations

    Google Scholar 

  62. Hanke M et al (2009) PyMVPA: a python toolbox for multivariate pattern analysis of fMRI data. Neuroinformatics 7(1):37–53

    Article  Google Scholar 

  63. Kachel A et al (2010) Infosel++: information based feature selection c++ library. In: International conference on artificial intelligence and soft computing. Springer, Berlin, Heidelberg

    Chapter  Google Scholar 

  64. Schaul T et al (2010) PyBrain. J Mach Learn Res 11(Feb):743–746

    Google Scholar 

  65. Alcalá-Fdez J et al (2011) Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J Mult-Valued Log Soft Comput 17

    Google Scholar 

  66. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O ... Vanderplas J (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12(Oct):2825–2830

    Google Scholar 

  67. Albanese D et al (2012) mlpy: Machine learning python. arXiv preprint arXiv:1202.6548 (2012)

  68. Curtin RR et al (2013) MLPACK: a scalable C++ machine learning library. J Mach Learn Res 14(Mar): 801–805

    Google Scholar 

  69. Demar J et al (2013) Orange: data mining toolbox in Python. J Mach Learn Res 14(1):2349–2353

    MATH  Google Scholar 

  70. Thüm T et al (2014) FeatureIDE: an extensible framework for feature-oriented software development. Sci Comput Program 79:70–85

    Article  Google Scholar 

  71. Soufan O et al (2015) DWFS: a wrapper feature selection tool based on a parallel genetic algorithm. PloS one 10(2):e0117988

    Article  Google Scholar 

  72. Roffo G (2016) Feature selection library (MATLAB toolbox). arXiv preprint arXiv:1607.01327 (2016)

  73. van Rossum G (1990–2004) Python programming language

    Google Scholar 

  74. Faris H, Aljarah I, Mirjalili S, Castillo PA, Guervés JJM (2016) EvoloPy: an open-source nature-inspired optimization framework in python. In: IJCCI (ECTA), pp 171–177

    Google Scholar 

  75. Faris H, Aljarah I, Al-Madi N, Mirjalili S (2016) Optimizing the learning process of feedforward neural networks using lightning search algorithm. Int J Artif Intell Tools 25(06):1650033

    Article  Google Scholar 

  76. Faris H, Aljarah I, Mirjalili S (2016) Training feedforward neural networks using multi-verse optimizer for binary classification problems. Appl Intell 45(2):322–332

    Article  Google Scholar 

  77. Aljarah I, Faris H, Mirjalili S (2018) Optimizing connection weights in neural networks using the whale optimization algorithm. Soft Comput 22(1):1–15

    Article  Google Scholar 

  78. Emary E, Zawbaa HM, Hassanien AE (2016) Binary grey wolf optimization approaches for feature selection. Neurocomputing 172(2016):371–381

    Article  Google Scholar 

  79. Xue B, Zhang M, Browne WN (2013) Novel initialisation and updating mechanisms in PSO for feature selection in classification. In: European conference on the applications of evolutionary computation. Springer, Berlin, Heidelberg

    Chapter  Google Scholar 

  80. Chuang L-Y et al (2008) Improved binary PSO for feature selection using gene expression data. Comput Biol Chem 32(1):29–38

    Article  MATH  Google Scholar 

  81. Huang CL, Dun JF (2008) A distributed PSO-SVM hybrid system with feature selection and parameter optimization. Appl Soft Comput 8:1381–1391

    Article  Google Scholar 

  82. Xue B, Zhang M, Browne WN (2012) New fitness functions in binary particle swarm optimisation for feature selection. In: IEEE congress on evolutionary computation (CEC2012) pp 2145–2152

    Google Scholar 

  83. Lin W et al (2016) ’An empirical study on the characteristics of Python fine-grained source code change types. In: 2016 IEEE international conference on software maintenance and evolution (ICSME). IEEE

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Seyedali Mirjalili .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Khurma, R.A., Aljarah, I., Sharieh, A., Mirjalili, S. (2020). EvoloPy-FS: An Open-Source Nature-Inspired Optimization Framework in Python for Feature Selection. In: Mirjalili, S., Faris, H., Aljarah, I. (eds) Evolutionary Machine Learning Techniques. Algorithms for Intelligent Systems. Springer, Singapore. https://doi.org/10.1007/978-981-32-9990-0_8

Download citation

Publish with us

Policies and ethics