Skip to main content
SpringerLink
Log in

Algorithmic cache of sorted tables for feature selection

Speeding up methods based on consistency and information theory measures

  • Published:
Data Mining and Knowledge Discovery Aims and scope Submit manuscript

Abstract

Feature selection is a mechanism used in Machine Learning to reduce the complexity and improve the speed of the learning process by using a subset of features from the data set. There are several measures which are used to assign a score to a subset of features and, therefore, are able to compare them and decide which one is the best. The bottle neck of consistence measures is having the information of the different examples available to check their class by groups. To handle it, this paper proposes the concept of an algorithmic cache, which stores sorted tables to speed up the access to example information. The work carries out an empirical study using 34 real-world data sets and four representative search strategies combined with different table caching strategies and three sorting methods. The experiments calculate four different consistency and one information measures, showing that the proposed sorted tables cache reduces computation time and it is competitive with hash table structures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  • Almuallim H, Dietterich TG (1991) Learning with many irrelevant features. In: Proceedings of the ninth national conference on artificial intelligence. AAAI Press, pp 547–552

  • Almuallim H, Dietterich TG (1994) Learning boolean concepts in the presence of many irrelevant features. Artif Intell 69(1–2):279–305

    Article  MathSciNet  MATH  Google Scholar 

  • Arauzo-Azofra A, Beníez JM, Castro JL (2008) Consistency measures for feature selection. J Intell Inf Syst 30(3):273–292. https://doi.org/10.1007/s10844-007-0037-0

    Article  Google Scholar 

  • Arauzo-Azofra A, Aznarte JL, Benítez JM (2011) Empirical study of feature selection methods based on individual feature evaluation for classification problems. Expert Syst Appl 38(7):8170–8177. https://doi.org/10.1016/j.eswa.2010.12.160

    Article  Google Scholar 

  • Atallah MJ, Fox S (eds) (1998) Algorithms and theory of computation handbook, 1st edn. CRC Press Inc, Boca Raton

    MATH  Google Scholar 

  • Auger N, Nicaud C, Pivoteau C (2015) Merge Strategies: from Merge Sort to TimSort, working paper or preprint. https://hal-upec-upem.archives-ouvertes.fr/hal-01212839. Accessed 06 Mar 2019.

  • Bharti KK, Singh PK (2015) Hybrid dimension reduction by integrating feature selection with feature extraction method for text clustering. Expert Syst Appl 42(6):3105–3114

    Article  Google Scholar 

  • Chen X, Fang T, Huo H, Li D (2011) Graph-based feature selection for object-oriented classification in vhr airborne imagery. IEEE Trans Geosci Remote Sens 49(1):353–365

    Article  Google Scholar 

  • Cover TM, Thomas JA (1991) Elements of information theory. Wiley, New York

    Book  MATH  Google Scholar 

  • Dash M, Liu H (2003) Consistency-based search in feature selection. Artif Intell 151(1–2):155–176. https://doi.org/10.1016/S0004-3702(03)00079-1

    Article  MathSciNet  MATH  Google Scholar 

  • Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  • Demšar J, Curk T, Erjavec A, Gorup Črt, Hočevar T, Milutinovič M, Možina M, Polajnar M, Toplak M, Starič A, Štajdohar M, Umek L, Žagar L, Žbontar J, Žitnik M, Zupan B (2013) Orange: data mining toolbox in python. J Mach Learn Res 14:2349–2353

    MATH  Google Scholar 

  • Fortin FA, De Rainville FM, Gardner MA, Parizeau M, Gagné C (2012) DEAP: evolutionary algorithms made easy. J Mach Learn Res 13:2171–2175

    MathSciNet  MATH  Google Scholar 

  • Frigo M, Leiserson CE, Prokop H, Ramachandran S (2012) Cache-oblivious algorithms. ACM Trans Algorithms 8(1):4:1–4:22. https://doi.org/10.1145/2071379.2071383

    Article  MathSciNet  MATH  Google Scholar 

  • García S, Luengo J, Herrera F (2016) Data preprocessing in data mining. Springer, Berlin

    Google Scholar 

  • Geng X, Liu TY, Qin T, Li H (2007) Feature selection for ranking. In: Proceedings of the 30th annual international ACM SIGIR conference on research and development in information retrieval. ACM, pp 407–414

  • Gui J, Sun Z, Ji S, Tao D, Tan T (2017) Feature selection based on structured sparsity: a comprehensive study. IEEE Trans Neural Netw Learn Syst 28(7):1490–1507

    Article  MathSciNet  Google Scholar 

  • Hastie T, Tibshirani R, Friedman J (2001) The elements of statistical learning. Springer series in statistics. Springer, New York Inc, New York

    Book  MATH  Google Scholar 

  • Kern R (2016) rkern/line\_profiler. https://github.com/rkern/line_profiler

  • Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. SCIENCE 220(4598):671–680

    Article  MathSciNet  MATH  Google Scholar 

  • Kohavi R, John GH (1997) Wrappers for feature subset selection. Artif Intell 97(1–2):273–324

    Article  MATH  Google Scholar 

  • Koprinska I, Rana M, Agelidis VG (2015) Correlation and instance based feature selection for electricity load forecasting. Knowl Based Syst 82:29–40

    Article  Google Scholar 

  • Kowarschik M, Weiß C (2003) An overview of cache optimization techniques and cache-aware numerical algorithms. In: Algorithms for memory hierarchies, pp 213–232

  • Lanaro G (2013) Python high performance programming. Packt Publishing, Birmingham

    Google Scholar 

  • Liu M, Zhang D (2016) Pairwise constraint-guided sparse learning for feature selection. IEEE Trans Cybern 46(1):298–310

    Article  MathSciNet  Google Scholar 

  • Marill T, Green D (1963) On the effectiveness of receptors in recognition systems. IEEE Trans Inf Theory 9(1):11–17

    Article  Google Scholar 

  • Molina L, Belanche L, Nebot A (2002) Feature selection algorithms: a survey and experimental evaluation. In: Proceedings 2002 IEEE international conference on data mining, 2002. ICDM 2002, pp 306–313. https://doi.org/10.1109/ICDM.2002.1183917

  • Newman CBD, Merz C (1998) UCI repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html. Accessed 25 Nov 2017

  • Onan A (2015) A fuzzy-rough nearest neighbor classifier combined with consistency-based subset evaluation and instance selection for automated diagnosis of breast cancer. Expert Syst Appl 42(20):6844–6852

    Article  Google Scholar 

  • Pawlak Z (1982) Rough sets. Int J Comput Inf Sci 11:341–356

    Article  MATH  Google Scholar 

  • Qian W, Shu W (2015) Mutual information criterion for feature selection from incomplete data. Neurocomputing 168:210–220. https://doi.org/10.1016/j.neucom.2015.05.105

    Article  Google Scholar 

  • Shin K, Miyazaki S (2016) A fast and accurate feature selection algorithm based on binary consistency measure. Comput Intell 32(4):646–667. https://doi.org/10.1111/coin.12072

    Article  MathSciNet  Google Scholar 

  • Shin K, Fernandes D, Miyazaki S (2011) Consistency measures for feature selection: A formal definition, relative sensitivity comparison, and a fast algorithm. In: Walsh T (ed) IJCAI, IJCAI/AAAI, pp 1491–1497. http://dblp.uni-trier.de/db/conf/ijcai/ijcai2011.html

  • Song Q, Ni J, Wang G (2013) A fast clustering-based feature subset selection algorithm for high-dimensional data. IEEE Trans Knowl Data Eng 25(1):1–14

    Article  Google Scholar 

  • Whitney AW (1971) A direct method of nonparametric measurement selection. IEEE Trans Comput 20(9):1100–1103. https://doi.org/10.1109/T-C.1971.223410

    Article  MATH  Google Scholar 

  • Zhao Z, Liu H (2007) Spectral feature selection for supervised and unsupervised learning. In: Proceedings of the 24th international conference on machine learning. ACM, pp 1151–1157

  • Zheng K, Wang X (2018) Feature selection method with joint maximal information entropy between features and class. Pattern Recognit 77:20–29. https://doi.org/10.1016/j.patcog.2017.12.008

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Engineering Sciences, Universidad de Cordoba, Córdoba, Spain

    Antonio Arauzo-Azofra, Alfonso Jiménez-Vílchez, José Molina-Baena & María Luque-Rodriguez

Authors
  1. Antonio Arauzo-Azofra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alfonso Jiménez-Vílchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. José Molina-Baena
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. María Luque-Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to María Luque-Rodriguez.

Additional information

Responsible editor: Dr. Fei Wang.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research is partially supported by Projects: TIN2013-47210-P of the Ministerio de Economía y Competitividad (Spain), P12-TIC-2958 and TIC1582 of the Consejeria de Economia, Innovacion, Ciencia y Empleo from Junta de Andalucia (Spain).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Arauzo-Azofra, A., Jiménez-Vílchez, A., Molina-Baena, J. et al. Algorithmic cache of sorted tables for feature selection. Data Min Knowl Disc 33, 964–994 (2019). https://doi.org/10.1007/s10618-019-00620-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10618-019-00620-8

Keywords

Search

Navigation