Advertisement

Quantum-enhanced feature selection with forward selection and backward elimination

  • Zhimin He
  • Lvzhou Li
  • Zhiming Huang
  • Haozhen Situ
Article
  • 114 Downloads

Abstract

Feature selection is a well-known preprocessing technique in machine learning, which can remove irrelevant features to improve the generalization capability of a classifier and reduce training and inference time. However, feature selection is time-consuming, particularly for the applications those have thousands of features, such as image retrieval, text mining and microarray data analysis. It is crucial to accelerate the feature selection process. We propose a quantum version of wrapper-based feature selection, which converts a classical feature selection to its quantum counterpart. It is valuable for machine learning on quantum computer. In this paper, we focus on two popular kinds of feature selection methods, i.e., wrapper-based forward selection and backward elimination. The proposed feature selection algorithm can quadratically accelerate the classical one.

Keywords

Feature selection Forward selection Backward elimination Quantum machine learning Grover’s algorithm 

Notes

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Grant Nos. 61472452, 61772565, 61602116, 61502179), the Natural Science Foundation of Guangdong Province of China (Grant No. 2017A030313378), the Science and Technology Program of Guangzhou City of China (No. 201707010194), the Fundamental Research Funds for the Central Universities (No. 17lgzd29) and the Research Foundation for Talented Scholars of Foshan University (No. gg040996).

References

  1. 1.
    Ordonez, V., Han, X., Kuznetsova, P., Kulkarni, G., Mitchell, M., Yamaguchi, K., Stratos, K., Goyal, A., Dodge, J., Mensch, A., et al.: Large scale retrieval and generation of image descriptions. Int. J. Comput. Vis. 119, 46–59 (2016)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Li, P., Shrivastava, A., Moore, J.L., König, A.C.: Hashing algorithms for large-scale learning. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P.L., Pereira, F., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, pp. 2672–2680. Curran Associates, Inc., Granada, Spain (2011)Google Scholar
  3. 3.
    Lee, C.P., Leu, Y.: A novel hybrid feature selection method for microarray data analysis. Appl. Soft Comput. 11, 208–213 (2011)CrossRefGoogle Scholar
  4. 4.
    Blum, B., Baker, D., Jordan, M.I., Bradley, P., Das, R., Kim, D.E.: Feature selection methods for improving protein structure prediction with rosetta. In: Platt, J.C., Koller, D., Singer, Y., Roweis, S.T. (eds.) Advances in Neural Information Processing Systems, pp. 137–144. Curran Associates, Inc., Vancouver, British Columbia, Canada (2008)Google Scholar
  5. 5.
    Dash, M., Liu, H.: Feature selection for classification. Intell. Data Anal. 1, 131–156 (1997)CrossRefGoogle Scholar
  6. 6.
    Saeys, Y., Inza, I., Larrañaga, P.: A review of feature selection techniques in bioinformatics. Bioinformatics 23, 2507–2517 (2007)CrossRefGoogle Scholar
  7. 7.
    Giovannetti, V., Lloyd, S., Maccone, L.: Quantum private queries. Phys. Rev. Lett. 100(23), 230502 (2008)ADSMathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Wei, C.Y., Cai, X.Q., Liu, B., Wang, T., Gao, F.: A generic construction of quantum-oblivious-key-transfer-based private query with ideal database security and zero failure. IEEE Trans. Comput. 67, 2–8 (2018)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Schuld, M., Sinayskiy, I., Petruccione, F.: An introduction to quantum machine learning. Contemp. Phys. 56, 172–185 (2015)ADSCrossRefzbMATHGoogle Scholar
  10. 10.
    Li, K., Qiu, D., Li, L., Zheng, S., Rong, Z.: Application of distributed semi-quantum computing model in phase estimation. Inf. Process. Lett. 120, 23–29 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Yu, C.H., Gao, F., Wang, Q.L., Wen, Q.Y.: Quantum algorithm for association rules mining. Phys. Rev. A 94, 042311 (2016)ADSCrossRefGoogle Scholar
  12. 12.
    Biamonte, J., Wittek, P., Pancotti, N., Rebentrost, P., Wiebe, N., Lloyd, S.: Quantum machine learning. Nature 549, 195 (2017)ADSCrossRefGoogle Scholar
  13. 13.
    Ruan, Y., Xue, X., Liu, H., Tan, J., Li, X.: Quantum algorithm for k-nearest neighbors classification based on the metric of hamming distance. Int. J. Theor. Phys. 56, 3496–3507 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Ezhov, A.A., Ventura, D.: Quantum neural networks. Future Directions Intell. Syst. Inf. Sci. 45, 213–235 (2000)Google Scholar
  15. 15.
    Cheng, S., Chen, J., Wang, L.: Quantum entanglement: from quantum states of matter to deep learning. Physics 46(7), 416–423 (2017)Google Scholar
  16. 16.
    Aimeur, E., Brassard, G., Gambs, S.: Quantum speed-up for unsupervised learning. Mach. Learn. 90, 261–287 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Harrow, A.W., Hassidim, A., Lloyd, S.: Quantum algorithm for linear systems of equations. Phys. Rev. Lett. 103, 150502 (2009)ADSMathSciNetCrossRefGoogle Scholar
  18. 18.
    Weinstein, M., Meirer, F., Hume, A., Sciau, P., Shaked, G., Hofstetter, R., Persi, E., Mehta, A., Horn, D.: Analyzing big data with dynamic quantum clustering. arXiv:1310.2700 (2013)
  19. 19.
    Dong, D., Chen, C., Chen, Z.: Quantum reinforcement learning. IEEE Trans. Syst. Man Cybern. B 38, 1207–1220 (2008)CrossRefGoogle Scholar
  20. 20.
    Duan, B., Yuan, J., Liu, Y., Li, D.: Quantum algorithm for support matrix machines. Phys. Rev. A 96, 032301 (2017)ADSCrossRefGoogle Scholar
  21. 21.
    Chen, C., Dong, D., Chen, Z.: Quantum computation for action selection using reinforcement learning. Int. J. Quantum Inf. 4, 1071–1083 (2006)CrossRefzbMATHGoogle Scholar
  22. 22.
    Chatterjee, R., Yu, T.: Generalized coherent states, reproducing kernels, and quantum support vector machines. Quantum Inf. Comput. 17, 1292 (2017)MathSciNetGoogle Scholar
  23. 23.
    Adachi, S.H., Henderson, M.P.: Application of quantum annealing to training of deep neural networks. arXiv:1510.06356 (2015)
  24. 24.
    Wiebe, N., Kapoor, A., Svore, K.M.: Quantum deep learning. arXiv:1412.3489 (2014)
  25. 25.
    Pan, J., Cao, Y., Yao, X., Li, Z., Ju, C., Chen, H., Peng, X., Kais, S., Du, J.: Experimental realization of quantum algorithm for solving linear systems of equations. Phys. Rev. A 89, 022313 (2014)ADSCrossRefGoogle Scholar
  26. 26.
    Cai, X.D., Wu, D., Su, Z.E., Chen, M.C., Wang, X.L., Li, L., Liu, N.L., Lu, C.Y., Pan, J.W.: Entanglement-based machine learning on a quantum computer. Phys. Rev. Lett. 114, 110504 (2015)ADSCrossRefGoogle Scholar
  27. 27.
    Chen, J., Wang, L., Charbon, E.: A quantum-implementable neural network model. Quantum Inf. Process. 16, 245 (2017)ADSMathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Lu, S., Braunstein, S.L.: Quantum decision tree classifier. Quantum Inf. Process. 13, 757–770 (2014)ADSMathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Zhang, G., Hu, L., Jin, W.: Resemblance coefficient and a quantum genetic algorithm for feature selection. In: International Conference on Discovery Science, pp. 155–168. Springer (2004)Google Scholar
  30. 30.
    Rebentrost, P., Mohseni, M., Lloyd, S.: Quantum support vector machine for big data classification. Phys. Rev. Lett. 113, 130503 (2014)ADSCrossRefGoogle Scholar
  31. 31.
    Li, Z., Liu, X., Xu, N., Du, J.: Experimental realization of a quantum support vector machine. Phys. Rev. Lett. 114(14), 140504 (2015)ADSCrossRefGoogle Scholar
  32. 32.
    Liu, Y., Zhang, S.: Fast quantum algorithms for least squares regression and statistic leverage scores. In: International Workshop on Frontiers in Algorithmics, pp. 204–216. Springer (2015)Google Scholar
  33. 33.
    Doquire, G., Verleysen, M.: Mutual information-based feature selection for multilabel classification. Neurocomputing 122, 148–155 (2013)CrossRefzbMATHGoogle Scholar
  34. 34.
    Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artif. Intell. 97(1–2), 273–324 (1997)CrossRefzbMATHGoogle Scholar
  35. 35.
    Maldonado, S., Weber, R.: A wrapper method for feature selection using support vector machines. Inf. Sci. 179, 2208–2217 (2009)CrossRefGoogle Scholar
  36. 36.
    Wang, A., An, N., Chen, G., Li, L., Alterovitz, G.: Accelerating wrapper-based feature selection with k-nearest-neighbor. Knowl. Based Syst. 83, 81–91 (2015)CrossRefGoogle Scholar
  37. 37.
    Apolloni, J., Leguizamón, G., Alba, E.: Two hybrid wrapper-filter feature selection algorithms applied to high-dimensional microarray experiments. Appl. Soft Comput. 38, 922–932 (2016)CrossRefGoogle Scholar
  38. 38.
    Mao, K.Z.: Orthogonal forward selection and backward elimination algorithms for feature subset selection. IEEE Trans. Syst. Man Cybern. B (Cybernetics) 34, 629–634 (2004)CrossRefGoogle Scholar
  39. 39.
    Hu, Y., Lan, W., Miller, D.: Handling high-dimension (high-feature) microrna data. In: Bioinformatics in MicroRNA Research. Methods in Molecular Biology, vol. 1617, pp. 179–186 (2017)Google Scholar
  40. 40.
    Durr, C., Hoyer, P.: A quantum algorithm for finding the minimum. arXiv:quant-ph/9607014 (1996)
  41. 41.
    Lloyd, S., Mohseni, M., Rebentrost, P.: Quantum principal component analysis. Nat. Phys. 10, 631–633 (2014)CrossRefGoogle Scholar
  42. 42.
    Bennett, C.H.: Logical reversibility of computation. IBM J. Res. Dev. 17, 525–532 (1973)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Boyer, M., Brassard, G., Høyer, P., Tapp, A.: Tight bounds on quantum searching. arXiv:quant-ph/9605034 (1996)

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Electronic and Information EngineeringFoshan UniversityFoshanChina
  2. 2.School of Data and Computer ScienceSun Yat-Sen UniversityGuangzhouChina
  3. 3.School of Economics and ManagementWuyi UniversityJiangmenChina
  4. 4.College of Mathematics and InformaticsSouth China Agricultural UniversityGuangzhouChina

Personalised recommendations