A Weighted KNN Algorithm Based on Entropy Method

  • Hui ZhangEmail author
  • Kaihu Hou
  • Zhou Zhou
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 924)


Aiming at the problem that the classification accuracy of K-nearest neighbor algorithm is not high, this paper proposes a K-nearest neighbor algorithm that uses the weighted entropy method of Extreme value (EEM-KNN algorithm). The entropy method assigns weight to the sample’s feature index, and then introduces the weight of the feature index when calculating the distance between the query sample vector and the training sample vector. The four groups of classification data sets are used as test samples to test the effectiveness of the improved KNN algorithm, it also compares the difference between the improved algorithm and the traditional algorithm under different K values. Algorithms are implemented and tested on the Jupyter Notebook interactive platform. The improved KNN algorithm is verified by experiments, and the classification accuracy is improved.


KNN algorithm Distance metric Entropy method Weighting 


  1. 1.
    Li, L., Weinberg, C.R., Darden, T.A., et al.: Gene selection for sample classification based on gene expression data: study of sensitivity to choice of parameters of the GA/KNN method. Bioinformatics 17(12), 1131 (2001)CrossRefGoogle Scholar
  2. 2.
    Beaudoin, A., Bernierp, Y., Guindon, L., et al.: Mapping attributes of Canada’s forests at moderate resolution through. Can. J. Forest Res. 44(5), 521–532 (2014)CrossRefGoogle Scholar
  3. 3.
    Pandya, D.H., Upadhyay, S.H., Harsha, S.P.: Fault diagnosis of rolling element bearing with intrinsic mode function of acoustic emission data using APF-KNN. Expert Syst. Appl. 40(10), 4137–4145 (2013)CrossRefGoogle Scholar
  4. 4.
    Aslam, M.W., Zhu, Z., Nandi, A.K.: Automatic modulation classification using combination of genetic programming and KNN. IEEE Trans. Wirel. Commun. 11(8), 2742–2750 (2012)Google Scholar
  5. 5.
    Steenwijk, M.D., Pouwels, P.J., Daams, M., et al.: Accurate white matter lesion segmentation by k nearest neighbor classification with tissue type priors (kNN-TTPs). Neuroimage Clin. 3(3), 462–469 (2013)CrossRefGoogle Scholar
  6. 6.
    Homaeinezhad, M.R., Atyabi, S.A., Tavakkoli, E., et al.: ECG arrhythmia recognition via a neuro-SVM-KNN hybrid classifier with virtual QRS image-based geometrical features. Expert Syst. Appl. 39(2), 2047–2058 (2012)CrossRefGoogle Scholar
  7. 7.
    Tan, S.: An effective refinement strategy for KNN text classifier. Expert Syst. Appl. 30(2), 290–298 (2006)CrossRefGoogle Scholar
  8. 8.
    Ren, J.T., Zhuo, X.L., Xu, S.C.: PSO based feature weighting algorithm for KNN. Comput. Sci. 34(5), 187–189 (2007)Google Scholar
  9. 9.
    Wang, Z.; Wang, K.: Improve KNN algorithm based on entropy method. Comput. Eng. Appl., 45, 129–131,160 (2009)Google Scholar
  10. 10.
    Zhou, J., Liu, J.: KNN algorithm based on feature entropy correlation difference. Comput. Eng. 37(17), 146–148 (2011)Google Scholar
  11. 11.
    Tong, X., Zhou, Z.: Enhancement of K-nearest neighbor algorithm based on information entropy of attribute value. Comput. Eng. Appl. 46(3), 115–117 (2010)Google Scholar
  12. 12.
    Sun, K., Gong, Y., Deng, Z.: An efficient K-value adaptive SA-KNN algorithm. Comput. Eng. Sci. 37(10), 1965–1970 (2015)Google Scholar
  13. 13.
    Zhou, J., Liu, J.: Improved K-nearest neighbor algorithm for feature union entropy. Comput. Appl., 31(7), 1785–1788,1792 (2011)Google Scholar
  14. 14.
    Wei, L., Zhao, X., Zhou, X.: An enhanced entropy-K-nearest neighbor algorithm based on attribute reduction. In: 2014 4th International Conference on Computer Engineering and Networks, pp 381–388 (2014)Google Scholar
  15. 15.
    Xiao, X., Ding, H.: Enhancement of K-nearest neighbor algorithm based on weighted entropy of attribute value. In: 2012 5th International Conference on BioMedical Engineering and Informatics, Chongqing, pp. 1261–1264 (2012)Google Scholar
  16. 16.
    Zhu, X., Wei, G.: Discussion on the non-dimensional method’s good standard in entropy method. Stat. Decis. 02, 12–15 (2015)Google Scholar
  17. 17.
    Mejdoub, M., Amar, C.B.: Classification improvement of local feature vectors over the KNN algorithm. Multimed. Tools Appl. 64, 197–218 (2013)CrossRefGoogle Scholar
  18. 18.
    Zou, Z.H., Yun, Y., Sun, J.N.: Entropy method for determination of weight of evaluating indicators in fuzzy synthetic evaluation for water quality assessment. J. Environ. Sci. 18(5), 1020–1023 (2006)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Kunming University of Science and TechnologyKunmingChina

Personalised recommendations