Entropic Feature Discrimination Ability for Pattern Classification Based on Neural IAL

  • Ting Wang
  • Sheng-Uei Guan
  • Fei Liu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7368)


Incremental Attribute Learning (IAL) is a novel machine learning strategy, where features are gradually trained in one or more according to some orderings. In IAL, feature ordering is a special preprocessing. Apart from time-consuming contribution-based feature ordering methods, feature ordering also can be derived by filter criteria. In this paper, a novel criterion based on Discriminability, a distribution-based metric, and Entropy is presented to give ranks for feature ordering, which has been validated in both two-category and multivariable classification problems by neural networks. Final experimental results show that the new metric is not only applicable for IAL, but also able to obtain better performance in lower error rates.


neural networks incremental attribute learning feature ordering entropy discrimination ability 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Liu, H.: Evolving feature selection. IEEE Intelligent Systems 20(6), 64–76 (2005)CrossRefGoogle Scholar
  2. 2.
    Weiss, S.H., Indurkhya, N.: Predictive data mining: a practical guide. Morgan Kaufmann Publishers, San Francisco (1998)zbMATHGoogle Scholar
  3. 3.
    Chao, S., Wong, F.: An incremental decision tree learning methodology regarding attributes in medical data mining. In: Proc. of the 8th Int’l Conf. on Machine Learning and Cybernetics, Baoding, pp. 1694–1699 (2009)Google Scholar
  4. 4.
    Agrawal, R.K., Bala, R.: Incremental Bayesian classification for multivariate normal distribution data. Pattern Recognition Letters 29(13), 1873–1876 (2008)CrossRefGoogle Scholar
  5. 5.
    Guan, S.U., Liu, J.: Feature selection for modular networks based on incremental training. Journal of Intelligent Systems 14(4), 353–383 (2005)CrossRefGoogle Scholar
  6. 6.
    Zhu, F., Guan, S.U.: Ordered incremental training for GA-based classifiers. Pattern Recognition Letters 26(14), 2135–2151 (2005)CrossRefGoogle Scholar
  7. 7.
    Guan, S.U., Li, S.: Incremental learning with respect to new incoming input attributes. Neural Processing Letters 14(3), 241–260 (2001)zbMATHCrossRefGoogle Scholar
  8. 8.
    Guan, S.U., Li, S.: Parallel growing and training of neural networks using output parallelism. IEEE Trans. on Neural Networks 13(3), 542–550 (2002)CrossRefGoogle Scholar
  9. 9.
    Bermejo, P., de la Ossa, L., Gámez, J.A., Puerta, J.M.: Fast wrapper feature subset selection in high-dimensional datasets by means of filter re-ranking. Knowledge-Based Systems 25(1), 35–44 (2012)CrossRefGoogle Scholar
  10. 10.
    Peng, H., Long, F., Ding, C.: Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(8), 1226–1238 (2005)CrossRefGoogle Scholar
  11. 11.
    Wang, T., Guan, S.U., Liu, F.: Ordered Incremental Attribute Learning based on mRMR and Neural Networks. International Journal of Design, Analysis and Tools for Integrated Circuits and Systems 2(2), 86–90 (2011)Google Scholar
  12. 12.
    Wang, T., Guan, S.-U., Liu, F.: Feature Discriminability for Pattern Classification Based on Neural Incremental Attribute Learning. In: Wang, Y.L., Li, T.R. (eds.) ISKE 2011. AISC, vol. 122, pp. 275–280. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  13. 13.
    Guan, S.U., Liu, J.: Incremental neural network training with an increasing input dimension. Journal of Intelligent Systems 13(1), 43–69 (2004)CrossRefGoogle Scholar
  14. 14.
    Ripley, B.D.: Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge (1996)zbMATHGoogle Scholar
  15. 15.
    Guan, S.U., Liu, J.: Incremental Ordered Neural Network Training. Journal of Intelligent Systems 12(3), 137–172 (2002)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ting Wang
    • 1
    • 2
  • Sheng-Uei Guan
    • 2
  • Fei Liu
    • 3
  1. 1.Department of Computer ScienceUniversity of LiverpoolLiverpoolUK
  2. 2.Department of Computer Science and Software EngineeringXi’an Jiaotong-Liverpool UniversitySuzhouChina
  3. 3.Department of Computer Science & Computer EngineeringLa Trobe UniversityVictoriaAustralia

Personalised recommendations