Features Learning and Transformation Based on Deep Autoencoders

  • Eric Janvier
  • Thierry Couronne
  • Nistor GrozavuEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9949)


Tag recommendation has become one of the most important ways of an organization to index online resources like articles, movies, and music in order to recommend it to potential users. Since recommendation information is usually very sparse, effective learning of the content representation for these resources is crucial to accurate the recommendation.

One of the issue of this problem is features transformation or features learning. In one hand, the projection methods allows to find new representations of the data, but it is not adapted for non-linear data or very sparse datasets. In another hand, unsupervised feature learning with deep networks has been widely studied in the recent years. Despite the progress, most existing models would be fragile to non-Gaussian noises, outliers or high dimensional sparse data. In this paper, we propose a study on the use of deep denoising autoencoders and other dimensional reduction techniques to learn relevant representations of the data in order to increase the quality of the clustering model.

In this paper, we propose an hybrid framework with a deep learning model called stacked denoising autoencoder (SDAE), the SVD and Diffusion Maps to learn more effective content representation. The proposed framework is tested on real tag recommendation dataset which was validated by using internal clustering indexes and by experts.


  1. 1.
    Bishop, C.M., Svensén, M., Williams, C.K.I.: GTM: The generative topographic mapping. Neural Comput. 10(1), 215–234 (1998)CrossRefzbMATHGoogle Scholar
  2. 2.
    Saporta, G.: Probabilits, analyse des donnes et statistiques. Editions Technip (2006)Google Scholar
  3. 3.
    Golub, G.H., Kahan, W.: Calculating the singular values and pseudo-inverse of a matrix. SIAM J. Numer. Anal. 2, 205–224 (1965)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Grozavu, N., Bennani, Y., Labiod, L.: Feature space transformation for transfer learning. In: The 2012 International Joint Conference on Neural Networks (IJCNN), Brisbane, 10–15 June 2012, pp. 1–6 (2012)Google Scholar
  5. 5.
    Grozavu, N., Bennani, Y., Lebbah, M.: From variable weighting to cluster characterization in topographic unsupervised learning. In: Proceedings of International Joint Conference on Neural Network. IJCNN (2009)Google Scholar
  6. 6.
    Kang, L., Lee, K.T., Eun, J., Park, S.E., Choi, S.: Stacked denoising autoencoders for face pose normalization. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) Neural Information Processing. Theoretical Computer Science and General Issues, vol. 8227, pp. 241–248. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  7. 7.
    Van der Maaten, L., Postma, E., Van den Herik, H.: Dimensionality reduction: a comparative review. Technical report TiCC TR 2009–005 (2009)Google Scholar
  8. 8.
    Qi, Y., Wang, Y., Zheng, X., Wu, Z.: Robust feature learning by stacked autoencoder with maximum correntropy criterion. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2014, Florence, 4–9 May 2014, pp. 6716–6720 (2014). doi: 10.1109/ICASSp.2014.6854900
  9. 9.
    Roth, V., Lange, T.: Feature selection in clustering problems. In: Thrun, S., Saul, L., Schölkopf, B. (eds.) Advances in Neural Information Processing Systems, vol. 16. MIT Press, Cambridge (2003)Google Scholar
  10. 10.
    Kohonen, T.: Self-organizing Maps. Springer, Heidelberg (2001)CrossRefzbMATHGoogle Scholar
  11. 11.
    Verbeek, J., Vlassis, N., Krose, B.: Self-organizing mixture models. Neurocomputing 63, 99–123 (2005)CrossRefGoogle Scholar
  12. 12.
    Verleysen, M., Francois, D., Simon, G., Wertz, V.: On the effects of dimensionality on data analysis with neural networks. In: Mira, J., Álvarez, J.R. (eds.) IWANN 2003. LNCS, vol. 2687, pp. 105–112. Springer, Heidelberg (2003). doi: 10.1007/3-540-44869-1_14 CrossRefGoogle Scholar
  13. 13.
    Vesanto, J., Alhoniemi, E.: Clustering of the self-organizing map. IEEE Trans. Neural Netw. 11(3), 586–600 (2000)CrossRefGoogle Scholar
  14. 14.
    Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning. ICML 2008, pp. 1096–1103. ACM, New York (2008)Google Scholar
  15. 15.
    Wang, H., Shi, X., Yeung, D.Y.: Relational stacked denoising autoencoder for tag recommendation. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. AAAI 2015, pp. 3052–3058. AAAI Press (2015).

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Eric Janvier
    • 1
  • Thierry Couronne
    • 1
  • Nistor Grozavu
    • 2
    Email author
  1. 1.MindlytixSaint-MandFrance
  2. 2.LIPN CNRS UMR 7030, CNRS - Université Paris 13VilletaneuseFrance

Personalised recommendations