Features Learning and Transformation Based on Deep Autoencoders
Tag recommendation has become one of the most important ways of an organization to index online resources like articles, movies, and music in order to recommend it to potential users. Since recommendation information is usually very sparse, effective learning of the content representation for these resources is crucial to accurate the recommendation.
One of the issue of this problem is features transformation or features learning. In one hand, the projection methods allows to find new representations of the data, but it is not adapted for non-linear data or very sparse datasets. In another hand, unsupervised feature learning with deep networks has been widely studied in the recent years. Despite the progress, most existing models would be fragile to non-Gaussian noises, outliers or high dimensional sparse data. In this paper, we propose a study on the use of deep denoising autoencoders and other dimensional reduction techniques to learn relevant representations of the data in order to increase the quality of the clustering model.
In this paper, we propose an hybrid framework with a deep learning model called stacked denoising autoencoder (SDAE), the SVD and Diffusion Maps to learn more effective content representation. The proposed framework is tested on real tag recommendation dataset which was validated by using internal clustering indexes and by experts.
- 2.Saporta, G.: Probabilits, analyse des donnes et statistiques. Editions Technip (2006)Google Scholar
- 4.Grozavu, N., Bennani, Y., Labiod, L.: Feature space transformation for transfer learning. In: The 2012 International Joint Conference on Neural Networks (IJCNN), Brisbane, 10–15 June 2012, pp. 1–6 (2012)Google Scholar
- 5.Grozavu, N., Bennani, Y., Lebbah, M.: From variable weighting to cluster characterization in topographic unsupervised learning. In: Proceedings of International Joint Conference on Neural Network. IJCNN (2009)Google Scholar
- 6.Kang, L., Lee, K.T., Eun, J., Park, S.E., Choi, S.: Stacked denoising autoencoders for face pose normalization. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) Neural Information Processing. Theoretical Computer Science and General Issues, vol. 8227, pp. 241–248. Springer, Heidelberg (2013)CrossRefGoogle Scholar
- 7.Van der Maaten, L., Postma, E., Van den Herik, H.: Dimensionality reduction: a comparative review. Technical report TiCC TR 2009–005 (2009)Google Scholar
- 8.Qi, Y., Wang, Y., Zheng, X., Wu, Z.: Robust feature learning by stacked autoencoder with maximum correntropy criterion. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2014, Florence, 4–9 May 2014, pp. 6716–6720 (2014). doi: 10.1109/ICASSp.2014.6854900
- 9.Roth, V., Lange, T.: Feature selection in clustering problems. In: Thrun, S., Saul, L., Schölkopf, B. (eds.) Advances in Neural Information Processing Systems, vol. 16. MIT Press, Cambridge (2003)Google Scholar
- 14.Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning. ICML 2008, pp. 1096–1103. ACM, New York (2008)Google Scholar
- 15.Wang, H., Shi, X., Yeung, D.Y.: Relational stacked denoising autoencoder for tag recommendation. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. AAAI 2015, pp. 3052–3058. AAAI Press (2015). http://dl.acm.org/citation.cfm?id=2888116.2888141