Transformation of Categorical Features into Real Using Low-Rank Approximations
Most of existing machine learning techniques can handle objects described by real but not categorical features. In this paper we introduce a simple unsupervised method for transforming categorical feature values into real ones. It is based on low-rank approximations of collaborative feature value frequencies. Once object descriptions are transformed, any common real-value machine learning technique can be applied for further data analysis. For example, it becomes possible to apply classic and powerful Random Forest predictor in supervised learning problems. Our experiments show that a combination of the proposed features transformation method with common real-value supervised algorithms leads to the results that are comparable to the state-of-the-art approaches like Factorization Machines.
KeywordsCategorical features Low-rank approximations Matrix factorization Feature extraction Factorization machines Sparse data
- 3.Breiman, L.: Mach. Learn. Random forests 45(1), 5–32 (2001)Google Scholar
- 5.Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.I.: Nonnegative matrix and tensor factorizations: applications to exploratory multi-way data analysis and blind source separation. Wiley, United Kingdom (2009)Google Scholar
- 6.D’yakonov, A.G.: Solution methods for classification problems with categorical attributes. Comput. Math. Model. 46, 1–21 (2015)Google Scholar
- 9.Hardy, M.A.: Regression with dummy variables. No. 93, Sage (1993)Google Scholar
- 10.Jurafsky, D., James, H.: Speech and language processing an introduction to natural language processing, computational linguistics, and speech (2000)Google Scholar
- 11.Koren, Y.: Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 426–434. ACM (2008)Google Scholar
- 14.Rendle, S.: Factorization machines. In: 2010 IEEE 10th International Conference on Data Mining (ICDM), pp. 995–1000. IEEE (2010)Google Scholar
- 15.Rendle, S.: Factorization machines with libfm. ACM Trans. Intell. Syst. Technol. (TIST) 3(3), 57 (2012)Google Scholar
- 16.Rendle, S., Freudenthaler, C., Schmidt-Thieme, L.: Factorizing personalized markov chains for next-basket recommendation. In: Proceedings of the 19th International Conference on World Wide Web, pp. 811–820. ACM (2010)Google Scholar
- 17.Rendle, S., Schmidt-Thieme, L.: Pairwise interaction tensor factorization for personalized tag recommendation. In: Proceedings of the Third ACM International Conference on Web Search and Data Mining, pp. 81–90. ACM (2010)Google Scholar
Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (http://creativecommons.org/licenses/by-nc/2.5/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.