Improving the Data Quality of the MovieLens Dataset Using Dimensionality Reduction Techniques

  • Hagar ElFikyEmail author
  • Wedad Hussein
  • Rania El Gohary
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1153)


The presence of recommendation systems within the current era is not controlled or even managed like before, for providing the optimum services to users. For that reason, these services need to be in an ideal and complete state since the real-world data is often not that much perfect. That optimum state could not be achieved without the interaction of preprocessing method. Preprocessing can offer a lot of insights about the data and its structure and quality. Preprocessing techniques ensure that the dataset has complete, consistent and integrant properties for further analysis. Dimensionality reduction is this paper is discussed as one of the preprocessing stages using the most popular matrix factorization methods. We have also made an experiment to show how the dimensionality reduction algorithms can overcome any constrained computational resources and prove better performance and efficiency for such demanding recommendation systems.


Preprocessing MovieLens Data quality Dimensionality reduction 


  1. 1.
  2. 2. Accessed 18 Oct 2019
  3. 3.
    Harper, F.M., Konstan, J.A.: The MovieLens datasets: history and context. ACM Trans. Interact. Intell. Syst. 5(4), 19:1–19:19 (2015)CrossRefGoogle Scholar
  4. 4.
  5. 5.
    Al-Garadi, M.A., Mohamed, A., Al-Ali, A., Du, X., Guizani, M.: A survey of machine and deep learning methods for Internet of Things (IoT) security. arXiv preprint (2018)Google Scholar
  6. 6.
    Vozalis, M.G., Konstantinos, G.M.: A recommender system using principal component analysis. In: Published in 11th Panhellenic Conference in Informatics, pp. 271–283. Citeseer (2007)Google Scholar
  7. 7.
    Juvonen, A., Sipola, T., Hämäläinen, T.: Online anomaly detection using dimensionality reduction techniques for HTTP log analysis. Comput. Netw. 91, 46–56 (2015)CrossRefGoogle Scholar
  8. 8.
    Lee, Y.J., Yeh, Y.R., Wang, Y.C.F.: Anomaly detection via online oversampling principal component analysis. IEEE Trans. Knowl. Data Eng. 25(7), 1460–1470 (2013)CrossRefGoogle Scholar
  9. 9.
    Bokde, D., Girase, S., Mukhopadhyay, D.: Matrix factorization model in collaborative filtering algorithms: a survey. In: Proceedings of 4th International Conference on Advances in Computing, Communication and Control, pp. 136–146 (2015)Google Scholar
  10. 10.
    Zhou, X., He, J., Huang, G., Zhang, Y.: SVD-based incremental approaches for recommender systems. J. Comput. Syst. Sci. 81(4), 717–733 (2014)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Zarzour, H., Al-Sharif, Z., Al-Ayyoub, M., Jararweh, Y.: A new collaborative filtering recommendation algorithm based on dimensionality reduction and clustering techniques. In: 9th International Conference on Information and Communication Systems (ICICS), Irbid, pp. 102–106 (2018)Google Scholar
  12. 12.
    Yang, H., Wang, Y.X., Xia, J.J., Zhang, Y.: China GDP prediction on traffic and transportation by PCA. In: Proceeding of the 24th International Conference on Industrial Engineering and Engineering Management, Singapore, pp. 388–394 (2018)Google Scholar
  13. 13.
    Vasan, K., Surendiran, B.: Dimensionality reduction using principal component analysis for network intrusion detection. Perspect. Sci. 8, 510–512 (2016)CrossRefGoogle Scholar
  14. 14.
    Nilashi, M., Ibrahim, O., Bagherifard, K.: A recommender system based on collaborative filtering using ontology and dimensionality reduction techniques. Expert Syst. Appl. 92, 507–520 (2018)CrossRefGoogle Scholar
  15. 15.
    Deldjoo, Y., Constantin, M.G., Eghbal-Zadeh, H., Ionescu, B., Schedl, M., Cremonesi, P.: Audiovisual encoding of multimedia content for enhancing movie recommendations. In: Proceedings of the 12th ACM Conference on Recommender Systems, RecSys 2018, Vancouver, BC, Canada, 2–7 October 2018, pp. 455–459 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Information Systems, Faculty of Computer and Information SciencesAin Shams UniversityCairoEgypt

Personalised recommendations