Design of normalized fractional SGD computing paradigm for recommender systems

Abstract

Fast and effective recommender systems are fundamental to fulfill the growing requirements of the e-commerce industry. The strength of matrix factorization procedure based on stochastic gradient descent (SGD) algorithm is exploited widely to solve the recommender system problem. Modern computing paradigms are designed by utilizing the concept of fractional gradient in standard SGD and outperform the standard counterpart. The performance of fractional SGD improves considerably by adaptively tuning the learning rate parameter. A nonlinear computing paradigm based on normalized version of fractional SGD is developed in this paper to investigate the adaptive behavior of learning rate with novel application to recommender systems. The accuracy of the proposed approach is verified through root mean square error metric by using different latent features, learning rates, fractional orders and datasets. The superiority of the designed method is validated through comparison with the state-of-the-art counterparts.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  1. 1.

    Bobadilla J, Ortega F, Hernando A, Gutiérrez A (2013) Recommender systems survey. Knowl Based Syst 46:109–132

    Google Scholar 

  2. 2.

    Logesh R, Subramaniyaswamy V, Malathi D, Sivaramakrishnan N, Vijayakumar V (2019) Enhancing recommendation stability of collaborative filtering recommender system through bio-inspired clustering ensemble method. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3891-5

    Article  Google Scholar 

  3. 3.

    Katarya R, Verma OP (2018) Recommender system with grey wolf optimizer and FCM. Neural Comput Appl 30(5):1679–1687

    Google Scholar 

  4. 4.

    Yin H, Wang W, Chen L, Du X, Nguyen QVH, Huang Z (2018) Mobi-SAGE-RS: a sparse additive generative model-based mobile application recommender system. Knowl Based Syst 157:68–80

    Google Scholar 

  5. 5.

    Zhou H, Hirasawa K (2019) Evolving temporal association rules in recommender system. Neural Comput Appl 31(7):2605–2619

    Google Scholar 

  6. 6.

    Katarya R (2018) Movie recommender system with metaheuristic artificial bee. Neural Comput Appl 30(6):1983–1990

    Google Scholar 

  7. 7.

    Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 8:30–37

    Google Scholar 

  8. 8.

    Luo X, Zhou M (2019) Effects of extended stochastic gradient descent algorithms on improving latent factor-based recommender systems. IEEE Robot Autom Lett 4(2):618–624

    Google Scholar 

  9. 9.

    Khan ZA, Chaudhary NI, Zubair S (2019) Fractional stochastic gradient descent for recommender systems. Electron Mark 29:275–285

    Google Scholar 

  10. 10.

    Chaudhary NI, Ahmed M, Khan ZA, Zubair S, Raja MAZ, Dedovic N (2018) Design of normalized fractional adaptive algorithms for parameter estimation of control autoregressive autoregressive systems. Appl Math Model 55:698–715

    MathSciNet  MATH  Google Scholar 

  11. 11.

    Chaudhary NI, Zubair S, Raja MAZ, Dedovic N (2019) Normalized fractional adaptive methods for nonlinear control autoregressive systems. Appl Math Model 66:457–471

    MathSciNet  MATH  Google Scholar 

  12. 12.

    Zhang Y, Abbas H, Sun Y (2019) Smart e-commerce integration with recommender systems. Electron Mark 29(2):219–220

    Google Scholar 

  13. 13.

    Kunaver M, Požrl T (2017) Diversity in recommender systems—a survey. Knowl Based Syst 123:154–162

    Google Scholar 

  14. 14.

    Aggarwal CC (2016) An introduction to recommender systems. In: Recommender systems. Springer, pp 1–28

  15. 15.

    Narayan S, Sathiyamoorthy E (2019) A novel recommender system based on FFT with machine learning for predicting and identifying heart diseases. Neural Comput Appl 31(1):93–102

    Google Scholar 

  16. 16.

    Xie X, Wang B (2018) Web page recommendation via twofold clustering: considering user behavior and topic relation. Neural Comput Appl 29(1):235–243

    MathSciNet  Google Scholar 

  17. 17.

    He C, Parra D, Verbert K (2016) Interactive recommender systems: a survey of the state of the art and future research challenges and opportunities. Expert Syst Appl 56:9–27

    Google Scholar 

  18. 18.

    Hong B, Yu M (2018) A collaborative filtering algorithm based on correlation coefficient. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3857-7

    Article  Google Scholar 

  19. 19.

    Wu X, Yuan X, Duan C, Wu J (2019) A novel collaborative filtering algorithm of machine learning by integrating restricted Boltzmann machine and trust information. Neural Comput Appl 31(9):4685–4692

    Google Scholar 

  20. 20.

    Schafer JB, Konstan JA, Riedl J (2001) E-commerce recommendation applications. In: Applications of data mining to electronic commerce. Springer, pp 115–153

  21. 21.

    Wang J, De Vries AP, Reinders MJ (2006) Unifying user-based and item-based collaborative filtering approaches by similarity fusion. In: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 501–508

  22. 22.

    Salakhutdinov R, Mnih A (2007) Probabilistic matrix factorization. In: Nips, vol 1, no. 1, pp 2-1

  23. 23.

    Wang S, Tang J, Wang Y, Liu H (2015) Exploring implicit hierarchical structures for recommender systems. In: IJCAI, pp 1813–1819

  24. 24.

    Gao L, Li C (2008) Hybrid personalized recommended model based on genetic algorithm. In: 4th International conference on wireless communications, networking and mobile computing, 2008. WiCOM’08. IEEE, pp 1–4

  25. 25.

    Wang S, Gong M, Li H, Yang J, Wu Y (2017) Memetic algorithm based location and topic aware recommender system. Knowl Based Syst 131:125–134

    Google Scholar 

  26. 26.

    Roh TH, Oh KJ, Han I (2003) The collaborative filtering recommendation based on SOM cluster-indexing CBR. Expert Syst Appl 25(3):413–423

    Google Scholar 

  27. 27.

    Park MH, Hong JH, Cho SB (2007) Location-based recommendation system using bayesian user’s preference model in mobile devices. In: International conference on ubiquitous intelligence and computing. Springer, Berlin, pp 1130–1139

  28. 28.

    Zhong J, Li X (2010) Unified collaborative filtering model based on combination of latent features. Expert Syst Appl 37(8):5666–5672

    Google Scholar 

  29. 29.

    Luo X, Xia Y, Zhu Q (2012) Incremental collaborative filtering recommender based on regularized matrix factorization. Knowl Based Syst 27:271–280

    Google Scholar 

  30. 30.

    Luo X, Xia Y, Zhu Q (2013) Applying the learning rate adaptation to the matrix factorization based collaborative filtering. Knowl Based Syst 37:154–164

    Google Scholar 

  31. 31.

    Funk S (2006) Netflix update: try this at home. http://sifter.org/simon/journal/20061211.html. Accessed 10 Dec 2017

  32. 32.

    Paterek A (2007) Improving regularized singular value decomposition for collaborative filtering. In: Proceedings of KDD cup and workshop, vol 2007, pp 5–8

  33. 33.

    Sarwar B, Karypis G, Konstan J, Riedl J (2000) Application of dimensionality reduction in recommender system-a case study (No. TR-00-043). Minnesota Univ Minneapolis Dept of Computer Science

  34. 34.

    Srebro N, Rennie JD, Jaakkola TS (2004) Maximum-margin matrix factorization. In: NIPS, vol 17, pp 1329–1336

  35. 35.

    Hofmann T (2004) Latent semantic models for collaborative filtering. ACM Trans Inf Syst (TOIS) 22(1):89–115

    Google Scholar 

  36. 36.

    Bell RM, Koren Y (2007) Scalable collaborative filtering with jointly derived neighborhood interpolation weights. In: Seventh IEEE international conference on data mining, 2007. ICDM 2007. IEEE, pp 43–52

  37. 37.

    Zhou Y, Wilkinson D, Schreiber R, Pan R (2008) Large-scale parallel collaborative filtering for the netflix prize. In: International conference on algorithmic applications in management. Springer, Berlin, pp 337–348

  38. 38.

    Takács G, Pilászy I, Németh B, Tikk D (2009) Scalable collaborative filtering approaches for large recommender systems. J Mach Learn Res 10(Mar):623–656

    Google Scholar 

  39. 39.

    Yu HF, Hsieh CJ, Si S, Dhillon I (2012) Scalable coordinate descent approaches to parallel matrix factorization for recommender systems. In 2012 IEEE 12th international conference on data mining (ICDM). IEEE, pp 765–774

  40. 40.

    Gemulla R, Nijkamp E, Haas PJ, Sismanis Y (2011) Large-scale matrix factorization with distributed stochastic gradient descent. In: Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 69–77

  41. 41.

    Chin WS, Zhuang Y, Juan YC, Lin CJ (2015) A fast parallel stochastic gradient method for matrix factorization in shared memory systems. ACM Trans Intell Syst Technol (TIST) 6(1):2

    Google Scholar 

  42. 42.

    Chin WS, Zhuang Y, Juan YC, Lin CJ (2015) A learning-rate schedule for stochastic gradient methods to matrix factorization. In: Pacific-Asia conference on knowledge discovery and data mining. Springer, Cham, pp 442–455

  43. 43.

    Sun R, Luo ZQ (2016) Guaranteed matrix completion via non-convex factorization. IEEE Trans Inf Theory 62(11):6535–6579

    MathSciNet  MATH  Google Scholar 

  44. 44.

    Jafari H, Jassim HK, Moshokoa SP, Ariyan VM, Tchier F (2016) Reduced differential transform method for partial differential equations within local fractional derivative operators. Adv Mech Eng 8(4):1687814016633013

    Google Scholar 

  45. 45.

    Jafari H, Jassim HK (2016) A new approach for solving a system of local fractional partial differential equations. Appl Appl Math 11:162–173

    MathSciNet  MATH  Google Scholar 

  46. 46.

    Hosseini VR, Chen W, Avazzadeh Z (2014) Numerical solution of fractional telegraph equation by using radial basis functions. Eng Anal Bound Elem 38:31–39

    MathSciNet  MATH  Google Scholar 

  47. 47.

    Hosseini VR, Shivanian E, Chen W (2016) Local radial point interpolation (MLRPI) method for solving time fractional diffusion-wave equation with damping. J Comput Phys 312:307–332

    MathSciNet  MATH  Google Scholar 

  48. 48.

    Chaudhary NI, Manzar MA, Raja MAZ (2019) Fractional Volterra LMS algorithm with application to Hammerstein control autoregressive model identification. Neural Comput Appl 31(9):5227–5240

    Google Scholar 

  49. 49.

    Chaudhary NI, Raja MAZ, Aslam MS, Ahmed N (2018) Novel generalization of Volterra LMS algorithm to fractional order with application to system identification. Neural Comput Appl 29(6):41–58

    Google Scholar 

  50. 50.

    Cheng S, Wei Y, Sheng D, Chen Y, Wang Y (2018) Identification for Hammerstein nonlinear ARMAX systems based on multi-innovation fractional order stochastic gradient. Sig Process 142:1–10

    Google Scholar 

  51. 51.

    Chaudhary NI, Aslam MS, Baleanu D, Raja MAZ (2019) Design of sign fractional optimization paradigms for parameter estimation of nonlinear Hammerstein systems. Neural Comput Appl. https://doi.org/10.1007/s00521-019-04328-0

    Article  Google Scholar 

  52. 52.

    Raja MAZ, Akhtar R, Chaudhary NI, Zhiyu Z, Khan Q, Rehman AU, Zaman F (2019) A new computing paradigm for the optimization of parameters in adaptive beamforming using fractional processing. Eur Phys J Plus 134(6):275

    Google Scholar 

  53. 53.

    Chaudhary NI, Zubair S, Aslam MS, Raja MAZ, Machado JT (2019) Design of momentum fractional LMS for Hammerstein nonlinear system identification with application to electrically stimulated muscle model. Eur Phys J Plus 134(8):407

    Google Scholar 

  54. 54.

    Shah SM, Samar R, Khan NM, Raja MAZ (2017) Design of fractional-order variants of complex LMS and NLMS algorithms for adaptive channel equalization. Nonlinear Dyn 88(2):839–858

    MATH  Google Scholar 

  55. 55.

    Chaudhary NI, Zubair S, Raja MAZ (2017) A new computing approach for power signal modeling using fractional adaptive algorithms. ISA Trans 68:189–202

    Google Scholar 

  56. 56.

    Zubair S, Chaudhary NI, Khan ZA, Wang W (2018) Momentum fractional LMS for power signal parameter estimation. Sig Process 142:441–449

    Google Scholar 

  57. 57.

    Podlubny I (1998) Fractional differential equations: an introduction to fractional derivatives, fractional differential equations, to methods of their solution and some of their applications, vol 198. Elsevier, Amsterdam

    Google Scholar 

  58. 58.

    Hajipour M, Jajarmi A, Baleanu D (2018) An efficient nonstandard finite difference scheme for a class of fractional chaotic systems. J Comput Nonlinear Dyn 13(2):021013

    Google Scholar 

  59. 59.

    Baleanu D, Machado JAT, Luo AC (eds) (2011) Fractional dynamics and control. Springer, Berlin

    Google Scholar 

  60. 60.

    Baleanu D (2012) Fractional calculus: models and numerical methods, vol 3. World Scientific, Singapore

    Google Scholar 

  61. 61.

    Valério D, Trujillo JJ, Rivero M, Machado JT, Baleanu D (2013) Fractional calculus: a survey of useful formulas. Eur Phys J Spec Top 222(8):1827–1846

    Google Scholar 

  62. 62.

    Zhang F, Yang C, Zhou X, Gui W (2018) Fractional-order PID controller tuning using continuous state transition algorithm. Neural Comput Appl 29(10):795–804

    Google Scholar 

  63. 63.

    Baleanu D, Yusuf A, Aliyu AI (2018) Time fractional third-order evolution equation: symmetry analysis, explicit solutions, and conservation laws. J Comput Nonlinear Dyn 13(2):021011

    Google Scholar 

  64. 64.

    Zheng Y, Huang M, Lu Y, Li W (2018) Fractional stochastic resonance multi-parameter adaptive optimization algorithm based on genetic algorithm. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3910-6

    Article  Google Scholar 

  65. 65.

    Maxwell HF, Konstan JA (2015) The movielens datasets: history and context. ACM Trans Interact Intell Syst (TIIS) 5:19

    Google Scholar 

  66. 66.

    Li K, Zhou X, Lin F, Zeng W, Alterovitz G (2019) Deep probabilistic matrix factorization framework for online collaborative filtering. IEEE Access 7:56117–56128

    Google Scholar 

  67. 67.

    Nguyen DM, Tsiligianni E, Deligiannis N (2018) Learning discrete matrix factorization models. IEEE Signal Process Lett 25(5):720–724

    Google Scholar 

  68. 68.

    Hoi SC, Sahoo D, Lu J, Zhao P (2018) Online learning: a comprehensive survey. arXiv preprint arXiv:1802.02871

  69. 69.

    Kim D, Park C, Oh J, Lee S, Yu H (2016) Convolutional matrix factorization for document context-aware recommendation. In: Proceedings of the 10th ACM conference on recommender systems. ACM, pp 233–240

  70. 70.

    Lin F, Zhou X, Zeng W (2016) Sparse online learning for collaborative filtering. Int J Comput Commun Control 11(2):248–258

    Google Scholar 

  71. 71.

    Huang J, Nie F, Huang H (2013) Robust discrete matrix completion. In: Twenty-seventh AAAI conference on artificial intelligence

  72. 72.

    Huo Z, Liu J, Huang H (2016) Optimal discrete matrix completion. In: Thirtieth AAAI conference on artificial intelligence

  73. 73.

    Xue HJ, Dai X, Zhang J, Huang S, Chen J (2017) Deep matrix factorization models for recommender systems. In: IJCAI, pp 3203–3209

  74. 74.

    Nguyen DM, Tsiligianni E, Deligiannis N (2018) Extendable neural matrix completion. In: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 6328–6332

  75. 75.

    Dassios IK, Baleanu DI (2018) Caputo and related fractional derivatives in singular systems. Appl Math Comput 337:591–606

    MathSciNet  MATH  Google Scholar 

  76. 76.

    Dassios I, Baleanu D (2018) Optimal solutions for singular linear systems of Caputo fractional differential equations. Math Methods Appl Sci. https://doi.org/10.1002/mma.5410

    Article  Google Scholar 

  77. 77.

    Dassios IK (2018) A practical formula of solutions for a family of linear non-autonomous fractional nabla difference equations. J Comput Appl Math 339:317–328

    MathSciNet  MATH  Google Scholar 

  78. 78.

    Atangana A (2016) On the new fractional derivative and application to nonlinear Fisher’s reaction–diffusion equation. Appl Math Comput 273:948–956

    MathSciNet  MATH  Google Scholar 

  79. 79.

    Atangana A, Baleanu D (2017) Caputo-Fabrizio derivative applied to groundwater flow within confined aquifer. J Eng Mech 143(5):D4016005

    Google Scholar 

  80. 80.

    Atangana A, Gómez-Aguilar JF (2018) Fractional derivatives with no-index law property: application to chaos and statistics. Chaos Solitons Fractals 114:516–535

    MathSciNet  MATH  Google Scholar 

  81. 81.

    Yang XJ, Gao F, Machado JA, Baleanu D (2017) A new fractional derivative involving the normalized sinc function without singular kernel. arXiv preprint arXiv:1701.05590

  82. 82.

    Firoozjaee MA, Jafari H, Lia A, Baleanu D (2018) Numerical approach of Fokker–Planck equation with Caputo–Fabrizio fractional derivative using Ritz approximation. J Comput Appl Math 339:367–373

    MathSciNet  MATH  Google Scholar 

  83. 83.

    Mehmood A, Chaudhary NI, Zameer A, Raja MAZ (2019) Novel computing paradigms for parameter estimation in Hammerstein controlled auto regressive auto regressive moving average systems. Appl Soft Comput 80:263–284

    Google Scholar 

  84. 84.

    Mehmood A, Chaudhary NI, Zameer A, Raja MAZ (2019) Novel computing paradigms for parameter estimation in power signal models. Neural Comput Appl. https://doi.org/10.1007/s00521-019-04133-9

    Article  Google Scholar 

  85. 85.

    Khan WU, Ye Z, Chaudhary NI, Raja MAZ (2018) Backtracking search integrated with sequential quadratic programming for nonlinear active noise control systems. Appl Soft Comput 73:666–683

    Google Scholar 

  86. 86.

    Mehmood A, Zameer A, Raja MAZ, Bibi R, Chaudhary NI, Aslam MS (2018) Nature-inspired heuristic paradigms for parameter estimation of control autoregressive moving average systems. Neural Comput Appli. https://doi.org/10.1007/s00521-018-3406-4

    Article  Google Scholar 

Download references

Acknowledgements

For the last author, this paper was supported by the Ministry of Education, Science and Technological Development of the Republic of Serbia (project: Improvement of the quality of tractors and mobile systems with the aim of increasing competitiveness and preserving soil and environment, no. TR-31046).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Nebojsa Dedovic.

Ethics declarations

Conflict of interest

All authors declared that there are no potential conflicts of interest.

Human and animal rights statements

All authors declared that there is no research involving human and/or animal.

Informed consent

All authors declared that there is no material that required informed consent.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Khan, Z.A., Zubair, S., Chaudhary, N.I. et al. Design of normalized fractional SGD computing paradigm for recommender systems. Neural Comput & Applic 32, 10245–10262 (2020). https://doi.org/10.1007/s00521-019-04562-6

Download citation

Keywords

  • Recommender systems
  • Normalized adaptive algorithms
  • E-commerce
  • Fractional calculus
  • Stochastic gradient descent