CDSFM: A Circular Distributed SGLD-Based Factorization Machines

  • Kankan Zhao
  • Jing Zhang
  • Liangfu Zhang
  • Cuiping LiEmail author
  • Hong Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10828)


Factorization Machines (FMs) offers attractive performance by combining low-rank data vectors and heuristic features. However, it suffers from the growth of the dataset and the model complexity. Although much efforts have been made to distribute FMs over multiple machines, the computation efficiency is still limited by the foundational master-slave framework. In this paper, we propose CDSFM, which leverages Stochastic Gradient Langevin Dynamics (SGLD) to optimize FMs, and is distributed into a completely new circular framework. Experiments on two genres of datasets show that CDSFM can achieves a 2.3–4.7\(\times \) speed-up over the comparison methods while obtains better performance.



This work is supported by National Key Research&Develop Plan (No. 2016YFB 100702), and NSFC under the grant No. (61772537, 61772536, 61702522, 61532021).


  1. 1.
    Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)zbMATHGoogle Scholar
  2. 2.
    Dai, W., Wei, J., Zheng, X., Jin, K.K., Lee, S., Yin, J., Ho, Q., Xing, E.P.: Petuum: a framework for iterative-convergent distributed ML. Eprint Arxiv (2013)Google Scholar
  3. 3.
    Dean, J., Corrado, G.S., Monga, R.: Large scale distributed deep networks. In: NIPS 2012, pp. 1223–1231 (2012)Google Scholar
  4. 4.
    Hirata, A., Komachi, M.: Sparse named entity classification using factorization machines. Eprint Arxiv (2017)Google Scholar
  5. 5.
    Jiang, J., Yu, L., Jiang, J., Liu, Y., Cui, B.: Angel: a new large-scale machine learning system. Natl. Sci. Rev. 5, 216–236 (2017)CrossRefGoogle Scholar
  6. 6.
    Lam, L., Suen, S.: Application of majority voting to pattern recognition: an analysis of its behavior and performance. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 27(5), 553–568 (1997)CrossRefGoogle Scholar
  7. 7.
    Li, M., Andersen, D.G., Park, J.W.: Scaling distributed machine learning with the parameter server. In: OSDI 2014, vol. 14, pp. 583–598 (2014)Google Scholar
  8. 8.
    Low, Y., Gonzalez, J., Kyrola, A., Bickson, D., Guestrin, C.: GraphLab: a distributed framework for machine learning in the cloud. Eprint Arxiv (2011)Google Scholar
  9. 9.
    Malewicz, G., Austern, M.H., Bik, A.J., Dehnert, J.C., Horn, I., Leiser, N.: Pregel: a system for large-scale graph processing. In: SIGMOD 2010, pp. 135–146 (2010)Google Scholar
  10. 10.
    Rendle, S.: Factorization machines. In: ICDM 2010, pp. 995–1000 (2010)Google Scholar
  11. 11.
    Sun, H., Wang, W., Shi, Z.: Parallel factorization machine recommended algorithm based on MapReduce. In: SKG 2014, pp. 120–123 (2014)Google Scholar
  12. 12.
    Tsai, M.F., Wang, C.J., Lin, Z.L.: Social influencer analysis with factorization machines. In: WebSci 2015, p. 50. ACM (2015)Google Scholar
  13. 13.
    Wang, S., Du, C., Zhao, K., Li, C., Li, Y.: Random partition factorization machines for context-aware recommendations. In: WAIM 2016. pp. 219–230 (2016)Google Scholar
  14. 14.
    Welling, M., Teh, Y.W.: Bayesian learning via stochastic gradient Langevin dynamics. In: ICML 2011, pp. 681–688 (2011)Google Scholar
  15. 15.
    Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: HotCloud 2010, p. 10 (2010)Google Scholar
  16. 16.
    Zhong, E., Shi, Y., Liu, N., Rajan, S.: Scaling factorization machines with parameter server. In: CIKM 2016, pp. 1583–1592 (2016)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Kankan Zhao
    • 1
    • 2
  • Jing Zhang
    • 1
    • 2
  • Liangfu Zhang
    • 1
    • 2
  • Cuiping Li
    • 1
    • 2
    Email author
  • Hong Chen
    • 1
    • 2
  1. 1.School of InformationRenmin University of ChinaBeijingChina
  2. 2.Key Laboratory of Data Engineering and Knowledge EngineeringBeijingChina

Personalised recommendations