Skip to main content
SpringerLink
Log in

A Dynamic Scheduling Strategy of ADMM Sub-problem Optimization Algorithm Based on Hierarchical Structure

  • Conference paper
  • First Online:
Algorithms and Architectures for Parallel Processing (ICA3PP 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12452))

  • 1564 Accesses

Abstract

The Alternating Direction Method of Multiplier (ADMM) is a simple algorithm to resolve decomposable convex optimization problems, especially effective in solving large-scale problems. However, this algorithm suffers from the straggler problem its updates have to be synchronized. Therefore, the asynchronous ADMM algorithm is proposed. However, the convergence speed of the ADMM algorithm is not very satisfactory. In this paper, we propose a dynamic scheduling strategy for sub-problems-automatically calling different algorithms at different iteration periods of each iteration, and combining this strategy with a hierarchical communication structure. The experiments based on ZiQiang 4000 cluster experimental environment show that the dynamic scheduling strategy based on hierarchical communication structure can solve the ADMM sub-problem and effectively improve the convergence speed and communication efficiency of the algorithm.

Supported by the Natural Science Foundation of China under grant No. U1811461.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Mateos, G., Bazerque, J.A., Giannakis, G.B.: Distributed sparse linear regression. IEEE Trans. Sig. Process. 58(10), 5262–5276 (2010)

    Article  MathSciNet  Google Scholar 

  2. Lubell-Doughtie, P., Sondag, J.: Practical distributed classification using the Alternating Direction Method of Multipliers algorithm. In: IEEE International Conference on Big Data, pp. 773–776. IEEE (2013)

    Google Scholar 

  3. Forero, P.A., Cano, A., Giannakis, G.B.: Consensus-based distributed support vector machines. J. Mach. Learn. Res. 11(3), 1663–1707 (2010)

    MathSciNet  MATH  Google Scholar 

  4. Boyd, S., Parikh, N., et al.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. (2012)

    Google Scholar 

  5. Valiant, L.G.: Bulk-synchrony: a bridging model for parallel computation. Commun. ACM 33(8), 103–111 (1990)

    Article  Google Scholar 

  6. Ho, Q., Cipar, J., Cui, H., et al.: More effective distributed ML via a stale synchronous parallel parameter server. In: International Conference on Neural Information Processing Systems, pp. 1223–1231. Curran Associates Inc. (2013)

    Google Scholar 

  7. Zhang, R., Kwok, J.T.: Asynchronous distributed ADMM for consensus optimization. In: International Conference on International Conference on Machine Learning (2014)

    Google Scholar 

  8. Chang, T.H., Hong, M., Liao, W.C., et al.: Asynchronous distributed alternating direction method of multipliers: Algorithm and convergence analysis. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 4781–4785. IEEE (2016)

    Google Scholar 

  9. Wang, H., Gao, Y., Shi, Y., Wang, R.: Group-based alternating direction method of multipliers for distributed linear classification. IEEE Trans. Cybern. 47(11), 3568–3582 (2017)

    Article  Google Scholar 

  10. Fang, L., Lei, Y.M.: An asynchronous distributed ADMM algorithm and efficient communication model. In: Dependable, Autonomic and Secure Computing, 14th International Conference on Pervasive Intelligence and Computing, 2nd International Conference on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech), 2016 IEEE 14th International Conference, pp. 136–140. IEEE, August 2016

    Google Scholar 

Download references

Acknowledgment

This research was supported in part by the National Natural Science Foundation of China under grant No. U1811461 and ZiQiang 4000 cluster experimental environment of Shanghai University.

Author information

Authors and Affiliations

  1. School of Computer Engineering and Science, Shanghai University, Shanghai, China

    Jiawei Ji, Yongmei Lei & Shenghong Jiang

Authors
  1. Jiawei Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongmei Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shenghong Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yongmei Lei .

Editor information

Editors and Affiliations

  1. Columbia University, New York, NY, USA

    Meikang Qiu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ji, J., Lei, Y., Jiang, S. (2020). A Dynamic Scheduling Strategy of ADMM Sub-problem Optimization Algorithm Based on Hierarchical Structure. In: Qiu, M. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2020. Lecture Notes in Computer Science(), vol 12452. Springer, Cham. https://doi.org/10.1007/978-3-030-60245-1_10

Download citation

Publish with us

Policies and ethics

Search

Navigation