Diffusion logistic regression algorithms over multiagent networks

Abstract

In this paper, a distributed scheme is proposed for ensemble learning method of bagging, which aims to address the classification problems for large dataset by developing a group of cooperative logistic regression learners in a connected network. Moveover, each weak learner/agent can share the local weight vector with its immediate neighbors through diffusion strategy in a fully distributed manner. Our diffusion logistic regression algorithms can effectively avoid overfitting and obtain high classification accuracy compared to the non-cooperation mode. Furthermore, simulations with a real dataset are given to demonstrate the effectiveness of the proposed methods in comparison with the centralized one.

This is a preview of subscription content, log in to check access.

References

  1. [1]

    R. M. Harbord, P. Whiting. Metand: meta-analysis of diagnostic accuracy using hierarchical logistic regression. The Stata Journal, 2009, 9(2): 211–229.

    Article  Google Scholar 

  2. [2]

    C. L. Tsien, H. S. Fraser, W. J. Long, et al. Using classification tree and logistic regression methods to diagnose myocardial infarction. Proceedings of the 9th World Congress on Medical Informatics, Seoul: IOS Press, 1998: 493–497.

    Google Scholar 

  3. [3]

    T. Ayer, J. Chhatwal, O. Alagoz, et al. Comparison of logistic regression and artificial neural network models in breast cancer risk estimation. Adiographics, 2010, 30(1): 13–22.

    Article  Google Scholar 

  4. [4]

    Y. Gong, Y. Fang, Y. Guo. Private data analytics on biomedical sensing data via distributed computation. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2016, 13(3): 431–444.

    Article  Google Scholar 

  5. [5]

    D. Singh, D. Roy, C. K. Mohan. DiP-SVM: distribution preserving kernel support vector machine for big data. IEEE Transactions on Big Data, 2016, 3(1): 79–90.

    Article  Google Scholar 

  6. [6]

    T. G. Dietterich. Ensemble learning. The Handbook of Brain Theory and Neural Networks. 2nd ed. Cambridge: MIT Press, 2002: 110–125.

    Google Scholar 

  7. [7]

    C. Zhang, Y. Ma (eds.). Ensemble Machine Learning: Methods and Applications. New York: Springer, 2012.

    Google Scholar 

  8. [8]

    L. Breiman. Bagging predictors. Machine learning, 1996, 24(2): 123–140.

    MATH  Google Scholar 

  9. [9]

    E. Bauer, R. Kohavi. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning, 1999, 36(1/2): 105–139.

    Article  Google Scholar 

  10. [10]

    L. Breiman. Random forests. Machine Learning, 2001, 45(1): 5–32.

    MATH  Article  Google Scholar 

  11. [11]

    A. Liaw, M. Wiener. Classification and regression by random forest. R News, 2002, 2(3): 18–22.

    Google Scholar 

  12. [12]

    T. N. Do, F. Poulet. Classifying one billion data with a new distributed SVM algorithm. International Conference on Research, Innovation and Vision for the Future, Vietnam: IEEE, 2006: 59–66.

    Google Scholar 

  13. [13]

    S. S. Ram, A. Nedic, V. V. Veeravalli. Distributed stochastic subgradient projection algorithms for convex optimization. Journal of Optimization Theory and Applications, 2010, 147(3): 516–545.

    MathSciNet  MATH  Article  Google Scholar 

  14. [14]

    D. Caragea, A. Silvescu, V. Honavar. Agents that learn from distributed dynamic data sources. Proceedings of the Workshop on Learning Agents, 2000: 53–61.

    Google Scholar 

  15. [15]

    F. Pourpanah, C. J. Tan, C. P. Lim, et al. A Q-learning-based multi-agent system for data classification. Applied Soft Computing, 2017, 52: 519–531.

    Article  Google Scholar 

  16. [16]

    Z. Tu, Y. Wang, Y. Hong. Distributed boosting algorithm over multi-agent networks. Proceedings of the 37th Chinese Control Conference, Wuhan: IEEE, 2018: 7153–7157.

    Google Scholar 

  17. [17]

    N. Couellan, S. Jan, T. Jorquera, et al. Self-adaptive support vector machine: a multi-agent optimization perspective. Expert Systems with Applications, 2015, 42(9): 4284–4298.

    Article  Google Scholar 

  18. [18]

    Y. Wang, P. Lin, H. Qin. Distributed classification learning based on nonlinear vector support machines for switching networks. Kybernetika, 2017, 53(4): 595–611.

    MathSciNet  MATH  Google Scholar 

  19. [19]

    J. Chen, A. H. Sayed. Diffusion adaptation strategies for distributed optimization and learning over networks. IEEE Transactions on Signal Processing, 2012, 60(8): 4289–4305.

    MathSciNet  MATH  Article  Google Scholar 

  20. [20]

    C. G. Lopes, A. H. Sayed. Diffusion least-mean squares over adaptive networks: Formulation and performance analysis. IEEE Transactions on Signal Processing, 2008, 56(7): 3122–3136.

    MathSciNet  MATH  Article  Google Scholar 

  21. [21]

    F. S. Cattivelli, A. H. Sayed. Diffusion LMS strategies for distributed estimation. IEEE Transactions on Signal Processing, 2009, 58(3): 1035–1048.

    MathSciNet  MATH  Article  Google Scholar 

  22. [22]

    S. Y. Tu, A. H. Sayed. Diffusion strategies outperform consensus strategies for distributed estimation over adaptive networks. IEEE Transactions on Signal Processing, 2012, 60(12): 6217–6234.

    MathSciNet  MATH  Article  Google Scholar 

  23. [23]

    A. H. Sayed. Adaptation, learning, and optimization over networks. Foundations and Trends in Machine Learning, 2014, 7(4/5): 311–801.

    MATH  Article  Google Scholar 

  24. [24]

    S. Boyd, L. Vandenberghe. Convex Optimization. Cambridge: Cambridge University Press, 2004.

    Google Scholar 

  25. [25]

    T. P. Minka. A comparison of numerical optimizers for logistic regression: https://www.microsoft.com/en-us/research/publication/comparison-numerical-optimizers-logistic-regression/.

  26. [26]

    D. Dua, C. Graff. UCI Machine Learning Repository: https://archive.ics.uci.edu/ml/index.php.

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Lijuan Jia.

Additional information

This work was supported in part by the National Natural Science foundation of China (No. 41927801).

Yan DU received her B.Sc. degree in Communication Engineering from Wuhan University of Technology, China in 2018. From 2018 to 2021, she is pursuing a M.Sc. degree at Beijing Institute of Technology. Her research interests include distributed signal processing, estimation and machine learning.

Lijuan JIA received her Ph.D. degree in Electric and Electronic Engineering from Kyushu University, Japan, in 2002. From 2002 to 2005, she was a lecturer at the Department of Electrical and Electronic Engineering, Kyushu University, Japan. Since 2005, she has been an associate professor of the School of Information and Electronics, Beijing Institute of Technology. From 2013 to 2014, she was a visiting scholar at the Department of Electrical Engineering, University of California, Los Angeles, U.S.A. Her research interests include multi-agent system theory, distributed adaptive networks, and statistical signal processing.

Shunshoku KANAE (M’00) received the Dr. Eng. degree from Kyushu University, Fukuoka, Japan, in 1995. From 1995 to 1998, he was a Research Associate at Kyushu Institute of Technology. Since October 1998, he has been a Research Associate in the Graduate School of Information Science and Electrical Engineering, Kyushu University. He is now a Professor with the Department of Medical Engineering, Faculty of Health Science, Junshin Gakune University, in Fukuoka-city, Japan. His research interests include system identification, mechatronics system control, and soft computing.

Zijiang YANG received his Dr. Eng. degree in 1992 from Kyushu University. From 1996 to 2000, he was an Associate Professor in the Faculty of Computer Engineering and System Science, Kyushu Institute of Technology, Japan. From 2000 to 2009, He was an Associate Professor in the Department of Electrical and Electronic Systems Engineering, Kyushu University, Japan. Since 2009, he has been a Professor of the Department of Mechanical Systems Engineering, Faculty of Engineering, Ibaraki University, Japan. His research interests include system identification and nonlinear system control.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Du, Y., Jia, L., Kanae, S. et al. Diffusion logistic regression algorithms over multiagent networks. Control Theory Technol. 18, 160–167 (2020). https://doi.org/10.1007/s11768-020-0009-2

Download citation

Keywords

  • Logistic regression
  • bagging
  • diffusion strategy
  • connected network