Advertisement

Convergence and correctness of belief propagation for the Chinese postman problem

  • Guowei Dai
  • Fengwei Li
  • Yuefang Sun
  • Dachuan Xu
  • Xiaoyan ZhangEmail author
Article
  • 7 Downloads

Abstract

Belief Propagation (BP), a distributed, message-passing algorithm, has been widely used in different disciplines including information theory, artificial intelligence, statistics and optimization problems in graphical models such as Bayesian networks and Markov random fields. Despite BP algorithm has a great success in many application fields and many progress about BP algorithm has been made, the rigorous analysis about the correctness and convergence of BP algorithm are known in only a few cases for arbitrary graph. In this paper, we will investigate the correctness and convergence of BP algorithm for determining the optimal solutions of the Chinese postman problem in both undirected and directed graphs. As a main result, we prove that BP algorithm converges to the optimal solution of the undirected Chinese postman problem within O(n) iterations where n represents the number of vertices, provided that the optimal solution is unique. For the directed case, we consider the directed Chinese postman problem with capacity and show that BP algorithm also converges to its optimal solution after O(n) iterations, as long as its corresponding linear programming relaxation has the unique optimal solution.

Keywords

Belief propagation (BP) Message-passing algorithm Min-sum algorithm Convergence Undirected Chinese postman problem Directed Chinese postman problem 

Notes

Acknowledgements

We are truly grateful to the reviewers’ comments and suggestions on improving the manuscript, which helped us greatly to improve the quality of our paper. The research was partially supported by the National Natural Science Foundation of China under Grant Nos. 11871280, 11471003, 11401389,11531014, 11871081, China Scholarship Council under Grant Nos. 201607910003, 201608330111, Zhejiang Provincial Natural Science Foundation of China under Grant Nos. LY17A010017, 11401389 and Qing Lan Project.

References

  1. 1.
    Achlioptas, D., Ricci-Tersenghi, F.: On the solution-space geometry of random constraint satisfaction problems. In: Proceedings of the 38th Annual ACM Symposium on Theory of Computing, pp. 130–139, ACM (2006)Google Scholar
  2. 2.
    Aji, S.M., Horn, G.B., Mceliece, R.J.: On the convergence of iterative decoding on graphs with a single cycle. In: Proceedings of IEEE International Symposium Information Theory, p. 276. Cambridge (1998)Google Scholar
  3. 3.
    Aji, S.M., Mceliece, R.J.: The generalized distributive law. IEEE Trans. Inf. Theory 46, 325–343 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bayati, M., Braunstein, A., Zecchina, R.: A rigorous analysis of the cavity equations for the minimum spanning tree. J. Math. Phys. 49, 857–883 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Bayati, M., Borgs, C., Chayes, J., Zecchina, R.: Belief propagation for weighted b-matchings on arbitrary graphs and its relation to linear programs with integer solutions. SIAM J. Discrete Math. 25, 989–1011 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Bayati, M., Shah, D., Sharma, M.: A simpler max-product maximum weight matching algorithm and the auction algorithm. In: IEEE International Symposium Information Theory, pp. 557–561 (2006)Google Scholar
  7. 7.
    Bayati, M., Shah, D., Sharma, M.: Max-product for maximum weight matching: convergence, correctness, and LP duality. IEEE Trans. Inf. Theory 54, 1241–1251 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Brunsch, T., Cornelissen, K., Manthey, B., Röglin, H.: Smoothed Analysis of Belief Propagation for Minimum-Cost Flow and Matching, WALCOM: algorithms and Computation, pp. 182–193. Springer, Berlin (2012)zbMATHGoogle Scholar
  9. 9.
    Chen, D.J., Lee, C.Y., Park, C.H., Mendes, P.: Parallelizing simulated annealing algorithms based on high-performance computer. J Glob. Optim. 39, 261–289 (2007)CrossRefzbMATHGoogle Scholar
  10. 10.
    Cheng, Y., Neely, M., Chugg, K.M.: Iterative message passing algorithm for bipartite maximum weighted matching. In: IEEE International Symposium Information Theory, Cambridge, pp. 1934–1938, (2006)Google Scholar
  11. 11.
    Coja-Oghlan, A., Mossel, E., Vilenchik, D.: A spectral approach to analysing belief propagation for 3-colouring. Comb. Probab. Comput. 18, 881–912 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Edmonds, J.: The Chinese postman problem. Oper. Res. 13, 1–73 (1965)CrossRefGoogle Scholar
  13. 13.
    Edmonds, J., Johnson, E.: Matching, Euler tour and Chinese postman. Math. Program. 5, 88–124 (1973)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Even, G., Halabi, N.: Analysis of the min-sum algorithm for packing and covering problems via linear programming. IEEE Trans. Inf. Theory 61, 5295–5305 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Feige, U., Mossel, E., Vilenchik, D.: Complete convergence of message passing algorithms for some satisfiability problems. Random 4110, 339–350 (2010)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Frey, B.J., Dueck, D.: Clustering by passing messages between data points. Science 315, 972–976 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Frey, B.J., Koetter, R., Forney Jr., G.D., Kschischang, F.R., Spielman, D.A.: Introduction to the special issue on codes on graphs and iterative algorithms. IEEE Trans. Inf. Theory 47, 493–497 (2001)CrossRefGoogle Scholar
  18. 18.
    Gallager, R.G.: Low density parity check codes. IEEE Trans. Inf. Theory 8, 21–28 (1962)Google Scholar
  19. 19.
    Gamarnik, D., Shah, D., Wei, Y.: Belief propagation for min-cost network flow: convergence and correctness. Oper. Res. 60, 410–428 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Guan, M.G.: Graphic programming using odd or even points. Chin. Math. 1, 273–277 (1962)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Kschischang, F.R., Frey, B.J., Loeliger, H.-A.: Factor graphs and sum-product algorithm. IEEE Trans. Inf. Theory 47, 498–519 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Mézard, M.: Passing messages between disciplines. Science 301, 1685–1686 (2003)CrossRefGoogle Scholar
  23. 23.
    Mézard, M., Parisi, G., Zecchina, R.: Analytic and algorithmic solution of random satisfiability problems. Science 297, 812–815 (2002)CrossRefGoogle Scholar
  24. 24.
    Mézard, M., Zecchina, R.: Random k-satisfiability problem: from an analytic solution to an efficient algorithm. Phys. Rev. 66, 249–264 (2002)Google Scholar
  25. 25.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Reasoning. Morgan Kaufman, San Mateo (1988)zbMATHGoogle Scholar
  26. 26.
    Richardson, T., Urbanke, R.: The capacity of low-density parity check codes under message-passing decoding. IEEE Trans. Inf. Theory 47, 599–618 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Sanghavi, S., Malioutov, D., Willsky, A.: Linear programming analysis of loopy belief propagation for weighted matching. In: Advances in Neural Information Processing Systems, pp. 1273–1280. Cambridge, (2007)Google Scholar
  28. 28.
    Sanghavi, S., Malioutov, D., Willsky, A.: Belief propagation and LP relaxation for weighted matching in general graphs. IEEE Trans. Inf. Theory 54, 2203–2212 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Sanghavi, S., Shah, D., Willsky, A.S.: Message passing for maximum weight independent set. IEEE Trans. Inf. Theory 55, 4822–4834 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Schrijver, A.: Combinatorial Optimization - Polyhedra and Efficiency. Springer, Berlin (2003)zbMATHGoogle Scholar
  31. 31.
    Vontobel, P.O., Koetter, R.: Graph-cover decoding and finite-length analysis of message passing iterative decoding of LDPC codes, arXiv preprint cs/0512078 (2005)Google Scholar
  32. 32.
    Wainwright, M., Jaakkola, T., Willsky, A.: Tree consistency and bounds on the performance of the max-product algorithm and its generalizations. Stat. Comput. 14, 143–166 (2004)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Weiss, Y.: Correctness of local probability propagation in graphical models with loops. Neural Comput. 12, 1–42 (2000)CrossRefGoogle Scholar
  34. 34.
    Weiss, Y., Freeman, W.: On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs. IEEE Trans. Inf. Theory 47, 736–744 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Weiss, Y., Freeman, W.: Correctness of belief propagation in Gaussian graphical models of arbitrary topology. Neural Comput. 13, 2173–2200 (2001)CrossRefzbMATHGoogle Scholar
  36. 36.
    Yedidia, J., Freeman, W., Weiss, Y.: Understanding belief propagation and its generalizations. Explor. Artif. Intell. New Millenn. 8, 236–239 (2003)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Institute of Mathematics and School of Mathematical ScienceNanjing Normal UniversityNanjingChina
  2. 2.Faculty of Mathematics and StatisticsCentral China Normal UniversityWuhanChina
  3. 3.Department of MathematicsShaoxing UniversityShaoxingChina
  4. 4.Department of Information and Operations Research, College of Applied SciencesBeijing University of TechnologyBeijingChina

Personalised recommendations