Improving Efficiency in Parallel Computing Leveraging Local Synchronization

  • Franco Cicirelli
  • Andrea GiordanoEmail author
  • Carlo Mastroianni
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11973)


In a parallel computing scenario, a complex task is typically split among many computing nodes, which are engaged to perform portions of the task in a parallel fashion. Except for a very limited class of application, computing nodes need to coordinate with each other in order to carry out the parallel execution in a consistent way. As a consequence, a synchronization overhead arises, which can significantly impair the overall execution performance. Typically, synchronization is achieved by adopting a centralized synchronization barrier involving all the computing nodes. In many application domains, though, such kind of global synchronization can be relaxed and a lean synchronization schema, namely local synchronization, can be exploited. By using local synchronization, each computing node needs to synchronize only with a subset of the other computing nodes. In this work, we evaluate the performance of the local synchronization mechanism when compared to the global synchronization approach. As a key performance indicator, the efficiency index is considered, which is defined as the ratio between useful computation time and total computation time, including the synchronization overhead. The efficiency trend is evaluated both analytically and through numerical simulation.


Parallel computing Efficiency Synchronization Max-Plus Algebra 


  1. 1.
    Cicirelli, F., Forestiero, A., Giordano, A., Mastroianni, C.: Transparent and efficient parallelization of swarm algorithms. ACM Trans. Auton. Adapt. Syst. 11(2), 14 (2016)CrossRefGoogle Scholar
  2. 2.
    Cicirelli, F., Forestiero, A., Giordano, A., Mastroianni, C.: Parallelization of space-aware applications: modeling and performance analysis. J. Netw. Comput. Appl. 122, 115–127 (2018)CrossRefGoogle Scholar
  3. 3.
    Gong, Z., Tang, W., Bennett, D.A., Thill, J.C.: Parallel agent-based simulation of individual-level spatial interactions within a multicore computing environment. Int. J. Geogr. Inf. Sci. 27(6), 1152–1170 (2013)CrossRefGoogle Scholar
  4. 4.
    Tang, W., Bennett, D.A., Wang, S.: A parallel agent-based model of land use opinions. J. Land Use Sci. 6(2–3), 121–135 (2011)CrossRefGoogle Scholar
  5. 5.
    Mastroianni, C., Cesario, E., Giordano, A.: Efficient and scalable execution of smart city parallel applications. Concurrency Comput. Pract. Experience 30(20), e4258 (2018)CrossRefGoogle Scholar
  6. 6.
    Zheng, Y., Capra, L., Wolfson, O., Yang, H.: Urban computing: concepts, methodologies, and applications. ACM Trans. Intell. Syst. Technol. 5(3), 1–55 (2014)Google Scholar
  7. 7.
    Blecic, I., Cecchini, A., Trunfio, G.A., Verigos, E.: Urban cellular automata with irregular space of proximities. J. Cell. Automata 9(2–3), 241–256 (2014)Google Scholar
  8. 8.
    Atzori, L., Iera, A., Morabito, G.: The internet of things: a survey. Comput. Netw. 54(15), 2787–2805 (2010)CrossRefGoogle Scholar
  9. 9.
    Lee, I., Lee, K.: The Internet of Things (IoT): applications, investments, and challenges for enterprises. Bus. Horiz. 58(4), 431–440 (2015)CrossRefGoogle Scholar
  10. 10.
    Krishnan, Y.N., Bhagwat, C.N., Utpat, A.P.: Fog computing–network based cloud computing. In: 2nd IEEE International Conference on Electronics and Communication Systems (ICECS), pp. 250–251 (2015)Google Scholar
  11. 11.
    Hu, P., Dhelim, S., Ning, H., Qiu, T.: Survey on fog computing: architecture, key technologies, applications and open issues. J. Netw. Comput. Appl. 98, 27–42 (2017)CrossRefGoogle Scholar
  12. 12.
    Ekanayake, J., Fox, G.: High performance parallel computing with clouds and cloud technologies. In: Avresky, D.R., Diaz, M., Bode, A., Ciciani, B., Dekel, E. (eds.) CloudComp 2009. LNICSSTE, vol. 34, pp. 20–38. Springer, Heidelberg (2010). Scholar
  13. 13.
    Fujimoto, R.: Parallel and Distributed Simulation Systems. Wiley, New York (2000)Google Scholar
  14. 14.
    Cicirelli, F., Forestiero, A., Giordano, A., Mastroianni, C., Razumchik, R.: Global and local synchronization in parallel space-aware applications. In: 32nd European Conference on Modelling and Simulation, ECMS 2018, Wilhelmshaven, Germany, May 2018Google Scholar
  15. 15.
    Borovkov, A.A.: Ergodicity and stability theorems for a class of stochastic equations and their applications. Theory Probab. Appl. 23(2), 227–247 (1979)CrossRefGoogle Scholar
  16. 16.
    David, H.A., Nagaraja, H.N.: Order Statistics, 3rd edn. Wiley, New York (2003)CrossRefGoogle Scholar
  17. 17.
    Ang, A.S., Tang, W.: Probability Concepts in Engineering Planning and Design, vol. II. Rainbow Bridge (1984)Google Scholar
  18. 18.
    Madala, S., Sinclair, J.B.: Performance of synchronous parallel algorithms with regular structures. IEEE Trans. Parallel Distrib. Syst. 2(1), 105–116 (1991)CrossRefGoogle Scholar
  19. 19.
    Whitt, W., Crow, C., Goldberg, D.: Two-moment approximations for maxima. Oper. Res. 55(3), 532–548 (2007)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Baccelli, F., Konstantopoulos, P.: Estimates of cycle times in stochastic petri nets. In: Karatzas, I., Ocone, D. (eds.) Applied Stochastic Analysis, pp. 1–20. Springer, Heidelberg (1992). Scholar
  21. 21.
    Lublin, U., Feitelson, D.G.: The workload on parallel supercomputers: modeling the characteristics of rigid jobs. J. Parallel Distrib. Comput. 63(11), 1105–1122 (2003)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.ICAR-CNRRendeItaly

Personalised recommendations