Advertisement

Abstract

In the Map-Reduce programming model for data parallel computation, a reducer computes an output from a list of input values associated with a key. The inputs however may not arrive at a reducer in a fixed order due to non-determinism in transmitting key-value pairs over the network. This gives rise to the reducer commutativity problem, that is, is the reducer computation independent of the order of its inputs? In this paper, we study the reducer commutativity problem formally. We introduce a syntactic subset of integer programs termed integer reducers to model real-world reducers. In spite of syntactic restrictions, we show that checking commutativity of integer reducers over unbounded lists of exact integers is undecidable. It remains undecidable even with input lists of a fixed length. The problem however becomes decidable for reducers over unbounded input lists of bounded integers. We propose an efficient reduction of commutativity checking to conventional assertion checking and report experimental results using various off-the-shelf program analyzers.

Keywords

Diophantine Equation Symbolic Execution Input List Read Head Normal Operation Mode 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Beyer, D., Keremoglu, M.E.: cPAchecker: A tool for configurable software verification. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 184–190. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  2. 2.
    Cadar, C., Dunbar, D., Engler, D.R.: KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In: OSDI, pp. 209–224. ACM (2008)Google Scholar
  3. 3.
    Clarke, E., Kroning, D., Lerda, F.: A tool for checking ANSI-C programs. In: Jensen, K., Podelski, A. (eds.) TACAS 2004. LNCS, vol. 2988, pp. 168–176. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  4. 4.
    Csallner, C., Fegaras, L., Li, C.: New ideas track: testing MapReduce-style programs. In: FSE, pp. 504–507 (2011)Google Scholar
  5. 5.
    de Moura, L., Bjørner, N.S.: Z3: An efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    Dimitrov, D., Raychev, V., Vechev, M., Koskinen, E.: Commutativity race detection. In: PLDI, p. 33. ACM (2014)Google Scholar
  7. 7.
    Felsing, D., Grebing, S., Klebanov, V., Rummer, P., Ulbrich, M.: Automating regression verification. In: ASE, pp. 349–360 (2014)Google Scholar
  8. 8.
    Ganesh, V., Dill, D.L.: A decision procedure for bit-vectors and arrays. In: Damm, W., Hermanns, H. (eds.) CAV 2007. LNCS, vol. 4590, pp. 519–531. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  9. 9.
    Halbwachs, N., Peron, M.: Discovering properties about arrays in simple programs. In: PLDI (2008)Google Scholar
  10. 10.
    Hawblitzel, C., Kawaguchi, M., Lahiri, S.K., Rebêlo, H.: Towards modularly comparing programs using automated theorem provers. In: Bonacina, M.P. (ed.) CADE 2013. LNCS, vol. 7898, pp. 282–299. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  11. 11.
    Hueske, F., Peters, M., Sax, M.J., Rheinländer, A., Bergmann, R., Krettek, A., Tzoumas, K.: Opening the black boxes in data flow optimization. VLDB Endowment 5(11) (2012)Google Scholar
  12. 12.
    Hungerford, T.W.: Algebra. Graduate Texts in Mathematics, vol. 73. Springer (2003)Google Scholar
  13. 13.
    Jones, J.P.: Universal diophantine equation. Journal of Symbolic Logic 47(3) (1982)Google Scholar
  14. 14.
    Kovacs, M., Seidl, H., Finkbeiner, B.: Relational abstract interpretation for the verification of 2-hypersafety properties. In: CCS, pp. 211–222. ACM (2013)Google Scholar
  15. 15.
    Kulkarni, M., Nguyen, D., Prountzos, D., Sui, X., Pingali, K.: Exploiting the commutativity lattice. ACM SIGPLAN Notices 46(6) (2011)Google Scholar
  16. 16.
    Rabin, M.O., Scott, D.: Finite automata and their decision problems. IBM Journal Res. Dev. 3(2) (1959)Google Scholar
  17. 17.
    Rinard, M., Diniz, P.C.: Commutativity analysis: A new analysis technique for parallelizing compilers. TOPLAS 19(6), 942–991 (1997)CrossRefGoogle Scholar
  18. 18.
    Sinha, N., Singhania, N., Chandra, S., Sridharan, M.: Alternate and learn: Finding witnesses without looking all over. In: Madhusudan, P., Seshia, S.A. (eds.) CAV 2012. LNCS, vol. 7358, pp. 599–615. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  19. 19.
    Xiao, T., Zhang, J., Zhou, H., Guo, Z., McDirmid, S., Lin, W., Chen, W., Zhou, L.: Nondeterminism in MapReduce considered harmful? an empirical study on non-commutative aggregators in MapReduce programs. In: Companion Proceedings of ICSE, pp. 44–53 (2014)Google Scholar
  20. 20.
    Xu, Z., Hirzel, M., Rothermel, G.: Semantic characterization of MapReduce workloads. In: IISWC, pp. 87–97 (2013)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Yu-Fang Chen
    • 1
  • Chih-Duo Hong
    • 1
  • Nishant Sinha
    • 2
  • Bow-Yaw Wang
    • 1
  1. 1.Institute of Information ScienceAcademia SinicaTaipeiTaiwan
  2. 2.IBM ResearchNew DelhiIndia

Personalised recommendations