On Using Erlang for Parallelization

Abstract

Erlang is a functional language that allows programmers to employ shared nothing processes and asynchronous message passing for parts of applications which can naturally execute concurrently. This paper reports on a non-trivial effort to use these concurrency features to parallelize a widely used application written in Erlang. More specifically, we present how Dialyzer, consisting of about 30,000 lines of quite complex and sequential Erlang code, has been parallelized using the language primitives and report on the challenges that were involved and lessons learned from engaging in this feat. In addition, we evaluate the performance improvements that were achieved on a variety of modern hardware. On a 32-core AMD “Bulldozer” machine, the parallel version of Dialyzer can now complete the analysis of Erlang/OTP’s code base, consisting of about two million lines of Erlang code, in about six minutes compared to more than one hour twenty minutes that the sequential version (still) requires.

This work has been partially supported by the European Union grant IST-2011-287510 RELEASE: A High-Level Paradigm for Reliable Large-scale Server Software.