Collective Communication and Computation

Abstract

Counting votes after elections is an early example of massively parallel computing. In a large country, there are millions and millions of ballots cast in thousands and thousands of polling places distributed all over the country. It is clearly a bad idea to ship all the ballot boxes to the capital and have them counted by a single clerk. Assuming the clerk can count one vote per second 24 h a day, counting 100 000 000 votes would take more than three years. Rather, the votes are counted in each station and the counts are aggregated in a hierarchy of election offices (e.g., polling place, city, county, state, capital). These counts are very compact and can be communicated by telephone in a few seconds. Overall, a well-organized counting process can yield a preliminary result in a few hours. We shall see that this is an example of a global reduction and that efficient parallel algorithms follow a very similar pattern.