Design and evaluation of a multi-threaded architecture for parallel graph reduction

Abstract

Main limitations of distributed memory machines involving thousands of processors, deal with network latencies for remote data and/or programs accesses. In this paper we present multithreading techniques for parallel graph reduction model that can tolerate latencies of thousands of cycles by dynamically creating a set of threads. Efficiency is achieved by introducing fast context switch, non preemptive threads and comparative long run-lenght threads (thousand of cycles). Conventional multithreaded techniques deals with statically defined threads (at compile time) and dynamically or statically scheduling of the work to be performed. Parallel graph reduction is an attractive model because of its simplicity and inherently distributed nature: parallelism is introduced by parallel evaluation of function parameters corresponding to dynamically created set of threads. The single assignment feature and the absence of side effect, since the internal representation of programs remains purely functional, overcomes the difficulties of synchronized accesses to shared data and scheduling of parallel activities.