Automatic reduction tree generation for fine-grain parallel architectures when iteration count is unknown

Abstract

Over the last few years, the research trend in future generation high-performance computing systems has been moving toward a multi-threaded parallel architectures. Thus the importance to exploit and control parallelism has growing parallel activities must be both synchronized and reduced. In fine-grain parallel computation, designing efficient micro synchronization, at the same level of granularity as the grain size, is essential for implementation. This article discusses methods of synchronizing parallel activities, focusing on the case when the number of activities to be gathered is determined at run time. A new reduction graph, without loss of parallelism, is proposed. It is especially useful if the number of parallel activities is determined dynamically. This method is basically developed for instruction-level dataflow computers. Its full potential should be realized when trends in parallel processing return to finer grain sizes.