Abstract
Single Assignment C (SaC) is a data parallel programming language that combines an imperative looking syntax, closely resembling that of C, with purely functional, state-free semantics. Again unlike the functional programming mainstream, that puts the emphasis on lists and trees, the focus of SaC as a data-parallel language is on (truly) multi-dimensional arrays. SaC arrays are not just loose collections of data cells or memory address ranges as in many imperative and object-oriented languages. Neither are they explicitly managed, stateful data objects as in some functional languages, may it be for language design or for performance considerations. SaC arrays are indeed purely functional, immutable, state-free, first-class values.
The array type system of SaC allows functions to abstract not only from the size of vectors or matrices but even from the number of array dimensions. Programs can and should be written in a mostly index-free style with functions consuming entire arrays as arguments and producing entire arrays as results. SaC supports a highly generic and compositional programming style that composes applications in layers of abstractions from universally applicable building blocks. The design of SaC aims at combining high productivity in software engineering of compute-intensive applications with high performance in program execution on today’s multi- and many-core computing systems.
These CEFP lecture notes provide a balanced introduction to language design and programming methodology of SaC, but our main focus is on the in-depth description and illustration of the associated compilation technology. It is literally a long way down from state-free, functional program code involving multi-dimensional, truly state-free arrays to efficient execution on modern computing machines. Fully compiler-directed parallelisation for symmetric multi-socket, multi-core, hyper-threaded server systems, CUDA-enabled graphics accelerators, workstation clusters or heterogeneous systems from a single architecture-agnostic source program adds a fair deal of complexity. Over the years, we have developed an intricate program transformation and optimisation machinery for this purpose. These lecture notes provide the first comprehensive presentation of our compilation technology as a whole.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sutter, H.: The free lunch is over: a fundamental turn towards concurrency in software. Dr. Dobb’s J. 30, 202–210 (2005)
Intel: Product Brief: Intel Xeon Processor 7500 Series. Technical report (2010)
AMD: AMD Opteron 6000 Series Platform Quick Reference Guide. Technical report, AMD (2011)
Sun/Oracle: Oracle’s SPARC T3-1, SPARC T3-2, SPARC T3-4 and SPARC T3-1B Server Architecture. Whitepaper, Oracle (2011)
Shin, J.L., Huang, D., Petrick, B., et al.: A 40 nm 16-core 128-thread SPARC SoC processor. IEEE J. Solid-State Circ. 46, 131–144 (2011)
Strohmaier, E., Dongarra, J., Simon, H., Meuer, M.: 48th top500 list. Technical report (2016). www.top500.org
Greenhalgh, P.: Big.LITTLE Processing with ARM Cortex-A15 and Cortex-A7. Technical report, EE Times (2011)
Jeff, B.: big.LITTLE Technology Moves Towards Fully Heterogeneous Global Task Scheduling. Arm whitepaper, ARM (2013)
Chrysos, G.: Intel Xeon Phi coprocessor (codename Knights Corner). In: Hot Chips 24 Symposium (HCS 2012), Cupertino, USA. IEEE (2012)
Chrysos, G.: Intel Xeon Phi Coprocessor: The Architecture. Technical report, Intel Corp. (2013)
Jeffers, J., Reinders, J.: Intel Xeon Phi Coprocessor High Performance Programming. Morgan Kaufmann, San Francisco (2013)
Grelck, C., Scholz, S.B.: SAC: a functional array language for efficient multithreaded execution. Int. J. Parallel Program. 34, 383–427 (2006)
Grelck, C., Scholz, S.B.: SAC: off-the-shelf support for data-parallelism on multicores. In: Glew, N., Blelloch, G. (eds.) 2nd Workshop on Declarative Aspects of Multicore Programming (DAMP 2007), Nice, France, pp. 25–33. ACM Press (2007)
Falkoff, A., Iverson, K.: The design of APL. IBM J. Res. Dev. 17, 324–334 (1973)
International Standards Organization: Programming Language APL, Extended. ISO N93.03. ISO (1993)
Hui, R.: An Implementation of J. Iverson Software Inc., Toronto (1992)
Jenkins, M.: Q’Nial: a portable interpreter for the nested interactive array language Nial. Softw. Pract. Exp. 19, 111–126 (1989)
Bousias, K., Guang, L., Jesshope, C., Lankamp, M.: Implementation and evaluation of a microthread architecture. J. Syst. Archit. 55, 149–161 (2009)
Macht, T., Grelck, C.: SAC goes cluster: from functional array programming to distributed memory array processing. In: Knoop, J. (ed.) 18th Workshop on Programming Languages and Foundations of Programming, Pörtschach am Wörthersee, Austria, Technical University of Vienna (2015)
Diogo, M., Grelck, C.: Towards heterogeneous computing without heterogeneous programming. In: Loidl, H.-W., Peña, R. (eds.) TFP 2012. LNCS, vol. 7829, pp. 279–294. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40447-4_18
Grelck, C.: Single Assignment C (SAC) high productivity meets high performance. In: Zsók, V., Horváth, Z., Plasmeijer, R. (eds.) CEFP 2011. LNCS, vol. 7241, pp. 207–278. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32096-5_5
Schildt, H.: American National Standards Institute, International Organization for Standardization, International Electrotechnical Commission, ISO/IEC JTC 1: The annotated ANSI C standard: American National Standard for Programming Languages C: ANSI/ ISO 9899-1990. McGraw-Hill (1990)
Kernighan, B., Ritchie, D.: The C Programming Language, 2nd edn. Prentice-Hall, Englewood Cliffs (1988)
Iverson, K.: A Programming Language. Wiley, Hoboken (1962)
Iverson, K.: Programming in J. Iverson Software Inc., Toronto (1991)
Burke, C.: J and APL. Iverson Software Inc., Toronto (1996)
Jenkins, M., Jenkins, W.: The Q’Nial Language and Reference Manual. Nial Systems Ltd., Ottawa (1993)
Mullin, L.R., Jenkins, M.: A comparison of array theory and a mathematics of arrays. In: Arrays, Functional Languages and Parallel Systems, pp. 237–269. Kluwer Academic Publishers (1991)
Mullin, L.R., Jenkins, M.: Effective data parallel computation using the Psi calculus. Concurr. - Pract. Exp. 8, 499–515 (1996)
Dagum, L., Menon, R.: OpenMP: an industry-standard API for shared-memory programming. IEEE Trans. Comput. Sci. Eng. 5, 46–55 (1998)
Chapman, B., Jost, G., van der Pas, R.: Using OpenMP: Portable Shared Memory Parallel Programming. MIT Press, Cambridge (2008)
Gropp, W., Lusk, E., Skjellum, A.: Using MPI: Portable Parallel Programming with the Message Passing Interface. MIT Press, Cambridge (1994)
Douma, R.: Nested arrays in single assignment C. Master’s thesis, University of Amsterdam, Amsterdam, Netherlands (2011)
Trojahner, K., Grelck, C.: Dependently typed array programs don’t go wrong. J. Log. Algebraic Program. 78, 643–664 (2009)
Trojahner, K.: QUBE – array programming with dependent types. Ph.D. thesis, University of Lübeck, Institute of Software Technology and Programming Languages, Lübeck, Germany (2011)
Grelck, C., Tang, F.: Towards hybrid array types in SAC. In: Stolz, V., Trancón Widemann, B. (eds.) 7. GI Arbeitstagung Programmiersprachen (ATPS 2014), Software Engineering Workshops (SE-WS 2014), Kiel, Germany, CEUR Workshop Proceedings, vol. 1129 (2014)
Chamberlain, B., Callahan, D., Zima, H.: Parallel programmability and the Chapel language. Int. J. High Perform. Comput. Appl. 21, 291–312 (2007)
Grelck, C., Scholz, S.B., Shafarenko, A.: Asynchronous stream processing with S-Net. Int. J. Parallel Prog. 38, 38–67 (2010)
Scholz, S.B.: Single Assignment C—efficient support for high-level array operations in a functional setting. J. Func. Program. 13, 1005–1059 (2003)
Grelck, C., van Deurzen, T., Herhut, S., Scholz, S.B.: Asynchronous adaptive optimisation for generic data-parallel array programming. Concurr. Comput.: Pract. Exp. 24, 499–516 (2012)
Grelck, C., Wiesinger, H.: Next generation asynchronous adaptive specialization for data-parallel functional array processing in SAC. In: Plasmeijer, R., Achten, P., Koopman, P. (eds.) 25th International Symposium Implementation and Application of Functional Languages (IFL 2013), Nijmegen, Netherlands. ACM (2014)
Scholz, S.B.: With-loop-folding in SAC - condensing consecutive array operations. In: Clack, C., Hammond, K., Davie, T. (eds.) IFL 1997. LNCS, vol. 1467, pp. 72–91. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0055425
Scholz, S.B.: A case study: effects of with-loop-folding on the NAS benchmark MG in Sac. In: Hammond, K., Davie, T., Clack, C. (eds.) IFL 1998. LNCS, vol. 1595, pp. 216–228. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-48515-5_14
Grelck, C., Hinckfuß, K., Scholz, S.B.: With-loop fusion for data locality and parallelism. In: Butterfield, A., Grelck, C., Huch, F. (eds.) IFL 2005. LNCS, vol. 4015, pp. 178–195. Springer, Heidelberg (2006). https://doi.org/10.1007/11964681_11
Grelck, C., Scholz, S.B., Trojahner, K.: With-loop scalarization – merging nested array operations. In: Trinder, P., Michaelson, G.J., Peña, R. (eds.) IFL 2003. LNCS, vol. 3145, pp. 118–134. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-27861-0_8
Grelck, C.: Improving cache effectiveness through array data layout manipulation in SAC. In: Mohnen, M., Koopman, P. (eds.) IFL 2000. LNCS, vol. 2011, pp. 231–248. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-45361-X_14
Grelck, C., Kreye, D., Scholz, S.B.: On code generation for multi-generator with-loops in SAC. In: Koopman, P., Clack, C. (eds.) IFL 1999. LNCS, vol. 1868, pp. 77–94. Springer, Heidelberg (2000). https://doi.org/10.1007/10722298_5
Kreye, D.: Zur Generierung von effizient ausführbarem Code aus SAC-spezifischen Schleifenkonstrukten (1998)
Bernecky, R., Herhut, S., Scholz, S.B., Trojahner, K., Grelck, C., Shafarenko, A.: Index vector elimination – making index vectors affordable. In: Horváth, Z., Zsók, V., Butterfield, A. (eds.) IFL 2006. LNCS, vol. 4449, pp. 19–36. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74130-5_2
Wilson, P.R.: Uniprocessor garbage collection techniques. In: Bekkers, Y., Cohen, J. (eds.) IWMM 1992. LNCS, vol. 637, pp. 1–42. Springer, Heidelberg (1992). https://doi.org/10.1007/BFb0017182
Jones, R.: Garbage Collection: Algorithms for Automatic Dynamic Memory Management. Wiley, New York City (1999)
Marlow, S., Harris, T., James, R.P., Peyton Jones, S.: Parallel generational-copying garbage collection with a block-structured heap. In: 7th International Symposium on Memory Management (ISMM 2008), pp. 11–20. ACM (2008)
Hudak, P., Bloss, A.: The aggregate update problem in functional programming systems. In: 12th ACM Symposium on Principles of Programming Languages (POPL 1985), New Orleans, Louisiana, USA, pp. 300–313. ACM Press (1985)
Collins, G.E.: A method for overlapping and erasure of lists. Commun. ACM 3, 655–657 (1960)
Grelck, C., Trojahner, K.: Implicit memory management for SAC. In: Grelck, C., Huch, F. (eds.) 16th International Workshop Implementation and Application of Functional Languages, IFL 2004, University of Kiel, Institute of Computer Science and Applied Mathematics, Technical Report 0408, pp. 335–348 (2004)
Grelck, C., Scholz, S.B.: Efficient heap management for declarative data parallel programming on multicores. In: Hermenegildo, M., Peterson, L., Glew, N. (eds.) 3rd Workshop on Declarative Aspects of Multicore Programming (DAMP 2008), San Francisco, CA, USA, pp. 17–31. ACM Press (2008)
Trinder, P., Hammond, K., Loidl, H.W., Jones, S.P.: Algorithm + Strategy = Parallelism. J. Funct. Program. 8, 23–60 (1998)
Grelck, C.: A multithreaded compiler backend for high-level array programming. In: Hamza, M.H. (eds.) 2nd International Conference on Parallel and Distributed Computing and Networks (PDCN 2003), Innsbruck, Austria, pp. 478–484. ACTA Press (2003)
Grelck, C.: Shared memory multiprocessor support for functional array processing in SAC. J. Funct. Program. 15, 353–401 (2005)
Kirk, D., Hwu, W.: Programming Massively Parallel Processors: A Hands-on Approach. Morgan Kaufmann, San Francisco (2010)
Guo, J., Thiyagalingam, J., Scholz, S.B.: Breaking the GPU programming barrier with the auto-parallelising SAC compiler. In: 6th Workshop on Declarative Aspects of Multicore Programming (DAMP 2011), Austin, USA, pp. 15–24. ACM Press (2011)
Bernard, T., Grelck, C., Jesshope, C.: On the compilation of a language for general concurrent target architectures. Parallel Process. Lett. 20, 51–69 (2010)
Grelck, C., et al.: Compiling the functional data-parallel language SAC for microgrids of self-adaptive virtual processors. In: 14th Workshop on Compilers for Parallel Computing (CPC’09). IBM Research Center, Zürich (2009)
Herhut, S., Joslin, C., Scholz, S.B., Grelck, C.: Truly nested data-parallelism: compiling SAC to the microgrid architecture. In: 21st Symposium on Implementation and Application of Functional Languages (IFL 2009), South Orange, NJ, USA. Number SHU-TR-CS-2009-09-1, Seton Hall University (2009)
Herhut, S., Joslin, C., Scholz, S.B., Poss, R., Grelck, C.: Concurrent non-deferred reference counting on the microgrid: first experiences. In: Hage, J., Morazán, M.T. (eds.) IFL 2010. LNCS, vol. 6647, pp. 185–202. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24276-2_12
Ramesh, B., Ribbens, C.J., Varadarajan, S.: Is it time to rethink distributed shared memory systems? In: 17th IEEE International Conference on Parallel and Distributed Systems (ICPADS 2011), pp. 212–219, ID 1 (2011)
Šinkarovs, A., Scholz, S., Bernecky, R., Douma, R., Grelck, C.: SAC/C formulations of the all-pairs N-body problem and their performance on SMPs and GPGPUs. Concurr. Comput.: Pract. Exp. 26, 952–971 (2014). https://doi.org/10.1002/cpe.3078
Wieser, V., et al.: Combining high productivity and high performance in image processing using Single Assignment C on multi-core CPUs and many-core GPUs. J. Electr. Imaging 21, 021116 (2012)
Grelck, C., Douma, R.: SAC on a Niagara T3-4 server: lessons and experiences. In: de Bosschere, K., D’Hollander, E., Joubert, G., Padua, D., Peters, F., Sawyer, M. (eds.) Applications, Tools and Techniques on the Road to Exascale Computing. Advances in Parallel Computing, vol. 22, pp. 289–296. IOS Press, Amsterdam (2012)
Rolls, D., Joslin, C., Kudryavtsev, A., Scholz, S.B., Shafarenko, A.: Numerical simulations of unsteady shock wave interactions using SaC and Fortran-90. In: Malyshkin, V. (ed.) PaCT 2009. LNCS, vol. 5698, pp. 445–456. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03275-2_44
Shafarenko, A., Scholz, S.B., Herhut, S., Grelck, C., Trojahner, K.: Implementing a numerical solution of the KPI equation using single assignment C: lessons and experiences. In: Butterfield, A., Grelck, C., Huch, F. (eds.) IFL 2005. LNCS, vol. 4015, pp. 160–177. Springer, Heidelberg (2006). https://doi.org/10.1007/11964681_10
Grelck, C., Scholz, S.B.: Towards an efficient functional implementation of the NAS benchmark FT. In: Malyshkin, V.E. (ed.) PaCT 2003. LNCS, vol. 2763, pp. 230–235. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-45145-7_20
Grelck, C.: Implementing the NAS benchmark MG in SAC. In: Prasanna, V.K., Westrom, G. (eds.) 16th International Parallel and Distributed Processing Symposium (IPDPS 2002), Fort Lauderdale, USA. IEEE Computer Society Press (2002)
Bailey, D., et al.: The NAS parallel benchmarks. Int. J. Supercomput. Appl. 5, 63–73 (1991)
Grelck, C., Scholz, S.B.: HPF vs. SAC—a case study. In: Bode, A., Ludwig, T., Karl, W., Wismüller, R. (eds.) Euro-Par 2000. LNCS, vol. 1900, pp. 620–624. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-44520-X_87
Groningen, J.H.G.: The implementation and efficiency of arrays in Clean 1.1. In: Kluge, W. (ed.) IFL 1996. LNCS, vol. 1268, pp. 105–124. Springer, Heidelberg (1997). https://doi.org/10.1007/3-540-63237-9_21
Zörner, T.: Numerical analysis and functional programming. In: Hammond, K., Davie, T., Clack, C. (eds.) 10th International Workshop on Implementation of Functional Languages (IFL 1998), pp. 27–48. University College, London (1998)
Chakravarty, M.M.T., Keller, G.: An approach to fast arrays in Haskell. In: Jeuring, J., Peyton Jones, S. (eds.) AFP 2002. LNCS, vol. 2638, pp. 27–58. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-44833-4_2
Chakravarty, M.M.T., Leshchinskiy, R., Peyton Jones, S., Keller, G., Marlow, S.: Data parallel Haskell: a status report. In: 2nd Workshop on Declarative Aspects of Multicore Programming (DAMP 2007), Nice, France. ACM Press (2007)
Peyton Jones, S., Leshchinskiy, R., Keller, G., Chakravarty, M.: Harnessing the multicores: nested data parallelism in Haskell. In: IARCS Annual Conference on Foundations of Software Technology and Theoretical Computer Science (FSTTCS 2008), Bangalore, India (2008) 383–414
Blelloch, G., Chatterjee, S., Hardwick, J., Sipelstein, J., Zagha, M.: Implementation of a portable nested data-parallel language. J. Parallel Distrib. Comput. 21, 4–14 (1994)
McGraw, J., Skedzielewski, S., Allan, S., Oldehoeft, R., et al.: SISAL: Streams and Iteration in a Single Assignment Language: Reference Manual Version 1.2. M 146. Lawrence Livermore National Laboratory, Livermore, California, USA (1985)
Cann, D.: Retire Fortran? A debate rekindled. Commun. ACM 35, 81–89 (1992)
Oldehoeft, R.: Implementing arrays in SISAL 2.0. In: 2nd SISAL Users Conference, San Diego, CA, USA, pp. 209–222. Lawrence Livermore National Laboratory (1992)
Feo, J., Miller, P., Skedzielewski, S.K., Denton, S., Solomon, C.: SISAL 90. In: Böhm, A., Feo, J. (eds.) Conference on High Performance Functional Computing (HPFC 1995), Denver, Colorado, USA, pp. 35–47. Lawrence Livermore National Laboratory, Livermore (1995)
Hammes, J.P., Draper, B.A., Böhm, W.: Sassy: a language and optimizing compiler for image processing on reconfigurable computing systems. ICVS 1999. LNCS, vol. 1542, pp. 83–97. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-49256-9_6
Najjar, W., Böhm, W., Draper, B., Hammes, J., et al.: High-level language abstraction for reconfigurable computing. IEEE Comput. 36, 63–69 (2003)
Bernecky, R.: The role of APL and J in high-performance computation. APL Quote Quad 24, 17–32 (1993)
van der Walt, S., Colbert, S., Varoquaux, G.: The NumPy array: a structure for efficient numerical computation. Comput. Sci. Eng. 13, 22–30 (2011)
Kristensen, M., Vinter, B.: Numerical Python for scalable architectures. In: 4th Conference on Partitioned Global Address Space Programming Model (PGAS 2010), New York, NY, USA. ACM Press (2010)
Scholz, S.B.: Single Assignment C - functional programming using imperative style. In: Glauert, J., (ed.) 6th International Workshop on Implementation of Functional Languages (IFL 1994), Norwich, England, UK, pp. 21.1-21.13. University of East Anglia, Norwich (1994)
Acknowledgements
The work described in this paper is the result of more than 15 years of research and development conducted by an ever changing group of people working at a variety of places. From the nucleus at the University of Kiel, Germany, in the mid-1990s the virus spread to the University of Lübeck, Germany, the University of Hertfordshire, England, the University of Toronto, Canada, the University of Amsterdam, Netherlands, and recently to Heriot-Watt University, Scotland. Apart from the internal funds of these universities, three European projects have been instrumental in supporting our activities: Æther, Apple-CORE and Advance.
First and foremost, I would like to thank Sven-Bodo Scholz for many years of intense and fruitful collaboration. The original proposal of a no-frills functional language with a C-like syntax and particular support for arrays was his [91]. Apart from the name and these three design principles not too much in today’s SaC resembles the original proposal, though.
My special thanks go to those who helped to shape SaC by years of continued work: Dietmar Kreye, Robert Bernecky, Stephan Herhut, Kai Trojahner and Artem Shinkarov. Over the years many more have contributed to advancing SaC to its current state. I take the opportunity to thank (in roughly temporal order) Henning Wolf, Arne Sievers, Sören Schwartz, Björn Schierau, Helge Ernst, Jan-Hendrik Schöler, Nico Marcussen-Wulff, Markus Bradtke, Borg Enders, Michael Werner, Karsten Hinckfuß, Steffen Kuthe, Florian Massel, Andreas Gudian, Jan-Henrik Baumgarten, Theo van Klaveren, Daoen Pan, Sonia Chouaieb, Florian Büther, Torben Gerhards, Carl Joslin, Jing Guo, Hraban Luyat, Abhishek Lal, Santanu Dash, Daniel Rolls, Zheng Zhangzheng, Aram Visser, Tim van Deurzen, Roeland Douma, Fangyong Tang, Pablo Rauzy, Miguel Diogo, Heinz Wiesinger, Jaroslav Sykora, Raphaël Poss, Victor Azizi, Stuart Gordon, Hans-Nikolai Viessmann, Thomas Macht, Cédric Blom, Nikolaos Sarris for their invaluable work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Grelck, C. (2019). Single Assignment C (SAC). In: Zsók, V., Porkoláb, Z., Horváth, Z. (eds) Central European Functional Programming School. CEFP 2015. Lecture Notes in Computer Science(), vol 10094. Springer, Cham. https://doi.org/10.1007/978-3-030-28346-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-28346-9_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-28345-2
Online ISBN: 978-3-030-28346-9
eBook Packages: Computer ScienceComputer Science (R0)