Abstract
Recent technology advances in high-density, byte-addressable non-volatile memory (NVM) and low-latency interconnects have enabled building large-scale systems with a large disaggregated fabric-attached memory (FAM) pool shared across decentralized compute nodes. The OpenFAM API is an API for programming with persistent FAM that is inspired by the OpenSHMEM partitioned global address space (PGAS) model. Unlike OpenSHMEM, where each node contributes local memory toward a logically shared global address space, FAM isn’t associated with a particular node and can be addressed directly from any node without the cooperation or involvement of another node. The OpenFAM API enables programmers to manage FAM allocations, access FAM-resident data structures, and order FAM operations. Because state in FAM can survive program termination, the API also provides interfaces for naming and managing data beyond the lifetime of a single program invocation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Asanovic, K.: FireBox: a hardware building block for 2020 warehouse-scale computers. Keynote at USENIX Conference on File and Storage Technologies (FAST) 2014
Binnig, C., Crotty, A., Galakatos, A., Kraska, T., Zamanian, E.: The end of slow networks: it’s time for a redesign. Proc. VLDB Endow. 9(7), 528–539 (2016)
Cavium, ThunderX2 ARM Processors. https://www.cavium.com/product-thunderx2-arm-processors.html. Accessed 22 July 2018
Cray: Cray XC Series GNI and DMAPP API User Guide (CLE 6.0.UP05) S-2446. https://pubs.cray.com/pdf-attachments/attachment?pubId=00478935-DA&attachmentId=pub_00478935-DA.pdf. Accessed 28 July 2018
Dulloor, S.R., et al.: Data tiering in heterogeneous memory systems. In: Proceedings of 11th ACM European Conference on Computer Systems (EuroSys) (2016)
Faraboschi, P., Keeton, K., Marsland, T., Milojicic, D.: Beyond processor-centric operating systems. In: Proceedings of Workshop on Hot Topics in Operating Systems (HotOS) (2015)
Funk, M.: Drilling down into the machine from HPE. The Next Platform, 4 January 2016. https://www.nextplatform.com/2016/01/04/drilling-down-into-the-machine-from-hpe/. Accessed 22 July 2018
Funk, M.: Drilling into the CCIX coherence standard. The Next Platform, 13 July 2016. https://www.nextplatform.com/2016/01/04/drilling-down-into-the-machine-from-hpe/. Accessed 22 July 2018
Gen-Z Consortium: Gen-Z Atomics, October 2017. https://genzconsortium.org/wp-content/uploads/2018/05/1711_Gen-Z-Atomics.pdf. Accessed 25 July 2018
Gen-Z Consortium: Gen-Z Coherency, October 2017. http://genzconsortium.org/wp-content/uploads/2017/08/Gen-Z-Coherency.pdf. Accessed 25 July 2018
Gen-Z Consortium: Gen-Z Core Specification. https://genzconsortium.org/specification/core-specification-1-0/. Accessed 22 July 2018
Gen-Z Consortium: Gen-Z ZMMU and Memory Interleave, June 2018. https://genzconsortium.org/wp-content/uploads/2018/06/Gen-Z-MMU-and-Memory-Interleave-1.pdf. Accessed 25 July 2018
Hewlett Packard Enterprise: HPE Persistent Memory: The Performance of Memory with the Persistence of Storage. https://www.hpe.com/us/en/servers/persistent-memory.html. Accessed 25 July 2018
Hewlett Packard Enterprise: New HPE Pointnext Capabilities Accelerate Transition to Memory-Driven Computing. https://news.hpe.com/new-hpe-pointnext-capabilities-accelerate-transition-to-memory-driven-computing/. Accessed 22 July 2018
Huawei: High Throughput Computing Data Center Architecture: Thinking of Data Center 3.0. http://www.huawei.com/ilink/en/download/HW_349607. Accessed 25 July 2018
Intel: Facebook Collaborate on Future Data Center Rack Technologies, Intel Newsroom, 16 January 2013. http://goo.gl/6h2Ut. Accessed 25 July 2018
Intel: Intel Omni-Path Architecture (Intel OPA) Driving Exascale Computing and HPC. http://www.intel.com/content/www/us/en/high-performance-computing-fabrics/omni-path-architecture-fabric-overview.html. Accessed 25 July 2018
Intel Rack Scale Design: Data Center Agility at Scale. http://www.intel.com/content/www/us/en/architecture-and-technology/rack-scale-design-overview.html. Accessed 25 July 2018
High Bandwidth Memory (HBM) DRAM, JEDEC Standard JESD235A, November 2015. http://www.jedec.org/standards-documents/results/jesd235. Accessed 25 July 2018
Keeton, K.: Memory-driven computing. Keynote at 33rd International Conference on Massive Storage Systems and Technology (MSST), May 2017. http://storageconference.us/2017/Presentations/Keeton.pdf
Kerrisk, M.: Linux/UNIX System Programming: POSIX Shared Memory, February 2015. http://man7.org/training/download/posix_shm_slides.pdf. Accessed 30 July 2018
Lim, K., et al.: System-level implications of disaggregated memory. In: Proceedings of International Symposium on High Performance Computer Architecture (HPCA) (2012)
Lockwood, G.: What’s so bad about POSIX I/O?. The Next Platform, 11 September 2017. https://www.nextplatform.com/2017/09/11/whats-bad-posix-io/. Accessed 29 July 2018
Loesing, S., Pilman, M., Etter, T., Kossmann, D.: On the design and scalability of distributed shared-data databases. In: Proceedings of 15th ACM SIGMOD International Conference on Management of Data (SIGMOD), pp. 663–676 (2015)
Novakovic, S., Daglis, A., Bugnion, E., Falsafi, B., Grot, B.: Scale-out NUMA. In: Proceedings of Symposium on Architectural Support for Programming Languages and Operating Systems (ASPLOS) (2014)
OpenCAPI Overview, 14 October 2016. http://opencapi.org/wp-content/uploads/2016/09/OpenCAPI-Overview.10.14.16.pdf. Accessed 25 July 2018
OpenSHMEM API Version 1.4. http://openshmem.org/site/sites/default/site_files/OpenSHMEM-1.4.pdf. Accessed 25 July 2018
The OpenFAM API. https://github.com/OpenFAM/API. Accessed 27 June 2018
pmem.io: Persistent Memory Programming. http://pmem.io. Accessed 29 July 2018
Qureshi, M.K., Srinivasan, V., Rivers, J.A.: Scalable high performance main memory system using phase-change memory technology. In: Proceedings of 36th International Symposium on Computer Architecture (ISCA), pp. 24–33 (2009)
RDMA Consortium: http://www.rdmaconsortium.org. Accessed 25 July 2018
Seyedi, M.A., Fiorentino, M.: Silicon photonics; ring modulator transmitters. In: Guenther, R., Steel, D. (eds.) Encyclopedia of Modern Optics, 2nd edn, vol. 4, pp. 216–223 (2018)
Shah, H., Marti, F., Noureddine, W., Eiriksson, A., Sharp, R.: Remote Direct Memory Access (RDMA) Protocol Extensions, IETF RFC 7306, June 2014. https://tools.ietf.org/html/rfc7306. Accessed 25 July 2018
Spelman, L.: Reimagining the Data Center Memory and Storage Hierarchy, Intel Newsroom, 30 May 2018. https://newsroom.intel.com/editorials/re-architecting-data-center-memory-storage-hierarchy/. Accessed 25 July 2018
Storage Networking Industry Association (SNIA): NVM Programming Model (NPM), Version 1.2. https://www.snia.org/sites/default/files/technical_work/final/NVMProgrammingModel_v1.2.pdf. Accessed 29 July 2018
Strukov, D.B., Snider, G.S., Steward, D.R., Williams, R.S.: The missing memristor found. Nature 453, 80–83 (2008)
Xie, Y.: Modeling, architecture and applications for emerging memory technologies. IEEE Design and Test of Computers 28(1), 44–51 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Keeton, K., Singhal, S., Raymond, M. (2019). The OpenFAM API: A Programming Model for Disaggregated Persistent Memory. In: Pophale, S., Imam, N., Aderholdt, F., Gorentla Venkata, M. (eds) OpenSHMEM and Related Technologies. OpenSHMEM in the Era of Extreme Heterogeneity. OpenSHMEM 2018. Lecture Notes in Computer Science(), vol 11283. Springer, Cham. https://doi.org/10.1007/978-3-030-04918-8_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-04918-8_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-04917-1
Online ISBN: 978-3-030-04918-8
eBook Packages: Computer ScienceComputer Science (R0)