Abstract
The NIX project targeted manycore architectures in which only a small subset of the cores could run kernel code. Consider N cores on a die arranged in a grid, with an on-die network providing a totally symmetric network and access to common memory. Let us assume that only \(\sqrt{N}\) of the cores can run a kernel and the rest can only run user mode code. NIX was intended to provide a kernel for such a system. The hypothetical target was a 1,024 core system, as several vendors hinted that was reasonable in the 2016 time frame. We picked this number as it was large enough to far exceed any possible number of user processes: the Plan 9 systems on which NIX was based use only a small fraction of this number. Further, the realities of memory bandwidth will limit the number of active cores to small single digits. We also wanted to have a system with good support for Go, the then-new language from Google. At the time, Go was a statically linked language and a typical binary was about 1 MiB. This influenced the design of the virtual memory system in NIX. The base page size was expanded from 4,096 bytes to 2 MiB. We further extended Plan 9’s virtual memory model to transparently support GiB pages, but via a simple heuristic that made it far less complex than existing systems. The target system had a 32:1 ratio of application cores to kernel cores (a close match to the Blue Gene/Q system at 16:1), and it was designed for space sharing, not time sharing, and had a minimum page size of 2 MiB; supported GiB pages; and yet, at the same time, looked in all other ways like a standard Plan 9 system.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
A good explication of the two types of systems can be found in (Padlipsky 1985).
- 2.
Not surprising since the idea for unsharing came from Plan 9 to Linux.
- 3.
Not the namespace of the node; the namespace of the process on the node. Again, the distinction is crucial.
- 4.
As mentioned above, mounts are not privileged operations in Plan 9, as they are not global. In Linux terms, every login process runs as though it had been started with a CLONE_NEWNS option.
- 5.
It is now a core, but the terminology was created long before multi-core CPUs existed.
- 6.
i.e. not Opterons with 4 TLB entries for 2 MiB pages.
- 7.
Some projects such as coreboot even require gnu11 compatibility, not c11!
References
Forsyth, C., McKie, J., Minnich, R., & Hensbergen, E. V. (2010). Night of the Lepus: A Plan 9 perspective on Blue Gene’s interconnects.
Hoare, C. A. R. (1978). Communicating sequential processes. Communications of the ACM, 21(8), 666–677.
Sottile, M., Minnich, R. (2004). Analysis of microbenchmarks for performance tuning of clusters. In IEEE international conference on cluster computing.
Minnich, R., & Mckie, J. (2009). Experiences porting the Plan 9 research operating system to the IBM Blue Gene supercomputers. Computer Science - Research and Development, 23(3–4), 117–124, June 2009.
Padlipsky, M. A. (1985). The Elements of Networking Style: And Other Essays and Animadversions on the Art of Intercomputer Networking. Upper Saddle River, NJ, USA: Prentice-Hall Inc.
Pike, R., Presotto, D. L., Dorward, S., Flandrena, B., Thompson, K., Trickey, H., et al. (1995). Plan 9 from Bell Labs. Computing Systems, 8(2), 221–254.
Acknowledgements
Jim McKie, of Bell Labs, provided support for HPC development on Plan 9 from 2005 to 2011, and kept us on track over the last year as we made NIX work. Francisco J. Ballesteros, Gorka Guardiola, and Enrique Soriano hosted me for the month of May 2011 at the Laboratorio de Sistemas and each contributed key ideas that made NIX successful; in particular, if memory serves, Francisco created the Inter-Core Calling structures and conventions. Charles Forsyth of Vita Nuova provided the port of the Plan 9 toolchain to 64-bit x86 in 2005 (in 6 weeks!) and was very active in the NIX work, pushing hard to make use of shared memory and take advantage of what it provided. Eric Van Hensbergen, then of IBM, worked closely with us from 2005 to 2011 to make the Blue Gene port of Plan 9 possible; this work was the foundation of NIX.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Minnich, R. (2019). NIX. In: Gerofi, B., Ishikawa, Y., Riesen, R., Wisniewski, R.W. (eds) Operating Systems for Supercomputers and High Performance Computing. High-Performance Computing Series, vol 1. Springer, Singapore. https://doi.org/10.1007/978-981-13-6624-6_16
Download citation
DOI: https://doi.org/10.1007/978-981-13-6624-6_16
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-6623-9
Online ISBN: 978-981-13-6624-6
eBook Packages: Computer ScienceComputer Science (R0)