N-Docker: A NVM-HDD Hybrid Docker Storage Framework to Improve Docker Performance

  • Lin Gu
  • Qizhi Tang
  • Song WuEmail author
  • Hai Jin
  • Yingxi Zhang
  • Guoqiang Shi
  • Tingyu Lin
  • Jia Rao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11783)


Docker has been widely adopted in production environment, but unfortunately deployment and cold-start of container are limited by the low speed of disk. The emerging non-volatile memory (NVM) technology, which has high speed and can store data permanently, brings a new chance to accelerate the deployment and cold-start of container. However, it is expensive to replace the whole hard disk driver (HDD) with NVM. To achieve the fastest deployment and cold-start with lowest cost, we conduct in-depth analysis on the Top-134 images in Docker Hub and obtain two main insights as: (1) the storing latency of layered image has become the bottleneck of container deployment; (2) only a few image layers are required for container cold-start. Based on these two findings, we propose a NVM-HDD hybrid docker storage framework as N-Docker. It can effectively accelerate container cold-start by detecting the bottleneck layers as well as cold-start required layers and storing them into NVM for faster container startup with limited NVM capacity. Experimental results show that N-Docker can accelerate the container deployment by 1.21X and cold-start by 2.96X. Compared to NVM-Docker, which stores all images into NVM, N-Docker achieves the same performance improvements while reducing the usage of NVM by 88.22%.


Container deployment Container cold-start Docker Image NVM 


  1. 1.
    Add support for NV-DIMMs to ext4.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
    Linux kernel virtual machine.
  6. 6.
  7. 7.
    Akkus, I.E., et al.: SAND: towards high-performance serverless computing. In: Proceedings of the 2018 USENIX Annual Technical Conference, pp. 923–935 (2018)Google Scholar
  8. 8.
    Belay, A., Bittau, A., Mashtizadeh, A.J., Terei, D., Mazières, D., Kozyrakis, C.: Dune: Safe user-level access to privileged CPU features. In: Proceedings of the 10th USENIX Symposium on Operating Systems Design and Implementation, pp. 335–348. USENIX Association (2012)Google Scholar
  9. 9.
    Du, L., Wo, T., Yang, R., Hu, C.: Cider: a rapid docker container deployment system through sharing network storage. In: Proceedings of 19th International Conference on High Performance Computing and Communications, pp. 332–339. IEEE (2017)Google Scholar
  10. 10.
    Harter, T., Salmon, B., Liu, R., Arpaci-Dusseau, A.C., Arpaci-Dusseau, R.H.: Slacker: fast distribution with lazy docker containers. In: Proceedings of the 14th USENIX Conference on File and Storage Technologies, pp. 181–195. USENIX Association (2016)Google Scholar
  11. 11.
    Madhavapeddy, A., Scott, D.J.: Unikernels: the rise of the virtual library operating system. Commun. ACM 57(1), 61–69 (2014)CrossRefGoogle Scholar
  12. 12.
    Narayanan, A.: Tupperware: containerized deployment at facebook (2014)Google Scholar
  13. 13.
    Oakes, E., et al.: Sock: rapid task provisioning with serverless-optimized containers. In: Proceedings of the 2018 USENIX Annual Technical Conference, pp. 57–70. USENIX Association (2018)Google Scholar
  14. 14.
    Thalheim, J., Bhatotia, P., Fonseca, P., Kasikci, B.: CNTR: lightweight OS containers. In: Proceedings of the 2018 USENIX Annual Technical Conference, pp. 199–212. USENIX Association (2018)Google Scholar
  15. 15.
    Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., Wilkes, J.: Large-scale cluster management at Google with Borg. In: Proceedings of the 10th European Conference on Computer Systems, p. 18. ACM (2015)Google Scholar
  16. 16.
    Wang, L., Li, M., Zhang, Y., Ristenpart, T., Swift, M.: Peeking behind the curtains of serverless platforms. In: Proceedings of the 2018 USENIX Annual Technical Conference, pp. 133–146. USENIX Association (2018)Google Scholar
  17. 17.
    Xu, J., et al.: NOVA-fortis: a fault-tolerant non-volatile main memory file system. In: Proceedings of the 26th Symposium on Operating Systems Principles, pp. 478–496. ACM (2017)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2019

Authors and Affiliations

  • Lin Gu
    • 1
  • Qizhi Tang
    • 1
  • Song Wu
    • 1
    Email author
  • Hai Jin
    • 1
  • Yingxi Zhang
    • 2
  • Guoqiang Shi
    • 2
  • Tingyu Lin
    • 2
  • Jia Rao
    • 3
  1. 1.National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab, Cluster and Grid Computing Lab, School of Computer Science and TechnologyHuazhong University of Science and TechnologyWuhanChina
  2. 2.State Key Laboratory of Intelligent Manufacturing System TechnologyBeijingChina
  3. 3.The University of Texas at ArlingtonArlingtonUSA

Personalised recommendations