Advertisement

LCCFS: a lightweight distributed file system for cloud computing without journaling and metadata services

  • Li WangEmail author
  • Jingling Xue
  • Xiangke Liao
  • Yunchuan Wen
  • Min Chen
Research Paper
  • 6 Downloads

Abstract

The major usage of a file system integrated with a cloud computing platform is to provide the storage for VM (virtual machine) instances. Distributed file systems, especially those implemented on top of object storage have many potential advantages over traditional local file systems for VM instance storage. In this paper, we make an investigation in the requirements imposed on a file system in cloud computing scenario, and claim that the implementation of a file system for VM instance storage could be reasonably simplified. We demonstrate that on top of an object storage with simple object-granularity transaction support, a lightweight distributed file system, which requires neither journaling nor dedicated metadata services, can be developed for cloud computing. We have implemented such a distributed file system, called LCCFS (lightweight cloud computing file system), based on the RADOS (reliable autonomic distributed object storage) object storage. Our experimental results show that for the main workloads in cloud computing, LCCFS achieves almost the same or slightly higher performance than CephFS (ceph filesystem), another published distributed file system based on RADOS. Compared to CephFS, LCCFS has only one tenth of its LOCs (lines of code). This theoretical simplicity makes it easy to implement LCCFS correctly and stably by avoiding the sheer design and implementation complexity behind CephFS, thereby making LCCFS a promising candidate in the cloud computing production environment.

Keywords

cloud computing distributed file system journaling metadata service garbage collection 

Notes

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant No. 61370018).

References

  1. 1.
    Mesnier M, Ganger G R, Riedel E. Object-based storage. IEEE Commun Mag, 2003, 41: 84–90CrossRefGoogle Scholar
  2. 2.
    Schwan P. Lustre: building a file system for 1000-node clusters. In: Proceedings of the Linux Symposium, Ottawa, 2003Google Scholar
  3. 3.
    Welch B, Gibson G. Managing scalability in object storage systems for HPC linux clusters. In: Proceedings of the 21st IEEE/12th NASA Goddard Conference on Mass Storage Systems and Technologies, Greenbelt, 2004. 433–445Google Scholar
  4. 4.
    Weil S A, Leung A W, Brandt S A, et al. Rados: a scalable, reliable storage service for petabyte-scale storage clusters. In: Proceedings of the 2nd International Workshop on Petascale Data Storage: Held in Conjunction with Supercomputing, Reno, 2007. 35–44CrossRefGoogle Scholar
  5. 5.
    Weil S A, Brandt S A, Miller E L, et al. Ceph: a scalable, high-performance distributed file system. In: Proceedings of the 7th Symposium on Operating Systems Design and Implementation, Seattle, 2006. 307–320Google Scholar
  6. 6.
    Weil S A, Pollack K T, Brandt S A, et al. Dynamic metadata management for petabyte-scale file systems. In: Proceedings of the 2004 ACM/IEEE Conference on Supercomputing, Pittsburgh, 2004. 6–12Google Scholar
  7. 7.
    Sevilla M. Mds has inconsistent performance. 2015. http://comments.gmane.org/gmane.comp.file-systems.ceph.devel/ 22674Google Scholar
  8. 8.
    Nagle D, Factor M E, Iren S, et al. The ANSI T10 object-based storage standard and current Implementations. IBM J Res Dev, 2008, 52: 401–411CrossRefGoogle Scholar
  9. 9.
    Rodeh O, Teperman A. zFS — a scalable distributed file system using object disks. In: Proceedings of the 20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, San Diego, 2003. 207–218Google Scholar
  10. 10.
    Abd-El-Malek M, Courtright II W V, Cranor C, et al. Ursa minor: versatile cluster-based storage. In: Proceedings of the 4th USENIX Conference on File and Storage Technologies, San Francisco, 2005. 59–72Google Scholar
  11. 11.
    Kubiatowicz J, Bindel D, Chen Y, et al. Oceanstore: an architecture for global-scale persistent storage. In: Proceedings of the 9th International Conference on Architectural Support for Programming Languages and Operating Systems, New York, 2000. 190–201Google Scholar
  12. 12.
    Adya A, Bolosky W J, Castro M, et al. Farsite: federated, available, and reliable storage for an incompletely trusted environment. In: Proceedings of the 5th Symposium on Operating Systems Design and Implementation, New York, 2002. 1–14Google Scholar
  13. 13.
    Haeberlen A, Mislove A, Druschel P. Glacier: highly durable, decentralized storage despite massive correlated failures. In: Proceedings of the 2nd Symposium on Networked Systems Design & Implementation, Berkeley, 2005. 143–158Google Scholar
  14. 14.
    Beaver D, Kumar S, Li H C, et al. Finding a needle in haystack: facebook’s photo storage. In: Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation, Vancouver, 2010. 1–8Google Scholar
  15. 15.
    Card R, Ts T, Tweedie S. Design and implementation of the second extended filesystem. In: Proceedings of the 1st Dutch International Symposium on Linux, Amsterdam, 1994Google Scholar
  16. 16.
    Mathur A, Cao M, Bhattacharya S, et al. The new ext4 lesystem: current status and future plans. In: Proceedings of the Linux Symposium, Ottawa, 2007. 21–33Google Scholar
  17. 17.
    Ts’o T Y, Tweedie S. Planned extensions to the Linux EXT2/EXT3 filesystem. In: Proceedings of the FREENIX Track: 2002 USENIX Annual Technical Conference, Berkeley, 2002. 235–243Google Scholar
  18. 18.
    Tweedie S. Ext3, journaling filesystem. In: Proceedings of the Linux Symposium, Ottawa, 2000Google Scholar
  19. 19.
    Konishi R, Amagai Y, Sato K, et al. The Linux implementation of a log-structured file system. SIGOPS Oper Syst Rev, 2006, 40: 102–107CrossRefGoogle Scholar
  20. 20.
    Rosenblum M, Ousterhout J K. The design and implementation of a log-structured file system. ACM Trans Comput Syst, 1992, 10: 26–52CrossRefGoogle Scholar
  21. 21.
    Hitz D, Lau J, Malcolm M. File system design for an NFS file server appliance. In: Proceedings of the USENIX Winter 1994 Technical Conference, San Francisco, 1994. 19Google Scholar
  22. 22.
    Rodeh O, Bacik J, Mason C. Btrfs: the linux b-tree filesystem. ACM Trans Storage, 2013, 9: 1–32CrossRefGoogle Scholar
  23. 23.
    Shen K, Park S, Zhu M. Journaling of journal is (almost) free. In: Proceedings of the 12th USENIX Conference on File and Storage Technologies, Santa Clara, 2014. 287–293Google Scholar
  24. 24.
    Fryer D, Qin D, Sun J, et al. Checking the integrity of transactional mechanisms. In: Proceedings of the 12th USENIX Conference on File and Storage Technologies, Berkeley, 2014. 295–308Google Scholar
  25. 25.
    Aghayev A, Theodore Y, Gibson G, et al. Evolving ext4 for shingled disks. In: Proceedings of the 15th USENIX Conference on File and Storage Technologies, Santa clara, 2017. 105–119Google Scholar
  26. 26.
    Wang L, Liao X K, Xue J L, et al. Enhancement of cooperation between file systems and applications–VFS extensions for optimized performance. Sci China Inf Sci, 2015, 58: 092104Google Scholar
  27. 27.
    Zhang S, Catanese H, Wang A I A. The composite-file file system: decoupling the one-to-one mapping of files and metadata for better performance. In: Proceedings of the 14th USENIX Conference on File and Storage Technologies, Santa Clara, 2016. 15–22Google Scholar
  28. 28.
    Xu Q, Arumugam R V, Yong K L, et al. Efficient and scalable metadata management in EB-scale file systems. IEEE Trans Parallel Distrib Syst, 2014, 25: 2840–2850CrossRefGoogle Scholar
  29. 29.
    Thomson A, Abadi D J. Calvinfs: consistent wan replication and scalable metadata management for distributed file systems. In: Proceedings of the 13th USENIX Conference on File and Storage Technologies, Santa Clara, 2015. 1–14Google Scholar
  30. 30.
    Niazi S, Ismail M, Haridi S, et al. Hopsfs: scaling hierarchical file system metadata using newsql databases. In: Proceedings of the 15th USENIX Conference on File and Storage Technologies, Santa Clara, 2017. 89–104Google Scholar

Copyright information

© Science China Press and Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Li Wang
    • 1
    Email author
  • Jingling Xue
    • 2
  • Xiangke Liao
    • 1
  • Yunchuan Wen
    • 3
  • Min Chen
    • 3
  1. 1.School of Computer Science and TechnologyNational University of Defense TechnologyChangshaChina
  2. 2.School of Computer Science and EngineeringUniversity of New South WalesSydneyAustralia
  3. 3.Kylin CorporationChangshaChina

Personalised recommendations