SP-TSRM: A Data Grouping Strategy in Distributed Storage System

  • Dongjie Zhu
  • Haiwen Du
  • Ning CaoEmail author
  • Xueming Qiao
  • Yanyan Liu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11334)


With the development of smart devices and social media, massive unstructured data is uploaded to distributed storage systems. Since the characteristics of multi-users and high concurrency the unstructured data accesses have, it brings new challenges to traditional distributed storage systems designed for large files. We propose a grouping strategy to analyze relevant data in access according to disk access logs in the real distributed storage systems environment. When any data in the group is accessed, the whole group is prefetched from disk to the cache. Firstly, we conduct statistical analysis on the access logs and propose a preliminary classification method to classify files in spatiotemporal locality. Secondly, a strength-priority tree structure relation model (SP-TSRM) is proposed to mine file group efficiently. Finally, experiments show that the proposed model can improve the cache hit rate significantly, thereby improving the read efficiency of distributed storage systems.


Prefetching model Storage optimization Unstructured data Distributed storage systems 



Thanks to the students of HIT at Weihai. This work is supported by project under Grant no. 520613170002, SGSDWH00YXJS1700522, SGSDWH00YXJS1700270, the Fundamental Research Funds for the Central Universities (Grant No. HIT.NSRIF.201714), Weihai Science and Technology Development Program (2016DXGJMS15) and Key Research and Development Program in Shandong Provincial (2017GGX90103).


  1. 1.
    Dong, B., Zheng, Q., Tian, F.: An optimized approach for storing and accessing small files on cloud storage. J. Netw. Comput. Appl. 35(6), 1847–1862 (2012)CrossRefGoogle Scholar
  2. 2.
    Zhu, Y., Zhang, X., Zhao, R., Dong, X.: Data De-duplication on similar file detection. In: Innovative Mobile and Internet Services in Ubiquitous Computing, pp. 66–73. IEEE Press (2014)Google Scholar
  3. 3.
    Cui, Y., Lai, Z., Wang, X., Dai, N.: QuickSync: improving synchronization efficiency for mobile cloud storage services. IEEE Trans. Mob. Comput. 16, 3513–3526 (2017)CrossRefGoogle Scholar
  4. 4.
    Dong, B., Qiu, J., Zheng, Q., Zhong, X., Li, J., Li, Y.: A novel approach to improving the efficiency of storing and accessing small files on hadoop: a case study by powerpoint files. In: Services Computing (SCC), pp. 65–72. IEEE Press (2010)Google Scholar
  5. 5.
    Bok, K., Lim, J., Oh, H., Yoo, J.: An efficient cache management scheme for accessing small files in distributed file systems. In: Big Data and Smart Computing (BigComp), pp. 151–155. IEEE Press (2017)Google Scholar
  6. 6.
    Lin, L., Li, X., Jiang, H., Zhu, Y., Tian, L., AMP: an affinity-based metadata prefetching scheme in large-scale distributed storage systems. In: Cluster Computing and the Grid, pp. 459–466. IEEE Press (2008)Google Scholar
  7. 7.
    Zhu, D., et al.: An access prefetching strategy for accessing small files based on swift. Procedia Comput. Sci. 131, 816–824 (2018)CrossRefGoogle Scholar
  8. 8.
    Cherubini, G., Kim, Y., Lantz, M., Venkatesan, V.: Data prefetching for large tiered storage systems. In: Data Mining (ICDM), pp. 823–828. IEEE Press (2017)Google Scholar
  9. 9.
    Kroeger, T.M., Long, D.D., Mogul, J.C.: Exploring the bounds of web latency reduction from caching and prefetching. In: USENIX Symposium on Internet Technologies and Systems, pp. 13–22 (1997)Google Scholar
  10. 10.
    Wildani, A., Miller, E.L.: Can we group storage? Statistical techniques to identify predictive groupings in storage system accesses. ACM Trans. Storage (TOS) 12(2), 7–40 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Dongjie Zhu
    • 1
  • Haiwen Du
    • 1
  • Ning Cao
    • 2
    Email author
  • Xueming Qiao
    • 3
  • Yanyan Liu
    • 3
  1. 1.School of Computer Science and TechnologyHarbin Institute of Technology at WeihaiWeihaiChina
  2. 2.College of Information EngineeringQingdao Binhai UniversityQingdaoChina
  3. 3.WeiHai Power Supply CompanyWeihaiChina

Personalised recommendations