Abstract
As a data-intensive computing application, high-energy physics requires to process and store massive data at the PB or EB level. It requires high performance data access and large volume of data storage as well. Some enterprises and research organizations are beginning to use tiered storage architectures, using tapes, disks or solid drives at the same time to reduce hardware purchase costs and power consumption. Tiered storage requires data management software to migrate less active data to lower cost storage devices. Thus an automated data migration strategy is very necessary. Data access requests are driven by the behavior of users or programs. There must be associations between different files that are accessed consecutively. This paper proposes a method to predict the heat of data access and use data heat trend as the basis criteria for data migration. This paper proposes a deep learning algorithm model to predict the evolution trend of data access heat. This paper discussed the implementation of some initial parts of the system. Then some preliminary experiments are conducted with these parts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cheng, Z., Li, H., Huang, Q., et al.: Research on elastic resource management for multi-queue under cloud computing environment. In: Journal of Physics Conference Series, p. 092003 (2017)
Li, A., Yu, D., Shu, J., et al.: A tiered storage system for massive data: TH-TS. J. Comput. Res. Dev. 48(6), 1089–1100 (2011)
Zhang, G., Chiu, L., Dickey, C., et al.: Automated lookahead data migration in SSD-enabled multi-tiered storage systems. In: IEEE Symposium on Mass Storage Systems & Technologies, pp. 1–6. IEEE Computer Society (2010)
Ari, I.: Using statistical correlation for dependency analysis of cache replacement policies. In: IEEE TOC, ACM TODS, ACM TOC, SIGMETRICS 2004, ICDCS 2003, FAST 2002, The Computer Journal (2004)
Jiang, S., Davis, K., Zhang, X.: Coordinated multilevel buffer cache management with consistent access locality quantification. IEEE Trans. Comput. 56(1), 95–108 (2007)
吴峰光, 奚宏生, 徐陈锋: 一种支持并发访问流的文件预取算法.软件学报 21(8), 1820–1833(2010)
刘爱贵, 陈刚: 一种基于用户的 LNS 文件预测模型. 计算机工程与应用 43(29), 14–16 (2007)
Palmer, M.L., Zdonik, S.B.: FIDO: a cache that learns to fetch. In: Proceedings of Conference on Very Large Data Bases (VLDB), Barcelona, pp. 255–264, September 1991
Griffioen, J., Appleton, R.: Reducing file system latency using a predictive approach. In: Proceedings of 1994 Summer USENIX Conference, pp. 197–207 (1994)
Lei, H., Duchamp, D.: An analytical approach to file prefetching. In: Proceedings of 1997 USENIX Annual Technical Conference, January 1997
Kang, S.J., Lee, S.W., Ko, Y.B.: A recent popularity based dynamic cache management for content centric networking. In: 2012 Fourth International Conference on Ubiquitous and Future Networks (ICUFN), pp. 219–224. IEEE (2012)
Catalina, T., Virgone, J., Blanco, E.: Development and validation of regression models to predict monthly heating demand for residential buildings. Energy Build. 40(10), 1825–1832 (2008)
Lehmann, A., Overton, J.M.C., Leathwick, J.R.: GRASP: generalized regression analysis and spatial prediction. Ecol. Model. 157(2–3), 189–207 (2002)
Braam, P.J.: The lustre storage architecture (2004)
Peters, A.J., Sindrilaru, E.A., Adde, G.: EOS as the present and future solution for data storage at CERN. In: Journal of Physics: Conference Series, vol. 664, no. 4, p. 042042. IOP Publishing (2015)
Dehdouh, K., Bentayeb, F., Boussaid, O., et al.: Using the column oriented NoSQL model for implementing big data warehouses. In: Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA). The Steering Committee of the World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp), p. 469 (2015)
Graves, A.: Long Short-term Memory, pp. 1735–1780. Springer, Berlin (2012)
Wei, X.: From recurrent neural network to long short term memory architecture (2013)
Abadi, M., Barham, P., Chen, J., et al.: TensorFlow: a system for large-scale machine learning. In: OSDI, vol. 16, pp. 265–283 (2016)
Acknowledgments
This work was supported by the National key Research Program of China “Scientific Big Data Management System” (No. 2016YFB1000605).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Cheng, Z. et al. (2019). Automated and Intelligent Data Migration Strategy in High Energy Physical Storage Systems. In: Li, J., Meng, X., Zhang, Y., Cui, W., Du, Z. (eds) Big Scientific Data Management. BigSDM 2018. Lecture Notes in Computer Science(), vol 11473. Springer, Cham. https://doi.org/10.1007/978-3-030-28061-1_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-28061-1_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-28060-4
Online ISBN: 978-3-030-28061-1
eBook Packages: Computer ScienceComputer Science (R0)