Abstract
In the era of big data and cloud computing, both external data process techniques and new storage mediums are proposed to process and accommodate the sheer amount of information with data-intensive applications. For instance, external sorting algorithms perform sorting operations directly on the storage devices to lower the data transfer latency and increase system performance. On the other hand, Shingled Magnetic Recording (SMR) is proposed to increase the areal density by overlapping tracks. However, the overlapping technique also introduces the random-write restriction because writing a track also destroys the valid data on overlapped tracks. This constraint could induce signification write amplification issue when performing external sorting on SMR drives. To mitigate the write amplification issue, this paper proposes a sort-friendly SMR drive design to lower the write amount of external sorting algorithms on SMR drives. The experimental results show that the proposed design could lower the external sorting latency by 61.99% when compared with the external merge sort algorithm.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Knuth, D.: The Art of Computer Programming, vol. 3, 2nd edn. Addison-Wesley (1998)
Quero, L.C., Lee, Y.S., Kim, J.S.: Self-sorting SSD: producing sorted data inside active SSDs. In: Mass Storage Systems and Technologies (MSST), 2015 31st Symposium, pp. 1–7 (2015)
Lee, Y.-S., Quero, L.C., Kim, S.-H., Kim, J.-S., Maeng, S.: ActiveSort: efficient external sorting using active SSDs in the MapReduce framework. In: Future Generation Computer Systems (2016)
Shiroishi, Y., et al.: Future options for HDD storage. IEEE Trans. Magn. 45(10), 3816–3822 (2009)
Yahoo: Yahoo! cloud serving benchmark @ONLINE. https://github.com/brianfrankcooper/YCSB/wiki (2015)
Seagate: Seagate archive hdd @ONLINE. http://www.seagate.com/www-content/product-content/hdd-fam/seagate-archive-hdd/en-us/docs/archive-hdd-ds1834-5c-1508us.pdf (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Liang, YP., Shen, MH., Lien, YH., Shih, WK. (2020). Facilitate External Sorting for Large-Scale Storage on Shingled Magnetic Recording Drives. In: Arai, K., Bhatia, R. (eds) Advances in Information and Communication. FICC 2019. Lecture Notes in Networks and Systems, vol 70. Springer, Cham. https://doi.org/10.1007/978-3-030-12385-7_80
Download citation
DOI: https://doi.org/10.1007/978-3-030-12385-7_80
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-12384-0
Online ISBN: 978-3-030-12385-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)