Advertisement

Facilitate External Sorting for Large-Scale Storage on Shingled Magnetic Recording Drives

  • Yu-Pei LiangEmail author
  • Min-Hong Shen
  • Yi-Han Lien
  • Wei-Kuan Shih
Conference paper
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 70)

Abstract

In the era of big data and cloud computing, both external data process techniques and new storage mediums are proposed to process and accommodate the sheer amount of information with data-intensive applications. For instance, external sorting algorithms perform sorting operations directly on the storage devices to lower the data transfer latency and increase system performance. On the other hand, Shingled Magnetic Recording (SMR) is proposed to increase the areal density by overlapping tracks. However, the overlapping technique also introduces the random-write restriction because writing a track also destroys the valid data on overlapped tracks. This constraint could induce signification write amplification issue when performing external sorting on SMR drives. To mitigate the write amplification issue, this paper proposes a sort-friendly SMR drive design to lower the write amount of external sorting algorithms on SMR drives. The experimental results show that the proposed design could lower the external sorting latency by 61.99% when compared with the external merge sort algorithm.

Keywords

Shingle magnetic recording External sorting Cloud computing 

References

  1. 1.
    Knuth, D.: The Art of Computer Programming, vol. 3, 2nd edn. Addison-Wesley (1998)Google Scholar
  2. 2.
    Quero, L.C., Lee, Y.S., Kim, J.S.: Self-sorting SSD: producing sorted data inside active SSDs. In: Mass Storage Systems and Technologies (MSST), 2015 31st Symposium, pp. 1–7 (2015)Google Scholar
  3. 3.
    Lee, Y.-S., Quero, L.C., Kim, S.-H., Kim, J.-S., Maeng, S.: ActiveSort: efficient external sorting using active SSDs in the MapReduce framework. In: Future Generation Computer Systems (2016)Google Scholar
  4. 4.
    Shiroishi, Y., et al.: Future options for HDD storage. IEEE Trans. Magn. 45(10), 3816–3822 (2009)CrossRefGoogle Scholar
  5. 5.
    Yahoo: Yahoo! cloud serving benchmark @ONLINE. https://github.com/brianfrankcooper/YCSB/wiki (2015)
  6. 6.

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Yu-Pei Liang
    • 1
    Email author
  • Min-Hong Shen
    • 1
  • Yi-Han Lien
    • 2
  • Wei-Kuan Shih
    • 1
  1. 1.Department of Computer ScienceNational Tsing Hua UniversityHsinchuTaiwan
  2. 2.Department of Electronic EngineeringNational Taipei University of TechnologyTaipeiTaiwan

Personalised recommendations