Skip to main content
SpringerLink
Log in

Hardware Reliability Requirements

  • Living reference work entry
  • First Online:
Encyclopedia of Big Data Technologies

  • 414 Accesses

Introduction

The development of big data applications has brought a great amount of opportunities yet challenges to both the IT industry and academia. Supporting platforms and architectures, such as Clouds, supercomputing centers and data centers, must develop corresponding technologies to meet the “4V” requirements of big data applications, which are veracity, velocity, volume, and variety. Serving as the fundamental layer of these platforms and architectures, the hardware, especially storage hardware, are of great importance to the big data applications, where the reliability of storage units not only have significantly influence the reliability of the data, but also have significant impact on the “4V” requirements of the big data applications. In this article, we discuss the hardware reliability requirements, specifically, storage hardware reliability requirements, for current big data platforms and architectures.

Big Data Storage Architecture

A typical big data application contains...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • CAI Y et al (2012) Error patterns in MLC NAND flash memory: measurement, characterization, and analysis. In: Design, automation & test in Europe conference & exhibition, Dresden

    Google Scholar 

  • Dabrowski C (2009) Reliability in grid computing systems. Concurr Comput Pract Experience 21(8):927–959

    Article  Google Scholar 

  • Furrer S, Lantz MA, Reininger P (2017) 201 Gb/in2 recording areal density on sputtered magnetic tape. IEEE Trans Magn 99:1–1

    Google Scholar 

  • Gersback T (2012) Technical article: Energy-efficient climate control “keep cool” in the data centre. RITTAL Pty, New South Wales

    Google Scholar 

  • Harris R (2014) Amazon’s Glacier secret: BDXL. [cited 2017]; Available from https://storagemojo.com/2014/04/25/amazons-glacier-secret-bdxl/

  • Huang C et al (2012) Erasure coding in Windows Azure storage. In: USENIX annual technical conference, Boston

    Google Scholar 

  • IDEMA (1998) R2–98: specification of hard disk drive reliability. IDEMA Standards

    Google Scholar 

  • Klein A (2017) Hard drive stats for Q2 2017. [cited 2017]; Available from https://www.backblaze.com/blog/hard-drive-failure-stats-q2-2017/

  • Li W, Yang Y, Yuan D (2015) Ensuring cloud data reliability with minimum replication by proactive replica checking. IEEE Trans Comput 65:1–13

    MathSciNet  MATH  Google Scholar 

  • Loken C et al (2010) SciNet: lessons learned from building a power-efficient top-20 system and data centre. J Phys Conf Ser 256(1):1–35

    MathSciNet  Google Scholar 

  • Mielke N et al (2008) Bit error rate in NAND flash memories. In: IEEE international reliability physics symposium, Phoenix

    Google Scholar 

  • Patterson D, Gibson G, Katz R (1988) A case for redundant arrays of inexpensive disks (RAID). In: ACM SIGMOD international conference on the management of data, Chicago

    Google Scholar 

  • Pinheiro E, Weber W, Barroso LA (2007) Failure trends in a large disk drive population. In: USENIX conference on file and storage technologies, San Jose

    Google Scholar 

  • Rashmi KV et al (2013) A solution to the network challenges of data recovery in erasure-coded distributed storage systems: a study on the Facebook warehouse cluster. In: USENIX HotStorage, San Jose

    Google Scholar 

  • Rashmi KV et al (2014) A hitchhiker’s guide to fast and efficient data reconstruction in erasure-coded data centers. In: SIGCOMM, Chicago

    Google Scholar 

  • Schroeder B, Gibson G (2007) Disk failures in the real world: what does an MTTF of 1,000,000 hours mean to you? In: USENIX conference on file and storage technologies. San Jose

    Google Scholar 

  • Schroeder B, Lagisetty R, Merchant A (2016) Flash reliability in production the expected and the unexpected. In: USENIX conference on file and storage technologies, Santa Clara

    Google Scholar 

  • Taylor C (2014) SSD vs. HDD: performance and reliability. [cited 2017]; Available from http://www.enterprisestorageforum.com/storage-hardware/ssd-vs.-hdd-performance-and-reliability-1.html

  • Vishwanath KV, Nagappan N (2010) Characterizing cloud computing hardware reliability. In: ACM symposium on cloud computing, Indianapolis

    Google Scholar 

  • Wood T et al (2010) Disaster recovery as a cloud service: economic benefits & deployment challenges. In: HotCloud, Boston

    Google Scholar 

  • Xin Q, Schwarz TJE, Miller EL (2005) Disk infant mortality in large storage systems. In: IEEE international symposium on modeling, analysis, and simulation of computer and telecommunication systems, Atlanta

    Google Scholar 

  • Zaman R (2016) SSD vs HDD: which one is more reliable? [cited 2017]; Available from https://therevisionist.org/reviews/ssd-vs-hdd-one-reliable/

Download references

Author information

Authors and Affiliations

  1. Advanced Computing Research Centre, Data to Decisions Cooperative Research Centre, University of South Australia, Adelaide, Australia

    Wenhao Li

Authors
  1. Wenhao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenhao Li .

Editor information

Editors and Affiliations

  1. School of Comp. Sci. and Engineering, University of New South Wales School of Comp. Sci. and Engineering, Eveleigh, New South Wales, Australia

    Sherif Sakr

  2. Sch of Info Techno, Building J12, University of Sydney Sch of Info Techno, Building J12, Sydney, Australia

    Albert Zomaya

Section Editor information

  1. No affiliation provided

    Bingsheng He

  2. No affiliation provided

    Behrooz Parhami

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Li, W. (2018). Hardware Reliability Requirements. In: Sakr, S., Zomaya, A. (eds) Encyclopedia of Big Data Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-63962-8_173-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-63962-8_173-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-63962-8

  • Online ISBN: 978-3-319-63962-8

  • eBook Packages: Springer Reference MathematicsReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Search

Navigation