Encyclopedia of Big Data Technologies

Living Edition
| Editors: Sherif Sakr, Albert Zomaya

Data Longevity and Compatibility

  • Behrooz ParhamiEmail author
Living reference work entry
DOI: https://doi.org/10.1007/978-3-319-63962-8_331-1


Definition of Entry

Whether stored locally or in the cloud, data ultimately resides on physical media that are subject to electrical and physical deterioration and format obsolescence, making it necessary to augment the physical storage of data with a logical organization to protect data integrity and ensure its longevity.


Like many other attributes of computing and digital systems, the volume of data produced in the world is rising exponentially (Denning and Lewis 2016; Hilbert and Gomez 2011), with a growth rate that is even higher than those of circuit density and processor performance, modeled by Moore’s law (Brock and Moore 2006). A few exabytes of data generation per day in the early 2010s is slated to rise to many yottabytes in the 2020s (Cisco Systems 2017; Jacobson 2013). As data gains ever-greater value in the operation of social and business enterprises, data management, integrity, and preservation...

This is a preview of subscription content, log in to check access.


  1. Arulraj J, Pavlo A (2017) How to build a non-volatile memory database management system. In: Proceedings of the ACM international conference on management of data, Chicago, pp 1753–1758Google Scholar
  2. Arulraj J, Pavlo A, Dulloor SR (2015) Let’s talk about storage & recovery methods for non-volatile memory database System. In: Proceedings of the ACM international conference on management of data, Melbourne, pp 707–722Google Scholar
  3. Bhati I, Chang M-T, Chishti Z, Lu S-L, Jacob B (2016) DRAM refresh mechanisms, penalties, and trade-offs. IEEE Trans Comput 65(1):108–121MathSciNetCrossRefGoogle Scholar
  4. Bornholt J, Lopez R, Carmean DM, Ceze L, Seelig G, Strauss K (2016) A DNA-based archival storage system. ACM SIGOPS Oper Syst Rev 50(2):637–649CrossRefGoogle Scholar
  5. Brock DC, Moore GE (eds) (2006) Understanding Moore’s law: four decades of innovation. Chemical Heritage Foundation, PhiladelphiaGoogle Scholar
  6. Chen PM, Lee EK, Gibson GA, Katz RH, Patterson DA (1994) RAID: high-performance reliable secondary storage. ACM Comput Surv 26(2):145–185CrossRefGoogle Scholar
  7. Chen T, Gao X, Chen G (2016) The features, hardware, and architectures of data center networks: a survey. J Parallel Distrib Comput 96:45–74CrossRefGoogle Scholar
  8. Coughlin T (2014) Keeping data for a long time, Forbes, available on-line at http://www.forbes.com/sites/tomcoughlin/2014/06/29/keeping-data-for-a-long-time/#aac168815e26
  9. Denning PJ, Lewis TG (2016) Exponential Laws of computing growth. Commun ACM 60(1):54–65CrossRefGoogle Scholar
  10. Dimakis AG, Ramachandran K, Wu Y, Suh C (2011) A survey on network codes for distributed storage. Proc IEEE 99(3):476–489CrossRefGoogle Scholar
  11. Goda K, Kitsuregawa M (2012) The history of storage systems. Proc IEEE 100:1433–1440CrossRefGoogle Scholar
  12. Goldman N, Bertone P, Chen S, Dessimoz C, LeProust EM, Sipos B, Birney E (2013) Towards practical, high-capacity, low-maintenance information storage in synthesized DNA. Nature 494(7435):77–80CrossRefGoogle Scholar
  13. Googology Wiki (2018) SI prefix. On-line document. http://googology.wikia.com/wiki/SI_prefix. Accessed 23 Feb 2018
  14. Hilbert M, Gomez P (2011) The World’s technological capacity to store, communicate, and compute information. Science 332:60–65CrossRefGoogle Scholar
  15. Iyengar A, Cahn R, Garay JA, Jutla C (1998) Design and implementation of a secure distributed data repository. IBM Thomas J. Watson Research Division, Yorktown HeightsGoogle Scholar
  16. Jacobi J (2018) M-Disc optical media reviewed: your data, good for a thousand years. PCWorld. On-line document. http://www.pcworld.com/article/2933478/storage/m-disc-optical-media-reviewed-your-data-good-for-a-thousand-years.html
  17. Jacobson R (2013) 2.5 quintillion bytes of data created every day: how does CPG & retail manage it? IBM Industry Insights. http://www.ibm.com/blogs/insights-on-business/consumer-products/2-5-quintillion-bytes-of-data-created-every-day-how-does-cpg-retail-manage-it/
  18. Lunt BM, Linford MR, Davis RC, Jamieson S, Pearson A, Wang H (2013) Toward permanence in digital data storage. Proc Arch Conf 1:132–136Google Scholar
  19. Menon AK, Gupta BK (1999) Nanotechnology: a data storage perspective. Nanostruct Mater 11(8):965–986CrossRefGoogle Scholar
  20. Parhami B (2018) Dependable computing: a multi-level approach, draft of book manuscript, available on-line at http://www.ece.ucsb.edu/~parhami/text_dep_comp.htm
  21. Petascale Data Storage Institute (2012) Analyzing failure data. Project Web site: http://www.pdl.cmu.edu/PDSI/FailureData/index.html
  22. Plank JS (2013) Erasure codes for storage stems: a brief primer. Usenix Mag 38(6):44–50MathSciNetGoogle Scholar
  23. Qian C, Huang L, Xie P, Xiao N, Wang Z (2015) Efficient data management on 3D stacked memory for big data applications. In: Proceedings of the 10th international design & test symposium, Dead Sea, pp 84–89Google Scholar
  24. Rabin M (1989) Efficient dispersal of information for security, load balancing, and fault tolerance. J ACM 36(2):335–348MathSciNetCrossRefGoogle Scholar
  25. Rao TRN, Fujiwara E (1989) Error-control coding for computer systems. Prentice Hall, Upper Saddle RiverGoogle Scholar
  26. Rizzo L (1997) Effective erasure codes for reliable computer communication protocols. ACM Comput Commun Rev 27(2):24–36CrossRefGoogle Scholar
  27. Schroeder B, Gibson GA (2007) Understanding disk failure rates: what does an MTTF of 1,000,000 hours mean to you? ACM Trans Storage 3(3):8, 31 ppCrossRefGoogle Scholar
  28. Strukov DB, Snider GS, Stewart DR, Williams RS (2008) The missing memristor found. Nature 453(7191):80–83CrossRefGoogle Scholar
  29. Svrcek I (2009) Accelerated life cycle comparison of Millenniata archival DVD. Final report for Naval Air Warfare Center Weapons Division, 75 ppGoogle Scholar
  30. Yan W et al (2017) ROS: a rack-based optical storage system with inline accessibility for long-term data preservation. In: Proceedings of the 12th European conference on computer systems, Belgrade, pp 161–174Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringUniversity of CaliforniaSanta BarbaraUSA

Section editors and affiliations

  • Bingsheng He
  • Behrooz Parhami
    • 1
  1. 1.Dept. of Electrical and Computer EngineeringUniversity of California, Santa BarbaraSanta BarbaraUSA