Erasure-Coded Network Backup System (ECNBS)

Abstract

Traditionally File Servers are used as storage medium as Network Attached Storage. However the need for backup servers is a necessity. As a rule of thumb, data in a network environment needs to be replicated on 3 different machines. The amount of data replicated and other issues such as consistency and concurrency maintenance becomes a huge overhead. As a solution to those issues, Erasure Coded Storage is tipped to be the next best alternative. In Erasure Coded Storage, an object (file) is broken down into n blocks, and encoded. Furthermore some redundant parity blocks are created using mathematical formulas, so that redundancy could be provided. In case of loss of original blocks of data, any parity block can be used for recovery. There exist several Erasure Code Techniques, namely Reed-Solomon, Hierarchical, Self-Repairing and Regenerating codes. Each code has its specificity, some look into diminishing bandwidth consumption, while other lesser computational loads. Till date, there are only a few implementations based on erasure codes. Our contribution is that we are proposing a novel architecture for Network Backup System using Erasure Codes in ECNBS that includes three layers. The Interface Layer presents the user with a layout of files similar to windows environment, whereby files are stored in folders and subfolders. The Intermediate Layer (Mapping Layer) stores information about the files and the locations of the related blocks. The Storage Layer is where blocks of data are physically stored. The newly implemented system is fully functional, and has also been compared to the traditionally used File Server.