Introduction

Abstract

In this chapter, we show why data deduplication is important by stressing data explosion and large amounts of redundancies. We elaborate on current solutions (including storage data deduplication, redundancy elimination, information-centric networking) for data deduplication and the limitations of current solutions. We introduce a deduplication framework that optimizes data from clients to servers through networks. The framework consists of three components based on the level of deduplication. The client component removes local redundancies that occur in a client, the network component removes redundant transfers coming from different clients using redundancy elimination (RE) devices, and the server component eliminates redundancies coming from different networks. Then we show the evolution of data storage. Data storage has evolved from storage devices attached to a single computer (direct-attached storage) into storage devices attached to computer networks (storage area network and network-attached storage). We discuss the different kinds of storage devices and how they differ from one another. A redundant array of inexpensive disks (RAID), which improves storage access performance, is explained, and direct-attached storage (DAS), where storage is incorporated into a computer, is illustrated. We elaborate on storage area networks (SANs) and network-attached storage (NAS), where data from computers are transferred to storage devices through a dedicated network (SAN) or a general local area network used for sending and receiving application data (NAS). SAN and NAS consolidate and efficiently provide storage without wasting storage space compared to a DAS device. We describe a storage virtualization technique known as software-defined storage.