Blockchain-Based Secure and Reliable Distributed Deduplication Scheme

  • Jingyi Li
  • Jigang WuEmail author
  • Long Chen
  • Jiaxing Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11334)


Due to the explosive growth of data volume on the Internet, deduplication techniques have been wildly used in cloud storage to save both disk space and network bandwidth. However, conventional deduplication schemes lead to problems with data reliability that can be attributed to the algorithm implementation where there is only one copy for each file stored in the cloud. Furthermore, the participation of trusted third party in most of the previous work has brought about the security challenge as single point of failure. In this paper, we propose a blockchain based deduplication scheme with high reliability and confidentiality in which the files are distributed to multiple servers and the information of files is recorded on the time-stamped blockchain whose central authorities are replaced to automatically decentralized smart contracts. Based on the proposed scheme, we present relevant protocols to achieve secure cloud storage derived from the consensus and incentive mechanism. Security analysis demonstrates that our deduplication scheme can achieve the proposed security goals while it has limited overhead proved by simulation experiments.


Deduplication Blockchain Distributed Cloud storage Security 



This work was supported by the National Natural Science Foundation of China under Grant Nos. 61702115 and 61672171, Natural Science Foundation of Guangdong, China under Grant No. 2018B030311007, and Major R&D Project of Educational Commission of Guangdong under Grant No. 2016KZDXM052, and China Postdoctoral Science Foundation Fund under Grant No. 2017M622632, and Opening Project of Guangdong Province Key Laboratory of Big Data Analysis and Processing No. 201805.


  1. 1.
    Anderson, P., Zhang, L.: Fast and secure laptop backups with encrypted de-duplication. In: International Conference on Large Installation System Administration, pp. 1–8 (2010)Google Scholar
  2. 2.
    Bellare, M., Keelveedhi, S., Ristenpart, T.: Message-locked encryption and secure deduplication. In: Johansson, T., Nguyen, P.Q. (eds.) EUROCRYPT 2013. LNCS, vol. 7881, pp. 296–312. Springer, Heidelberg (2013). Scholar
  3. 3.
  4. 4.
    Douceur, J.R., Adya, A., Bolosky, W.J., Simon, P.: Reclaiming space from duplicate files in a serverless distributed file system. In: Proceedings of International Conference on Distributed Computing Systems, pp. 617–624 (2002)Google Scholar
  5. 5.
  6. 6.
    Dropbox: A file-storage and sharing service.
  7. 7.
    Filecoin: A decentralized storage network.
  8. 8.
    Harnik, D., Pinkas, B., Shulman-Peleg, A.: Side channels in cloud services: deduplication in cloud storage. IEEE Secur. Priv. 8(6), 40–47 (2010)CrossRefGoogle Scholar
  9. 9.
    Kamara, S., Lauter, K.: Cryptographic cloud storage. In: Sion, R., et al. (eds.) FC 2010. LNCS, vol. 6054, pp. 136–149. Springer, Heidelberg (2010). Scholar
  10. 10.
    Li, J., Liu, Z., Chen, L., Chen, P., Wu, J.: Blockchain-based security architecture for distributed cloud storage. In: 2017 IEEE International Conference on Ubiquitous Computing and Communications (ISPA/IUCC) and 2017 IEEE International Symposium on Parallel and Distributed Processing with Applications, pp. 408–411. IEEE (2017)Google Scholar
  11. 11.
    Li, J., Wu, J., Chen, L.: Block-secure: blockchain based scheme for secure P2P cloud storage. Inf. Sci. 465, 219–231 (2018).,
  12. 12.
    Li, J., et al.: Secure distributed deduplication systems with improved reliability. IEEE Trans. Comput. 64(12), 3569–3579 (2015)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Li, J., Chen, X., Li, M., Li, J., Lee, P.P.C., Lou, W.: Secure deduplication with efficient and reliable convergent key management. IEEE Trans. Parallel Distrib. Syst. 25(6), 1615–1625 (2014)CrossRefGoogle Scholar
  14. 14.
    Mozy: A file-storage and sharing service.
  15. 15.
    Nakamoto, S.: Bitcoin: a peer-to-peer electronic cash system. (2008)
  16. 16.
    Shin, Y., Koo, D., Hur, J.: A survey of secure data deduplication schemes for cloud storage systems. ACM (2017)Google Scholar
  17. 17.
    Shin, Y., Koo, D., Yun, J., Hur, J.: Decentralized server-aided encryption for secure deduplication in cloud storage. IEEE Trans. Serv. Comput. PP(99), 1 (1939)Google Scholar
  18. 18.
    Sia: Simple decentralized storage.
  19. 19.
    Storer, M.W., Greenan, K., Long, D.D.E., Miller, E.L.: Secure data deduplication. In: ACM International Workshop on Storage Security and Survivability, pp. 1–10 (2008)Google Scholar
  20. 20.
    Storj: A peer-to-peer cloud storage network.
  21. 21.
    Weber, I., Xu, X., Riveret, R., Governatori, G., Ponomarev, A., Mendling, J.: Untrusted business process monitoring and execution using blockchain. In: La Rosa, M., Loos, P., Pastor, O. (eds.) BPM 2016. LNCS, vol. 9850, pp. 329–347. Springer, Cham (2016). Scholar
  22. 22.
    Wilcox-O’Hearn, Z., Warner, B.: Tahoe: the least-authority filesystem. In: Proceedings of the 4th ACM International Workshop on Storage Security and Survivability, pp. 21–26. ACM (2008)Google Scholar
  23. 23.
    Wood, G.: Ethereum: a secure decentralised generalised transaction ledger (2014).

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Guangdong University of TechnologyGuangzhouChina
  2. 2.Guangdong Key Laboratory of Big Data Analysis and ProcessingGuangzhouPeople’s Republic of China

Personalised recommendations