A Method of Recovering HBase Records from HDFS Based on Checksum File
Data recovery is a key problem in disaster recovery and digital forensics fields. The HDFS (Hadoop Distributed File System) is widely used for storing high-volume, velocity and variety dataset. However, previous work about data recovery mainly focuses on personal computers or mobile phones, and few attentions have been taken to HFDS. This paper analyzes the feature of HDFS and proposes a recovery method based on checksum file in order to address the records recovery problem of HBase, which is a common application on HDFS. We first carve out the Data blocks of HFile (HBase data file) using the corresponding checksum file, then analyze the format of HBase table records to extract them from the carved Data blocks. The experiments demonstrate that our method can restore HBase records effectively. The recovery rate is nearly 100% when the cluster size is 4 KB and 2 KB.
KeywordsHBase HDFS Records recovery File carving HFile
This work is supported by the Natural Science Foundation of China under Grant Nos. 61070212 and 61572165, the State Key Program of Zhejiang Province Natural Science Foundation of China under Grant No. LZ15F020003 and Key Lab of Information Network Security, Ministry of Public Security.
- 1.Richard III, G.G., Roussev, V.: Scalpel: a frugal, high performance file carver. In: DFRWS (2005)Google Scholar
- 4.EnCase Forensic. http://guidancesoftware.com/encase-forensic.htm
- 5.Adroit Photo Forensics. http://digital-assembly.com/products/adroit-photo-forensics/
- 7.Autopsy/The Sleuth Kit. http://sleuthkit.org
- 8.X-Ways Forensics. http://x-ways.net/forensics
- 9.Cohen, M.: Advanced jpeg carving. In: Proceedings of the 1st International Conference on Forensic Applications and Techniques in Telecommunications. Information, and Multimedia and Workshop, pp. 16:1–16:6 (2008)Google Scholar
- 14.Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In IEEE Symposium on Mass Storage Systems (2010)Google Scholar
- 18.Karresand, M., Shahmehri, N.: Fileprints: identifying file type by n-gram analysis. In: Proceedings of 7th IEEE Systems, Man and Cybernetics Information Assurance Workshop, pp. 64–71. IEEE (2006)Google Scholar
- 19.Veenman, C.J.: Statistical disk cluster classification for file carving. In: Proceedings of 3rd International Symposium on Information Assurance and Security, pp. 393–398. IEEE Computer Society (2007)Google Scholar