Efficient Storage and Processing of Video Data for Moving Object Detection Using Hadoop/MapReduce
Technological advances and easy availability of low cost video camera has further encouraged users for deploying network of surveillance systems. These systems generate massive data. Thus, storage and processing of the huge video data for application such as video forensics, post event investigation etc., has emerged as a challenging problem to the research community. In this paper we propose a powerful approach that makes use of Hadoop Distributed File System (HDFS) for efficient storage of video data and programming model called MapReduce for data intensive computing. The proposed approach detects moving objects and provides their coordinates which can be used for localizing post event investigation. We have analyzed the storage and processing of video data of varying resolution and size to assess the performance of proposed approach.
KeywordsMotion detection Hadoop distributed file system MapReduce
- 1.Dean, J. and Ghemawat, S.: Map Reduce: Simplified Data Processing on Large Cluster. Operating Systems Design and Implementation (OSDI). San Francisco, United States (2004)137–150.Google Scholar
- 2.Pereira, R. and Breitman, K.: A Cloud Based Architecture for Improving Video Compression Time Efficiency: The Split &Merge Approach. 3rd IEEE Int. Conf. on Cloud Computing (CLOUD). Miami, United States July (2010) 482–489.Google Scholar
- 3.Schmidt, R. and Rella, M.: An approach for processing large and non-uniform media objects on MapReduce-based clusters. 13th Int. Conf. on Asia-Pacific Digital Libraries, (ICADL). Beijing, China (2011).Google Scholar
- 4.Tan, H. and Chen, L.: An approach for fast and parallel video processing on Apache Hadoop clusters. IEEE Int. Conf on Multimedia and Expo (ICME). Chengdu, China (2014) 1–6.Google Scholar
- 5.Yamamoto, M. and Kaneko, K.: Parallel image database processing with MapReduce and performance evaluation in pseudo distributed mode. Int. J. of Electronic Commerce Studies, Vol. 3, No. 2 (2012) 211–228.Google Scholar
- 6.Ryu, C., Lee, C. M. Jhang, Kim, C., Seo, E.: Extensible video processing framework in apache hadoop. in Proc. IEEE 5th Int. Conf on Cloud Computing Technology and Science (CloudCom), Vol. 2, Bristol, England, Dec (2013) 305–310.Google Scholar
- 7.Zhang, W., Xu, L., Duan, P., Gong, W., Liu, X. and Q. Lu.: Towards a High Speed Video Cloud Based on Batch Processing Integrated with Fast Processing. in Proc. of Int. Conf of Identification, Information and Knowledge in the Internet of Things (IIKI), Beijing, China (2014) 28–33.Google Scholar
- 8.Zhao, X., H. Ma, H., Zhang, H., Tang, Y. and Fu, G.: Metadata extraction and correction for large-scale traffic surveillance videos. in Proc. on Big Data (Big Data), Washington, DC, United States (2014) 412–420.Google Scholar
- 9.Zhu, H., Shen, Z., Shang, L. and Pang, X.: Parallel Image Texture Feature Extraction under Hadoop Cloud Platform. Intelligent Computing Theory. Springer International Publishing (2014) 459–465.Google Scholar
- 10.Premchaiswadi, W., Tungkatsathan, A., Intarasema, S. and Premchaiswadi, A.: Improving performance of content-based image retrieval schemes using Hadoop MapReduce. In Int. Conf. of High Performance Computing and Simulation (HPCS). Helsinki, Finland (2013) 615–620.Google Scholar
- 11.Gamage, T.D, Samarvikrama, J.G., Rodrigo, R.and Pasqual, A. A.: Image filtering with MapReduce in pseudo-distribution mode. in IEEE Conf. of Moratuwa Engineering Research Conference (MERCon). Moratuwa, Sri Lanka (2015) 160–164.Google Scholar
- 12.APACHE, 2010. Hadoop MapReduce framework. Available: http://hadoop.apache.org/mapreduce/.
- 13.T. White. Hadoop: “The Definitive Guide”. Yahoo Press, 2010.Google Scholar
- 14.Holmes, Alex, “ Hadoop in practice”, Manning Publications Co., 2012.Google Scholar
- 15.Grimson, W.E.L. and Stauffer, C.: Adaptive background mixture models for real-time tracking, in Proc. Of IEEE Conf. Computer Vision and Pattern Recognition, Vol. 2, Fort Collins, United States (1999) 22–29.Google Scholar
- 16.Lim, S., Apostolopoulos, J.G. and Gamal, A.E.: Optical flow estimation using temporally oversampled video, IEEE Trans. of Image Processing, Vol. 14, No. 8(2005) 1074–1087.Google Scholar
- 17.Yu, Y. and Chen, Y.: A real-time motion detection algorithm for traffic monitoring systems based on consecutive temporal difference. 7th Conf of Asian Control Conference (ACC). Hong Kong, China (2009) 1594–1599.Google Scholar
- 18.Gangodkar, D., Kumar, P. and Mittal, A.: Segmentation of moving objects in visible and thermal videos. In Int. Conf of Computer Communication and Informatics (ICCCI), Coimbatore, India (2012) 1–5.Google Scholar
- 19.Open Source Computer Vision (OpenCV) [Online]. Available: http://opencv.willowgarage.com/wiki/.
- 20.Performance Evaluation of Tracking and Surveillance, (PETS). Available: http://www.cvg.cs.rdg.ac.uk/slides/pets.html.
- 21.Ecole Polytechnique, Montreal, LITIV Datasets Available: http://www.polymtl.ca/litiv/en/vid/index.php.
- 22.OTCBVS Benchmark Dataset Collection, Ohio State University. Available: http://www.cse.ohio-state.edu/otcbvs-bench/.
- 23.Context Aware Vision Using Image-Based Active Recognition (CAVIAR).Available: http://groups.inf.ed.ac.uk/vision/CAVIAR/.html.