Deep convolution network for surveillance records super-resolution

  • Pourya Shamsolmoali
  • Masoumeh Zareapoor
  • Deepak Kumar Jain
  • Vinay Kumar Jain
  • Jie Yang


The aim of image super resolution (SR) is to recover low resolution (LR) input image or video to a visually desirable high-resolution (HR) one. The task of identifying an object in surveillance records is interesting, yet challenging due to the low resolution of the video. This paper, proposed a deep learning method for resolution recovery, the low-resolution objects and points in the surveillance records are up-sampled using a deep Convolutional Neural Network (CNN) to avoid problems of image boundary the data padded with zeros. The network is trained and tested on two surveillance datasets. Dissimilar to the outdated methods which operate components individually, our model performs combined optimization for all the layers. The proposed CNN model has a lightweight structure and minimal data pre-processing and computation cost. Testing our model and comparing with advanced techniques, we observed promising results. The code is accessible at


Super-resolution Convolution neural networks Surveillance records Deep learning 


  1. 1.
    Al-Najjar YAY, Soong DDC (2012) Comparison of image quality assessment: PSNR, HVS, SSIM, UIQI. International Journal of Scientific and Engineering Research 3(8):1–5Google Scholar
  2. 2.
    Bengio Y, Goodfellow IJ, Courville A (2015) Deep learning. Book in preparation for MIT Press. 2015Google Scholar
  3. 3.
    Cai D, Chen K, Qian Y, Kämäräinen JK (2017) Convolutional low-resolution fine-grained classification. Pattern recognition letters.
  4. 4.
    Cui Z, Chang H, Shan S, Zhong B, Chen X (2014) Deep network cascade for image super-resolution. In: European Conference on Computer Vision, pp 49–64Google Scholar
  5. 5.
    Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: ECCVGoogle Scholar
  6. 6.
    Dong C, Loy CC, He K, Tang X (2015) Image super resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine IntelligenceGoogle Scholar
  7. 7.
    Fu Z, Li Z, Ding L, Nguyen T (2014) Translation invariance-based super resolution method for mixed resolution multiview video. In: ICIPGoogle Scholar
  8. 8.
    Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. IEEE International Conference on Computer Vision, pp 349–356Google Scholar
  9. 9.
    Grgic M, Delac K, Grgic S (2011) SCface - surveillance cameras face database. Multimedia Tools and Applications 51(3):863–879CrossRefGoogle Scholar
  10. 10.
    He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Proc. Eur. Conf. Computer vision, pp 346–361Google Scholar
  11. 11.
    Huang J-B, Singh A, Ahuja N (2015) Single image super resolution using transformed self-exemplars. In: CVPRGoogle Scholar
  12. 12.
    Hung EM, Dorea CC, Garcia DC, Queiroz RL (2010) Transform-domain super resolution for multiview images using depth information. In: EUSIPCOGoogle Scholar
  13. 13.
    Irani M, Peleg S (1991) Improving resolution by image registration. CVGIP: Graphical models and image processing 53(3):231–239, 1991Google Scholar
  14. 14.
    Jain AK, Nguyen TQ (2013) Video super resolution for mixed resolution stereo. In: ICIPGoogle Scholar
  15. 15.
    Jin Z, Tillo T, Yao C, Xiao J, Zhao Y (2015) Virtual view assisted video super-resolution and enhancement. IEEE transactions on circuits and Systems for Video Technology, pp 467–478Google Scholar
  16. 16.
    Joachimiak M, Aflaki P, Hannuksela MM, Gabbouj M (2014) Evaluation of depth-based super resolution on compressed mixed resolution 3d video. In: ACCVGoogle Scholar
  17. 17.
    Kim KI, Kown Y (2010) Single-image super-resolution using sparse regression and natural image prior. IEEE Trans Pattern Anal Mach Intell 32(6):1127–1133CrossRefGoogle Scholar
  18. 18.
    Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. CoRR, abs/1412.6980Google Scholar
  19. 19.
    Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. In Proc. Adv Neural Inf Process Syst, pp 1097–1105Google Scholar
  20. 20.
    LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradientbased learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRefGoogle Scholar
  21. 21.
    Liao R, Tao X, Li R, Ma Z, Jia J (2015) Video superresolution via deep draft-ensemble learning. IEEE International Conference on Computer Vision, pp 531–539Google Scholar
  22. 22.
    Liu C, Sum D (2014) On bayesian adaptive video super resolution. IEEE Trans Pattern Anal Mach Intell 36(2):346–360CrossRefGoogle Scholar
  23. 23.
    Marco Bevilacqua CG, Roumy A, Morel M-LA (2012) Low-complexitysingle-imagesuper-resolutionbased on nonnegative neighbor embedding. In: BMVCGoogle Scholar
  24. 24.
    Na Z, Liao R, Tao X, Xu L, Jia J, Wu E (2015) Handling motion blur in multi-frame super-resolution. CVPRGoogle Scholar
  25. 25.
    Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: Proc. Int. Conf. Mach Learn, pp 807–814Google Scholar
  26. 26.
    Ouyang W, Wang X, Zeng X, Qiu S, Luo P, Tian Y, Li H, Yang S, Wang Z, Loy C-C, Tang X (2015) Deepid-net: deformable deep convolutional neural networks for object detection. In: Proc. IEEE Conf. Comput. Vis. Pattern Recogn. pp 2403–2412Google Scholar
  27. 27.
    Razvan P, Tomas M, Yoshua B (2013) On the difficulty of training recurrent neural networks. ICMLGoogle Scholar
  28. 28.
    Schulter S, Leistner C, Bischof H (2015) Fast and accurate image upscaling with super-resolution forests. In: CVPRGoogle Scholar
  29. 29.
    Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: ICLR. arXiv:1409.1556v6Google Scholar
  30. 30.
    Song X, Dai Y, Qin X (2016) Deep depth super-resolution: learning depth super-resolution using deep convolutional neural network. Computer Vision – ACCV 2016 pp 360–376Google Scholar
  31. 31.
    Stelmach L, Tom WJ, Meegan D, Vincent A (2000) Stereo image quality: effects of mixed spatio-temporal resolution. IEEE Transactions on Circuits and Systems for Video Technology 10(2):188–193CrossRefGoogle Scholar
  32. 32.
    Sun Y, Chen Y, Wang X, Tang X (2014) Deep learning face representation by joint identification-verification. In: Proc. Adv Neural Inf Process Syst, pp 1988–1996Google Scholar
  33. 33.
    Sutskever I, Martens J, Dahl G, Hinton G (2013) On the importance of initialization and momentum in deep learning. Proceedings of the 30th international conference on Mach Learn, pp 1139–1147Google Scholar
  34. 34.
    Timofte R, Smet VD, Gool LV (2013) Anchored neighborhood regression for fast example-based super-resolution. IEEE International Conference on Computer Vision, pp 1920–1927Google Scholar
  35. 35.
    Wang S, Zhang L, Liang Y, Pan Q (2012) Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In: CVPRGoogle Scholar
  36. 36.
    Wong Y, Chen S, Mau S, Sanderson C, Lovell BC (2011) Patch-based probabilistic image quality assessment for face selection and improved video-based face recognition. IEEE Biometrics Workshop, Computer Vision and Pattern Recognition (CVPR), pp 81–88Google Scholar
  37. 37.
    Xie Y, Xiao J, Tillo T, Wei Y, Zhao Y (2016) 3D video super resolution using fully convolutional neural networks. IEEE International Conference on Multimedia and Expo (ICME)Google Scholar
  38. 38.
    Yang J, Wright J, Huang T, Ma Y (2010) Image super resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Yang C-Y, Ma C, Yang MH (2014) Single-image super resolution: a benchmark. European Conference on Computer Vision, pp 372–386Google Scholar
  40. 40.
    Zeyde R, Elad M, Protter M (2012) On single image scale-up using sparse-representations. In: Curves and Surfaces, pp 711–730Google Scholar
  41. 41.
    Zhang N, Donahue J, Girshick R, Darrell T (2014) Part-based RCNNs for fine-grained category detection. In: Proc. Eur. Conf. Comput. Vis., pp 834–849Google Scholar
  42. 42.
    Zhao Y, Wang R, Dong W, Jia W, Yang J, Liu X, Gao W (2017) GUN: Gradual Upsampling Network for single image super-resolution. Computer Vision and Pattern Recognition (CVPR). arXiv:1703.04244Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institute of Image Processing and Pattern RecognitionShanghai Jiao Tong UniversityShanghaiChina
  2. 2.Advanced Scientific Computing DivisionEuro-Mediterranean Centre on Climate ChangeLecceItaly
  3. 3.Institute of AutomationChinese Academy of SciencesBeijingChina
  4. 4.Jaypee University of Engineering and TechnologyGunaIndia

Personalised recommendations