Abstract
Occlusion poses as a critical challenge in computer vision for a long time. Camera array based synthetic aperture photography has been regarded as a promising way to address the problem of occluded object imaging. However, the application of this technique is limited by the building cost and the immobility of the camera array system. In order to build a more practical synthetic aperture photography system, in this paper, a novel multiple moving camera based collaborative synthetic aperture photography is proposed. The main characteristics of our work include: (1) to the best of our knowledge, this is the first multiple moving camera based collaborative synthetic aperture photography system; (2) by building a sparse 3D map of the occluded scene using one camera, the information from the subsequent cameras can be incrementally utilized to estimate the warping induced by the focal plane; (3) the compatibility of different types of cameras, such as the hand-held action cameras or the quadrotor on-board cameras, shows the generality of the proposed framework. Extensive experiments have demonstrated the see-through-occlusion performance of the proposed approach in different scenarios.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Vaish, V., Wilburn, B., Joshi, N., Levoy, M.: Using plane+ parallax for calibrating dense camera arrays. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 1, p. I-2. IEEE (2004)
Vaish, V., Garg, G., Talvala, E.V., Antunez, E., Wilburn, B., Horowitz, M., Levoy, M.: Synthetic aperture focusing using a shear-warp factorization of the viewing transform. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, CVPR Workshops, p. 129. IEEE (2005)
Yang, T., Zhang, Y., Tong, X., Zhang, X., Yu, R.: Continuously tracking and see-through occlusion based on a new hybrid synthetic aperture imaging model. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3409–3416. IEEE (2011)
Zhang, X., Zhang, Y., Yang, T., Song, Z.: Calibrate a moving camera on a linear translating stage using virtual plane + parallax. In: Yang, J., Fang, F., Sun, C. (eds.) IScIDE 2012. LNCS, vol. 7751, pp. 48–55. Springer, Heidelberg (2013). doi:10.1007/978-3-642-36669-7_7
Yang, T., Ma, W., Wang, S., Li, J., Yu, J., Zhang, Y.: Kinect based real-time synthetic aperture imaging through occlusion. Multimedia Tools Appl. 75(12), 6925–6943 (2016)
Yang, T., Zhang, Y., Yu, J., Li, J., Ma, W., Tong, X., Yu, R., Ran, L.: All-in-focus synthetic aperture imaging. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 1–15. Springer, Cham (2014). doi:10.1007/978-3-319-10599-4_1
Zhang, X., Zhang, Y., Yang, T., Yang, Y.H.: Synthetic aperture photography using a moving camera-IMU system. Pattern Recogn. 62, 175–188 (2017)
Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2161–2168. IEEE (2006)
Gálvez-López, D., Tardos, J.D.: Bags of binary words for fast place recognition in image sequences. IEEE Trans. Robot. 28, 1188–1197 (2012)
Mur-Artal, R., Tardós, J.D.: Fast relocalisation and loop closing in keyframe-based SLAM. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 846–853. IEEE (2014)
Mur-Artal, R., Montiel, J., Tardós, J.D.: ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans. Robot. 31, 1147–1163 (2015)
Lepetit, V., Moreno-Noguer, F., Fua, P.: EPnP: an accurate \(o(n)\) solution to the PnP problem. Int. J. Comput. Vis. 81, 155–166 (2009)
Penate-Sanchez, A., Andrade-Cetto, J., Moreno-Noguer, F.: Exhaustive linearization for robust camera pose and focal length estimation. IEEE Trans. Pattern Anal. Mach. Intell. 35, 2387–2400 (2013)
Acknowledgments
This work is supported the National Natural Science Foundation of China (No. 61272288, No. 61301192, No. 61303123, No. 61672429, No. 61231016 (Key Project)), NPU New People and New Directions Foundation (No.1 3GH014604), The Fundamental Research Funds for the Central Universities (No. 3102015AX007), NPU New Ao Xiang Star (No. G2015KY0301), and Shen Zhen Science and Technology Foundation (JCYJ20160229172932237).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhang, X., Zhang, Y., Yang, T., Li, Z., Tao, D. (2016). Occluded Object Imaging Based on Collaborative Synthetic Aperture Photography. In: Zhang, Z., Huang, K. (eds) Intelligent Visual Surveillance. IVS 2016. Communications in Computer and Information Science, vol 664. Springer, Singapore. https://doi.org/10.1007/978-981-10-3476-3_1
Download citation
DOI: https://doi.org/10.1007/978-981-10-3476-3_1
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-3475-6
Online ISBN: 978-981-10-3476-3
eBook Packages: Computer ScienceComputer Science (R0)