Camera Pose Free Depth Sensing Based on Focus Stacking
Abstract
Binocular or multi-view depth imaging usually fails when camera pose is fixed, but depth need to be sensed with single fixed camera in some scenarios. To tackle this problem, we present a camera pose free depth sensing based on focus stacking. We first compute the mapping between scene depth and focus ring. A sharpness function based on discrete Fourier transform (DFT) is provided to calculate the in-focus parts, and parallax method is used to obtain the mapping between object space and image space. After the camera mapping calculation, we calculate the depth of the scene from the image distance map (IDM), which is generated by fusing in-focus areas of different images in the focal stacks. To reconstruct the scene, a depth map and an all-in-focus (AiF) image are combined from the focal stacks and IDM. Experimental results show that the proposed method is effective and robust compared to some binocular stereo and focus stacking method, and the depth sensing accuracy of our method is over 98%.
Keywords
Depth sensing Focus staking Camera poseReferences
- 1.Akira, K., Kiyoharu, A., Tsuhan, C.: Reconstructing dense light field from array of multifocus images for novel view synthesis. IEEE Trans. Image Process. 16(1), 269 (2007). A Publication of the IEEE Signal Processing SocietyMathSciNetCrossRefGoogle Scholar
- 2.Alonso, J.R., Fernández, A., Ferrari, J.A.: Reconstruction of perspective shifts and refocusing of a three-dimensional scene from a multi-focus image stack. Appl. Opt. 55(9), 2380 (2016)CrossRefGoogle Scholar
- 3.Fan, C., Weng, C., Lin, Y., Cheng, P.: Surface profiling measurement using varifocal lens based on focus stacking. In: 2018 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), pp. 1–5, May 2018. https://doi.org/10.1109/I2MTC.2018.8409820
- 4.Grossmann, P.: Depth from focus. Pattern Recogn. Lett. 5(1), 63–69 (1987)MathSciNetCrossRefGoogle Scholar
- 5.Gulbins, J., Gulbins, R.: Photographic Multishot Techniques: High Dynamic Range, Super-Resolution, Extended Depth of Field, Stitching. Rocky Nook, Inc., Santa Barbara (2009)Google Scholar
- 6.Heiko, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2007)Google Scholar
- 7.Konolige, K.: Small vision systems: hardware and implementation. In: Shirai, Y., Hirose, S. (eds.) Robotics Research, pp. 203–212. Springer, London (1998). https://doi.org/10.1007/978-1-4471-1580-9_19CrossRefGoogle Scholar
- 8.Kuthirummal, S., Nagahara, H., Zhou, C., Nayar, S.K.: Flexible depth of field photography. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 58–71 (2010)CrossRefGoogle Scholar
- 9.Levin, A., Lischinski, D., Weiss, Y.: Colorization using optimization. ACM Trans. Graph. 23(3), 689–694 (2004)CrossRefGoogle Scholar
- 10.Levin, A., Lischinski, D., Weiss, Y.: A closed form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30, 228–242 (2007) CrossRefGoogle Scholar
- 11.Lischinski, D., Farbman, Z., Uyttendaele, M., Szeliski, R.: Interactive local adjustment of tonal values, pp. 646–653 (2006)Google Scholar
- 12.Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M.F., Toyama, K.: Digital photography with flash and no-flash image pairs. ACM Trans. Graph. 23(3), 664–672 (2004)CrossRefGoogle Scholar
- 13.Sakurikar, P., Narayanan, P.J.: Focal stack representation and focus manipulation. In: 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR) (2017)Google Scholar
- 14.Xu, G., Quan, Y., Ji, H.: Estimating defocus blur via rank of local patches. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5371–5379 (2017)Google Scholar
- 15.Zhang, L.X., Sun, H.Y., Guo, H.C., Fan, Y.C.: Auto focusing algorithm based on largest gray gradient summation. Acta Photonica Sinica 42(5), 605–610 (2013)CrossRefGoogle Scholar
- 16.Zhao, H., Bao, G.T., Wei, T.: Experimental research and analysis of automatic focusing function for imaging measurement. Opt. Precis. Eng. 12, 531–536 (2004)Google Scholar
- 17.Zhou, S., Lou, Z., Yu, H.H., Jiang, H.: Multiple view image denoising using 3D focus image stacks. In: Signal & Information Processing (2016)Google Scholar
- 18.Zhuo, S., Sim, T.: Defocus map estimation from a single image. Pattern Recogn. 44(9), 1852–1858 (2011)CrossRefGoogle Scholar