Abstract
We propose a novel framework for joint 2D segmentation and 3D pose and 3D shape recovery, for images coming from a single monocular source. In the past, integration of all three has proven difficult, largely because of the high degree of ambiguity in the 2D - 3D mapping. Our solution is to learn nonlinear and probabilistic low dimensional latent spaces, using the Gaussian Process Latent Variable Models dimensionality reduction technique. These act as class or activity constraints to a simultaneous and variational segmentation – recovery – reconstruction process. We define an image and level set based energy function, which we minimise with respect to 3D pose and shape, 2D segmentation resulting automatically as the projection of the recovered shape under the recovered pose. We represent 3D shapes as zero levels of 3D level set embedding functions, which we project down directly to probabilistic 2D occupancy maps, without the requirement of an intermediary explicit contour stage. Finally, we detail a fast, open-source, GPU-based implementation of our algorithm, which we use to produce results on both real and artificial video sequences.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Rosenhahn, B., Brox, T., Weickert, J.: Three-dimensional shape knowledge for joint image segmentation and pose tracking. IJCV 73, 243–262 (2007)
Schmaltz, C., Rosenhahn, B., Brox, T., Cremers, D., Weickert, J., Wietzke, L., Sommer, G.: Region-Based Pose Tracking. In: Martí, J., Benedí, J.M., Mendonça, A.M., Serrat, J. (eds.) IbPRIA 2007. LNCS, vol. 4478, pp. 56–63. Springer, Heidelberg (2007)
Dambreville, S., Sandhu, R., Yezzi, A., Tannenbaum, A.: Robust 3D Pose Estimation and Efficient 2D Region-Based Segmentation from a 3D Shape Prior. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 169–182. Springer, Heidelberg (2008)
Prisacariu, V.A., Reid, I.: PWP3D: Real-Time Segmentation and Tracking of 3D Objects. IJCV, 1–20
Vese, L.A., Chan, T.F.: A multiphase level set framework for image segmentation using the mumford and shah model. IJCV 50, 271–293 (2002)
Rousson, M., Paragios, N.: Prior Knowledge, Level Set Representations & Visual Grouping. IJCV 76, 231–243 (2008)
Riklin-raviv, T., Kiryati, N., Sochen, N.: Prior-based segmentation and shape registration in the presence of projective distortion. IJCV 72, 309–328 (2007)
Tsai, A., Yezzi, A., Wells, W., Tempany, C., Tucker, D., Fan, A., Grimson, E., Willsky, A.: A shape-based approach to the segmentation of medical imagery using level sets. T-MI 22, 137–154 (2003)
Dambreville, S., Rathi, Y., Tannenbaum, A.: A framework for image segmentation using shape models and kernel space shape priors. T-PAMI 30, 1385–1399 (2008)
Prisacariu, V., Reid, I.: Nonlinear shape manifolds as shape priors in level set segmentation and tracking. In: CVPR 2011, pp. 2185–2192 (2011)
Sandhu, R., Dambreville, S., Yezzi, A., Tannenbaum, A.: A Nonrigid Kernel-Based Framework for 2D-3D Pose Estimation and 2D Image Segmentation. T-PAMI 33, 1098–1115 (2011)
Prisacariu, V., Reid, I.: Shared shape spaces. In: ICCV 2011 (2011)
Santner, J., Unger, M., Pock, T., Leistner, C., Saffari, A., Bischof, H.: Interactive Texture Segmentation using Random Forests and Total Variation. In: BMVC 2009 (2009)
Lawrence, N.: Probabilistic non-linear principal component analysis with gaussian process latent variable models. JMLR 6, 1783–1816 (2005)
NVIDIA: NVIDIA CUDA Programming Guide 4.1 (2012)
Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: ISMAR 2007, pp. 1–10 (2007)
Bibby, C., Reid, I.: Robust Real-Time Visual Tracking Using Pixel-Wise Posteriors. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 831–844. Springer, Heidelberg (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Prisacariu, V.A., Segal, A.V., Reid, I. (2013). Simultaneous Monocular 2D Segmentation, 3D Pose Recovery and 3D Reconstruction. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds) Computer Vision – ACCV 2012. ACCV 2012. Lecture Notes in Computer Science, vol 7724. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37331-2_45
Download citation
DOI: https://doi.org/10.1007/978-3-642-37331-2_45
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-37330-5
Online ISBN: 978-3-642-37331-2
eBook Packages: Computer ScienceComputer Science (R0)