Abstract
Convex relaxation techniques have become a popular approach to a variety of image segmentation problems as they allow to compute solutions independent of the initialization. In this paper, we propose a novel technique for the segmentation of RGB-D images using convex function optimization. The function that we propose to minimize considers both the color image and the depth map for finding the optimal segmentation. We extend the objective function by moment constraints, which allow to include prior knowledge on the 3D center, surface area or volume of the object in a principled way. As we show in this paper, the relaxed optimization problem is convex, and thus can be minimized in a globally optimal way leading to high-quality solutions independent of the initialization. We validated our approach experimentally on four different datasets, and show that using both color and depth substantially improves segmentation compared to color or depth only. Further, 3D moment constraints significantly robustify segmentation which proves in particular useful for object tracking.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Boykov, Y., Jolly, M.-P.: Interactive organ segmentation using graph cuts. In: Delp, S.L., DiGoia, A.M., Jaramaz, B. (eds.) MICCAI 2000. LNCS, vol. 1935, pp. 276–286. Springer, Heidelberg (2000)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. on Patt. Anal. and Mach. Intell. 23(11), 1222–1239 (2001)
Chambolle, A., Cremers, D., Pock, T.: A convex approach for computing minimal partitions. Communications on Pure and Applied Mathematics (2008)
Chan, T., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of denoising and segmentation models. SIAM Journal on Applied Mathematics 66(5), 1632–1648 (2006)
Chan, T., Esedoḡlu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM Journal on Applied Mathematics 66(5), 1632–1648 (2006)
Chan, T., Vese, L.: A level set algorithm for minimizing the Mumford–Shah functional in image processing. In: IEEE Workshop on Variational and Level Set Methods, Vancouver, CA, pp. 161–168 (2001)
Cremers, D., Osher, S.J., Soatto, S.: Kernel density estimation and intrinsic alignment for shape priors in level set segmentation. Int. J. of Computer Vision 69(3), 335–351 (2006)
Etyngier, P., Segonne, F., Keriven, R.: Shape priors using manifold learning techniques. In: IEEE Int. Conf. on Computer Vision. Rio de Janeiro (October 2007)
Foulonneau, A., Charbonnier, P., Heitz, F.: Affine-invariant geometric shape priors for region-based active contours. IEEE Trans. on Patt. Anal. and Mach. Intell. 28(8), 1352–1357 (2006)
Gordon, G., Darrell, T., Harville, M., Woodfill, J.: Background estimation and removal based on range and color. In: Int. Conf. on Computer Vision and Pattern Recognition, pp. 459–464 (1999)
Greig, D.M., Porteous, B.T., Seheult, A.H.: Exact maximum a posteriori estimation for binary images. J. Roy. Statist. Soc., Ser. B. 51(2), 271–279 (1989)
Grenander, U., Chow, Y., Keenan, D.M.: Hands: A Pattern Theoretic Study of Biological Shapes. Springer, New York (1991)
Klodt, M., Cremers, D.: A convex framework for image segmentation with moment constraints. In: IEEE Int. Conf. on Computer Vision (2011)
Lellmann, J., Kappes, J., Yuan, J., Becker, F., Schnörr, C.: Convex multi-class image labeling by simplex-constrained total variation. In: Tai, X.-C., Mørken, K., Lysaker, M., Lie, K.-A. (eds.) SSVM 2009. LNCS, vol. 5567, pp. 150–162. Springer, Heidelberg (2009)
Osher, S.J., Sethian, J.A.: Fronts propagation with curvature dependent speed: Algorithms based on Hamilton–Jacobi formulations. J. of Comp. Phys. 79, 12–49 (1988)
Schoenemann, T., Cremers, D.: A combinatorial solution for model-based image segmentation and real-time tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence (2009)
Taylor, C., Cowley, A.: Fast scene analysis using image and range data. In: Proc. of the Intl. Conf. on Robotics and Automation, ICRA (2011)
Unger, M., Pock, T., Cremers, D., Bischof, H.: TVSeg - interactive total variation based image segmentation. In: British Machine Vision Conference (BMVC), Leeds, UK (September 2008)
Veksler, O.: Star shape prior for graph-cut image segmentation. In: Europ. Conf. on Computer Vision., pp. 454–467 (2008)
Zach, C., Gallup, D., Frahm, J.M., Niethammer, M.: Fast global labeling for real-time stereo using multiple plane sweeps. In: Vision, Modeling and Visualization Workshop VMV 2008(October 2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Klodt, M., Sturm, J., Cremers, D. (2013). Scale-Aware Object Tracking with Convex Shape Constraints on RGB-D Images. In: Weickert, J., Hein, M., Schiele, B. (eds) Pattern Recognition. GCPR 2013. Lecture Notes in Computer Science, vol 8142. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40602-7_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-40602-7_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40601-0
Online ISBN: 978-3-642-40602-7
eBook Packages: Computer ScienceComputer Science (R0)