Journal of Zhejiang University SCIENCE C

, Volume 13, Issue 6, pp 428–439 | Cite as

Convex relaxation for a 3D spatiotemporal segmentation model using the primal-dual method



A method based on 3D videos is proposed for multi-target segmentation and tracking with a moving viewing system. A spatiotemporal energy functional is built up to perform motion segmentation and estimation simultaneously. To overcome the limitation of the local minimum problem with the level set method, a convex relaxation method is applied to the 3D spatiotemporal segmentation model. The relaxed convex model is independent of the initial condition. A primal-dual algorithm is used to improve computational efficiency. Several indoor experiments show the validity of the proposed method.

Key words

3D spatiotemporal segmentation Motion estimation Total variation Primal-dual 

CLC number

TP391.7 TP317.4 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Alvarez, L., Castano, C.A., Garcia, M., Krissian, K., Mazorra, L., Salgado, A., Sanchez, J., 2009. A new energy-based method for 3D motion estimation of incompressible PIV flows. Comput. Vis. Image Understand., 113(7):802–810. [doi:10.1016/j.cviu.2009.01.005]CrossRefGoogle Scholar
  2. Boykov, Y., Funka-Lea, G., 2006. Graph cuts and efficient N-D image segmentation. Int. J. Comput. Vis., 70(2):109–131. [doi:10.1007/s11263-006-7934-5]CrossRefGoogle Scholar
  3. Boykov, Y., Kolmogorov, V., 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell., 26(9):1124–1137. [doi:10.1109/TPAMI.2004.60]CrossRefGoogle Scholar
  4. Bresson, X., Esedoglu, S., Vandergheynst, P., Thiran, J., Osher, S., 2007. Fast global minimization of the active contour/snake model. J. Math. Imag. Vis., 28(2):151–167. [doi:10.1007/s10851-007-0002-0]MathSciNetCrossRefGoogle Scholar
  5. Caselles, V., Kimmell, R., Sapiro, G., 1997. Geodesic active contours. Int. J. Comput. Vis., 22(1):61–79. [doi:10.1023/A:1007979827043]MATHCrossRefGoogle Scholar
  6. Chambolle, A., 2004. An algorithm for total variation minimization and applications. J. Math. Imag. Vis., 20(1–2): 89–97. [doi:10.1023/B:JMIV.0000011325.36760.1e]MathSciNetGoogle Scholar
  7. Chan, T.F., Vese, L.A., 2001. Active contours without edges. IEEE Trans. Image Process., 10(2):266–277. [doi:10.1109/83.902291]MATHCrossRefGoogle Scholar
  8. Chan, T.F., Golub, G.H., Mulet, P., 1999. A nonlinear primal-dual method for total variation-based image restoration. SIAM J. Sci. Comput., 20(6):1964–1977. [doi:10.1137/S1064827596299767]MathSciNetMATHCrossRefGoogle Scholar
  9. Chan, T.F., Esedoglu, S., Nikolova, M., 2004. Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math., 66:1632–1648. [doi:10.1137/040615286]MathSciNetGoogle Scholar
  10. Dufaux, F., Moccagatta, I., Moscheni, F., Nicolas, H., 1994. Vector quantization-based motion field segmentation under the entropy criterion. J. Vis. Commun. Image Represent., 5:356–369. [doi:10.1006/jvci.1994.1034]CrossRefGoogle Scholar
  11. Feghali, R., Mitiche, A., 2004. Spatiotemporal motion boundary detection and motion boundary velocity estimation for tracking moving objects with a moving camera: a level sets PDEs approach with concurrent camera motion compensation. IEEE Trans. Image Process., 13(11): 1473–1490. [doi:10.1109/TIP.2004.836158]CrossRefGoogle Scholar
  12. Goldstein, T., Bresson, X., Osher, S., 2009. Geometric applications of the split Bregman method: segmentation and surface reconstruction. J. Sci. Comput., 45(1–3):272–293. [doi:10.1007/s10915-009-9331-z]MathSciNetGoogle Scholar
  13. Heeger, D.J., Jepson, A.D., 1992. Subspace methods for recovering rigid motion I: algorithm and implementation. Int. J. Comput. Vis., 7(2):95–117. [doi:10.1007/BF00128130]CrossRefGoogle Scholar
  14. Horn, B.K.P., Schunck, B.G., 1981. Determining optical flow. Artif. Intell., 17(1–3):185–203. [doi:10.1016/0004-3702(81)90024-2]CrossRefGoogle Scholar
  15. Jonasson, L., Bresson, X., Hagmann, P., Cuisenaire, O., Meuli, R., Thiran, J.P., 2005. White matter fiber tract segmentation in DT-MRI using geometric flows. Med. Image Anal., 9(3):223–236. [doi:10.1016/]CrossRefGoogle Scholar
  16. Kass, M., Witkin, A., Terzopoulos, D., 1988. Snakes: active contour models. Int. J. Comput. Vis., 1(4):321–331. [doi:10.1007/BF00133570]CrossRefGoogle Scholar
  17. Leventon, M.E., Grimson, W.E.L., Faugeras, O., 2000. Statistical Shape Influence in Geodesic Active Contours. IEEE Conf. on Computer Vision and Pattern Recognition, 1:316–323. [doi:10.1109/CVPR.2000.855835]Google Scholar
  18. Longuet-Higgins, H.C., Prazdny, K., 1980. The interpretation of a moving retinal image. Proc. R. Soc. Lond. B, 208(1173):385–397. [doi:10.1098/rspb.1980.0057]CrossRefGoogle Scholar
  19. Malladi, R., Kimmel, R., Adalsteinsson, D., Sapiro, G., Caselles, V., Sethian, J.A., 1996. A Geometric Approach to Segmentation and Analysis of 3D Medical Images. Proc. Workshop on Mathematical Methods in Biomedical Image Analysis, p.244–252. [doi:10.1109/MMBIA.1996.534076]Google Scholar
  20. Merriman, B., Bence, J.K., Osher, S.J., 1994. Motion of multiple junctions: a level set approach. J. Comput. Phys., 112(2):334–363. [doi:10.1006/jcph.1994.1105]MathSciNetCrossRefGoogle Scholar
  21. Mémin, E., Pérez, P., 2002. Hierarchical estimation and segmentation of dense motion fields. Int. J. Comput. Vis., 46(2):129–155. [doi:10.1023/A:1013539930159]MATHCrossRefGoogle Scholar
  22. Mumford, D., Shah, J., 1989. Optimal approximations of piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math., 42(5):577–685. [doi:10.1002/cpa.3160420503]MathSciNetMATHCrossRefGoogle Scholar
  23. Ohno, K., Nomura, T., Tadokoro, S., 2006. Real-Time Robot Trajectory Estimation and 3D Map Construction Using 3D Camera. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, p.5279–5285. [doi:10.1109/IROS.2006.282027]Google Scholar
  24. Osher, S., Sethian, J., 1988. Fronts propagating with curvature dependent speed: algorithms based on the Hamilton-Jacobi formulation. J. Comput. Phys., 79(1):12–49. [doi:10.1016/0021-9991(88)90002-2]MathSciNetMATHCrossRefGoogle Scholar
  25. Paragios, N., Deriche, R., 2005. Geodesic active regions and level set methods for motion estimation and tracking. Comput. Vis. Image Understand., 97(3):259–282. [doi:10.1016/j.cviu.2003.04.001]CrossRefGoogle Scholar
  26. Rudin, L.I., Osher, S., Fatemi, E., 1992. Nonlinear total variation based noise removal algorithms. Phys. D, 60(1–4): 259–268. [doi:10.1016/0167-2789(92)90242-F]MATHCrossRefGoogle Scholar
  27. Schnorr, C., 1991. Determining optical flow for irregular domains by minimizing quadratic functionals of a certain class. Int. J. Comput. Vis., 6(1):25–38. [doi:10.1007/BF00127124]MathSciNetCrossRefGoogle Scholar
  28. Wang, S., Yu, H., 2011. A Variational Approach for Ego-motion Estimation and Segmentation Based on 3D TOF Camera. 4th Int. Congress on Image and Signal Processing, p.1160–1164. [doi:10.1109/CISP.2011.6100402]Google Scholar
  29. Ye, C., Hegde, G.P.M., 2009. Robust Edge Extraction for SwissRange SR-3000 Range Image. Proc. IEEE Int. Conf. on Robotics and Automation, p.2437–2442. [doi:10.1109/ROBOT.2009.5152559]Google Scholar

Copyright information

© Journal of Zhejiang University Science Editorial Office and Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.Department of Information Science & Electronic EngineeringZhejiang UniversityHangzhouChina

Personalised recommendations