Skip to main content
SpringerLink
Log in

Let’s Take a Walk on Superpixels Graphs: Deformable Linear Objects Segmentation and Model Estimation

  • Conference paper
  • First Online:
Computer Vision – ACCV 2018 (ACCV 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11362))

Included in the following conference series:

Abstract

While robotic manipulation of rigid objects is quite straightforward, coping with deformable objects is an open issue. More specifically, tasks like tying a knot, wiring a connector or even surgical suturing deal with the domain of Deformable Linear Objects (DLOs). In particular the detection of a DLO is a non-trivial problem especially under clutter and occlusions (as well as self-occlusions). The pose estimation of a DLO results into the identification of its parameters related to a designed model, e.g. a basis spline. It follows that the stand-alone segmentation of a DLO might not be sufficient to conduct a full manipulation task. This is why we propose a novel framework able to perform both a semantic segmentation and b-spline modeling of multiple deformable linear objects simultaneously without strict requirements about environment (i.e. the background). The core algorithm is based on biased random walks over the Region Adiacency Graph built on a superpixel oversegmentation of the source image. The algorithm is initialized by a Convolutional Neural Networks that detects the DLO’s endcaps. An open source implementation of the proposed approach is also provided to easy the reproduction of the whole detection pipeline along with a novel cables dataset in order to encourage further experiments.

This work was supported by the European Commissions Seventh Framework Programme (FP7/2007-2013) under grant agreement no. 601116.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    In [18] the authors deal with a thin flexible manipulator which may be described as a cable due to the high number of degrees of freedom.

  2. 2.

    https://github.com/ntnu-bioopt/libfrangi.

  3. 3.

    https://github.com/kapcom01/Curviliniar_Detector.

  4. 4.

    https://github.com/m4nh/ariadne.

  5. 5.

    https://github.com/m4nh/cables_dataset.

References

  1. Javdani, S., Tandon, S., Tang, J., O’Brien, J.F., Abbeel, P.: Modeling and perception of deformable one-dimensional objects. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 1607–1614. IEEE (2011)

    Google Scholar 

  2. Jackson, R.C., Yuan, R., Chow, D.L., Newman, W., Çavuşoğlu, M.C.: Automatic initialization and dynamic tracking of surgical suture threads. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 4710–4716. IEEE (2015)

    Google Scholar 

  3. Saha, M., Isto, P.: Manipulation planning for deformable linear objects. IEEE Trans. Robot. 23, 1141–1150 (2007)

    Article  Google Scholar 

  4. Lui, W.H., Saxena, A.: Tangled: learning to untangle ropes with RGB-D perception. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 837–844. IEEE (2013)

    Google Scholar 

  5. Nair, A., et al.: Combining self-supervised learning and imitation for vision-based rope manipulation. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 2146–2153 (2017)

    Google Scholar 

  6. Schulman, J., Lee, A., Ho, J., Abbeel, P.: Tracking deformable objects with point clouds. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 1130–1137. IEEE (2013)

    Google Scholar 

  7. Hopcroft, J.E., Kearney, J.K., Krafft, D.B.: A case study of flexible object manipulation. Int. J. Robot. Res. 10, 41–50 (1991)

    Article  Google Scholar 

  8. Remde, A., Henrich, D., Wörn, H.: Picking-up deformable linear objects with industrial robots (1999)

    Google Scholar 

  9. Yue, S., Henrich, D.: Manipulating deformable linear objects: sensor-based fast manipulation during vibration. In: Proceedings of the IEEE International Conference on Robotics and Automation, ICRA 2002, vol. 3, pp. 2467–2472. IEEE (2002)

    Google Scholar 

  10. Alvarez, N., Yamazaki, K., Matsubara, T.: An approach to realistic physical simulation of digitally captured deformable linear objects. In: IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR), pp. 135–140. IEEE (2016)

    Google Scholar 

  11. Koo, K.m., Jiang, X., Kikuchi, K., Konno, A., Uchiyama, M.: Development of a robot car wiring system. In: IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2008, pp. 862–867. IEEE (2008)

    Google Scholar 

  12. Gregorio, D.D., Zanella, R., Palli, G., Pirozzi, S., Melchiorri, C.: Integration of robotic vision and tactile sensing for wire-terminal insertion tasks. IEEE Trans. Autom. Sci. Eng., 1–14 (2018)

    Google Scholar 

  13. Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6517–6525. IEEE (2017)

    Google Scholar 

  14. Huang, J., et al.: Speed/accuracy trade-offs for modern convolutional object detectors (2017)

    Google Scholar 

  15. Trémeau, A., Colantoni, P.: Regions adjacency graph applied to color image segmentation. IEEE Trans. Image Process. 9, 735–744 (2000)

    Article  Google Scholar 

  16. Bibiloni, P., González-Hidalgo, M., Massanet, S.: A survey on curvilinear object segmentation in multiple applications. Pattern Recogn. 60, 949–970 (2016)

    Article  Google Scholar 

  17. Jiang, X., Koo, K.M., Kikuchi, K., Konno, A., Uchiyama, M.: Robotized assembly of a wire harness in a Car production line. Adv. Robot. 25, 473–489 (2011)

    Article  Google Scholar 

  18. Camarillo, D.B., Loewke, K.E., Carlson, C.R., Salisbury, J.K.: Vision based 3-D shape sensing of flexible manipulators. In: IEEE International Conference on Robotics and Automation, ICRA 2008, pp. 2940–2947. IEEE (2008)

    Google Scholar 

  19. Schulman, J., Ho, J., Lee, C., Abbeel, P.: Learning from demonstrations through the use of non-rigid registration. In: Inaba, M., Corke, P. (eds.) Robotics Research. STAR, vol. 114, pp. 339–354. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-28872-7_20

    Chapter  Google Scholar 

  20. Moll, M., Kavraki, L.E.: Path planning for deformable linear objects. IEEE Trans. Robot. 22, 625–636 (2006)

    Article  Google Scholar 

  21. Padoy, N., Hager, G.D.: 3D thread tracking for robotic assistance in tele-surgery. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2102–2107. IEEE (2011)

    Google Scholar 

  22. Frangi, A.F., Niessen, W.J., Vincken, K.L., Viergever, M.A.: Multiscale vessel enhancement filtering. In: Wells, W.M., Colchester, A., Delp, S. (eds.) MICCAI 1998. LNCS, vol. 1496, pp. 130–137. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0056195

    Chapter  Google Scholar 

  23. Liskowski, P., Krawiec, K.: Segmenting retinal blood vessels with deep neural networks. IEEE Trans. Med. Imaging 35, 2369–2380 (2016)

    Article  Google Scholar 

  24. Melinščak, M., Prentašić, P., Lončarić, S.: Retinal vessel segmentation using deep neural networks. In: VISAPP 2015 (10th International Conference on Computer Vision Theory and Applications) (2015)

    Google Scholar 

  25. Li, Q., Feng, B., Xie, L., Liang, P., Zhang, H., Wang, T.: A cross-modality learning approach for vessel segmentation in retinal images. IEEE Trans. Med. Imaging 35, 109–118 (2016)

    Article  Google Scholar 

  26. Staal, J., Abràmoff, M.D., Niemeijer, M., Viergever, M.A., Van Ginneken, B.: Ridge-based vessel segmentation in color images of the retina. IEEE Trans. Med. Imaging 23, 501–509 (2004)

    Article  Google Scholar 

  27. Pătrăucean, V., Gurdjos, P., von Gioi, R.G.: A parameterless line segment and elliptical arc detector with enhanced ellipse fitting. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, pp. 572–585. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33709-3_41

    Chapter  Google Scholar 

  28. Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59, 167–181 (2004)

    Article  Google Scholar 

  29. Vedaldi, A., Soatto, S.: Quick shift and kernel methods for mode seeking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5305, pp. 705–718. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88693-8_52

    Chapter  Google Scholar 

  30. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2274–2282 (2012)

    Article  Google Scholar 

  31. Ning, J., Zhang, L., Zhang, D., Wu, C.: Interactive image segmentation by maximal similarity based region merging. Pattern Recogn. 43, 445–456 (2010)

    Article  Google Scholar 

  32. Predoehl, A., Morris, S., Barnard, K.: A statistical model for recreational trails in aerial images. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 337–344. IEEE (2013)

    Google Scholar 

  33. Dierckx, P.: Curve and Surface Fitting with Splines. Oxford University Press, New York (1995)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. DISI, University of Bologna, 40136, Bologna, Italy

    Daniele De Gregorio & Luigi Di Stefano

  2. DEI, University of Bologna, 40136, Bologna, Italy

    Gianluca Palli

Authors
  1. Daniele De Gregorio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gianluca Palli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luigi Di Stefano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniele De Gregorio .

Editor information

Editors and Affiliations

  1. IIIT Hyderabad, Hyderabad, India

    C. V. Jawahar

  2. ANU, Canberra, ACT, Australia

    Hongdong Li

  3. Simon Fraser University, Burnaby, BC, Canada

    Greg Mori

  4. ETH Zurich, Zurich, Zürich, Switzerland

    Konrad Schindler

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

De Gregorio, D., Palli, G., Di Stefano, L. (2019). Let’s Take a Walk on Superpixels Graphs: Deformable Linear Objects Segmentation and Model Estimation. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds) Computer Vision – ACCV 2018. ACCV 2018. Lecture Notes in Computer Science(), vol 11362. Springer, Cham. https://doi.org/10.1007/978-3-030-20890-5_42

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-20890-5_42

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-20889-9

  • Online ISBN: 978-3-030-20890-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation