Skip to main content
SpringerLink
Log in

Nonprehensile Manipulation Control and Task Planning for Deformable Object Manipulation: Results from the RoDyMan Project

  • Conference paper
  • First Online:
Informatics in Control, Automation and Robotics (ICINCO 2018)

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 613))

Abstract

This chapter aims the broadcasting of the results achieved by the RoDyMan project about the task planning manipulation of deformable objects, and the nonprehensile manipulation control. The final demonstrator of the project is a pizza-making process. After an introduction to the general topic of nonprehensile manipulation, the mechatronic design and the high-level software architecture are described. Then, the smoothed particle hydrodynamic formulation is briefly introduced, along with the description of a detection method for a deformable object. The task planning for stretching a modelling clay, emulating the pizza dough, is sketched. After, the problematic control objective is split into several nonprehensile motion primitives: holonomic and nonholonomic rolling, friction-induced manipulation, and tossing are the described primitives. This chapter highlights the achievements reached so far by the project, and pave the way towards future research directions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.rodyman.eu.

  2. 2.

    https://www.youtube.com/user/ThePRISMAlab.

  3. 3.

    http://rtc.nagoya.riken.jp/RIBA/index-e.html.

  4. 4.

    https://cordis.europa.eu/project/rcn/98813_en.html.

References

  1. Amjid MR, Shehzad A, Hussain S, Shabbir MA, Khan MR, Shoaib M (2013) A comprehensive review on wheat flour dough rheology. Pak J Food Sci 23:105–123

    Google Scholar 

  2. Anshelevich E, Owens S, Lamiraux F, Kavraki LE (2000) Deformable volumes in path planning applications. In: 2000 IEEE international conference on robotics and automation, San Francisco, CA, pp 2290–2285

    Google Scholar 

  3. Bätz G, Mettin U, Schimdts A, Scheint M, Wollherr D, Shiriaev A (2010) Ball dribbling with an underactuated continuous-time control phase: theory & experiments. In: 2010 IEEE/RSJ international conference on intelligent robots and systems, Taipei, TW, pp 2890–2895

    Google Scholar 

  4. Bätz G, Yaqub A, Wu H, Kuhnlenz K, Wollherr D, Buss M (2010) Dnamic manipulation: nonprehensile ball catching. In: 18th mediterranean conference on control and automation, Marrakech, MA, pp 365–370

    Google Scholar 

  5. Bay H, Ess A, Tuytelaars T, Van Gool L (2006) Surf: speeded up robust features. In: European conference on computer vision, Graz, AT, pp 404–417

    Google Scholar 

  6. Bender J, Koschier D (2015) Divergence-free smoothed particle hydrodynamics. In: 14th ACM SIGGRAPH/eurographics symposium on computer animation, Los Angeles, CA, USA, pp 147–155

    Google Scholar 

  7. Bender J, Koschier D (2017) Divergence-free SPH for incompressible and viscous fluids. IEEE Trans Vis Comput Graph 23:1193–1206

    Article  Google Scholar 

  8. Bittanti S, Laub A, Willems J (1991) The Riccati equation. Springer, New York

    Book  MATH  Google Scholar 

  9. Bloch AM, Crouch PE (1995) Nonholonomic control systems on riemannian manifolds. SIAM J Control Optim 33(1):126–148

    Article  MathSciNet  MATH  Google Scholar 

  10. Boothby WM (1986) An introduction to differentiable manifolds and Riemannian geometry, vol 120. Academic Press

    Google Scholar 

  11. Cacace J, Finzi A, Lippiello V, Loianno G, Sanzone D (2015) Aerial service vehicles for industrial inspection: task decomposition and plan execution. Appl Intell 42:49–62

    Article  Google Scholar 

  12. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell PAMI 8(6):679–698

    Article  Google Scholar 

  13. Chitta S, Sucan I, Cousins S (2012) Moveit! [ros topics]. IEEE Robot Autom Mag 19:18–19

    Article  Google Scholar 

  14. Chung TJ (2010) Computational fluid dynamics, 2nd edn. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  15. Cretu AM, Payeur P, Petriu E (2008) Neural network mapping and clustering of elastic behavior from tactile and range imaging for virtualized reality applications. IEEE Trans Instrum Meas 57(9):1918–1928

    Article  Google Scholar 

  16. Donaire A, Ruggiero F, Buonocore L, Lippiello V, Siciliano B (2016) Passivity-based control for a rolling-balancing system: the nonprehensile disk-on-disk. IEEE Trans Control Syst Technol 25(6):2135–2142

    Article  Google Scholar 

  17. Eymard R, Gallouët TR, Herbin R (2000) The finite volume method. In: Handbook of numerical analysis, vol. 7, pp 713–1020

    Google Scholar 

  18. Frank B, Schmedding R, Stachniss C, Teschner M, Burgard W (2010) Learning the elasticity parameters of deformable objects with a manipulation robot. In: 2010 IEEE/RSJ international conference on intelligent robots and systems, Taipei, TW, pp 1877–1883

    Google Scholar 

  19. Frank B, Stachniss C, Abdo N, Burgard W (2011) Efficient motion planning for manipulation robots in environments with deformable objects. In: 2011 IEEE/RSJ international conference on intelligent robots and systems, San Francisco, CA, USA, pp 2180–2185

    Google Scholar 

  20. Frisken S, Gibson F, Mirtich B (1997) A survey of deformable modeling in computer graphics. Techincal report, Brigham and Women’s Hospital

    Google Scholar 

  21. Gonzalez RC, Woods RE (2006) Digital image processing, 3rd edn. Prentice-Hall Inc., Upper Saddle River

    Google Scholar 

  22. Gutiérrez-Giles A, Ruggiero F, Lippiello V, Siciliano B (2017) Modelling and control of a robotic hula-hoop system without velocity measurements. In: 20th world congress of the international federation of automatic control, Toulouse, FR, pp 9808–9814

    Google Scholar 

  23. Gutiérrez-Giles A, Ruggiero F, Lippiello V, Siciliano B (2018) Nonprehensile manipulation of an underactuated mechanical system with second-order nonholonomic constraints: the robotic hula-hoop. IEEE Robot Autom Lett 3(2):1136–1143

    Article  Google Scholar 

  24. Harris C, Stephens M (1988) A combined corner and edge detector. In: 4th Alvey vision conference, Manchester, UK, pp 147–151

    Google Scholar 

  25. Hartley R, Zisserman A (2004) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  26. Higashimori M, Utsumi K, Kaneko M (2008) Dexterous hyper plate inspired by pizza manipulation. In: 2008 IEEE international conference on robotics and automation, Pasadena, CA, USA, pp 399–406

    Google Scholar 

  27. Higashimori M, Utsumi K, Omoto Y, Kaneko M (2009) Dynamic manipulation inspired by the handling of a pizza peel. IEEE Trans. Robot. 25(4):829–838

    Article  Google Scholar 

  28. Higashimori M, Yoshimoto K, Kaneko M (2010) Active shaping of an unknown rheological object based on deformation decomposition into elasticity and plasticity. In: 2010 IEEE international conference on robotics and automation, Anchorage, AK, USA, pp 5120–5126

    Google Scholar 

  29. Irving G, Teran J, Fedkiw R (2004) Invertible finite elements for robust simulation of large deformation. In: 2004 ACM SIGGRAPH/eurographics symposium on computer animation, Grenoble, FR, pp 131–140

    Google Scholar 

  30. Kato H, Billinghurst M (1999) Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: 2nd IEEE and ACM international workshop on augmented reality, San Francisco, CA, USA, pp 85–94

    Google Scholar 

  31. Lang J (2004) An acquisition method for interactive deformable models. In: Second international conference on creating, connecting and collaborating through computing, Kyoto, JP, pp 165–170

    Google Scholar 

  32. Lippiello V, Ruggiero F, Siciliano B (2016) The effects of shapes in input-state linearization for stabilization of nonprehensile planar rolling dynamic manipulation. Robot Autom Lett 1(1):492–499

    Article  Google Scholar 

  33. Liu GR, Zhang J, Lam KY, Li H, Xu G, Zhong ZH, Li GY, Han X (2008) A gradient smoothing method (GSM) with directional correction for solid mechanics problems. Comput Mech 41(3):457–472

    Article  MATH  Google Scholar 

  34. Liu MB, Liu GR (2010) Smoothed particle hydrodynamics (SPH): An overview and recent developments. Arch. Comput. Methods in Eng. 17(1):25–76

    Article  MathSciNet  MATH  Google Scholar 

  35. Lowe DG (1999) Object recognition from local scale-invariant features. In: International conference on computer vision, Kerkyra, GR, vol 2, pp 1150–1157

    Google Scholar 

  36. Lynch KM, Mason MT (1999) Dynamic nonprehensile manipulation: controllability, planning, and experiments. Int J Robot. Res. 18(1):64–92

    Article  Google Scholar 

  37. Lynch KM, Murphey TD (2003) Control of nonprehensile manipulation. In: Bicchi A, Prattichizzo D, Christensen H (eds) Control problems in robotics, vol 4. Springer tracts in advanced robotics. Springer, Heidelberg, pp 39–57

    Chapter  MATH  Google Scholar 

  38. Mitsoulis E (2008) Numerical simulation of calendering viscoplastic fluids. J Non-Newton Fluid Mech 154:77–88

    Article  MATH  Google Scholar 

  39. Mitsoulis E, Hatzikiriakos SG (2009) Rolling of bread dough: experiments and simulations. Food Bioprod Process 87(2):124–138

    Article  Google Scholar 

  40. Monaghan JJ (1992) Smoothed particle hydrodynamics. Annu. Rev. Astron. Astrophys. 30:543–574

    Article  Google Scholar 

  41. Monaghan JJ (1994) Simulating free surface flows with SPH. J. Comput. Phys. 110(2):399–406

    Article  MATH  Google Scholar 

  42. Monaghan JJ (2005) Smoothed particle hydrodynamics. Rep Progress Phys 68:1703–1759

    Article  MathSciNet  MATH  Google Scholar 

  43. Murray RM, Li Z, Sastry S (1994) A mathematical introduction to robotic manipulation. CRC Press

    Google Scholar 

  44. Nealen A, Müller M, Keiser R, Boxerman R, Carlson M (2005) Physically based deformable models in computer graphics. Eurographics: State of the Art Report, pp 71–94

    Google Scholar 

  45. Peer A, Ihmsen M, Cornelis J, Teschner M (2015) An implicit viscosity formulation for SPH fluids. ACM Trans Graph 34(4):1–10

    Article  Google Scholar 

  46. Petit A, Lippiello V, Siciliano B (2015) Real-time tracking of 3D elastic objects with an RGB-D sensor. In: 2015 IEEE/RSJ international conference on intelligent robots and systems, Hamburg, GE, pp. 3914–3921

    Google Scholar 

  47. Phillips-Grafflin C, Berenson D (2014) A representation of deformable objects for motion planning with no physical simulation. In: 2014 IEEE International conference on robotics and automation, Hong Kong, CN, pp 98–105

    Google Scholar 

  48. Prattichizzo D, Trinkle J (2016) Grasping. In: Siciliano B, Khatib O (eds) Springer handbook of robotics, pp 955–988. Springer

    Google Scholar 

  49. Reist P, D’Andrea R (2012) Design and analysis of a blind juggling robot. IEEE Trans Robot 28(6):1228–1243

    Article  Google Scholar 

  50. Reznik DS, Canny JF (2001) C’mon part, do the local motion! In: 2001 IEEE international conference on robotics and automation, vol 3, Seul, KR, pp 2235–2242

    Google Scholar 

  51. Rosten E, Drummond T(2006) Machine learning for high-speed corner detection. In: European conference on computer vision, Graz, AT, pp 430–443

    Google Scholar 

  52. Ruggiero F, Lippiello V, Siciliano B (2018) Nonprehensile dynamic manipulation: a survey. IEEE Robot Autom Lett 3(3):1711–1718

    Article  Google Scholar 

  53. Satici A, Ruggiero F, Lippiello V, Siciliano B (2016) A coordinate-free framework for robotic pizza tossing and catching. In: 2016 IEEE international conference on robotics and automation, Stockholm, SE, pp 3932–3939

    Google Scholar 

  54. Serra D, Ferguson J, Ruggiero F, Siniscalco A, Petit A, Lippiello V, Siciliano B (2018) On the experiments about the nonprehensile reconfiguration of a rolling sphere on a plate. In: 26th mediterranean conference on control and automation, Zadar, HR

    Google Scholar 

  55. Serra D, Ruggiero F, Donaire A, Buonocore L, Lippiello V, Siciliano B (2019) Control of nonprehensile planar rolling manipulation: a passivity based approach. IEEE Trans Robot 35(2):317–329. https://doi.org/10.1109/TRO.2018.2887356

    Article  Google Scholar 

  56. Serra D, Satici A, Ruggiero F, Lippiello V, Siciliano B (2016) An optimal trajectory planner for a robotic batting task: the table tennis example. In: 13th international conference on informatics in control, automation and robotics, Lisbon, PT, pp 90–101 (2016)

    Google Scholar 

  57. Shao S, Lo E (2003) Incompressible SPH method for simulating newtonian and non-newtonian flows with a free surface. Adv Water Resour 26(7):787–800

    Article  Google Scholar 

  58. Shiriaev A, Perram J, Canudas-de Wit C (2005) Constructive tool for orbital stabilization of underactuated nonlinear systems: virtual constraints approach. IEEE Trans Autom Control 50(8):1164–1176

    Article  MathSciNet  MATH  Google Scholar 

  59. Sofou S, Muliawan EB, Hatzikiriakos SG, Mitsoulis E (2008) Rheological characterization and constitutive modeling of bread dough. Rheologica Acta 47(4):369–381

    Article  Google Scholar 

  60. Sousa CD, Cortesão R (2014) Physically feasibility of robot base inertial parameters identification: a linear matrix inequality approach. Int J Robot Res 33(6):931–944

    Article  Google Scholar 

  61. de Souza Andrade LF, Sandim M, Petronetto F, Pagliosa P, Paiva A (2014) SPH fluids for viscous jet buckling. In: 27th conference on graphics, patterns and images, Rio de Janeiro, BR, pp 65–72

    Google Scholar 

  62. de Souza Andrade LF, Sandim M, Petronetto F, Pagliosa P, Paiva A (2015) Particle-based fluids for viscous jet buckling. Comput Graph 52:106–115

    Article  Google Scholar 

  63. Spong MW (1994) Partial feedback linearization of underactuated mechanical systems. In: IEEE/RSJ/GI international conference on intelligent robots and systems, Munich, DE, pp 314–321

    Google Scholar 

  64. Takahashi T, Dobashi Y, Fujishiro I, Nishita T, Lin MC (2015) Implicit formulation for SPH-based viscous fluids. Comput Graph Forum 34(2):493–502

    Article  Google Scholar 

  65. Tokumoto S, Hirai S (2002) Deformation control of rheological food dough using a forming process model. In: 2002 IEEE international conference on robotics and automation, Washington, DC, USA, pp 1457–1464

    Google Scholar 

  66. Van Bockstaele F, De Leyn I, Eeckhout M, Dewettinck K (2008) Rheological properties of wheat flour dough and the relationship with bread volume. i. creep-recovery measurements. Cereal Chem J 85(6):753

    Article  Google Scholar 

  67. Vose TH, Umbanhowar P, Lynch KM (2009) Friction-induced lines of attraction and repulsion for parts sliding on an oscillated plate. IEEE Trans Autom Sci Eng 6(4):685–699

    Article  Google Scholar 

  68. Vose TH, Umbanhowar P, Lynch KM (2012) Sliding manipulation of rigid bodies on a controlled 6-dof plate. Int J Robot Res 31(7):819–838

    Article  Google Scholar 

  69. Vose T, Umbanhowar P, Lynch K (2009) Friction-induced velocity fields for point parts sliding on a rigid oscillated plate. Int J Robot Res 28(8):1020–1039

    Article  Google Scholar 

  70. Wada T, Hirai S, Kawamura S, Kamiji N (2001) Robust manipulation of deformable objects by a simple PID feedback. In: 2001 IEEE international conference on robotics and automation, Seoul, KR, pp 85–90

    Google Scholar 

  71. Wieser H (2007) Chemistry of gluten proteins. Food Microbiol 24(2):115–119

    Article  Google Scholar 

  72. Woodruff J, Lynch K (2017) Planning and control for dynamic, nonprehensile, and hybrid manipulation tasks. In: 2017 IEEE international conference on robotics and automation, Singapore, pp 4066–4073

    Google Scholar 

  73. Zienkiewicz O, Taylor R, Zhu J (2005) The finite element method set, 6th edn. Butterworth-Heinemann, Oxford

    Google Scholar 

Download references

Acknowledgements

The research leading to these results has been supported by the RoDyMan project, which has received funding from the European Research Council FP7 Ideas under Advanced Grant agreement number 320992.

Author information

Authors and Affiliations

  1. CREATE Consortium and Department of Electrical Engineering and Information Technology, University of Naples Federico II, Via Claudio 21, 80125, Naples, Italy

    Fabio Ruggiero, Jung-Tae Kim, Jonathan Cacace, Giuseppe Andrea Fontanelli, Vincenzo Lippiello & Bruno Siciliano

  2. Center for Research and Advanced Studies of the National Polytechnic Institute, Av. IPN 2508, San Pedro Zacatenco, 07300, Mexico City, Mexico

    Alejandro Gutierrez-Giles

  3. Mechanical and Biomedical Engineering, Boise State University, 1910 University Drive, Boise, 83725, ID, USA

    Aykut C. Satici

  4. School of Engineering, The University of Newcastle, University Drive, Callaghan, NSW, 2308, Australia

    Alejandro Donaire

  5. CERN, Router de Meyrin 385, Geneve 23, Switzerland

    Luca Rosario Buonocore

Authors
  1. Fabio Ruggiero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jung-Tae Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alejandro Gutierrez-Giles
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aykut C. Satici
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alejandro Donaire
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jonathan Cacace
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Luca Rosario Buonocore
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Giuseppe Andrea Fontanelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Vincenzo Lippiello
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Bruno Siciliano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fabio Ruggiero .

Editor information

Editors and Affiliations

  1. Ford Research and Advanced Engineer, Dearborn, MI, USA

    Oleg Gusikhin

  2. University of Paris-EST Créteil (UPEC), Créteil, France

    Kurosh Madani

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ruggiero, F. et al. (2020). Nonprehensile Manipulation Control and Task Planning for Deformable Object Manipulation: Results from the RoDyMan Project. In: Gusikhin, O., Madani, K. (eds) Informatics in Control, Automation and Robotics. ICINCO 2018. Lecture Notes in Electrical Engineering, vol 613. Springer, Cham. https://doi.org/10.1007/978-3-030-31993-9_4

Download citation

Publish with us

Policies and ethics

Search

Navigation