Skip to main content
SpringerLink
Log in

Toward Effective Soft Robot Control via Reinforcement Learning

  • Conference paper
  • First Online:
Intelligent Robotics and Applications (ICIRA 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10462))

Included in the following conference series:

Abstract

A soft robot is a kind of robot that is constructed with soft, deformable and elastic materials. Control of soft robots presents complex modeling and planning challenges. We introduce a new approach to accomplish that, making two key contributions: designing an abstract representation of the state of soft robots, and developing a reinforcement learning method to derive effective control policies. The reinforcement learning process can be trained quickly by ignoring the specific materials and structural properties of the soft robot. We apply the approach to the Honeycomb PneuNets Soft Robot and demonstrate the effectiveness of the training method and its ability to produce good control policies under different conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Notes

  1. 1.

    http://optitrack.com/.

References

  1. Bai, A., Wu, F., Chen, X.: Bayesian mixture modelling and inference based thompson sampling in monte-carlo tree search. In: Proceedings of NIPS, pp. 1646–1654 (2013)

    Google Scholar 

  2. Bai, A., Wu, F., Chen, X.: Online planning for large markov decision processes with hierarchical decomposition. ACM Trans. Intell. Syst. Technol. 6(4) (2015). Article No. 45

    Google Scholar 

  3. Baird, L., et al.: Residual algorithms: Reinforcement learning with function approximation. In: Proceedings of ICML, pp. 30–37 (1995)

    Google Scholar 

  4. Chen, Y., Wu, F., Shuai, W., Wang, N., Chen, R., Chen, X.: Kejia robot - an attractive shopping mall guider. In: Proceedings of ICSR, pp. 145–154 (2015)

    Google Scholar 

  5. Chen, Y., Wu, F., Wang, N., Tang, K., Cheng, M., Chen, X.: KeJia-LC: a low-cost mobile robot platform - champion of demo challenge on benchmarking service robots at RoboCup 2015. In: Proceedings of RoboCup, vol. 9513, pp. 60–71 (2015)

    Google Scholar 

  6. Cheng, B., Sun, H., Chen, X.: Evolving honeycomb pneumatic finger in bullet physics engine. Robot Intell. Tech. App. 3, 411–423 (2015)

    Google Scholar 

  7. Cheng, M., Chen, X., Tang, K., Wu, F., Kupcsik, A., Iocchi, L., Chen, Y., Hsu, D.: Synthetical benchmarking of service robots: a first effort on domestic mobile platforms. In: Almeida, L., Ji, J., Steinbauer, G., Luke, S. (eds.) RoboCup 2015. LNCS (LNAI), vol. 9513, pp. 377–388. Springer, Cham (2015). doi:10.1007/978-3-319-29339-4_32

    Chapter  Google Scholar 

  8. Duriez, C.: Control of elastic soft robots based on real-time finite element method. In: Proceedings of ICRA, pp. 3982–3987 (2013)

    Google Scholar 

  9. Giorelli, M., Renda, F., Ferri, G., Laschi, C.: A feed-forward neural network learning the inverse kinetics of a soft cable-driven manipulator moving in three-dimensional space. In: Proceedings of IROS, pp. 5033–5039 (2013)

    Google Scholar 

  10. Hannan, M.W., Walker, I.D.: Kinematics and the implementation of an elephant’s trunk manipulator and other continuum style robots. J. Robotic Syst. 20(2), 45–63 (2003)

    Article  MATH  Google Scholar 

  11. Inoue, T., Hirai, S.: Modeling of soft fingertip for object manipulation using tactile sensing. In: Proceedings of IROS, pp. 2654–2659 (2003)

    Google Scholar 

  12. Kober, J., Peters, J.: Reinforcement learning in robotics: a survey. In: Reinforcement Learning, pp. 579–610 (2012)

    Google Scholar 

  13. Laschi, C., Cianchetti, M., Mazzolai, B., Margheri, L., Follador, M., Dario, P.: Soft robot arm inspired by the octopus. Adv. Robot. 26(7), 709–727 (2012)

    Article  Google Scholar 

  14. Laschi, C., Mazzolai, B., Cianchetti, M.: Soft robotics: technologies and systems pushing the boundaries of robot abilities. Sci. Robot. (2016)

    Google Scholar 

  15. Lu, D., Zhou, Y., Wu, F., Zhang, Z., Chen, X.: Integrating answer set programming with semantic dictionaries for robot task planning. In: Proceedings of IJCAI (2017)

    Google Scholar 

  16. Luo, M., Pan, Y., Skorina, E.H., Tao, W., Chen, F., Ozel, S., Onal, C.D.: Slithering towards autonomy: a self-contained soft robotic snake platform with integrated curvature sensing. Bioinspir. Biomim. 10(5), 055001 (2015)

    Article  Google Scholar 

  17. Mazzolai, B., Margheri, L., Cianchetti, M., Dario, P., Laschi, C.: Soft-robotic arm inspired by the octopus: II. From artificial requirements to innovative technological solutions. Bioinspir. Biomim. 7(2), 025005 (2012)

    Article  Google Scholar 

  18. McMahan, W., Chitrakaran, V., Csencsits, M., Dawson, D., Walker, I.D., Jones, B.A., Pritts, M., Dienno, D., Grissom, M., Rahn, C.D.: Field trials and testing of the OctArm continuum manipulator. In: Proceedings of ICRA, pp. 2336–2341 (2006)

    Google Scholar 

  19. Pfeifer, R.: soft robotics - the next generation of intelligent machines. Invited talk on IJCAI (2013)

    Google Scholar 

  20. Ramchurn, S.D., Huynh, T.D., Wu, F., Ikuno, Y., Flann, J., Moreau, L., Fischer, J.E., Jiang, W., Rodden, T., Simpson, E., Reece, S., Roberts, S., Jennings, N.R.: A disaster response system based on human-agent collectives. J. Artif. Intell. Res. 57, 661–708 (2016)

    MATH  Google Scholar 

  21. Renda, F., Giorelli, M., Calisti, M., Cianchetti, M., Laschi, C.: Dynamic model of a multibending soft robot arm driven by cables. IEEE Trans. Robot. 30(5), 1109–1122 (2014)

    Article  Google Scholar 

  22. Rus, D., Tolley, M.T.: Design, fabrication and control of soft robots. Nature 521(7553), 467–475 (2015)

    Article  Google Scholar 

  23. Shibata, M., Hirai, S.: Soft object manipulation by simultaneous control of motion and deformation. In: Proceedings ICRA, pp. 2460–2465 (2006)

    Google Scholar 

  24. Sun, H., Chen, X.-P.: Towards honeycomb pneunets robots. In: Kim, J.-H., Matson, E.T., Myung, H., Xu, P., Karray, F. (eds.) Robot Intelligence Technology and Applications 2. AISC, vol. 274, pp. 331–340. Springer, Cham (2014). doi:10.1007/978-3-319-05582-4_28

    Chapter  Google Scholar 

  25. Tolley, M.T., Shepherd, R.F., Mosadegh, B., Galloway, K.C., Wehner, M., Karpelson, M., Wood, R.J., Whitesides, G.M.: A resilient, untethered soft robot. Soft Robot. 1(3), 213–223 (2014)

    Article  Google Scholar 

  26. Webster, R.J., Jones, B.: Design and kinematic modeling of constant curvature continuum robots: a review. Int. J. Robot. Res. 29(13), 1661–1683 (2010)

    Article  Google Scholar 

  27. Wu, F., Chen, X.: Solving large-scale and sparse-reward DEC-POMDPs with correlation-MDPs. In: Proceedings of RoboCup, pp. 208–219 (2007)

    Google Scholar 

  28. Wu, F., Ramchurn, S., Chen, X.: Coordinating human-UAV teams in disaster response. In: Proceedings of IJCAI, pp. 524–530 (2016)

    Google Scholar 

Download references

Acknowledgments

Feng Wu was supported in part by National Natural Science Foundation of China (No. 61603368), the Youth Innovation Promotion Association of CAS (No. 2015373), and Natural Science Foundation of Anhui Province (No. 1608085QF134).

Author information

Authors and Affiliations

  1. University of Science and Technology of China, Hefei, 230027, Anhui, China

    Haochong Zhang, Rongyun Cao, Feng Wu & Xiaoping Chen

  2. University of Massachusetts Amherst, Amherst, MA, 01003-9264, USA

    Shlomo Zilberstein

Authors
  1. Haochong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rongyun Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shlomo Zilberstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Feng Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaoping Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoping Chen .

Editor information

Editors and Affiliations

  1. School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China

    YongAn Huang

  2. School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China

    Hao Wu

  3. Institute of Industrial Research, University of Portsmouth, Portsmouth, United Kingdom

    Honghai Liu

  4. School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China

    Zhouping Yin

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Zhang, H., Cao, R., Zilberstein, S., Wu, F., Chen, X. (2017). Toward Effective Soft Robot Control via Reinforcement Learning. In: Huang, Y., Wu, H., Liu, H., Yin, Z. (eds) Intelligent Robotics and Applications. ICIRA 2017. Lecture Notes in Computer Science(), vol 10462. Springer, Cham. https://doi.org/10.1007/978-3-319-65289-4_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-65289-4_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-65288-7

  • Online ISBN: 978-3-319-65289-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation