Configuration Lattices for Planar Contact Manipulation Under Uncertainty

  • Michael KovalEmail author
  • David Hsu
  • Nancy Pollard
  • Siddhartha S. Srinivasa
Part of the Springer Proceedings in Advanced Robotics book series (SPAR, volume 13)


This work addresses the challenge of a robot using real-time feedback from contact sensors to reliably manipulate a movable object on a cluttered tabletop. We formulate this task as a partially observable Markov decision process (POMDP) in the joint space of robot configurations and object poses. This formulation enables the robot to explicitly reason about uncertainty and all major types of kinematic constraints: reachability, joint limits, and collision. We solve the POMDP using DESPOT, a state-of-the-art online POMDP solver, by leveraging two key ideas for computational efficiency. First, we lazily construct a discrete lattice in the robot’s configuration space. Second, we guide the search with heuristics derived from an unconstrained relaxation of the problem. We empirically show that our approach outperforms several baselines on a simulated seven degree-of-freedom manipulator.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dynamic Animation and Robotics Toolkit. (2013)
  2. 2.
    Bai, H., Hsu, D., Lee, W., Ngo, V.: Monte Carlo value iteration for continuousstate POMDPs. In: WAFR (2011)Google Scholar
  3. 3.
    Bohlin, R., Kavraki, L.: Path planning using lazy PRM. In: IEEE ICRA. pp. 521–528 (2000)Google Scholar
  4. 4.
    Boots, B., Byravan, A., Fox, D.: Learning predictive models of a depth camera & manipulator from raw execution traces. In: IEEE ICRA (2014)Google Scholar
  5. 5.
    Brokowski, M., Peshkin, M., Goldberg, K.: Curved fences for part alignment. In: IEEE ICRA (1993)Google Scholar
  6. 6.
    Calli, B., Singh, A., Walsman, A., Srinivasa, S., Abbeel, P., Dollar, A.: The YCB object and model set: Towards common benchmarks for manipulation research. In: ICAR (2015)Google Scholar
  7. 7.
    Catto, E.: Box2D. (2010)
  8. 8.
    Cohen, B., Chitta, S., Likhachev, M.: Single-and dual-arm motion planning with heuristic search. IJRR (2013)Google Scholar
  9. 9.
    Dogar, M., Hsiao, K., Ciocarlie, M., Srinivasa, S.: Physics-based grasp planning through clutter. In: R:SS (2012)Google Scholar
  10. 10.
    Dogar, M., Srinivasa, S.: Push-grasping with dexterous hands: Mechanics and a method. In: IEEE/RSJ IROS (2010)Google Scholar
  11. 11.
    Dogar, M., Srinivasa, S.: A planning framework for non-prehensile manipulation under clutter and uncertainty. AuRo 33(3), 217–236 (2012)Google Scholar
  12. 12.
    Duff, D., Wyatt, J., Stolkin, R.: Motion estimation using physical simulation. In: IEEE ICRA (2010)Google Scholar
  13. 13.
    Emery-Montemerlo, R., Gordon, G., Schneider, J., Thrun, S.: Approximate solutions for partially observable stochastic games with common payoffs. In: AAMAS (2004)Google Scholar
  14. 14.
    Erdmann, M., Mason, M.: An exploration of sensorless manipulation. IEEE T-RA (1988)Google Scholar
  15. 15.
    Hauser, K.: Lazy collision checking in asymptotically-optimal motion planning. In: IEEE ICRA (2015)Google Scholar
  16. 16.
    Hebert, P., Howard, T., Hudson, N., Ma, J., Burdick, J.: The next best touch for model-based localization. In: IEEE ICRA (2013)Google Scholar
  17. 17.
    Horowitz, M., Burdick, J.: Interactive non-prehensile manipulation for grasping via POMDPs. In: IEEE ICRA (2013)Google Scholar
  18. 18.
    Hsiao, K.: Relatively robust grasping. Ph.D. thesis, MIT (2009)Google Scholar
  19. 19.
    Hsiao, K., Lozano-Pérez, T., Kaelbling, L.: Robust belief-based execution of manipulation programs. In: WAFR (2008)Google Scholar
  20. 20.
    Javdani, S., Bagnell, J., Srinivasa, S.: Shared autonomy via hindsight optimization. In: R:SS (2015)Google Scholar
  21. 21.
    Javdani, S., Klingensmith, M., Bagnell, J., Pollard, N., Srinivasa, S.: Efficient touch based localization through submodularity. In: IEEE ICRA (2013)Google Scholar
  22. 22.
    Klingensmith, M., Galluzzo, T., Dellin, C., Kazemi, M., Bagnell, J., Pollard, N.: Closed-loop servoing using real-time markerless arm tracking. In: IEEE ICRA Humanoids Workshop (2013)Google Scholar
  23. 23.
    Koval, M., King, J., Pollard, N., Srinivasa, S.: Robust trajectory selection for rearrangement planning as a multi-armed bandit problem. In: IEEE/RSJ IROS (2015)Google Scholar
  24. 24.
    Koval, M., Pollard, N., Srinivasa, S.: Pose estimation for planar contact manipulation with manifold particle filters. IJRR 34(7), 922–945 (2015)Google Scholar
  25. 25.
    Koval, M., Pollard, N., Srinivasa, S.: Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. IJRR (2015), in pressGoogle Scholar
  26. 26.
    Kurniawati, H., Hsu, D., Lee, W.: SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. In: R:SS (2008)Google Scholar
  27. 27.
    LaValle, S., Hutchinson, S.: An objective-based framework for motion planning under sensing and control uncertainties. IJRR (1998)Google Scholar
  28. 28.
    Li, Q., Schürmann, C., Haschke, R., Ritter, H.: A control framework for tactile servoing. In: R:SS (2013)Google Scholar
  29. 29.
    Likhachev, M., Ferguson, D.: Planning long dynamically feasible maneuvers for autonomous vehicles. IJRR 28(8), 933–945 (2009)Google Scholar
  30. 30.
    Lim, Z., Hsu, D., Sun, L.: Monte Carlo value iteration with macro-actions. In: NIPS (2011)Google Scholar
  31. 31.
    Littman, M., Cassandra, A., Kaelbling, L.: Learning policies for partially observable environments: Scaling up. ICML (1995)Google Scholar
  32. 32.
    Lynch, K., Maekawa, H., Tanie, K.: Manipulation and active sensing by pushing using tactile feedback. In: IEEE/RSJ IROS (1992)Google Scholar
  33. 33.
    Ng, A., Jordan, M.: PEGASUS: A policy search method for large MDPs and POMDPs. In: UAI (2000)Google Scholar
  34. 34.
    Pastor, P., Righetti, L., Kalakrishnan, M., Schaal, S.: Online movement adaptation based on previous sensor experiences. In: IEEE/RSJ IROS (2011)Google Scholar
  35. 35.
    Petrovskaya, A., Khatib, O.: Global localization of objects via touch. IEEE T-RO 27(3), 569–585 (2011)Google Scholar
  36. 36.
    Pivtoraiko, M., Kelly, A.: Efficient constrained path planning via search in state lattices. In: i-SAIRAS (2005)Google Scholar
  37. 37.
    Platt, R., Fagg, A., Grupen, R.: Nullspace grasp control: theory and experiments. IEEE T-RO 26(2), 282–295 (2010)Google Scholar
  38. 38.
    Ross, S., Pineau, J., Paquet, S., Chaib-Draa, B.: Online planning algorithms for POMDPs. JAIR (2008)Google Scholar
  39. 39.
    Salisbury, K., Townsend, W., Eberman, B., DiPietro, D.: Preliminary design of a whole-arm manipulation system (WAMS). In: IEEE ICRA (1988)Google Scholar
  40. 40.
    Seiler, K., Kurniawati, H., Singh, S.: GPS-ABT: An online and approximate solver for POMDPs with continuous action space. In: IEEE ICRA (2015)Google Scholar
  41. 41.
    Silver, D., Veness, J.: Monte-Carlo planning in large POMDPs. In: NIPS (2010)Google Scholar
  42. 42.
    Smallwood, R., Sondik, E.: The optimal control of partially observable Markov processes over a finite horizon. Operations Research 21(5), 1071–1088 (1973)Google Scholar
  43. 43.
    Somani, A., Ye, N., Hsu, D., Lee, W.: DESPOT: Online POMDP planning with regularization. In: NIPS (2013)Google Scholar
  44. 44.
    Srinivasa, S., Berenson, D., Cakmak, M., Collet, A., Dogar, M., Dragan, A., Knepper, R., Niemueller, T., Strabala, K., VandeWeghe,M.: HERB 2.0: Lessons learned from developing a mobile manipulator for the home. Proc. IEEE 100(8), 1–19 (2012)Google Scholar
  45. 45.
    Stulp, F., Theodorou, E., Buchli, J., Schaal, S.: Learning to grasp under uncertainty. In: IEEE ICRA. pp. 5703–5708 (2011)Google Scholar
  46. 46.
    Townsend, W.: The BarrettHand grasper-programmably flexible part handling and assembly. Industrial Robot: An International Journal 27(3), 181–188 (2000)Google Scholar
  47. 47.
    Zhang, H., Chen, N.: Control of contact via tactile sensing. IEEE T-RA 16(5), 482–495 (2000)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Michael Koval
    • 1
    Email author
  • David Hsu
    • 2
  • Nancy Pollard
    • 1
  • Siddhartha S. Srinivasa
    • 1
  1. 1.The Robotics InstituteCarnegie Mellon UniversityPittsburghUnited States
  2. 2.Department of Computer ScienceNational University of Singapore SingaporeSingapore

Personalised recommendations