Advertisement

A Table Tennis Robot System Using an Industrial KUKA Robot Arm

  • Jonas TebbeEmail author
  • Yapeng GaoEmail author
  • Marc Sastre-RienietzEmail author
  • Andreas ZellEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11269)

Abstract

In recent years robotic table tennis has become a popular research challenge for image processing and robot control. Here we present a novel table tennis robot system with high accuracy vision detection and fast robot reaction. Our system is based on an industrial KUKA Agilus R900 sixx robot with 6 DOF. Four cameras are used for ball position detection at 150 fps. We employ a multiple-camera calibration method, and use iterative triangulation to reconstruct the 3D ball position with an accuracy of 2.0 mm. In order to detect the flying ball with higher velocities in real-time, we combine color and background thresholding. For predicting the ball’s trajectory we test both a curve fitting approach and an extended Kalman filter. Our robot is able to play rallies with a human counting up to 50 consequential strokes and has a general hitting rate of 87%.

Keywords

Table tennis robot Ball detection Trajectory prediction 

Notes

Acknowledgement

This work was supported in part by the Vector Stiftung and KUKA.

References

  1. 1.
    Acosta, L., Rodrigo, J.J., Mendez, J.A., Marichal, G.N., Sigut, M.: Ping-pong player prototype. IEEE Robot. Autom. Mag. 10(4), 44–52 (2003).  https://doi.org/10.1109/MRA.2003.1256297CrossRefGoogle Scholar
  2. 2.
    Agarwal, S., Mierle, K., et al.: Ceres solver. http://ceres-solver.org
  3. 3.
    Anderson, R.L.: A Robot Ping-Pong Player: Experiment in Real-time Intelligent Control. MIT Press, Cambridge (1988)Google Scholar
  4. 4.
    Arun, K.S., Huang, T.S., Blostein, S.D.: Least-squares fitting of two 3-D point sets. IEEE Trans. Pattern Anal. Mach. Intell. PAMI 9(5), 698–700 (1987).  https://doi.org/10.1109/TPAMI.1987.4767965CrossRefGoogle Scholar
  5. 5.
    Billingsley, J.: Robot ping pong, May 1983Google Scholar
  6. 6.
    Bradski, G.: The OpenCV Library. Dr. Dobb’s J. Softw. Tools (2000)Google Scholar
  7. 7.
    Fässler, H., Beyer, H.A., Wen, J.T.: Robot ping pong player: optimized mechanics, high performance 3D vision, and intelligent sensor control. Robotersysteme 6, 161–170 (1990)Google Scholar
  8. 8.
    Group, K.R.: The duel: timo boll vs. kuka robot, March 2014. https://www.youtube.com/watch?v=tIIJME8-au8
  9. 9.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004). ISBN 0521540518CrossRefGoogle Scholar
  10. 10.
    Hartley, R.I., Sturm, P.: Triangulation. Comput. Vis. Image Underst. 68(2), 146–157 (1997).  https://doi.org/10.1006/cviu.1997.0547CrossRefGoogle Scholar
  11. 11.
    Hoffmann, U.: Mann gegen maschine - ulf hoffmann tischtennis roboter (uhttr-1), March 2014. https://www.youtube.com/watch?v=imVNg9j7rvU
  12. 12.
    Kawakami, S., Ikumo, M., Oya, T.: Omron table tennis robot forpheus. https://www.omron.com/innovation/forpheus.html
  13. 13.
    Lampert, C.H., Peters, J.: Real-time detection of colored objects in multiple camera streams with off-the-shelf hardware components. J. Real-Time Image Process. 7(1), 31–41 (2012).  https://doi.org/10.1007/s11554-010-0168-3CrossRefGoogle Scholar
  14. 14.
    Li, H., Wu, H., Lou, L., Kühnlenz, K., Ravn, O.: Ping-pong robotics with high-speed vision system. In: 2012 12th International Conference on Control Automation Robotics Vision (ICARCV), pp. 106–111 (2012).  https://doi.org/10.1109/ICARCV.2012.6485142
  15. 15.
    Lourakis, M., Argyros, A.: SBA: a generic sparse bundle adjustment C/C++ package based on the Levenberg-Marquardt algorithm, January 2008Google Scholar
  16. 16.
    Miyazaki, F., Matsushima, M., Takeuchi, M.: Learning to dynamically manipulate: a table tennis robot controls a ball and rallies with a human being. In: Kawamura, S., Svinin, M. (eds.) Advances in Robot Control, pp. 317–341. Springer, Heidelberg (2006).  https://doi.org/10.1007/978-3-540-37347-6_15CrossRefzbMATHGoogle Scholar
  17. 17.
    Müling, K., Kober, J., Peters, J.: A biomimetic approach to robot table tennis. Adapt. Behav. 19(5), 359–376 (2011).  https://doi.org/10.1177/1059712311419378CrossRefGoogle Scholar
  18. 18.
    Mülling, K., Kober, J., Peters, J.: Simulating human table tennis with a biomimetic robot setup. In: Doncieux, S., Girard, B., Guillot, A., Hallam, J., Meyer, J.-A., Mouret, J.-B. (eds.) SAB 2010. LNCS (LNAI), vol. 6226, pp. 273–282. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-15193-4_26CrossRefGoogle Scholar
  19. 19.
    Mülling, K., Kober, J., Kroemer, O., Peters, J.: Learning to select and generalize striking movements in robot table tennis. Int. J. Robot. Res. 32(3), 263–279 (2013).  https://doi.org/10.1177/0278364912472380CrossRefGoogle Scholar
  20. 20.
    Nakashima, A., Ito, D., Hayakawa, Y.: An online trajectory planning of struck ball with spin by table tennis robot. In: 2014 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp. 865–870, July 2014.  https://doi.org/10.1109/AIM.2014.6878188
  21. 21.
    Nakashima, A., Ogawa, Y., Kobayashi, Y., Hayakawa, Y.: Modeling of rebound phenomenon of a rigid ball with friction and elastic effects. In: Proceedings of the 2010 American Control Conference, pp. 1410–1415, June 2010.  https://doi.org/10.1109/ACC.2010.5530520
  22. 22.
    Nakashima, A., Nonomura, J., Liu, C., Hayakawa, Y.: Hitting back-spin balls by robotic table tennis system based on physical models of ball motion. IFAC Proc. Vol. 45(22), 834–841 (2012).  https://doi.org/10.3182/20120905-3-HR-2030.00107. 10th IFAC Symposium on Robot Control. http://www.sciencedirect.com/science/article/pii/S1474667016337132CrossRefGoogle Scholar
  23. 23.
    Quigley, M., et al.: ROS: an open-source robot operating system. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) Workshop on Open Source Robotics, Kobe, Japan, May 2009Google Scholar
  24. 24.
    SIASUN: Siasun table tennis robot pongbot. https://youtu.be/Ov8jwAKucmk
  25. 25.
    Silva, R., Melo, F.S., Veloso, M.: Towards table tennis with a quadrotor autonomous learning robot and onboard vision. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 649–655, September 2015.  https://doi.org/10.1109/IROS.2015.7353441
  26. 26.
    Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment—a modern synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) IWVA 1999. LNCS, vol. 1883, pp. 298–372. Springer, Heidelberg (2000).  https://doi.org/10.1007/3-540-44480-7_21CrossRefGoogle Scholar
  27. 27.
    Wang, Q., Zhang, K., Wang, D.: The trajectory prediction and analysis of spinning ball for a table tennis robot application. In: The 4th Annual IEEE International Conference on Cyber Technology in Automation, Control and Intelligent, pp. 496–501, June 2014.  https://doi.org/10.1109/CYBER.2014.6917514
  28. 28.
    Xiong, R., Sun, Y., Zhu, Q., Wu, J., Chu, J.: Impedance control and its effects on a humanoid robot playing table tennis. Int. J. Adv. Robot. Syst. 9(5), 178 (2012).  https://doi.org/10.5772/51924CrossRefGoogle Scholar
  29. 29.
    Zhang, H., Wu, Y., Yang, F.: Ball detection based on color information and hough transform. In: 2009 International Conference on Artificial Intelligence and Computational Intelligence, vol. 2, pp. 393–397 (2009).  https://doi.org/10.1109/AICI.2009.21
  30. 30.
    Zhang, Y., Xiong, R., Zhao, Y., Wang, J.: Real-time spin estimation of ping-pong ball using its natural brand. IEEE Trans. Instrum. Measure. 64(8), 2280–2290 (2015).  https://doi.org/10.1109/TIM.2014.2385173CrossRefGoogle Scholar
  31. 31.
    Zhang, Y., Zhao, Y., Xiong, R., Wang, Y., Wang, J., Chu, J.: Spin observation and trajectory prediction of a ping-pong ball. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 4108–4114, May 2014.  https://doi.org/10.1109/ICRA.2014.6907456
  32. 32.
    Zhang, Y., Xiong, R., Zhao, Y., Chu, J.: An adaptive trajectory prediction method for ping-pong robots. In: Su, C.-Y., Rakheja, S., Liu, H. (eds.) ICIRA 2012. LNCS (LNAI), vol. 7508, pp. 448–459. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33503-7_44CrossRefGoogle Scholar
  33. 33.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000).  https://doi.org/10.1109/34.888718CrossRefGoogle Scholar
  34. 34.
    Zhang, Z., Xu, D., Tan, M.: Visual measurement and prediction of ball trajectory for table tennis robot. IEEE Trans. Instrum. Measure. 59(12), 3195–3205 (2010).  https://doi.org/10.1109/TIM.2010.2047128CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Cognitive SystemsEberhard Karls UniversityTübingenGermany

Personalised recommendations