A Table Tennis Robot System Using an Industrial KUKA Robot Arm

  • Jonas TebbeEmail author
  • Yapeng GaoEmail author
  • Marc Sastre-RienietzEmail author
  • Andreas ZellEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11269)


In recent years robotic table tennis has become a popular research challenge for image processing and robot control. Here we present a novel table tennis robot system with high accuracy vision detection and fast robot reaction. Our system is based on an industrial KUKA Agilus R900 sixx robot with 6 DOF. Four cameras are used for ball position detection at 150 fps. We employ a multiple-camera calibration method, and use iterative triangulation to reconstruct the 3D ball position with an accuracy of 2.0 mm. In order to detect the flying ball with higher velocities in real-time, we combine color and background thresholding. For predicting the ball’s trajectory we test both a curve fitting approach and an extended Kalman filter. Our robot is able to play rallies with a human counting up to 50 consequential strokes and has a general hitting rate of 87%.


Table tennis robot Ball detection Trajectory prediction 



This work was supported in part by the Vector Stiftung and KUKA.


  1. 1.
    Acosta, L., Rodrigo, J.J., Mendez, J.A., Marichal, G.N., Sigut, M.: Ping-pong player prototype. IEEE Robot. Autom. Mag. 10(4), 44–52 (2003). Scholar
  2. 2.
    Agarwal, S., Mierle, K., et al.: Ceres solver.
  3. 3.
    Anderson, R.L.: A Robot Ping-Pong Player: Experiment in Real-time Intelligent Control. MIT Press, Cambridge (1988)Google Scholar
  4. 4.
    Arun, K.S., Huang, T.S., Blostein, S.D.: Least-squares fitting of two 3-D point sets. IEEE Trans. Pattern Anal. Mach. Intell. PAMI 9(5), 698–700 (1987). Scholar
  5. 5.
    Billingsley, J.: Robot ping pong, May 1983Google Scholar
  6. 6.
    Bradski, G.: The OpenCV Library. Dr. Dobb’s J. Softw. Tools (2000)Google Scholar
  7. 7.
    Fässler, H., Beyer, H.A., Wen, J.T.: Robot ping pong player: optimized mechanics, high performance 3D vision, and intelligent sensor control. Robotersysteme 6, 161–170 (1990)Google Scholar
  8. 8.
    Group, K.R.: The duel: timo boll vs. kuka robot, March 2014.
  9. 9.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004). ISBN 0521540518CrossRefGoogle Scholar
  10. 10.
    Hartley, R.I., Sturm, P.: Triangulation. Comput. Vis. Image Underst. 68(2), 146–157 (1997). Scholar
  11. 11.
    Hoffmann, U.: Mann gegen maschine - ulf hoffmann tischtennis roboter (uhttr-1), March 2014.
  12. 12.
    Kawakami, S., Ikumo, M., Oya, T.: Omron table tennis robot forpheus.
  13. 13.
    Lampert, C.H., Peters, J.: Real-time detection of colored objects in multiple camera streams with off-the-shelf hardware components. J. Real-Time Image Process. 7(1), 31–41 (2012). Scholar
  14. 14.
    Li, H., Wu, H., Lou, L., Kühnlenz, K., Ravn, O.: Ping-pong robotics with high-speed vision system. In: 2012 12th International Conference on Control Automation Robotics Vision (ICARCV), pp. 106–111 (2012).
  15. 15.
    Lourakis, M., Argyros, A.: SBA: a generic sparse bundle adjustment C/C++ package based on the Levenberg-Marquardt algorithm, January 2008Google Scholar
  16. 16.
    Miyazaki, F., Matsushima, M., Takeuchi, M.: Learning to dynamically manipulate: a table tennis robot controls a ball and rallies with a human being. In: Kawamura, S., Svinin, M. (eds.) Advances in Robot Control, pp. 317–341. Springer, Heidelberg (2006). Scholar
  17. 17.
    Müling, K., Kober, J., Peters, J.: A biomimetic approach to robot table tennis. Adapt. Behav. 19(5), 359–376 (2011). Scholar
  18. 18.
    Mülling, K., Kober, J., Peters, J.: Simulating human table tennis with a biomimetic robot setup. In: Doncieux, S., Girard, B., Guillot, A., Hallam, J., Meyer, J.-A., Mouret, J.-B. (eds.) SAB 2010. LNCS (LNAI), vol. 6226, pp. 273–282. Springer, Heidelberg (2010). Scholar
  19. 19.
    Mülling, K., Kober, J., Kroemer, O., Peters, J.: Learning to select and generalize striking movements in robot table tennis. Int. J. Robot. Res. 32(3), 263–279 (2013). Scholar
  20. 20.
    Nakashima, A., Ito, D., Hayakawa, Y.: An online trajectory planning of struck ball with spin by table tennis robot. In: 2014 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp. 865–870, July 2014.
  21. 21.
    Nakashima, A., Ogawa, Y., Kobayashi, Y., Hayakawa, Y.: Modeling of rebound phenomenon of a rigid ball with friction and elastic effects. In: Proceedings of the 2010 American Control Conference, pp. 1410–1415, June 2010.
  22. 22.
    Nakashima, A., Nonomura, J., Liu, C., Hayakawa, Y.: Hitting back-spin balls by robotic table tennis system based on physical models of ball motion. IFAC Proc. Vol. 45(22), 834–841 (2012). 10th IFAC Symposium on Robot Control. Scholar
  23. 23.
    Quigley, M., et al.: ROS: an open-source robot operating system. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) Workshop on Open Source Robotics, Kobe, Japan, May 2009Google Scholar
  24. 24.
    SIASUN: Siasun table tennis robot pongbot.
  25. 25.
    Silva, R., Melo, F.S., Veloso, M.: Towards table tennis with a quadrotor autonomous learning robot and onboard vision. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 649–655, September 2015.
  26. 26.
    Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment—a modern synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) IWVA 1999. LNCS, vol. 1883, pp. 298–372. Springer, Heidelberg (2000). Scholar
  27. 27.
    Wang, Q., Zhang, K., Wang, D.: The trajectory prediction and analysis of spinning ball for a table tennis robot application. In: The 4th Annual IEEE International Conference on Cyber Technology in Automation, Control and Intelligent, pp. 496–501, June 2014.
  28. 28.
    Xiong, R., Sun, Y., Zhu, Q., Wu, J., Chu, J.: Impedance control and its effects on a humanoid robot playing table tennis. Int. J. Adv. Robot. Syst. 9(5), 178 (2012). Scholar
  29. 29.
    Zhang, H., Wu, Y., Yang, F.: Ball detection based on color information and hough transform. In: 2009 International Conference on Artificial Intelligence and Computational Intelligence, vol. 2, pp. 393–397 (2009).
  30. 30.
    Zhang, Y., Xiong, R., Zhao, Y., Wang, J.: Real-time spin estimation of ping-pong ball using its natural brand. IEEE Trans. Instrum. Measure. 64(8), 2280–2290 (2015). Scholar
  31. 31.
    Zhang, Y., Zhao, Y., Xiong, R., Wang, Y., Wang, J., Chu, J.: Spin observation and trajectory prediction of a ping-pong ball. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 4108–4114, May 2014.
  32. 32.
    Zhang, Y., Xiong, R., Zhao, Y., Chu, J.: An adaptive trajectory prediction method for ping-pong robots. In: Su, C.-Y., Rakheja, S., Liu, H. (eds.) ICIRA 2012. LNCS (LNAI), vol. 7508, pp. 448–459. Springer, Heidelberg (2012). Scholar
  33. 33.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000). Scholar
  34. 34.
    Zhang, Z., Xu, D., Tan, M.: Visual measurement and prediction of ball trajectory for table tennis robot. IEEE Trans. Instrum. Measure. 59(12), 3195–3205 (2010). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Cognitive SystemsEberhard Karls UniversityTübingenGermany

Personalised recommendations