Motion Planning of the Cooperative Robot with Visual Markers

  • Michał Spławski
  • Rafał Staszak
  • Filip Jarecki
  • Jakub Chudziński
  • Piotr Kaczmarek
  • Paweł Drapikowski
  • Dominik BelterEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1140)


In recent years numerous robotic solutions have been developed to improve production processes. However, most of the robotic arms are applied in large factories where they perform repetitive tasks supporting mass production. The complexity of programming and adapting robots for changing production is still a barrier for the application of robot arms in flexible and small-scale production. In this paper, we present the system which allows programming the robot using visual cameras and a set of visual markers. We develop a method for flexible motion planning for the cooperative robot to quickly define the trajectory of the robot in the 3D space. We present the architecture of the system, calibration method and the performance of the system. We also show how to significantly improve the detection range of the visual markers using Convolutional Neural Networks for object detection.


Motion planning Cooperative robot Visual programming 



This work was supported by the European Regional Development Fund (ERDF) Wielkopolska Regional Operational Programme for 2014–2020, Measure 1.2.


  1. 1.
    Andersen, R.E., Hansen, E.B., Cerny, D., Madsen, S., Pulendralingam, B., Bøgh, S., Chrysostomou, D.: Integration of a skill-based collaborative mobile robot in a smart cyber-physical environment. Proc. Manuf. 11, 114–123 (2017)Google Scholar
  2. 2.
    Andersen, R.S., Damgaard, J.S., Madsen, O., Moeslund, T.B.: Fast calibration of industrial mobile robots to workstations using QR codes. In: IEEE International Symposium on Robotics, pp. 1–6 (2013)Google Scholar
  3. 3.
    Babinec, A., Jurišica, L., Hubinský, P., Duchoň, F.: Visual localization of mobile robot using artificial markers. Proc. Eng. 96(6), 1–9 (2014)CrossRefGoogle Scholar
  4. 4.
    Cieślak, W., Rodykow, S., Belter, D.: Teleoperation of a six-legged walking robot using a hand tracking interface. In: Silva, M.E., et al. (eds.) Human-Centric Robotics, pp. 527–536. World Scientific, Singapore (2017)Google Scholar
  5. 5.
    Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F., Medina-Carnicer, R.: Generation of fiducial marker dictionaries using mixed integer linear programming. Pattern Recogn. 51, 481–491 (2016)CrossRefGoogle Scholar
  6. 6.
    Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F., Marín-Jiménez, M.: Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn. 47(6), 2280–2292 (2014)CrossRefGoogle Scholar
  7. 7.
    George, L., Mazel, A.: Humanoid robot indoor navigation based on 2D bar codes: application to the NAO robot. In: IEEE-RAS International Conference on Humanoid Robots (Humanoids), pp. 329–335 (2013)Google Scholar
  8. 8.
    He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. In: IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)Google Scholar
  9. 9.
    Hornung, A., Wurm, K.M., Bennewitz, M., Stachniss, C., Burgard, W.: OctoMap: an efficient probabilistic 3d mapping framework based on octrees. Auton. Robots 34(3), 189–206 (2013)CrossRefGoogle Scholar
  10. 10.
    Krupke, D., Starke, S., Einig, L., Zhang, J., Steinicke, F.: Prototyping of immersive HRI scenarios. In: Silva , M. E., et al. (eds.) Human-Centric Robotics. World Scientific, Singapore (2017)Google Scholar
  11. 11.
    Li, Y., Zhu, S., Yu, Y., Wang, Z.: An improved graph-based visual localization system for indoor mobile robot using newly designed markers. Int. J. Adv. Robot. Syst. 15(2), 1–15 (2018)Google Scholar
  12. 12.
    Mersch, D.P., Crespi, A., Keller, L.: Tracking individuals shows spatial fidelity is a key regulator of ant social organization. Science 340(6136), 1090–1093 (2013)CrossRefGoogle Scholar
  13. 13.
    Nissler, C., Büttner, S., Marton, Z.-C., Beckmann, L., Thomas, U.: Evaluation and improvement of global pose estimation with multiple AprilTags for industrial manipulators. In: IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), pp. 1–8 (2016)Google Scholar
  14. 14.
    Olson, E.: Apriltag: a robust and flexible visual fiducial system. In: IEEE International Conference on Robotics and Automation, pp. 3400–3407 (2011)Google Scholar
  15. 15.
    Ouaviani, E., Pavan, A., Bottazzi, M., Brunelli, E., Caselli, F., Guerrero, M.: A common image processing framework for 2d barcode reading. In: 7th International Conference on Image Processing and its Applications, Conference Publishing, vol. 2, no. 465, pp. 652–655 (1999)Google Scholar
  16. 16.
    Rostkowska, M., Skrzypczyński, P.: A practical application of QR-codes for mobile robot localization in home environment. In: Silva, M.E., et al. (eds.) Human-Centric Robotics, pp. 311–318. World Scientific, Singapore (2017)Google Scholar
  17. 17.
    Silaghi, H., Rohde, U., Spoial, V., Silaghi, A., Gergely, E., Nagy, Z.: Voice command of an industrial robot in a noisy environment. In: International Symposium on Fundamentals of Electrical Engineering (ISFEE), pp. 1–5 (2014)Google Scholar
  18. 18.
    Pires, J.N.: Experiments on commanding an industrial robot using the human voice. Ind. Robot: Int. J. 32(6), 5–11 (2005)MathSciNetGoogle Scholar
  19. 19.
    Wang, J., Olson, E.: AprilTag 2: efficient and robust fiducial detection. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4193–4198 (2016)Google Scholar
  20. 20.
    Wilson, F., Cueva, C., Hugo, S., Torres, M., Kern, M.J.: 7 DOF industrial robot controlled by hand gestures using microsoft kinect v2. In: IEEE 3rd Colombian Conference on Automatic Control, pp. 1–6 (2017)Google Scholar
  21. 21.
    Xavier, R.S., da Silva, B.M.F., Gonzalves, L.M.G.: Accuracy analysis of augmented reality markers for visual mapping and localization. In: Workshop of Computer Vision, pp. 73–77 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Michał Spławski
    • 2
  • Rafał Staszak
    • 1
  • Filip Jarecki
    • 2
  • Jakub Chudziński
    • 2
  • Piotr Kaczmarek
    • 1
    • 2
  • Paweł Drapikowski
    • 1
    • 2
  • Dominik Belter
    • 1
    • 2
    Email author
  1. 1.Institute of Control, Robotics and Information EngineeringPoznan University of TechnologyPoznanPoland
  2. 2.A.S. Adrian SternPoznanPoland

Personalised recommendations