Abstract
In this paper, we address the problem of real-time human pose-based robust person tracking for a person following shopping support robot. We achieve this by cropping the target person’s body from the image and then apply a color histogram matching algorithm for tracking a unique person. After tracking the person, we used an omnidirectional camera and ultrasonic sensor to find the target person’s location and distance from the robot. When the target person is fixed in front of shopping shelves the robot finds the fixed distance between the robot and target person. In this situation our robot finds the target person’s body movement orientation using our proposed methodology. According to the body orientation our robot assumes a suitable position so that the target person can easily put his shopping product in the basket. Our proposed system was verified in real time environments and it shows that our robot system is highly effective at following a given target person and provides proper support while shopping the target person.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Moeslund, T.B., Hilton, A., Krüger, V., Sigal, L.: Visual Analysis of Humans. Springer, London (2011). https://doi.org/10.1007/978-0-85729-997-0
Ahmad, M.F., Alhady, S.S.N., Kaharuddin, S., Othman, W.A.F.W.: Visual based sensor cart follower for wheelchair by using microcontroller. In: ICCSCE (2015)
Rawashdeh, N.A., Haddad, R.M., Jadallah, O.A., To’ma, A.E.: A person-following robotic cart controlled via a smartphone application: design and evaluation. In: REM (2017)
Ilias, B., Nagarajan, R., Murugappan, M., Helmy, K., Awang Omar, A.S., Abdul Rahman, M.A.: Hospital nurse following robot: hardware development and sensor integration. Int. J. Med. Eng. Inf. 6(1), 1–13 (2014)
Ng, Y.L., Lim, C.S., Danapalasingam, K.A., Tan, M.L.P., Tan, C.W.: Automatic human guided shopping trolley with smart shopping system. Jurnal Teknologi 73(3) (2015)
Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: CVPR (2017)
Montemerlo, M., Thrun, S., Whittaker, W.: Conditional particle filters for simultaneous mobile robot localization and people-tracking. In: Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292) (2002)
Grimson, W.E.L., Stauffer, C., Romano, R., Lee, L.: Using adaptive tracking to classify and monitor activities in a site. In: Proceedings of the 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No. 98CB36231) (1998)
Beymer, D.J.: Real-time tracking of multiple people using stereo. In: FRAME-RATE: Frame-rate Applications, Methods and Experiences with Regularly Available Technology and Equipment (1999)
Kleinehagenbrock, M., Lang, S., Fritsch, J., Lomker, F., Fink, G.A., Sagerer, G.: Person tracking with a mobile robot based on multi-modal anchoring. In: Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication (2002)
Kobilarov, M., Sukhatme, G., Hyams, J., Batavia, P.: People tracking and following with mobile robot using an omnidirectional camera and a laser. In: International Conference on Robotics and Automation (2006)
Bianco, R., Caretti, M., Nolfi, S.: Developing a robot able to follow a human target in a domestic environment. In: Proceedings of the First RoboCare Workshop (2003)
Lu, W., Tan, Y.P.: A color histogram-based people tracking system. In: International Symposium on Circuits and Systems (ISCAS) (2001)
Mason, M., Duric, Z.: Using histograms to detect and track objects in color video. In: Applied Imagery Pattern Recognition Workshop (AIPR) (2001)
Wojtaszek, D., Laganiere, R.: Using color histograms to recognize people in real time visual surveillance. In: International Conference on Multimedia, Internet and Video Technologies (2002)
Bajramovic, F., Deutsch, B., Grabl, C., Denzler, J.: Efficient adaptive combination of histograms for real-time tracking. EURASIP J. Image Video Process. 2008 (2008). Article ID 528297
Fang, Y., Wang, H., Mao, S., Wu, X.: Multi-object tracking based on region corresponding and improved color-histogram matching. In: International Symposium on Signal Processing and Information Technology (2007)
Swain, M.J., Ballard, D.H.: Color indexing. Int. J. Comput. Vis. 7(1), 11–32 (1991)
Acknowledgement
This work was supported by JSPS KAKENHI Grant Number JP26240038.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Islam, M.M., Lam, A., Fukuda, H., Kobayashi, Y., Kuno, Y. (2019). A Person-Following Shopping Support Robot Based on Human Pose Skeleton Data and LiDAR Sensor. In: Huang, DS., Huang, ZK., Hussain, A. (eds) Intelligent Computing Methodologies. ICIC 2019. Lecture Notes in Computer Science(), vol 11645. Springer, Cham. https://doi.org/10.1007/978-3-030-26766-7_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-26766-7_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-26765-0
Online ISBN: 978-3-030-26766-7
eBook Packages: Computer ScienceComputer Science (R0)