Abstract
Tracking the pose of objects is a relevant topic in computer vision, which potentially allows to recover meaningful information for other applications such as task supervision, robot manipulation or activity recognition. In the last years, RGB-D cameras have been widely adopted for this problem with impressive results. However, there are certain objects whose surface properties or complex shapes prevents the depth sensor from returning good depth measurements, and only color-based methods can be applied. In this work, we show how the depth information of the surroundings of the object can still be useful in the object pose tracking with RGB-D even in this situation. Specifically, we propose using the depth information to handle occlusions in a state of the art region-based object pose tracking algorithm. Experiments with recordings of humans naturally interacting with difficult objects have been performed, showing the advantages of our contribution in several image sequences.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Asif, U., Bennamoun, M., Sohel, F.: Real-time pose estimation of rigid objects using RGB-D imagery. In: IEEE Conference on Industrial Electronics and Applications, pp. 1692–1699 (2013)
Azad, P., Asfour, T., Dillmann, R.: Combining appearance-based and model-based methods for real-time object recognition and 6D localization. In: IEEE International Conference on Intelligent Robots and Systems, pp. 5339–5344 (2006)
Azad, P., Munch, D., Asfour, T., Dillmann, R.: 6-DoF model-based tracking of arbitrarily shaped 3D objects. In: IEEE International Conference on Robotics and Automation, pp. 5204–5209 (2011)
Bibby, C., Reid, I.: Robust real-time visual tracking using pixel-wise posteriors. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5303, pp. 831–844. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88688-4_61
Seo, B.-K., Park, H., Park, J.-I., Hinterstoisser, S., Ilic, S.: Optimal local searching for fast and robust textureless 3d object tracking in highly cluttered backgrounds. IEEE Trans. Visual. Comput. Graph. 20(1), 99–110 (2014)
Choi, C., Christensen, H.I.: Robust 3D visual tracking using particle filtering on the SE(3) group. In: IEEE International Conference on Robotics and Automation, vol. 31, pp. 4384–4390 (2011)
Choi, C., Christensen, H.I.: 3D textureless object detection and tracking: an edge-based approach. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3877–3884 (2012)
Dambreville, S., Sandhu, R., Yezzi, A., Tannenbaum, A.: Robust 3D pose estimation and efficient 2D region-based segmentation from a 3D shape prior. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5303, pp. 169–182. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88688-4_13
Drummond, T., Cipolla, R.: Real-time visual tracking of complex structures. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 932–946 (2002)
Grundmann, T., Eidenberger, R., Schneider, M., Fiegert, M., Wichert, G.: Robust high precision 6D pose determination in complex environments for robotic manipulation. In: Workshop Best Practice in 3D Perception and Modeling for Mobile Manipulation at the IEEE International Conference of Robotics and Automation (2010)
Grundmann, T., Fiegert, M., Burgard, W.: Probabilistic rule set joint state update as approximation to the full joint state estimation applied to multi object scene analysis. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2047–2052 (2010)
Hinterstoisser, S., Holzer, S., Cagniart, C., Ilic, S., Konolige, K., Navab, N., Lepetit, V.: Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes. In: IEEE International Conference on Computer Vision, pp. 858–865 (2011)
Imperoli, M., Pretto, A.: D\(^{2}\)CO: fast and robust registration of 3D textureless objects using the directional chamfer distance. In: Nalpantidis, L., Krüger, V., Eklundh, J.-O., Gasteratos, A. (eds.) ICVS 2015. LNCS, vol. 9163, pp. 316–328. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20904-3_29
Kim, K., Lepetit, V., Woo, W.: Keyframe-based modeling and tracking of multiple 3D objects. In: IEEE International Symposium on Mixed and Augmented Reality, pp. 193–198. IEEE (2010)
Klank, U., Carton, D., Beetz, M.: Transparent object detection and reconstruction on a mobile platform. In: IEEE International Conference on Robotics and Automation, pp. 5971–5978 (2011)
Lepetit, V., Fua, P.: Monocular Model-Based 3D Tracking of Rigid Objects: A Survey. Now Publishers Inc., Hanover (2005)
Liu, M.Y., Tuzel, O., Veeraraghavan, A., Chellappa, R.: Fast directional chamfer matching. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1696–1703 (2010)
Lowrey, K., Kolev, S., Tassa, Y., Erez, T., Todorov, E.: Physically-consistent sensor fusion in contact-rich behaviors. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1656–1662 (2014)
Lysenkov, I., Eruhimov, V., Bradski, G.: Recognition and Pose Estimation of Rigid Transparent Objects with a Kinect Sensor. In: Robotics: Science and Systems (2012)
Phillips, C.J., Derpanis, K.G., Daniilidis, K.: A novel stereoscopic cue for figure-ground segregation of semi-transparent objects. In: IEEE International Conference on Computer Vision Workshops, vol. 1, pp. 1100–1107 (2011)
Prisacariu, V.A., Reid, I.D.: PWP3D: Real-time segmentation and tracking of 3D objects. Int. J. Comput. Vis. 98(3), 335–354 (2012)
Rosenhahn, B., Brox, T., Weickert, J.: Three-dimensional shape knowledge for joint image segmentation and pose tracking. Int. J. Comput. Vis. 73(3), 243–262 (2007)
Schmidt, T., Hertkorn, K., Newcombe, R., Marton, Z., Suppa, M., Fox, D.: Depth-based tracking with physical constraints for robot manipulation. In: IEEE International Conference on Robotics and Automation., pp. 119–126 (2015)
Schmidt, T., Newcombe, R., Fox, D.: DART: dense articulated real-time tracking with consumer depth cameras. Auton. Robot. 39(3), 239–258 (2015)
Schulman, J., Lee, A., Ho, J., Abbeel, P., Berkeley, U.C.: Tracking deformable objects with point clouds. In: IEEE International Conference on Robotics and Automation, pp. 1122–1129 (2013)
Ulrich, M., Wiedemann, C., Steger, C.: CAD-based recognition of 3D objects in monocular images. In: IEEE International Conference on Robotics and Automation, pp. 1191–1198 (2009)
Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv. (CSUR) 38(4) (2006)
Acknowledgments
This work was supported by Projects DPI2014-61792-EXP and DPI2015-65962-R (MINECO/FEDER, UE) and grant BES-2013-065834 (MINECO).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Perez-Yus, A., Puig, L., Lopez-Nicolas, G., Guerrero, J.J., Fox, D. (2018). RGB-D Based Tracking of Complex Objects. In: Wannous, H., Pala, P., Daoudi, M., Flórez-Revuelta, F. (eds) Understanding Human Activities Through 3D Sensors. UHA3DS 2016. Lecture Notes in Computer Science(), vol 10188. Springer, Cham. https://doi.org/10.1007/978-3-319-91863-1_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-91863-1_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91862-4
Online ISBN: 978-3-319-91863-1
eBook Packages: Computer ScienceComputer Science (R0)