Abstract
Visual and tactile measurements offer complementary properties that make them particularly suitable for fusion. It is helpful for the robust and accurate recognition of objects, which is a necessity in many automation systems. In this chapter, a visual–tactile fusion framework is developed for object recognition tasks. This work uses the multivariate time series model to represent the tactile sequence, and the covariance descriptor to characterize the image. Further, a joint group kernel sparse coding method is designed to tackle the intrinsically weak-pairing problem in visual–tactile data samples. Finally, a visual–tactile dataset is developed, which is composed of 18 household objects for validation. The experimental results show that considering both visual and tactile input is beneficial and the proposed method indeed provides an effective strategy for fusion.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Mizone and Pocari are just two trademarks.
References
Allen, P.K.: Integrating vision and touch for object recognition tasks. Int. J. Robot. Res. 7(6), 15–33 (1988)
Beauchamp, M.S.: See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex. Curr. Opin. Neurobiol. 15(2), 145–153 (2005)
Drimus, A., Kootstra, G., Bilberg, A., Kragic, D.: Design of a flexible tactile sensor for classification of rigid and deformable objects. Robot. Auton. Syst. 62(1), 3–15 (2014)
Ernst, M.O., Banks, M.S.: Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870), 429–433 (2002)
Gao, S., Tsang, I.W., Chia, L.T.: Sparse representation with kernels. IEEE Trans. Image Process. 22(2), 423–434 (2013)
Heller, M.A.: Visual and tactual texture perception: intersensory cooperation. Atten. Percept. Psychophys. 31(4), 339–344 (1982)
Kroemer, O., Lampert, C.H., Peters, J.: Learning dynamic tactile sensing with robust vision-based training. IEEE Trans. Robot. 27(3), 545–557 (2011)
Lacey, S., Campbell, C., Sathian, K.: Vision and touch: multiple or multisensory representations of objects? Perception 36(10), 1513–1521 (2007)
Lederman, S.J., Klatzky, R.L.: Multisensory texture perception. Handb. Multisens. Process., 107–122 (2004)
Liu, H., Qin, J., Cheng, H., Sun, F.: Robust kernel dictionary learning using a whole sequence convergent algorithm. IJCAI 1(2), 5 (2015)
Liu, H., Yu, Y., Sun, F., Gu, J.: Visual-tactile fusion for object recognition. IEEE Trans. Autom. Sci. Eng. 14(2), 996–1008 (2017)
Natale, L., Metta, G., Sandini, G.: Learning haptic representation of objects. In: International Conference on Intelligent Manipulation and Grasping (2004)
Nene, S.A., Nayar, S.K., Murase, H., et al.: Columbia Object Image Library (coil-20) (1996)
Norman, J.F., Norman, H.F., Clayton, A.M., Lianekhammy, J., Zielke, G.: The visual and haptic perception of natural object shape. Atten. Percept. Psychophys. 66(2), 342–351 (2004)
Shekhar, S., Patel, V.M., Nasrabadi, N.M., Chellappa, R.: Joint sparse representation for robust multimodal biometrics recognition. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 113–126 (2014)
Shrivastava, A., Patel, V.M., Chellappa, R.: Multiple kernel learning for sparse representation-based classification. IEEE Trans. Image Process. 23(7), 3013–3024 (2014)
Sprechmann, P., Ramirez, I., Sapiro, G., Eldar, Y.C.: C-hilasso: a collaborative hierarchical sparse modeling framework. IEEE Trans. Signal Process. 59(9), 4183–4198 (2011)
Wang, D., Zhang, Y., Zhou, W., Zhao, H., Chen, Z.: Collocation accuracy of visuo-haptic system: metrics and calibration. IEEE Trans. Haptics 4(4), 321–326 (2011)
Woods, A.T., Newell, F.N.: Visual, haptic and cross-modal recognition of objects and scenes. J. Physiol. Paris 98(1), 147–159 (2004)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Liu, H., Sun, F. (2018). Visual–Tactile Fusion Object Recognition Using Joint Sparse Coding. In: Robotic Tactile Perception and Understanding. Springer, Singapore. https://doi.org/10.1007/978-981-10-6171-4_7
Download citation
DOI: https://doi.org/10.1007/978-981-10-6171-4_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-6170-7
Online ISBN: 978-981-10-6171-4
eBook Packages: Computer ScienceComputer Science (R0)