Abstract
For robots, tactile perception is a key function utilized to obtain information from environment. Unlike vision sensors, tactile sensors can directly measure various physical properties of objects and the environment. Similarly, humans also use touch sensory receptors as an important approach to perceive and interact with the environment. In this chapter, a detailed discussion associated with tactile object recognition is presented. Current studies on tactile object recognition are divided into three sub-categories, and detailed analyses are provided. In addition, some advanced topics such as visual–tactile fusion, exploratory procedure, and datasets are discussed.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
This image is adopted from the website http://robohub.org/team-rbo-from-berlin-wins-amazon-picking-challenge-convincingly/.
- 3.
- 4.
- 5.
- 6.
References
Abderrahmane, Z., Ganesh, G., Cherubini, A., Crosnier, A.: Zero-shot object recognition based on haptic attributes (2017)
Aggarwal, A., Kampmann, P., Lemburg, J., Kirchner, F.: Haptic object recognition in underwater and deep-sea environments. J. Field Robot. 32(1), 167–185 (2015)
Allen, P.K., Roberts, K.S.: Haptic object recognition using a multi-fingered dextrous hand. In: Robotics and Automation, 1989. Proceedings., 1989 IEEE International Conference on, pp. 342–347 (1989)
Alt, N., Steinbach, E.: Navigation and manipulation planning using a visuo-haptic sensor on a mobile platform. IEEE Trans. Instrum. Meas. 63(11), 2570–2582 (2014)
Argall, B.D., Billard, A.G.: A survey of tactile human-robot interactions. Robot. Auton. Syst. 58(10), 1159–1176 (2010)
Asif, U., Bennamoun, M., Sohel, F.A.: Rgb-d object recognition and grasp detection using hierarchical cascaded forests. IEEE Trans, Robot (2017)
Baishya, S.S., Bäuml, B.: Robust material classification with a tactile skin using deep learning. In: Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, pp. 8–15 (2016)
Bekiroglu, Y., Detry, R., Kragic, D.: Learning tactile characterizations of object-and pose-specific grasps. In: Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on, pp. 1554–1560 (2011)
Bekiroglu, Y., Laaksonen, J., Jorgensen, J.A., Kyrki, V., Kragic, D.: Assessing grasp stability based on learning and haptic data. IEEE Trans. Robot. 27(3), 616–629 (2011)
Bhattacharjee, T., Shenoi, A.A., Park, D., Rehg, J.M., Kemp, C.C.: Combining tactile sensing and vision for rapid haptic mapping. In: Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, pp. 1200–1207 (2015)
Bjorkman, M., Bekiroglu, Y., Hogman, V., Kragic, D.: Enhancing visual perception of shape through tactile glances. In: Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on, pp. 3180–3186 (2013)
Bohg, J., Morales, A., Asfour, T., Kragic, D.: Data-driven grasp synthesisła survey. IEEE Trans. Robot. 30(2), 289–309 (2014)
Boshra, M., Zhang, H.: Localizing a polyhedral object in a robot hand by integrating visual and tactile data. Pattern Recognit. 33(3), 483–501 (2000)
Burka, A., Hu, S., Helgeson, S., Krishnan, S., Gao, Y., Hendricks, L.A., Darrell, T., Kuchenbecker, K.J.: Proton: A visuo-haptic data acquisition system for robotic learning of surface properties. In: Multisensor Fusion and Integration for Intelligent Systems (MFI), 2016 IEEE International Conference on, pp. 58–65 (2016)
Burka, A., Kuchenbecker, K.J.: How much haptic surface data is enough? (2017)
Cao, L., Kotagiri, R., Sun, F., Li, H., Huang, W., Aye, Z.M.M.: Efficient spatio-temporal tactile object recognition with randomized tiling convolutional networks in a hierarchical fusion strategy. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp. 3337–3345. AAAI Press (2016)
Chathuranga, D.S., Wang, Z., Noh, Y., Nanayakkara, T., Hirai, S.: Robust real time material classification algorithm using soft three axis tactile sensor: evaluation of the algorithm. In: Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, pp. 2093–2098 (2015)
Chebotar, Y., Hausman, K., Su, Z., Molchanov, A., Kroemer, O., Sukhatme, G., Schaal, S.: Bigs: Biotac grasp stability dataset. In: ICRA 2016 Workshop on Grasping and Manipulation Datasets (2016)
Chebotar, Y., Hausman, K., Su, Z., Sukhatme, G.S., Schaal, S.: Self-supervised regrasping using spatio-temporal tactile features and reinforcement learning. In: Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, pp. 1960–1966. IEEE (2016)
Cheng, Y., Su, C., Jia, Y., Xi, N.: Data correlation approach for slippage detection in robotic manipulations using tactile sensor array. In: Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, pp. 2717–2722 (2015)
Chitta, S., Sturm, J., Piccoli, M., Burgard, W.: Tactile sensing for mobile manipulation. IEEE Trans. Robot. 27(3), 558–568 (2011)
Chu, V., McMahon, I., Riano, L., McDonald, C.G., He, Q., Perez-Tejada, J.M., Arrigo, M., Darrell, T., Kuchenbecker, K.J.: Robotic learning of haptic adjectives through physical interaction. Robot. Auton. Syst. 63, 279–292 (2015)
Cockburn, D., Roberge, J.P., Le, T.H.L., Maslyczyk, A., Duchaine, V.: Grasp stability assessment through unsupervised feature learning of tactile images. In: Robotics and Automation (ICRA), 2017 IEEE International Conference on. IEEE (2017)
Corradi, T., Hall, P., Iravani, P.: Bayesian tactile object recognition: learning and recognising objects using a new inexpensive tactile sensor. In: Robotics and Automation (ICRA), 2015 IEEE International Conference on, pp. 3909–3914 (2015)
Culbertson, H., Lopez Delgado, J.J., Kuchenbecker, K.J.: The penn haptic texture toolkit for modeling, rendering, and evaluating haptic virtual textures (2014)
Dahiya, R.S., Metta, G., Cannata, G., Valle, M.: Guest editorial special issue on robotic sense of touch. IEEE Trans. Robot. 27(3), 385–388 (2011)
Dahiya, R.S., Metta, G., Valle, M., Sandini, G.: Tactile sensingłfrom humans to humanoids. IEEE Trans. Robot. 26(1), 1–20 (2010)
Dahiya, R.S., Mittendorfer, P., Valle, M., Cheng, G., Lumelsky, V.J.: Directions toward effective utilization of tactile skin: a review. IEEE Sens. J. 13(11), 4121–4138 (2013)
Dallaire, P., Giguère, P., Émond, D., Chaib-Draa, B.: Autonomous tactile perception: a combined improved sensing and bayesian nonparametric approach. Robot. Auton. Syst. 62(4), 422–435 (2014)
Dang, H.: Stable and semantic robotic grasping using tactile feedback (2013)
Decherchi, S., Gastaldo, P., Dahiya, R.S., Valle, M., Zunino, R.: Tactile-data classification of contact materials using computational intelligence. IEEE Trans. Robot. 27(3), 635–639 (2011)
Denei, S., Maiolino, P., Baglini, E., Cannata, G.: On the development of a tactile sensor for fabric manipulation and classification for industrial applications. In: Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, pp. 5081–5086 (2015)
Drimus, A., Kootstra, G., Bilberg, A., Kragic, D.: Design of a flexible tactile sensor for classification of rigid and deformable objects. Robot. Auton. Syst. 62(1), 3–15 (2014)
van Erp, J.B., van Veen, H.A.: Touch down: the effect of artificial touch cues on orientation in microgravity. Neurosci. Lett. 404(1), 78–82 (2006)
Falco, P., Lu, S., Cirillo, A., Natale, C., Pirozzi, S., Lee, D.: Cross-modal visuo-tactile object recognition using robotic active exploration
Gao, Y., Hendricks, L.A., Kuchenbecker, K.J., Darrell, T.: Deep learning for tactile understanding from visual and haptic data. In: Robotics and Automation (ICRA), 2016 IEEE International Conference on, pp. 536–543 (2016)
Gemici, M.C., Saxena, A.: Learning haptic representation for manipulating deformable food objects. In: Intelligent Robots and Systems (IROS 2014), 2014 IEEE/RSJ International Conference on, pp. 638–645 (2014)
Goldfeder, C., Ciocarlie, M., Dang, H., Allen, P.K.: The columbia grasp database. In: Robotics and Automation, 2009. ICRA’09. IEEE International Conference on, pp. 1710–1716 (2009)
Güler, P., Bekiroglu, Y., Gratal, X., Pauwels, K., Kragic, D.: What’s in the container? classifying object contents from vision and touch. In: Intelligent Robots and Systems (IROS 2014), 2014 IEEE/RSJ International Conference on, pp. 3961–3968 (2014)
Henshaw, C.G.: Touch sensing for space robotics. Aerospace Conference, 2012 IEEE, pp. 1–13 (2012)
Heyneman, B., Cutkosky, M.R.: Biologically inspired tactile classification of object-hand and object-world interactions. In: Robotics and Biomimetics (ROBIO), 2012 IEEE International Conference on, pp. 167–173 (2012)
Hollis, B., Patterson, S., Trinkle, J.: Compressed sensing for scalable robotic tactile skins (2017). arXiv:1705.05247
van Hoof, H., Chen, N., Karl, M., van der Smagt, P., Peters, J.: Stable reinforcement learning with autoencoders for tactile and visual data. In: Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, pp. 3928–3934. IEEE (2016)
Hsiao, K., Kaelbling, L., Lozano-Pérez, T.: Task-driven tactile exploration (2010)
Hughes, D., Krauthammer, A., Correll, N.: Recognizing social touch gestures using recurrent and convolutional neural networks. In: Robotics and Automation (ICRA), 2017 IEEE International Conference on. IEEE (2017)
Ilonen, J., Bohg, J., Kyrki, V.: Fusing visual and tactile sensing for 3-d object reconstruction while grasping. In: Robotics and Automation (ICRA), 2013 IEEE International Conference on, pp. 3547–3554 (2013)
Izatt, G., Mirano, G., Adelson, E., Tedrake, R.: Tracking objects with point clouds from vision and touch
Jamali, N., Sammut, C.: Majority voting: Material classification by tactile sensing using surface texture. IEEE Trans. Robot. 27(3), 508–521 (2011)
Jin, M., Gu, H., Fan, S., Zhang, Y., Liu, H.: Object shape recognition approach for sparse point clouds from tactile exploration. In: Robotics and Biomimetics (ROBIO), 2013 IEEE International Conference on, pp. 558–562 (2013)
Johansson, R.S., Flanagan, J.R.: Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat. Rev. Neurosci. 10(5), 345–359 (2009)
Kaboli, M., Walker, R., Cheng, G.: Re-using prior tactile experience by robotic hands to discriminate in-hand objects via texture properties. In: Robotics and Automation (ICRA), 2016 IEEE International Conference on, pp. 2242–2247. IEEE (2016)
Kappassov, Z., Corrales, J.A., Perdereau, V.: Tactile sensing in dexterous robot hands. Robot. Auton. Syst. 74, 195–220 (2015)
Kim, J., Forsythe, S.: Adoption of sensory enabling technology for online apparel shopping. Eur. J. Mark. 43(9/10), 1101–1120 (2009)
Kroemer, O., Lampert, C.H., Peters, J.: Learning dynamic tactile sensing with robust vision-based training. IEEE Trans. Robot. 27(3), 545–557 (2011)
Krug, R., Lilienthal, A.J., Kragic, D., Bekiroglu, Y.: Analytic grasp success prediction with tactile feedback. In: Robotics and Automation (ICRA), 2016 IEEE International Conference on, pp. 165–171. IEEE (2016)
Lai, K., Bo, L., Ren, X., Fox, D.: A large-scale hierarchical multi-view rgb-d object dataset. In: Robotics and Automation (ICRA), 2011 IEEE International Conference on, pp. 1817–1824 (2011)
Lederman, S.J., Klatzky, R.L.: Hand movements: a window into haptic object recognition. Cogn. Psychol. 19(3), 342–368 (1987)
Lee, M.H., Nicholls, H.R.: Review article tactile sensing for mechatronicsła state of the art survey. Mechatronics 9(1), 1–31 (1999)
Lester, D.H., Forman, A.M., Loyd, D.: Internet shopping and buying behavior of college students. Serv. Mark. Q. 27(2), 123–138 (2006)
Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., Quillen, D.: Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. In: The International Journal of Robotics Research, p. 0278364917710318 (2016)
Li, R., Adelson, E.H.: Sensing and recognizing surface textures using a gelsight sensor. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1241–1247 (2013)
Liarokapis, M.V., Calli, B., Spiers, A.J., Dollar, A.M.: Unplanned, model-free, single grasp object classification with underactuated hands and force sensors. In: Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, pp. 5073–5080 (2015)
Liu, H., Song, X., Nanayakkara, T., Seneviratne, L.D., Althoefer, K.: A computationally fast algorithm for local contact shape and pose classification using a tactile array sensor. In: Robotics and Automation (ICRA), 2012 IEEE International Conference on, pp. 1410–1415 (2012)
Liu, H., Yu, Y., Sun, F., Gu, J.: Visual-tactile fusion for object recognition. IEEE Trans. Autom. Sci. Eng. 14(2), 996–1008 (2017)
Luo, S., Mou, W., Althoefer, K., Liu, H.: Localizing the object contact through matching tactile features with visual map. In: Robotics and Automation (ICRA), 2015 IEEE International Conference on, pp. 3903–3908 (2015)
Luo, S., Mou, W., Althoefer, K., Liu, H.: Novel tactile-sift descriptor for object shape recognition. IEEE Sens. J. 15(9), 5001–5009 (2015)
Ma, R., Liu, H., Sun, F., Yang, Q., Gao, M.: Linear dynamic system method for tactile object classification. Sci. China Inf. Sci. 57(12), 1–11 (2014)
Madry, M., Bo, L., Kragic, D., Fox, D.: St-hmp: unsupervised spatio-temporal feature learning for tactile data. In: Robotics and Automation (ICRA), 2014 IEEE International Conference on, pp. 2262–2269 (2014)
Maldonado, A., Alvarez, H., Beetz, M.: Improving robot manipulation through fingertip perception. In: Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pp. 2947–2954 (2012)
Miller, A.T., Allen, P.K.: Graspit! a versatile simulator for robotic grasping. IEEE Robot. Autom. Mag. 11(4), 110–122 (2004)
Nagatani, T., Noda, A., Hirai, S., et al.: What can be inferred from a tactile arrayed sensor in autonomous in-hand manipulation? In: Automation Science and Engineering (CASE), 2012 IEEE International Conference on, pp. 461–468 (2012)
Navarro, S.E., Gorges, N., Wörn, H., Schill, J., Asfour, T., Dillmann, R.: Haptic object recognition for multi-fingered robot hands. In: Haptics Symposium (HAPTICS), 2012 IEEE, pp. 497–502 (2012)
Petrovskaya, A., Khatib, O.: Global localization of objects via touch. IEEE Trans. Rob. 27(3), 569–585 (2011)
Pezzementi, Z., Plaku, E., Reyda, C., Hager, G.D.: Tactile-object recognition from appearance information. IEEE Trans. Rob. 27(3), 473–487 (2011)
Pinto, L., Gupta, A.: Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours. In: Robotics and Automation (ICRA), 2016 IEEE International Conference on, pp. 3406–3413. IEEE (2016)
Prats, M., Sanz, P.J., Del Pobil, A.P.: Vision-tactile-force integration and robot physical interaction. In: Robotics and Automation, 2009. ICRA’09. IEEE International Conference on, pp. 3975–3980 (2009)
Roberge, J.P., Rispal, S., Wong, T., Duchaine, V.: Unsupervised feature learning for classifying dynamic tactile events using sparse coding. In: Robotics and Automation (ICRA), 2016 IEEE International Conference on, pp. 2675–2681. IEEE (2016)
Romano, J.M., Hsiao, K., Niemeyer, G., Chitta, S., Kuchenbecker, K.J.: Human-inspired robotic grasp control with tactile sensing. IEEE Trans. Robot. 27(6), 1067–1079 (2011)
Romano, J.M., Kuchenbecker, K.J.: Methods for robotic tool-mediated haptic surface recognition. In: Haptics Symposium (HAPTICS), 2014 IEEE, pp. 49–56 (2014)
Russell, R.A., Wijaya, J.A.: Object location and recognition using whisker sensors. In: Australasian Conference on Robotics and Automation, pp. 761–768 (2003)
Schmitz, A., Bansho, Y., Noda, K., Iwata, H., Ogata, T., Sugano, S.: Tactile object recognition using deep learning and dropout. In: Humanoid Robots (Humanoids), 2014 14th IEEE-RAS International Conference on, pp. 1044–1050 (2014)
Schneider, A., Sturm, J., Stachniss, C., Reisert, M., Burkhardt, H., Burgard, W.: Object identification with tactile sensors using bag-of-features. In: Intelligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ International Conference on, pp. 243–248 (2009)
Sinapov, J., Sukhoy, V., Sahai, R., Stoytchev, A.: Vibrotactile recognition and categorization of surfaces by a humanoid robot. IEEE Trans. Robot. 27(3), 488–497 (2011)
Soh, H., Demiris, Y.: Incrementally learning objects by touch: Online discriminative and generative models for tactile-based recognition. IEEE Trans. Haptics 7(4), 512–525 (2014)
Son, J.S., Howe, R., Wang, J., Hager, G.D.: Preliminary results on grasping with vision and touch. In: Intelligent Robots and Systems’96, IROS96, Proceedings of the 1996 IEEE/RSJ International Conference on, vol. 3, pp. 1068–1075 (1996)
Song, D., Ek, C.H., Huebner, K., Kragic, D.: Task-based robot grasp planning using probabilistic inference. IEEE Trans. Robot. 31(3), 546–561 (2015)
Spiers, A.J., Liarokapis, M.V., Calli, B., Dollar, A.M.: Single-grasp object classification and feature extraction with simple robot hands and tactile sensors. IEEE Trans. Haptics 9(2), 207–220 (2016)
Strese, M., Lee, J.Y., Schuwerk, C., Han, Q., Kim, H.G., Steinbach, E.: A haptic texture database for tool-mediated texture recognition and classification. In: Haptic, Audio and Visual Environments and Games (HAVE), 2014 IEEE International Symposium on, pp. 118–123 (2014)
Strese, M., Schuwerk, C., Iepure, A., Steinbach, E.: Multimodal feature-based surface material classification. IEEE Trans, Haptics (2017)
Sun, F., Liu, C., Huang, W., Zhang, J.: Object classification and grasp planning using visual and tactile sensing. IEEE Trans. Syst. Man Cybern. Syst. 46(7), 969–979 (2016)
Taira, R., Saga, S., Okatani, T., Deguchi, K.: 3d reconstruction of reflective surface on reflection type tactile sensor using constraints of geometrical optics. In: SICE Annual Conference 2010, Proceedings of, pp. 3144–3149 (2010)
Wan, Q., Adams, R.P., Howe, R.D.: Variability and predictability in tactile sensing during grasping. In: Robotics and Automation (ICRA), 2016 IEEE International Conference on, pp. 158–164. IEEE (2016)
Westling, G., Johansson, R.: Factors influencing the force control during precision grip. Exp. Brain Res. 53(2), 277–284 (1984)
Woods, A.T., Newell, F.N.: Visual, haptic and cross-modal recognition of objects and scenes. J. Physiol. Paris 98(1), 147–159 (2004)
Xu, D., Loeb, G.E., Fishel, J.A.: Tactile identification of objects using bayesian exploration. In: Robotics and Automation (ICRA), 2013 IEEE International Conference on, pp. 3056–3061 (2013)
Yuan, W., Wang, S., Dong, S., Adelson, E.: Connecting look and feel: associating the visual and tactile properties of physical materials (2017). arXiv:1704.03822
Zhang, M.M., Atanasov, N., Daniilidis, K.: Active tactile object recognition by monte carlo tree search (2017). arXiv:1703.00095
Zhang, M.M., Detry, R., Larry, M., Kostas, D.: Tactile-vision integration for task-compatible fine-part manipulation. In: Robotics: Science and Systems (RSS) Workshop on Revisiting Contact C Turning a Problem into a Solution (2017)
Zhang, M.M., Kennedy, M.D., Hsieh, M.A., Daniilidis, K.: A triangle histogram for object classification by tactile sensing. In: Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, pp. 4931–4938. IEEE (2016)
Zhang, Y.F., Liu, H.: Tactile sensor based varying contact point manipulation strategy for dexterous robot hand manipulating unknown objects. In: Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pp. 4756–4761 (2012)
Zheng, H., Fang, L., Ji, M., Strese, M., Özer, Y., Steinbach, E.: Deep learning for surface material classification using haptic and visual information. IEEE Trans. Multimed. 18(12), 2407–2416 (2016)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Liu, H., Sun, F. (2018). Introduction. In: Robotic Tactile Perception and Understanding. Springer, Singapore. https://doi.org/10.1007/978-981-10-6171-4_1
Download citation
DOI: https://doi.org/10.1007/978-981-10-6171-4_1
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-6170-7
Online ISBN: 978-981-10-6171-4
eBook Packages: Computer ScienceComputer Science (R0)