Abstract
The skill to find objects in a real world situation is important for mobile robots. Existing works of robotic vision-based object finding is based on the traditional training and classification paradigm, which means that a robot can only detect objects with the fixed and pre-trained classification labels. It is of great challenge for robots to find an untrained object, even if a complex description of the object has been given. In this paper, we proposed a vision-based object detection approach for robotic finding names Generative Search. It is inspired by the object detection model that when an unfamiliar object needs to be found through a complex description, human would “imagine” the object in his or her brain and then find the object which is mostly like the imagined object profile. By adopting a Generative Adversarial Network (GAN), our approach enables the robot to generate the object virtually according to the given description. Then, we use pre-trained deep neural networks to match the generated image with images in the robotic vision. At the implementation level, we adopt the cloud robotic architecture to promote the algorithm efficiency. The experiments on both open datasets and real robotic scenarios have proved the significant promotion of object finding accuracy when a robot searching an unfamiliar object with a complex description.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Szegedy, C., Toshev, A., Erhan, D.: Deep neural networks for object detection. In: Advances in Neural Information Processing Systems, pp. 2553–2561 (2013)
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software (2009)
Aydemir, A., Sjöö, K., Jensfelt, P.: Object search on a mobile robot using relational spatial information. In: Proceedings of International Conference on Intelligent Autonomous Systems, pp. 111–120 (2010)
Saigol, Z., Ridder, B., Wang, M., Dearden, R., Fox, M., Hawes, N., Lane, D.M., Long, D.: Efficient search for known objects in unknown environments using autonomous indoor robots. In: IROS Workshop on Task Planning for Intelligent Robots in Service and Manufacturing (2015)
Garvey, T.D.: Perceptual strategies for purposive vision (1976)
Torralba, A., Murphy, K.P., Freeman, W.T., Rubin, M.A., et al.: Context-based vision system for place and object recognition. In: ICCV, vol. 3, pp. 273–280 (2003)
Li, Y., Wang, H., Ding, B., Shi, P., Liu, X.: Toward QoS-aware cloud robotic applications: a hybrid architecture and its implementation. In: 2016 International IEEE Conferences Ubiquitous Intelligence & Computing, pp. 33–40. IEEE (2016)
López, D.G., Sjo, K., Paul, C., Jensfelt, P.: Hybrid laser and vision based object search and localization. In: IEEE International Conference on Robotics and Automation, ICRA 2008, pp. 2636–2643. IEEE (2008)
Kim, H.S., Jain, R., Volz, R.: Object recognition using multiple views. In: Proceedings 1985 IEEE International Conference on Robotics and Automation, vol. 2, pp. 28–33. IEEE (1985)
Zhang, H., Xu, T., Li, H., Zhang, S., Huang, X., Wang, X., Metaxas, D.: Stackgan: text to photo-realistic image synthesis with stacked generative adversarial networks. arXiv preprint arXiv:1612.03242 (2016)
Santana, E., Hotz, G.: Learning a driving simulator. arXiv preprint arXiv:1608.01230 (2016)
Ho, J., Ermon, S.: Generative adversarial imitation learning. In: Advances in Neural Information Processing Systems, pp. 4565–4573 (2016)
Lin, K., Yang, H.F., Hsiao, J.H., Chen, C.S.: Deep learning of binary hash codes for fast image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 27–35 (2015)
Uijlings, J.R., Van De Sande, K.E., Gevers, T., Smeulders, A.W.: Selective search for object recognition. Int. J. Comput. Vision 104(2), 154–171 (2013)
Hu, B., Wang, H., Zhang, P., Ding, B., Che, H.: Cloudroid: a cloud framework for transparent and QoS-aware robotic computation outsourcing. arXiv preprint arXiv:1705.05691 (2017)
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The caltech-ucsd birds-200-2011 dataset (2011)
Nilsback, M.E., Zisserman, A.: Automated flower classification over a large number of classes. In: Sixth Indian Conference on Computer Vision, Graphics & Image Processing, ICVGIP 2008, pp. 722–729. IEEE (2008)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)
Acknowledgments
This work is partially supported by the National Natural Science Foundation of China (nos. 91118008 and 61202117), the special program for the applied basic research of the National University of Defense Technology (no. ZDYYJCYJ20140601), and the Jiangsu Future Networks Innovation Institute Prospective Research Project on Future Networks (no. BY2013095-2-08).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Che, H., Hu, B., Ding, B., Wang, H. (2017). Enabling Imagination: Generative Adversarial Network-Based Object Finding in Robotic Tasks. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10639. Springer, Cham. https://doi.org/10.1007/978-3-319-70136-3_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-70136-3_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70135-6
Online ISBN: 978-3-319-70136-3
eBook Packages: Computer ScienceComputer Science (R0)