Abstract
The paper describes the control system of an autonomous mobile service robot using a human-machine dialogue to control the robot in a way natural for human with the help of syntactic structures that define the sequence of certain actions. The interaction with the robot by means of natural language in the formation of target designations is provided by the multimodal representation of the working area in the form of a hybrid map of the surrounding space with highlited work areas and locations of key objects complemented by a set of semantic relations that characterize features of the robot operation and impact of the environmental changes on its operation. The article discusses the control system of the mobile service robo, performing transport and logistics operations in a public place in direct contact with lots of people around. The proposed approach to the dialog control of the robot based on the multimodal representation of the robot operation area not only simplifies the process of determining the sequence of operations, but also enhances the level of safety for other people, objects and the robot itself in the process of fulfilling its service tasks.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Lim, G.H., Suh, I.H., Suh, H.: Ontology-based unified robot knowledge for service robots in indoor environments. Trans. IEEE Syst. Man Cybern. Part A Syst. Hum. 41(3), 492–509 (2011)
Heriberto, C., Nina, D., Kai-Florian, R., Thora, T., John, B.: A dialogue system for indoor wayfinding using text-based natural language. Int. J. Comput. Linguist. Appl. 1(1–2), 285–304 (2010)
Skubic, M., Perzanowski, D., Schultz, A., Adams, W.: Using spatial language in a human-robot dialog. In: IEEE International Conference on Robotics and Automation, vol. 4, pp. 4143–4148 (2002)
Zhonin, A., Kargin, D., Sergiyevskiy, N., Kharlamov, A.: Model mira robota na osnove sensoriki blokov navigatsii, tekhnicheskogo zreniya i rechevogo dialoga. Trudy mezhdunarodnoy molodezhnoy konferentsii Informatsionnyye sistemy i tekhnologii, pp. 68–69 (2012)
Ermishin, K., Vorotnikov, S.: Intellektualnaya sistema upravleniya servisnym mobilnym robotom. Politekhnika-servis, Ekstremalnaya robototekhnika. Trudy vserossiyskoy nauchno-tekhnicheskoy konferentsii, Sankt-Peterburg (2012)
Ermishin, K., Vorotnikov, S.: Multiagentnaya sensornaya sistema servisnogo mobilnogo robota. Vestnik MGTU im. N. E. Baumana. Ser. Priborostroyeniye 6, 50–59 (2012)
Kharlamov, A.: Neyrosetevaya tekhnologiya predstavleniya i obrabotki informatsii (estestvennoye predstavleniye znaniy). Radiotekhnika (2006)
Acknowledgement
The research was performed under the project Study of the mechanism of associative links in human verbal and cogitative activity using the method of neural network modeling in the analysis of textual information (financially supported by the Russian Foundation for Basic Research, grant N 14-06-00363).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Kharlamov, A., Ermishin, K. (2016). Voice Dialogue with a Collaborative Robot Driven by Multimodal Semantics. In: Ronzhin, A., Rigoll, G., Meshcheryakov, R. (eds) Interactive Collaborative Robotics. ICR 2016. Lecture Notes in Computer Science(), vol 9812. Springer, Cham. https://doi.org/10.1007/978-3-319-43955-6_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-43955-6_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-43954-9
Online ISBN: 978-3-319-43955-6
eBook Packages: Computer ScienceComputer Science (R0)