Advertisement

Cyber-Enhanced Rescue Canine

  • Kazunori OhnoEmail author
  • Ryunosuke Hamada
  • Tatsuya Hoshi
  • Hiroyuki Nishinoma
  • Shumpei Yamaguchi
  • Solvi Arnold
  • Kimitoshi Yamazaki
  • Takefumi Kikusui
  • Satoko Matsubara
  • Miho Nagasawa
  • Takatomi Kubo
  • Eri Nakahara
  • Yuki Maruno
  • Kazushi Ikeda
  • Toshitaka Yamakawa
  • Takeshi Tokuyama
  • Ayumi Shinohara
  • Ryo Yoshinaka
  • Diptarama Hendrian
  • Kaizaburo Chubachi
  • Satoshi Kobayashi
  • Katsuhito Nakashima
  • Hiroaki Naganuma
  • Ryu Wakimoto
  • Shu Ishikawa
  • Tatsuki Miura
  • Satoshi Tadokoro
Chapter
Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 128)

Abstract

This chapter introduces cyber-enhanced rescue canines that digitally strengthen the capability of search and rescue (SAR) dogs using robotics technology. A SAR dog wears a cyber-enhanced rescue canine (CRC) suit equipped with sensors (Camera, IMUs, and GNSS). The activities of the SAR dog and its surrounding view and sound are measured by the sensors mounted on the CRC suit. The sensor data are used to visualize the viewing scene of the SAR dog, its trajectory, its behavior (walk, run, bark, among others), and its internal state via cloud services (Amazon Web Services (AWS), Google Maps, and camera server). The trajectory can be plotted on an aerial photograph captured by flying robots or disaster response robots. The visualization results can be confirmed in real time via the cloud servers on the tablet terminal located in the command headquarters and with the handler. We developed various types of CRC suits that can measure the activities of large- and medium-size SAR dogs through non-invasive sensors on the CRC suits, and we visualized the activities from the sensor data. In addition, a practical CRC suit was developed with a company and evaluated using actual SAR dogs certified by the Japan Rescue Dog Association (JRDA). Through the ImPACT Tough Robotics Challenge, tough sensing technologies for CRC suits are developed to visualize the activities of SAR dogs. The primary contributions of our research include the following six topics. (1) Lightweight CRC suits were developed and evaluated. (2) Objects left by victims were automatically found using images from a camera mounted on the CRC suits. A deep neural network was used to find suitable features for searching for objects left by victims. (3) The emotions (positive as well as negative) of SAR dogs were estimated from their heart rate variation, which was measured by CRC inner suits. (4) The behaviors of SAR dogs were estimated from an IMU sensor mounted on the CRC suit. (5) The visual SLAM and inertial navigation systems for SAR dogs were developed to estimate trajectory in non-GNSS environments. These emotions, movements, and trajectories are used to visualize the search activities of the SAR dogs. (6) The dog was trained to search an area by controlling the dog with the laser light sources mounted on the CRC suit.

Notes

Acknowledgements

This work was supported by Impulsing Paradigm Change through Disruptive Technologies (ImPACT) Tough Robotics Challenge program of Japan Science and Technology (JST) Agency.

References

  1. 1.
    Akselrod, S., Gordon, D., Ubel, F.A., Shannon, D.C., Berger, A.C., Cohen, R.J.: Power spectrum analysis of heart rate fluctuation: a quantitative probe of beat-to-beat cardiovascular control. Science 213, 220–222 (1981)CrossRefGoogle Scholar
  2. 2.
    Appelhans, B.M., Luecken, L.J.: Heart rate variability as an index of regulated emotional responding. Rev. Gen. Psychol. 10, 229 (2006)CrossRefGoogle Scholar
  3. 3.
    Boissy, A., Manteuffel, G., Jensen, M.B., Moe, R.O., Spruijt, B., Keeling, L.J., Winckler, C., Forkman, B., Dimitr ov, I., Langbein, J.: Assessment of positive emotions in animals to improve their welfare. Physiol. Behav. 92, 375–397 (2007)Google Scholar
  4. 4.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001).  https://doi.org/10.1023/A:1010933404324CrossRefzbMATHGoogle Scholar
  5. 5.
    Browne, C., Stafford, K., Fordham, R.: The use of scent-detection dogs. Ir. Vet. J. 59, 97 (2006)Google Scholar
  6. 6.
    Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: The 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794 (2016).  https://doi.org/10.1145/2939672.2939785
  7. 7.
    Chiu, W., Arnold, J., Shih, Y., Hsiung, K., Chi, H., Chiu, C., Tsai, W., Huang, W.C.: A survey of international ur ban search-and-rescue teams following the Ji Ji earthquake. Disasters 26, 85–94 (2002)CrossRefGoogle Scholar
  8. 8.
    den Uijl, I., Álvarez, C.B., Bartram, D., Dror, Y., Holland, R., Cook, A.: External validation of a collar-mounted triaxial accelerometer for second-by-second monitoring of eight behavioural states in dogs. Plos One 12(11), e0188,481 (2017).  https://doi.org/10.1371/journal.pone.0188481CrossRefGoogle Scholar
  9. 9.
    Dreiseitl, S., Ohno-Machado, L.: Logistic regression and artificial neural network classification models: a methodology review. J. Biomed. Inform. 35(5–6), 352–359 (2002)CrossRefGoogle Scholar
  10. 10.
    Ekman, P., Levenson, R.W., Friesen, W.V.: Autonomic nervous system activity distinguishes among emotions. Science 221, 1208–1210 (1983)CrossRefGoogle Scholar
  11. 11.
    ELAN (version 5.2). https://tla.mpi.nl/tools/tla-tools/elan/. Max Planck Institute for Psycholinguistics, The Language Archive, Nijmegen, The Netherlands
  12. 12.
    Ferworn, A., Sadeghian, A., Barnum, K., Ostrom, D., Rahnama, H., Woungang, I.: Canine as robot in directed search. In: Proceedings of IEEE/SMC International Conference on System of Systems Engineering, Los Angeles, CA, USA (2006)Google Scholar
  13. 13.
    Ferworn, A., Sadeghian, A., Barnum, K., Rahnama, H., Pham, H., Erickson, C., Ostrom, D., L. Dell’Agnese: Urban: search and rescue with canine augmentation technology. In: Proceedings of IEEE/SMC International Conference on System of Systems Engineering, Los Angeles, CA, USA (2006)Google Scholar
  14. 14.
    Ferworn, A., Waismark, B., Scanlan, M.: CAT 360 – Canine augmented technology 360-degree video system. In: 2015 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) (2015)Google Scholar
  15. 15.
    Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th International Conference on Machine Learning, ICML, pp. 1126–1135 (2017)Google Scholar
  16. 16.
    Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 25, 1189–1232 (2001)Google Scholar
  17. 17.
    Fukushima, K.: Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern. 36(4), 193–202 (1980)CrossRefGoogle Scholar
  18. 18.
    Gerencsér, L., Vásárhelyi, G., Nagy, M., Vicsek, T., Miklósi, A.: Identification of behaviour in freely moving dogs (Canis familiaris) using inertial sensors. PLoS One 8(10), e77,814 (2013).  https://doi.org/10.1371/journal.pone.0077814CrossRefGoogle Scholar
  19. 19.
    Hamada, R., Ohno, K., Matsubara, S., Hoshi, T., Nagasawa, M., Kikusui, T., Kubo, T., Nakahara, E., Ikeda, K., Yamaguchi, S.: Real-time emotional state estimation system for Canines based on heart rate variability. In: CBS, pp. 298–303 (2017)Google Scholar
  20. 20.
    Hearst, M.A., Dumais, S.T., Osuna, E., Platt, J., Scholkopf, B.: Support vector machines. IEEE Intell. Syst. Appl. 13(4), 18–28 (1998)CrossRefGoogle Scholar
  21. 21.
    Inagaki, H., Kuwahara, M., Tsubone, H.: Changes in autonomic control of heart associated with classical appet itive conditioning in rats. Exp. Anim. 54, 61–69 (2005)CrossRefGoogle Scholar
  22. 22.
    Jonathan, M., Ueli, M., Dan C., J’urgen, S.: Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction, Artificial Neural Networks and Machine Learning— ICANN 2011. Lecture Notes in Computer Science (2011)Google Scholar
  23. 23.
  24. 24.
    Katayama, M., Kubo, T., Mogi, K., Ikeda, K., Nagasawa, M., Kikusui, T.: Heart rate variability predicts the emotional state in dogs. Behav. Proc. 128, 108–112 (2016)CrossRefGoogle Scholar
  25. 25.
    Komori, Y., Fujieda, T., Ohno, K., Suzuki, T., Tadokoro, S.: 1a1-u10 search and rescue dogs’ barking detection from audio and inertial sensor. In: The Proceedings of JSME Annual Conference on Robotics and Mechatronics (ROBOMECH), pp. 1A1- U10\(\_\)1–1A1- U10\(\_\)4. The Japan Society of Mechanical Engineers (2015).  https://doi.org/10.1299/jsmermd.2015._1A1-U10_1Google Scholar
  26. 26.
    Kreibig, S.D.: Autonomic nervous system activity in emotion: a review. Biol. Psychol. 84, 394–421 (2010)CrossRefGoogle Scholar
  27. 27.
    Kruijff, G.J.M., Kruijff-Korbayová, I., Keshavdas, S., Larochelle, B., Janíček, M., Colas, F., Liu, M., Pomerleau, F., Siegwart, R., Neerincx, M.A., Looije, R., Smets, N.J.J.M, Mioch, T., van Diggelen, J., Pirri, F., Gianni, M., Ferri, F., Menna, M., Worst, R., Linder, T., Tretyakov, V., Surmann, H., Svoboda, T., Reinštein, M., Zimmermann, K., Petříček, T., Hlaváč, V.: Designing, developing, and deploying systems to support human—robot teams in disaster response. Adv. Robot. Taylor & Francis 28(23), 1547–1570 (2014).  https://doi.org/10.1080/01691864.2014.985335CrossRefGoogle Scholar
  28. 28.
    Ladha, C., Belshaw, Z., J, O., Asher, L.: A step in the right direction: an open-design pedometer algorithm for dogs. Bmc. Vet. Res. 14(1), 107 (2018).  https://doi.org/10.1186/s12917-018-1422-3
  29. 29.
    Lane, R.D., McRae, K., Reiman, E.M., Chen, K., Ahern, G.L., Thayer, J.F.: Neural correlates of heart rate variab ility during emotion. Neuroimage 44, 213–222 (2009)CrossRefGoogle Scholar
  30. 30.
    LeCun, Y., Boser, B., Denker, J.S., Howard, R.E., Habbard, W., Jackel, L.D., Henderson, D.: Handwritten digit recognition with a back-propagation network. Adv. Neural Inf. Process. Syst. 2, 396–404 (1990)Google Scholar
  31. 31.
    LeDoux, J.: Rethinking the emotional brain. Neuron 73, 653–676 (2012)CrossRefGoogle Scholar
  32. 32.
    Michael, N., Shen, S., Mohta, K., Mulgaonkar, Y., Kumar, V., Nagatani, K., Okada, Y., Kiribayashi, S., Otake, K., Yoshida, K., Ohno, K., Takeuchi, E., Tadokoro, S.: Collaborative mapping of an earthquake-damaged building via ground and aerial robots. J. Field Robot 29(4), 832–841 (2012)CrossRefGoogle Scholar
  33. 33.
    Murphy, R.: Disaster Robotics. MIT Press, Cambridge (2014)CrossRefGoogle Scholar
  34. 34.
    Nagatani, K., Kiribayashi, S., Okada, Y., Otake, K., Yoshida, K., Tadokoro, S., Nishimura, T., Yoshida, T., Koyanagi, E., Fukushima, M., Kawatsuma, S.: Emergency response to the nuclear accident at the fukushima daiichi nuclear power plants using mobile rescue robots. J. Field Robot. 30(1), 44–63 (2013)CrossRefGoogle Scholar
  35. 35.
    Narisada, S., Mashiko, S., Shimizu, S., Ohori, Y., Sugawara, K., Sakuma, S., Sato, I., Ueki, Y., Hamada, R., Yamaguchi, S., Hoshi, T., Ohno, K., Yoshinaka, R., Shinohara, A., Tokuyama, T.: Behavior identification of search and rescue dogs based on inertial sensors. In: The Proceedings of JSME annual Conference on Robotics and Mechatronics (ROBOMECH). The Japan Society of Mechanical Engineers (2017).  https://doi.org/10.1299/jsmermd.2017.2A1-Q04CrossRefGoogle Scholar
  36. 36.
    Ohno, K., Yamaguchi, S., Nishinoma, H., Hoshi, T., Hamada, R., Matsubara, S., Nagasawa, M., Kikusui, T., Tadokor, S.: Control of Canine’s Moving Direction by Using On-suit Laser Beams, IEEE CBS (2018)Google Scholar
  37. 37.
    Reefmann, N., Wechsler, B., Gygax, L.: Behavioural and physiological assessment of positive and negative emot ion in sheep. Anim. Behav. 78, 651–659 (2009)CrossRefGoogle Scholar
  38. 38.
    Sakaguchi, N., Ohno, K., Takeuchi, E., Tadokoro, S.: Precise velocity estimation for dog using its gait. In: Proceedings of The 9th Conference on Field and Service Robotics (2013)Google Scholar
  39. 39.
    Slensky, K.A., Drobatz, K.J., Downend, A.B., Otto, C.M.: Deployment morbidity among search-and-rescue dogs use d after the September 11, 2001, terrorist attacks. J. Am. Vet. Med. Assoc. 225, 868–873 (2004)CrossRefGoogle Scholar
  40. 40.
    Tran, J., Ferworn, A., Ribeiro, C., Denko, M.: Enhancing canine disaster search. In: Proceedings of IEEE/SMC International Conference on System of Systems Engineering Monterey, CA, USA (2008)Google Scholar
  41. 41.
    Tsoumakas, G., Katakis, I., Vlahavas, I.P.: Mining multi-label data. In: Data Mining and Knowledge Discovery Handbook, 2nd ed., pp. 667–685 (2010).  https://doi.org/10.1007/978-0-387-09823-4_34CrossRefGoogle Scholar
  42. 42.
    Wagner, J., Kim, J., André E.: From physiological signals to emotions: implementing and comparing selected methods for feature extraction and classification. In: IEEE/ICME, pp. 940–943 (2005)Google Scholar
  43. 43.
    Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., Sloetjes, H.: ELAN: a professional framework for multimodality research. In: Proceedings of 5th International Conference on Language Resources and Evaluation (LREC 2006), pp. 1556–1559 (2006)Google Scholar
  44. 44.
    Yamaguchi, S., Ohno, K., Okada, Y., Suzuki, T., Tadokoro, S.: Sharing of search and rescue dog’s investigation activities by using cloud services and mobile communication service. In: The Proceedings of JSME annual Conference on Robotics and Mechatronics (ROBOMECH), p. 1A1-09a2. The Japan Society of Mechanical Engineers (2016).  https://doi.org/10.1299/jsmermd.2016.1A1-09a2Google Scholar
  45. 45.
    Yamakawa, T., Fujiwara, K., Miyajima, M., Abe, E., Kano, M., Ueda, Y.: Real-time heart rate variability monitoring em ploying a wearable telemeter and a smartphone. In: APSIPA-ASC, pp. 1–4 (2014)Google Scholar
  46. 46.
    Yonezawa, K., Miyaki, T., Rekimoto, J.: Cat@Log: sensing device attachable to pet cats for supporting human-pet interaction. In: Proceedings of International Conference on Advances in Computer Entertainment Technology, pp. 149–156 (2009)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Kazunori Ohno
    • 1
    Email author
  • Ryunosuke Hamada
    • 2
  • Tatsuya Hoshi
    • 3
  • Hiroyuki Nishinoma
    • 3
  • Shumpei Yamaguchi
    • 3
  • Solvi Arnold
    • 4
  • Kimitoshi Yamazaki
    • 4
  • Takefumi Kikusui
    • 5
  • Satoko Matsubara
    • 5
  • Miho Nagasawa
    • 5
  • Takatomi Kubo
    • 6
  • Eri Nakahara
    • 7
  • Yuki Maruno
    • 8
  • Kazushi Ikeda
    • 6
  • Toshitaka Yamakawa
    • 9
  • Takeshi Tokuyama
    • 10
  • Ayumi Shinohara
    • 11
  • Ryo Yoshinaka
    • 11
  • Diptarama Hendrian
    • 11
  • Kaizaburo Chubachi
    • 11
  • Satoshi Kobayashi
    • 11
  • Katsuhito Nakashima
    • 11
  • Hiroaki Naganuma
    • 11
  • Ryu Wakimoto
    • 11
  • Shu Ishikawa
    • 11
  • Tatsuki Miura
    • 11
  • Satoshi Tadokoro
    • 3
  1. 1.NICHeTohoku University/RIKEN AIPSendai-shiJapan
  2. 2.NICHeTohoku UniversitySendai-shiJapan
  3. 3.GSISTohoku UniversitySendai-shiJapan
  4. 4.Shinshu UniversityNaganoJapan
  5. 5.Azabu UniversitySagamiharaJapan
  6. 6.GSSTNara Institute of Science and TechnologyIkoma-shiJapan
  7. 7.GSISNara Institute of Science and TechnologyIkoma-shiJapan
  8. 8.Faculty for the Study of Contemporary SocietyKyoto Women’s UniversityKyotoJapan
  9. 9.Kumamoto UniversityKumamoto-shiJapan
  10. 10.GSISTohoku UniversitySendai-shiJapan
  11. 11.GSISTohoku UniversitySendai-shiJapan

Personalised recommendations