Biological Cybernetics

, Volume 112, Issue 3, pp 209–225 | Cite as

Leveraging variable sensor spatial acuity with a homogeneous, multi-scale place recognition framework

  • Adam Jacobson
  • Zetao Chen
  • Michael Milford
Original Article


Most robot navigation systems perform place recognition using a single-sensor modality and one, or at most two heterogeneous map scales. In contrast, mammals perform navigation by combining sensing from a wide variety of modalities including vision, auditory, olfactory and tactile senses with a multi-scale, homogeneous neural map of the environment. In this paper, we develop a multi-scale, multi-sensor system for mapping and place recognition that combines spatial localization hypotheses at different spatial scales from multiple different sensors to calculate an overall place recognition estimate. We evaluate the system’s performance over three repeated 1.5-km day and night journeys across a university campus spanning outdoor and multi-level indoor environments, incorporating camera, WiFi and barometric sensory information. The system outperforms a conventional camera-only localization system, with the results demonstrating not only how combining multiple sensing modalities together improves performance, but also how combining these sensing modalities over multiple scales further improves performance over a single-scale approach. The multi-scale mapping framework enables us to analyze the naturally varying spatial acuity of different sensing modalities, revealing how the multi-scale approach captures each sensing modality at its optimal operation point where a single-scale approach does not, and enables us to then weight sensor contributions at different scales based on their utility for place recognition at that scale.


Place recognition Place cells Sensor fusion Robotics 


  1. Andreasson H, Duckett T, Lilienthal A (2008) A minimalistic approach to appearance-based visual SLAM. IEEE Trans Robot 24(5):1–11CrossRefGoogle Scholar
  2. Arras KO, Tomatis N, Jensen BT, Siegwart R (2001) Multisensor on-the-fly localization:: precision and reliability for applications. Robot Auton Syst 34(23):131–143. ISSN 0921-8890. European Workshop on Advanced Mobile Robots
  3. Barry C, Burgess N (2014) Neural mechanisms of self-location. Curr Biol 24(8):R330–R339CrossRefPubMedPubMedCentralGoogle Scholar
  4. Bay H, Tuytelaars T, Van Gool L (2006) SURF: speeded up robust features. In: Computer vision ECCV 2006Google Scholar
  5. Berkvens R, Jacobson A, Milford M, Peremans H, Weyn M (2014) Biologically inspired SLAM using Wi-Fi. In: IEEE intelligent robots and systemsGoogle Scholar
  6. Bonardi F, Ainouz S, Boutteau R, Dupuis Y, Savatier X, Vasseur P (2017) PHROG: a multimodal feature for place recognition. Sensors 17(5):1167CrossRefPubMedCentralGoogle Scholar
  7. Bosse M, Newman P, Leonard J, Soika M, Feiten W, Teller S (2003) An atlas framework for scalable mapping. In: International conference on robotics and automation, vol 2. IEEE, Taipei, pp 1899–1906. ISBN 0780377362Google Scholar
  8. Burgess N, O’Keefe J (1996) Neuronal computations underlying the firing of place cells and their role in navigation. Hippocampus 6(6):749–762CrossRefPubMedGoogle Scholar
  9. Chen Z, Jacobson A, Erdem UM, Hasselmo ME, Milford M (2014) Multi-scale bio-inspired place recognition. In: 2014 IEEE international conference on robotics and automation (ICRA)Google Scholar
  10. Cummins M, Newman P (2011) Appearance-only SLAM at large scale with FAB-MAP 2.0. Int J Robot Res 30:1100–1123CrossRefGoogle Scholar
  11. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Computer vision and pattern recognition. CVPR 2005. IEEE computer society conference on, vol 1. IEEE, pp 886–893Google Scholar
  12. Davison AJ, Reid ID, Molton ND, Stasse O (2007) MonoSLAM: real-time single camera SLAM. IEEE Trans Pattern Anal Mach Intell 29(6):1052–1067CrossRefPubMedGoogle Scholar
  13. Faragher RM, Sarno C, Newman M (2012) Opportunistic radio SLAM for indoor navigation using smartphone sensors. In: Position location and navigation symposium (PLANS), 2012 IEEE/ION. IEEE, Myrtle Beach, pp 120–128. ISBN 978-1-4673-0387-3
  14. Fenton AA, Kao H-Y, Neymotin SA, Olypher A, Vayntrub Y, Lytton WW, Ludvig N (2008) Unmasking the CA1 ensemble place code by exposures to small and large environments: more place cells and multiple, irregularly arranged, and expanded place fields in the larger space. J Neurosci 28(44):11250–11262CrossRefPubMedPubMedCentralGoogle Scholar
  15. Ferris B, Fox D, Lawrence N (2007) WiFi-SLAM using Gaussian process latent variable models. In: Proceedings of the 20th international joint conference on artificial intelligence, pp 2480–2485Google Scholar
  16. Frost BJ, Mouritsen H (2006) The neural mechanisms of long distance animal navigation. Curr Opin Neurobiol 16(4):481–488CrossRefPubMedGoogle Scholar
  17. Geva-Sagiv M, Las L, Yovel Y, Ulanovsky N (2015) Spatial cognition in bats and rats: from sensory acquisition to multiscale maps and navigation. Nat Rev Neurosci 16(2):94–108CrossRefPubMedGoogle Scholar
  18. Glover A, Maddern W, Warren M, Reid S, Milford M, Wyeth G (2012) OpenFABMAP: an open source toolbox for appearance-based loop closure detection. In: Robotics and automation (ICRA), 2012 IEEE international conference on, pp 4730–4735. ISBN 1050-4729
  19. Hargreaves EL, Rao G, Lee I, Knierim JJ (2005) Major dissociation between medial and lateral entorhinal input to dorsal hippocampus. Science 308(5729):1792CrossRefPubMedGoogle Scholar
  20. Huang J, Millman D, Quigley M, Stavens D, Thrun S, Aggarwal A (2011) Efficient, generalized indoor WiFi GraphSLAM. In: International conference on robotics and automation. ISBN 978-1-61284-386-5
  21. Jacobson A, Chen Z, Milford M (2015a) Autonomous multisensor calibration and closed-loop fusion for SLAM. J Field Robot 32(1):85–122Google Scholar
  22. Jacobson A, Chen Z, Milford M (2015b) Online place recognition calibration for out-of-the-box SLAM. In: IEEE/RSJ international conference on intelligent robots and systemsGoogle Scholar
  23. Jung MW, Wiener SI, McNaughton BL (1994) Comparison of spatial firing characteristics of units in dorsal and ventral hippocampus of the rat. J Neurosci 14(12):7347–7356CrossRefPubMedGoogle Scholar
  24. Kawewong A, Tongprasit N, Tangruamsub S, Hasegawa O (2010) Online and incremental appearance-based SLAM in highly dynamic environments. Int J Robot Res 30(1):33–55Google Scholar
  25. Keefe JO, Burgess N (1996) Geometric determinants of the place fields of hippocampal neurons. Nature 381(6581):425CrossRefGoogle Scholar
  26. Konolige K, Agrawal M (2008) FrameSLAM: from bundle adjustment to real-time visual mapping. IEEE Trans Robot 24:1066–1077CrossRefGoogle Scholar
  27. Kuipers B, Byun YT (1991) A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations. Robot Auton Syst 8(1):47–63CrossRefGoogle Scholar
  28. Kuipers B, Modayil J, Beeson P, MacMahon M, Savelli F (2004) Local metrical and global topological maps in the hybrid spatial semantic hierarchy. In: International conference on robotics and automationGoogle Scholar
  29. Lategahn H, Beck J, Kitt B, Stiller C (2013) How to learn an illumination robust image feature for place recognition. In: Intelligent vehicles symposium (IV), 2013 IEEE. IEEE, pp 285–291Google Scholar
  30. Li B, Harvey B, Gallagher T (2013) Using barometers to determine the height for indoor positioning. In: Indoor positioning and indoor navigation (IPIN), 2013 international conference on. IEEE, pp 1–7Google Scholar
  31. Lowe DG (1999) Object recognition from local scale-invariant features. In: International conference on computer vision, KerkyraGoogle Scholar
  32. Maaswinkel H, Whishaw IQ (1999) Homing with locale, taxon, and dead reckoning strategies by foraging rats: sensory hierarchy in spatial navigation. Behav Brain Res 99(2):143–152. ISSN 0166-4328
  33. McNaughton B, Barnes C, Gerrard J, Gothard K, Jung M, Knierim J, Kudrimoti H, Qin Y, Skaggs W, Suster M et al (1996) Deciphering the hippocampal polyglot: the hippocampus as a path integration system. J Exp Biol 199(1):173–185PubMedGoogle Scholar
  34. Milford M, Wyeth G (2012) SeqSLAM: visual route-based navigation for sunny summer days and stormy winter nights. In: IEEE international conference on robotics and automation, St PaulGoogle Scholar
  35. Muller RU, Kubie JL, Ranck JB (1987) Spatial firing patterns of hippocampal complex-spike cells in a fixed environment. J Neurosci 7(7):1935–1950CrossRefPubMedGoogle Scholar
  36. Nakazawa K, McHugh TJ, Wilson MA, Tonegawa S (2004) NMDA receptors, place cells and hippocampal spatial memory. Nat Rev Neurosci 5(5):361–372CrossRefPubMedGoogle Scholar
  37. Olivia A (2005) Gist of the scene. In: Itti L, Rees G, Tsotsos JK (eds) Neurobiology of attention. Elsevier, AmsterdamGoogle Scholar
  38. Paz LM, Pinies P, Tardos JD, Neira J (2008) Large-scale 6-DOF SLAM with stereo-in-hand. IEEE Trans Robot 24:946–957CrossRefGoogle Scholar
  39. Ravassard P, Kees A, Willers B, Ho D, Aharoni D, Cushman J, Aghajan ZM, Mehta MR (2013) Multisensory control of hippocampal spatiotemporal selectivity. Science 340(6138):1342–1346CrossRefPubMedPubMedCentralGoogle Scholar
  40. Rossier J, Haeberli C, Schenk F (2000) Auditory cues support place navigation in rats when associated with a visual cue. Behav Brain Res 117(1–2):209–214. ISSN 01664328
  41. Sheppard JP, Raposo D, Churchland AK (2013) Dynamic weighting of multisensory stimuli shapes decision-making in rats and humans. J Vis 13(6):4CrossRefPubMedPubMedCentralGoogle Scholar
  42. Skaggs WE, McNaughton BL (1996) Theta phase precession in hippocampal. Hippocampus 6:149–172CrossRefPubMedGoogle Scholar
  43. Solstad T, Moser EI, Einevoll GT (2006) From grid cells to place cells: a mathematical model. Hippocampus 16:1026–1031CrossRefPubMedGoogle Scholar
  44. Spiers HJ, Barry C (2015) Neural systems supporting navigation. Curr Opin Behav Sci 1:47–55CrossRefGoogle Scholar
  45. Vanderelst D, Steckel J, Boen A, Peremans H, Holderied MW (2016) Place recognition using batlike sonar. eLife 5:e14188CrossRefPubMedPubMedCentralGoogle Scholar
  46. Weyn M (2011) Opportunistic seamless localization, Ph.D. thesis. Universiteit AntwerpenGoogle Scholar
  47. Wiener SI, Berthoz A, Zugaro MB (2002) Multisensory processing in the elaboration of place and head direction responses by limbic system neurons. Cogn Brain Res 14(1):75–90CrossRefGoogle Scholar
  48. Wikenheiser AM, Redish AD (2015) Hippocampal theta sequences reflect current goals. Nat Neurosci 18(2):289–294CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Electrical Engineering and Computer ScienceQueensland University of TechnologyBrisbaneAustralia

Personalised recommendations