Maps, Objects and Contexts for Robots

  • James J. Little
  • Tristram Southey
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 8)

When you have brought your new Apple iRobot home for the first time, you are faced with the challenging task of introducing the robot to its new home/workspace. Of course the robot knows about homes and typical tasks. That’s why you bought the sleek, stylish robot, in addition to the fact that it promised a simple interface. It prompts you to take it on a tour of the house, naming the rooms, pointing out the appliances, and identifying the occupants of the house.

The promise of the now discontinued Aibo whose communication and basic behaviours show that even simple visual sensors using strong features (SIFT[17]) can enable visual tracking and recognition. Built in to the home robot will be the necessary concepts – tasks, objects, contexts, locations. Your home vacuum robot “knows” only about stairs, objects, infrared walls, and random search. Your iRobot knows about kitchens, doors, stairs, bedrooms, beer (you have the party version of the iRobot that can bring you beer in the entertainment room). How does the robot tie the sensory flow it receives to its plans, names, and goals in its repertoire?

This fanciful thought experiment is not so far in the future. For some applications such as assistive technologies[21, 1] which operate in contexts where rich visual sensing is deployed, the range of objects may be limited to a care facility where patients’ rooms are typically constrained in their contents. Here it may be effective to learn the connection between features of the visual stream and the object in the scene, and how they influence the actions of the robot.


Mobile Robot Assistive Technology Appearance Model Service Robot Stereo Camera 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Boger, J., Hoey, J., Poupart, P., Boutilier, C., Fernie, G., Mihailidis, A.: A planning system based on markov decision processes to guide people with dementia through activities of daily living. IEEE Transactions on Information Technology in BioMedicine 10(2), 323-333 (2006)CrossRefGoogle Scholar
  2. 2.
    Davison, A.: Real-time simultaneous localisation and mapping with a single camera. In: Proceedings of the IEEE Int. Conf. on Computer Vision, pp. 1403-1410 (2003)Google Scholar
  3. 3.
    Ekvall, S., Jensfelt, P., Kragic, D.: Integrating active mobile robot object recognition and slam in natural environments. In: Proc. of the IEEE/RSJ International Conference on Robotics and Automation (IROS’06). Beijing, China (2006)Google Scholar
  4. 4.
    Elder, J., Prince, S., Hou, Y., Sizintsev, M., Olevskiy, E.: Pre-f and attentive detection of humans in wide-field scenes. International Journal of Computer Vision 72(1), 47-66 (2007)CrossRefGoogle Scholar
  5. 5.
    Elfes, A.: Using occupancy grids for mobile robot perception and navigation. IEEE Computer 22(6),46-57 (1990)Google Scholar
  6. 6.
    Elinas, P., Hoey, J., Lahey, D., Montgomery, J.D., Murray, D., Se, S., Little, J.J.: Waiting with jose, a vision-based mobile robot. IEEE International Conference on Robotics and Automation pp. 3698-3075 (2002)Google Scholar
  7. 7.
    Elinas, P., Hoey, J., Little, J.J.: Homer: Human oriented messenger robot. AAAI Spring Symposium on Human Interaction with Autonomous Systems in Complex Environments pp. 45-51 (2003)Google Scholar
  8. 8.
    Fergus, R., Perona, P., Zisserman, A.: Weakly supervised scale-invariant learning of models for visual recognition. International Journal of Computer Vision 71(3), 273-303 (2007)CrossRefGoogle Scholar
  9. 9.
    Ferrari, V., Tuytelaars, T., Van Gool, L.: Simultaneous object recognition and segmentation from single or multiple model views. International Journal of Computer Vision 67(2), 159-188 (2006)CrossRefGoogle Scholar
  10. 10.
    Gates, B.: A robot in every home. Scientific American 296(1), 58-65 (2007)CrossRefGoogle Scholar
  11. 11.
    Hoey, J., Little, J.J.: Value-directed human behavior analysis from video using partially observable markov decision processes. IEEE Trans. Pattern Anal. Mach. Intell. 29(7), 1118-1132 (2007)CrossRefGoogle Scholar
  12. 12.
    Itti, L., Baldi, P.: A principled approach to detecting surprising events in video. In: CVPR ’05: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05) - Volume 1, pp. 631-637. IEEE Computer Society, Washington, DC, USA (2005)Google Scholar
  13. 13.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254-1259 (1998)CrossRefGoogle Scholar
  14. 14.
    Kuipers, B.: The spatial semantic hierarchy. Artificial Intelligence 119, 191-233 (2000)MATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Lensch, H.P.A., Kautz, J., Goesele, M., Heidrich, W., Seidel, H.P.: Image-based reconstruction of spatial appearance and geometric detail. ACM Trans. Graph. 22(2), 234-257 (2003). DOI
  16. 16.
    Little, J.J., Hoey, J., , Elinas, P.: Visual capabilities in an interactive autonomous robot. In: H.I. Christensen, H.H. Nagel (eds.) Cognitive vision systems : sampling the spectrum of approaches, 365. Springer Verlag, Berlin et al. (2006)Google Scholar
  17. 17.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91-110 (2004)CrossRefGoogle Scholar
  18. 18.
    Mark, D.M., Smith, B., Tversky, B.: Ontology and geographic objects: An empirical study of cognitive categorization. In: COSIT ’99: Proceedings of the International Conference on Spatial Information Theory: Cognitive and Computational Foundations of Geographic Information Science, pp. 283-298. Springer-Verlag, London, UK (1999)Google Scholar
  19. 19.
    Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. Proceedings of British Machine Vision Conference, September 2002 I, 384-393 (2002)Google Scholar
  20. 20.
    Medioni, G.G., Francois, A.R., Siddiqui, M., Kim, K., Yoon, H.: Robust real-time vision for a personal service robot. Computer Vision and Image Understanding ((to appear))Google Scholar
  21. 21.
    Mihailidis, A., Elinas, P., an Jesse Hoey, J.N.B.: Pervasive computing to enable the mobility of older adults with cognitive impairments: An anti-collision system for a powered wheelchairGoogle Scholar
  22. 22.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615-1630 (2005)CrossRefGoogle Scholar
  23. 23.
    Mueller, A., Beetz, M.: Designing and implementing a plan library for a simulated household robot. In: AAAI Cognitive Robotics Workshop (2006)Google Scholar
  24. 24.
    Pentney, W., Popescu, A.M., Wang, S., Kautz, H.A., Philipose, M.: Sensor-based understanding of daily life via large-scale use of common sense. In: AAAI. AAAI Press (2006)Google Scholar
  25. 25.
    Rekleitis, I., Meger, D., Dudek, G.: Simultaneous planning, localization, and mapping in a camera sensor network. Robotics and Autonomous Systems 54(11), 921-932 (2006)CrossRefGoogle Scholar
  26. 26.
    Rimey, R.D., Brown, C.M.: Control of selective perception using bayes nets and decision theory. Int. J. Comput. Vision 12(2-3), 173-207 (1994)CrossRefGoogle Scholar
  27. 27.
    Se, S., Lowe, D.G., Little, J.J.: Mobile robot localization and mapping with uncertainty using scale-invariant visual landmarks. Intl. Jour. Robotic Research 21(8), 735-760 (2002)CrossRefGoogle Scholar
  28. 28.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 888-905 (2000)CrossRefGoogle Scholar
  29. 29.
    Sim, R., Elinas, P., Little, J.: A study of the rao-blackwellised particle filter for efficient and accurate vision-based slam. International Journal of Computer Vision 74(3), 303-318 (2007)CrossRefGoogle Scholar
  30. 30.
    Sim, R., Little, J.J.: Autonomous vision-based exploration and mapping using hybrid maps and Rao-Blackwellised particle filters. In: Proceedings of the IEEE/RSJ Conference on Intelligent Robots and Systems (IROS), pp. 2082-2089. IEEE/RSJ, IEEE Press, Beijing (2006)Google Scholar
  31. 31.
    Sivic, J., Schaffalitzky, F., Zisserman, A.: Object level grouping for video shots. International Journal of Computer Vision 67(2), 189-210 (2006)CrossRefGoogle Scholar
  32. 32.
    Smyth, C., Poole, D.: Qualitative probabilistic matching with hierarchical descriptions. In: KR, pp. 479-487 (2004)Google Scholar
  33. 33.
    Southey, T., Little, J.J.: Object discovery using motion, appearance and shape. In: AAAI Cognitive Robotics Workshop (2006)Google Scholar
  34. 34.
    Tapia, E.M., Choudhury, T., Philipose, M.: Building reliable activity models using hierarchical shrinkage and mined ontology. In: Pervasive, pp. 17-32 (2006)Google Scholar
  35. 35.
    Thrun, S., Fox, D., Burgard, W.: Monte carlo localization with mixture proposal distribution. In: Proceedings of the 2000 National Conference of the American Association for Artificial Intelligence (AAAI), pp. 859-865 (2000)Google Scholar
  36. 36.
    Torralba, A., Murphy, K., Freeman, W.: Sharing visual features for multiclass and multiview object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(5), 854-869 (2007)CrossRefGoogle Scholar
  37. 37.
    Torralba, A., Murphy, K., Freeman, W., Rubin, M.: Context-based vision system for place and object recognition. In: Nice, France, Oct 13-16, pp. 273-280 (2003)Google Scholar
  38. 38.
    Tsotsos, J., Culhane, S., Wai, W., Lai, Y., Davis, N., Nuflo, F.: Modeling visual-attention via selective tuning. Artificial Intelligence 78(1-2), 507-545 (1995)CrossRefGoogle Scholar
  39. 39.
    Vasudevan, S., G ächter, S., Nguyen, V., Siegwart, R.: Cognitive maps for mobile robots-an object based approach. Robot. Auton. Syst. 55(5), 359-371 (2007)CrossRefGoogle Scholar
  40. 40.
    Vogel, J., Murphy, K.: A non-myopic approach to visual search. In: CRV ’07: Proceedings of the Fourth Canadian Conference on Computer and Robot Vision, pp. 227-234. IEEE Computer Society, Washington, DC, USA (2007). DOI
  41. 41.
    Winn, J., Shotton, J.: The layout consistent random field for recognizing and segmenting partially occluded objects. In: CVPR ’06: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 37-44. IEEE Computer Society, Washington, DC, USA (2006)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • James J. Little
    • 1
  • Tristram Southey
    • 1
  1. 1.Department of Computer ScienceUniversity of British ColumbiaVancouverCanada

Personalised recommendations