Skip to main content
SpringerLink
Log in

Augmenting Robot Knowledge Consultants with Distributed Short Term Memory

  • Conference paper
  • First Online:
Social Robotics (ICSR 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11357))

Included in the following conference series:

Abstract

Human-robot communication in situated environments involves a complex interplay between knowledge representations across a wide variety of modalities. Crucially, linguistic information must be associated with representations of objects, locations, people, and goals, which may be represented in very different ways. In previous work, we developed a Consultant Framework that facilitates modality-agnostic access to information distributed across a set of heterogeneously represented knowledge sources. In this work, we draw inspiration from cognitive science to augment these distributed knowledge sources with Short Term Memory Buffers to create an STM-augmented algorithm for referring expression generation. We then discuss the potential performance benefits of this approach and insights from cognitive science that may inform future refinements in the design of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alvarez, G.A., Cavanagh, P.: The capacity of visual short-term memory is set both by visual information load and by number of objects. Psychol. Sci. 15, 106–111 (2004)

    Article  Google Scholar 

  2. Baddeley, A.: Working memory. Science 255(5044), 556–559 (1992)

    Article  Google Scholar 

  3. Baddeley, A.D., Thomson, N., Buchanan, M.: Word length and the structure of short-term memory. J. Verbal Learn. Verbal Behav. 14, 575–589 (1975)

    Article  Google Scholar 

  4. Brennan, S.E.: Lexical entrainment in spontaneous dialog. ISSD 96, 41–44 (1996)

    Google Scholar 

  5. Brennan, S.E., Clark, H.H.: Conceptual pacts and lexical choice in conversation. J. Exp. Psychol.: Learn. Mem. Cogn. 22, 1482 (1996)

    Google Scholar 

  6. Briggs, G., Scheutz, M.: A hybrid architectural approach to understanding and appropriately generating indirect speech acts. In: Proceedings of AAAI (2013)

    Google Scholar 

  7. Carbary, K., Tanenhaus, M.: Conceptual pacts, syntactic priming, and referential form. In: Proceedings of CogSci Workshop on Production of Referring Expressions (2011)

    Google Scholar 

  8. Clocksin, W., Mellish, C.S.: Programming in PROLOG. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-642-55481-0

    Book  MATH  Google Scholar 

  9. Dale, R., Reiter, E.: Computational interpretations of the gricean maxims in the generation of referring expressions. Cognit. Sci. 19(2), 233–263 (1995)

    Article  Google Scholar 

  10. Dzifcak, J., Scheutz, M., Baral, C., Schermerhorn, P.: What to do and how to do it: translating natural language directives into temporal and dynamic logic representation for goal management and action execution. In: Proceedings of ICRA (2009)

    Google Scholar 

  11. Fougnie, D., Zughni, S., Godwin, D., Marois, R.: Working memory storage is intrinsically domain specific. J. Exp. Psychol.: Gen. 144, 30 (2015)

    Article  Google Scholar 

  12. Goudbeek, M., Krahmer, E.: Alignment in interactive reference production: content planning, modifier ordering, and referential overspecification. TiCS 4, 269–289 (2012)

    Google Scholar 

  13. Just, M.A., Carpenter, P.A.: A capacity theory of comprehension: individual differences in working memory. Psychol. Rev. 99(1), 122 (1992)

    Article  Google Scholar 

  14. Krause, E., Zillich, M., Williams, T., Scheutz, M.: Learning to recognize novel objects in one shot through human-robot interactions in natural language dialogues. In: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence (2014)

    Google Scholar 

  15. Logie, R.H.: Visuo-Spatial Working Memory. Psychology Press, London (2014)

    Google Scholar 

  16. Ma, W.J., Husain, M., Bays, P.M.: Changing concepts of working memory. Nat. Neurosci. 17(3), 347 (2014)

    Article  Google Scholar 

  17. Mathy, F., Feldman, J.: What’s magic about magic numbers? chunking and data compression in short-term memory. Cognition 122(3), 346–362 (2012)

    Article  Google Scholar 

  18. Mavridis, N.: A review of verbal and non-verbal human-robot interactive communication. Robot. Auton. Syst. 63, 22–35 (2015)

    Article  MathSciNet  Google Scholar 

  19. Miller, G.A.: The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol. Rev. 63, 81 (1956)

    Article  Google Scholar 

  20. Oberauer, K., Farrell, S., Jarrold, C., Lewandowsky, S.: What limits working memory capacity? Psychol. Bull. 142, 758 (2016)

    Article  Google Scholar 

  21. Oberauer, K., Kliegl, R.: A formal model of capacity limits in working memory. J. Mem. Lang. 55(4), 601–626 (2006)

    Article  Google Scholar 

  22. Oberauer, K., Lewandowsky, S., Farrell, S., Jarrold, C., Greaves, M.: Modeling working memory: an interference model of complex span. Psychon. Bull. Rev. 19, 779–819 (2012)

    Article  Google Scholar 

  23. Popescu-Belis, A., Robba, I., Sabah, G.: Reference resolution beyond coreference: a conceptual frame and its application. In: Proceedings of COLING (1998)

    Google Scholar 

  24. Saito, S., Miyake, A.: On the nature of forgetting and the processing-storage relationship in reading span performance. J. Mem. Lang. 50, 425–443 (2004)

    Article  Google Scholar 

  25. Schermerhorn, P.W., Kramer, J.F., Middendorff, C., Scheutz, M.: DIARC: a testbed for natural human-robot interaction. In: Proceedings of AAAI (2006)

    Google Scholar 

  26. Schermerhorn, P.W., Scheutz, M.: The utility of affect in the selection of actions and goals under real-world constraints. In: IC-AI, pp. 948–853 (2009)

    Google Scholar 

  27. Scheutz, M.: Ade: steps toward a distributed development and runtime environment for complex robotic agent architectures. Appl. Artif. Intell. 20, 275–304 (2006)

    Article  Google Scholar 

  28. Scheutz, M., Krause, E., Oosterveld, B., Frasca, T., Platt, R.: Spoken instruction-based one-shot object and action learning in a cognitive robotic architecture. In: 16th Conference on Autonomous Agents and Multiagent Systems (AAMAS) (2017)

    Google Scholar 

  29. Scheutz, M., Williams, T., Krause, E., Oosterveld, B., Sarathy, V., Frasca, T.: An overview of the distributed integrated cognition affect and reflection DIARC architecture. In: Cognitive Architectures (2018, in press)

    Google Scholar 

  30. Schweickert, R., Boruff, B.: Short-term memory capacity: magic number or magic spell? J. Exp. Psychol.: Learn. Mem. Cogn. 12, 419 (1986)

    Google Scholar 

  31. Taylor, R., Thomson, H., Sutton, D., Donkin, C.: Does working memory have a single capacity limit? J. Mem. Lang. 93, 67–81 (2017)

    Article  Google Scholar 

  32. Tolins, J., Zeamer, C., Fox Tree, J.E.: Overhearing dialogues and monologues: how does entrainment lead to more comprehensible referring expressions? Discourse Process. 55, 1–21 (2017)

    Google Scholar 

  33. Walker, W., Lamere, P., Kwok, P., et al.: Sphinx-4: a flexible open source framework for speech recognition. Technical report (2004)

    Google Scholar 

  34. Wang, B., Cao, X., Theeuwes, J., Olivers, C.N., Wang, Z.: Separate capacities for storing different features in visual working memory. J. Exp. Psychol.: Learn. Mem. Cogn. 43, 26 (2017)

    Google Scholar 

  35. Williams, T.: A consultant framework for natural language processing in integrated robot architectures. IEEE Intell. Inform. Bull. (2017)

    Google Scholar 

  36. Williams, T., Acharya, S., Schreitter, S., Scheutz, M.: Situated open world reference resolution for human-robot dialogue. In: Proceedings of HRI (2016)

    Google Scholar 

  37. Williams, T., Krause, E., Oosterveld, B., Scheutz, M.: Towards givenness and relevance-theoretic open world reference resolution. In: RSS Workshop on Models and Representations for Natural Human-Robot Communication (2018)

    Google Scholar 

  38. Williams, T., Krause, E., Oosterveld, B., Thielstrom, R., Scheutz, M.: Towards robot knowledge consultants augmented with distributed short term memory. In: RSS Workshop on Models and Representations for Natural Human-Robot Communication (2018)

    Google Scholar 

  39. Williams, T., Scheutz, M.: A framework for resolving open-world referential expressions in distributed heterogeneous knowledge bases. In: Proceedings of AAAI (2016)

    Google Scholar 

  40. Williams, T., Scheutz, M.: Referring expression generation under uncertainty: algorithm and evaluation framework. In: Proceedings of the 10th International Conference on Natural Language Generation (INLG) (2017)

    Google Scholar 

  41. Williams, T., Scheutz, M.: Resolution of referential ambiguity in human-robot dialogue using dempster-Shafer theoretic pragmatics. In: Proceedings of RSS (2017)

    Google Scholar 

  42. Williams, T., Scheutz, M.: Reference in robotics: a givenness hierarchy theoretic approach. In: The Oxford Handbook of Reference (2018, in press)

    Google Scholar 

  43. Yu, Q., Shim, W.: Occipital, parietal, and frontal cortices selectively maintain task-relevant features of multi-feature objects in visual working memory (2017)

    Google Scholar 

Download references

Acknowledgments

This work was in part supported by ONR grant N00014-16-1-0278.

Author information

Authors and Affiliations

  1. Colorado School of Mines MIRRORLab, Golden, CO, USA

    Tom Williams

  2. Tufts University Human-Robot Interaction Lab, Medford, MA, USA

    Ravenna Thielstrom, Evan Krause, Bradley Oosterveld & Matthias Scheutz

Authors
  1. Tom Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ravenna Thielstrom
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Evan Krause
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bradley Oosterveld
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Matthias Scheutz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tom Williams .

Editor information

Editors and Affiliations

  1. The National University of Singapore, Singapore, Singapore

    Shuzhi Sam Ge

  2. Qatar University, Doha, Qatar

    John-John Cabibihan

  3. University Carlos III de Madrid, Madrid, Spain

    Miguel A. Salichs

  4. Canterbury University, Auckland, New Zealand

    Elizabeth Broadbent

  5. Wichita State University, Wichita, KS, USA

    Hongsheng He

  6. The Pennsylvania State University, University Park, PA, USA

    Alan R. Wagner

  7. Universidad Carlos III de Madrid, Madrid, Spain

    Álvaro Castro-González

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Williams, T., Thielstrom, R., Krause, E., Oosterveld, B., Scheutz, M. (2018). Augmenting Robot Knowledge Consultants with Distributed Short Term Memory. In: Ge, S., et al. Social Robotics. ICSR 2018. Lecture Notes in Computer Science(), vol 11357. Springer, Cham. https://doi.org/10.1007/978-3-030-05204-1_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-05204-1_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-05203-4

  • Online ISBN: 978-3-030-05204-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation