Advertisement

Modeling Success, Failure, and Intent of Multi-Agent Activities Under Severe Noise

  • Adam Sadilek
  • Henry Kautz

Abstract

This chapter takes on the task of understanding human interactions, attempted interactions, and intentions from noisy sensor data in a fully relational multi-agent setting. We use a real-world game of capture the flag to illustrate our approach in a well-defined domain that involves many distinct cooperative and competitive joint activities. We model the domain using Markov logic, a statistical-relational language, and learn a theory that jointly denoises the data and infers occurrences of high-level activities, such as a player capturing an enemy. We show that while it may be impossible to directly detect a multi-agent activity due to sensor noise or malfunction, the occurrence of the activity can still be inferred by considering its impact on the behavior of the people involved. Further, we show that given a model of successfully performed multi-agent activities, along with a set of examples of failed attempts at the same activities, our system automatically learns an augmented model that is capable of recognizing success and failure, as well as goals of people’s actions with high accuracy. We compare our approach with other alternatives and show that our unified model, which takes into account not only relationships among individual players, but also relationships among activities over the entire length of a game, although more computationally costly, is significantly more accurate. Finally, we demonstrate that interesting game segments and key players can be efficiently identified in an automated fashion. Our system exhibits a strong agreement with human judgement about the game situations at hand.

Keywords

Global Position System Activity Recognition Global Position System Data Conditional Random Field Dynamic Bayesian Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

This chapter extends work published in Sadilek and Kautz [72, 73, 74]. We thank Sebastian Riedel for his help with theBeast, and to Radka Sadílková and Wendy Beatty for their helpful comments. This work was supported by ARO grant #W911NF-08-1-0242, DARPA SBIR Contract #W31P4Q-08-C-0170, and a gift from Kodak.

References

  1. 1.
    Abowd, G. D., Atkeson, C. G., Hong, J., Long, S., Kooper, R., & Pinkerton, M. (1997). Cyberguide: a mobile context-aware tour guide. Wireless Networks, 3(5), 421–433. CrossRefGoogle Scholar
  2. 2.
    Allwein, E. L., Schapire, R. E., & Singer, Y. (2001). Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research, 1, 113–141. MathSciNetMATHGoogle Scholar
  3. 3.
    Ashbrook, D., & Starner, T. (2003). Using GPS to learn significant locations and predict movement across multiple users. Personal and Ubiquitous Computing, 7, 275–286. CrossRefGoogle Scholar
  4. 4.
    Baker, C., Tenenbaum, J., & Saxe, R. (2006). Bayesian models of human action understanding. Advances in Neural Information Processing Systems, 18, 99. Google Scholar
  5. 5.
    Baker, C. L., Tenenbaum, J. B., & Saxe, R. R. (2007). Goal inference as inverse planning. In Proceedings of the 29th annual meeting of the cognitive science society. Google Scholar
  6. 6.
    Baker, C. L., Goodman, N. D., & Tenenbaum, J. B. (2008). Theory-based social goal inference. In Proceedings of the thirtieth annual conference of the cognitive science society (pp. 1447–1452). Google Scholar
  7. 7.
    Baker, C. L., Saxe, R. R., & Tenenbaum, J. B. (2011). Bayesian theory of mind: Modeling joint belief-desire attribution. In Proceedings of the thirty-second annual conference of the cognitive science society. Google Scholar
  8. 8.
    Baldwin, D. A., & Baird, J. A. (2001). Discerning intentions in dynamic human action. Trends in Cognitive Sciences, 5(4), 171–178. CrossRefGoogle Scholar
  9. 9.
    Barbuceanu, M., & Fox, M. S. (1995). COOL: a language for describing coordination in multi agent systems. In Proceedings of the first international conference on multi-agent systems (ICMAS-95) (pp. 17–24). Google Scholar
  10. 10.
    Bell, R., Koren, Y., & Volinsky, C. (2007). Modeling relationships at multiple scales to improve accuracy of large recommender systems. In KDD (pp. 95–104). New York: ACM. Google Scholar
  11. 11.
    Biba, M., Ferilli, S., & Esposito, F. (2008). Discriminative structure learning of Markov logic networks (pp. 59–76). Berlin: Springer. Google Scholar
  12. 12.
    Biswas, R., Thrun, S., & Fujimura, K. (2007). Recognizing activities with multiple cues. In Workshop on human motion (pp. 255–270). Google Scholar
  13. 13.
    Bui, H. H. (2003). A general model for online probabilistic plan recognition. In Eighteenth international joint conference on artificial intelligence (IJCAI-2003). Google Scholar
  14. 14.
    Busetta, P., Serafini, L., Singh, D., & Zini, F. (2001). Extending multi-agent cooperation by overhearing. In Cooperative information systems (pp. 40–52). Berlin: Springer. CrossRefGoogle Scholar
  15. 15.
    Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys, 41, 15–11558. CrossRefGoogle Scholar
  16. 16.
    Culotta, A., & McCallum, A. (2005). Joint deduplication of multiple record types in relational data. In Proceedings of the 14th ACM international conference on information and knowledge management (pp. 257–258). New York: ACM. Google Scholar
  17. 17.
    Davis, J., & Domingos, P. (2009). Deep transfer via second-order Markov logic. In Proceedings of the 26th annual international conference on machine learning (pp. 217–224). New York: ACM. Google Scholar
  18. 18.
    De Raedt, L. (2008). Logical and relational learning. New York: Springer. MATHCrossRefGoogle Scholar
  19. 19.
    De Raedt, L., & Kersting, K. (2008). Probabilistic inductive logic programming. In L. De Raedt, P. Frasconi, K. Kersting, & S. Muggleton (Eds.), Lecture notes in computer science: Vol. 4911. Probabilistic inductive logic programming—theory and applications (pp. 1–27). Berlin: Springer. CrossRefGoogle Scholar
  20. 20.
    De Raedt, L., Frasconi, P., Kersting, K., & Muggleton, S. (Eds.) (2008). Lecture notes in computer science: Vol. 4911. Probabilistic inductive logic programming—theory and applications. Berlin: Springer. MATHGoogle Scholar
  21. 21.
    Denis, P., & Baldridge, J. (2007). Joint determination of anaphoricity and coreference resolution using integer programming. In Proceedings of NAACL HLT (pp. 236–243). Google Scholar
  22. 22.
    Domingos, P. (2004). Multi-relational record linkage. In Proceedings of the KDD-2004 workshop on multi-relational data mining. Google Scholar
  23. 23.
    Domingos, P., Kok, S., Lowd, D., Poon, H., Richardson, M., & Singla, P. (2008). Markov logic. In L. De Raedt, P. Frasconi, K. Kersting, & S. Muggleton (Eds.), Lecture notes in computer science: Vol. 4911. Probabilistic inductive logic programming—theory and applications (pp. 92–117). Berlin: Springer. CrossRefGoogle Scholar
  24. 24.
    Eagle, N., & Pentland, A. (2006). Reality mining: sensing complex social systems. Personal and Ubiquitous Computing, 10(4), 255–268. CrossRefGoogle Scholar
  25. 25.
    Eagle, N., & Pentland, A. S. (2009). Eigenbehaviors: Identifying structure in routine. Behavioral Ecology and Sociobiology, 63(7), 1057–1066. CrossRefGoogle Scholar
  26. 26.
    Eagle, N., Pentland, A., & Lazer, D. (2009). Inferring social network structure using mobile phone data. In Proceedings of the National Academy of Sciences. Google Scholar
  27. 27.
    Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378. CrossRefGoogle Scholar
  28. 28.
    Foote, J. (2000). Automatic audio segmentation using a measure of audio novelty. In IEEE international conference on multimedia and expo. ICME 2000 (Vol. 1, pp. 452–455). New York: IEEE. Google Scholar
  29. 29.
    Friedman, N., Getoor, L., Koller, D., & Pfeffer, A. (1999). Learning probabilistic relational models. In International joint conference on artificial intelligence (Vol. 16, pp. 1300–1309). Google Scholar
  30. 30.
    Goutte, C., & Gaussier, E. (2005). A probabilistic interpretation of precision, recall and f-score, with implication for evaluation (pp. 345–359). Berlin: Springer. Google Scholar
  31. 31.
    Gupta, A., Srinivasan, P., Shi, J., & Davis, L. S. (2009). Understanding videos, constructing plots: Learning a visually grounded storyline model from annotated videos. In CVPR. Google Scholar
  32. 32.
    Gutmann, B., & Kersting, K. (2006). TildeCRF: conditional random fields for logical sequences. In Machine learning: ECML 2006 (pp. 174–185). Berlin: Springer. CrossRefGoogle Scholar
  33. 33.
    Helaoui, R., Niepert, M., & Stuckenschmidt, H. (2010). A statistical-relational activity recognition framework for ambient assisted living systems. In Ambient intelligence and future trends—international symposium on ambient intelligence (ISAmI 2010) (pp. 247–254). Berlin: Springer. CrossRefGoogle Scholar
  34. 34.
    Hong, J. (2001). Goal recognition through goal graph analysis. The Journal of Artificial Intelligence Research, 15, 1–30. MATHGoogle Scholar
  35. 35.
    Horvitz, E., Apacible, J., Sarin, R., & Liao, L. (2005). Prediction, expectation, and surprise: Methods, designs, and study of a deployed traffic forecasting service. In Twenty-first conference on uncertainty in artificial intelligence. Google Scholar
  36. 36.
    Hu, D. H., Pan, S. J., Zheng, V. W., Liu, N. N., & Yang, Q. (2008). Real world activity recognition with multiple goals. In UbiComp (Vol. 8, pp. 30–39). CrossRefGoogle Scholar
  37. 37.
    Huynh, T. N., & Mooney, R. J. (2008). Discriminative structure and parameter learning for Markov logic networks. In Proceedings of the 25th international conference on machine learning (pp. 416–423). New York: ACM. CrossRefGoogle Scholar
  38. 38.
    Jaeger, M. (1997). Relational Bayesian networks. In Proceedings of the 13th conference on uncertainty in artificial intelligence (pp. 266–273). Google Scholar
  39. 39.
    Jordan, M. I. (1998). Learning in graphical models. Dordrecht: Kluwer Academic. MATHGoogle Scholar
  40. 40.
    Kamar, E., & Horvitz, E. (2009). Collaboration and shared plans in the open world: Studies of ridesharing. In IJCAI. Google Scholar
  41. 41.
    Kaminka, G. A., Tambe, D. V. P. M., Pynadath, D. V., & Tambe, M. (2002). Monitoring teams by overhearing: A multi-agent plan-recognition approach. Journal of Artificial Intelligence Research, 17. Google Scholar
  42. 42.
    Kersting, K., & De Raedt, L. (2000). Bayesian logic programs. In Proceedings of the work-in-progress track at the 10th international conference on inductive logic programming. Google Scholar
  43. 43.
    Kersting, K., De Raedt, L., & Raiko, T. (2006). Logical hidden Markov models. The Journal of Artificial Intelligence Research, 25(1), 425–456. MATHGoogle Scholar
  44. 44.
    Kok, S., & Domingos, P. (2005). Learning the structure of Markov logic networks. In Proceedings of the 22nd international conference on machine learning (pp. 441–448). New York: ACM. CrossRefGoogle Scholar
  45. 45.
    Kok, S., & Domingos, P. (2007). Statistical predicate invention. In Proceedings of the 24th international conference on machine learning (pp. 433–440). New York: ACM. CrossRefGoogle Scholar
  46. 46.
    Kok, S., & Domingos, P. (2009). Learning Markov logic network structure via hypergraph lifting. In Proceedings of the 26th annual international conference on machine learning (pp. 505–512). New York: ACM. Google Scholar
  47. 47.
    Koller, D. (1999). Probabilistic relational models. In Inductive logic programming (pp. 3–13). Berlin: Springer. CrossRefGoogle Scholar
  48. 48.
    Lafferty, J. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In International conference on machine learning (ICML) (pp. 282–289). San Mateo: Morgan Kaufmann. Google Scholar
  49. 49.
    Landwehr, N., Gutmann, B., Thon, I., Philipose, M., & De Raedt, L. (2007). Relational transformation-based tagging for human activity recognition. In Proceedings of the 6th international workshop on multi-relational data mining (MRDM07) (pp. 81–92). Google Scholar
  50. 50.
    Liao, L., Fox, D., & Kautz, H. (2004). Learning and inferring transportation routines. In Proceedings of the nineteenth national conference on artificial intelligence. Google Scholar
  51. 51.
    Liao, L., Fox, D., & Kautz, H. (2005). Location-based activity recognition using relational Markov networks. In IJCAI. Google Scholar
  52. 52.
    Liao, L., Patterson, D. J., Fox, D., & Kautz, H. (2007). Learning and inferring transportation routines. Artificial Intelligence, 171(5–6), 311–331. MathSciNetMATHCrossRefGoogle Scholar
  53. 53.
    Limketkai, B., Fox, D., & Liao, L. (2007). CRF-filters: Discriminative particle filters for sequential state estimation. In IEEE international conference on robotics and automation (pp. 3142–3147). Google Scholar
  54. 54.
    Ling, X., & Weld, D. S. (2010). Temporal information extraction. In Proceedings of the twenty fifth national conference on artificial intelligence. Google Scholar
  55. 55.
    Ma, Z. (2008). Modelling with PRISM of intelligent system. MSc. Thesis, Linacre College, University of Oxford. Google Scholar
  56. 56.
    Manfredotti, C. (2009). Modeling and inference with relational dynamic Bayesian networks. In Advances in artificial intelligence (pp. 287–290). Berlin: Springer. CrossRefGoogle Scholar
  57. 57.
    Manfredotti, C., & Messina, E. (2009). Relational dynamic Bayesian networks to improve multi-target tracking. In Advanced concepts for intelligent vision systems (pp. 528–539). Berlin: Springer. CrossRefGoogle Scholar
  58. 58.
    Manfredotti, C., Hamilton, H., & Zilles, S. (2010). Learning RDBNs for activity recognition. In Neural information processing systems. Google Scholar
  59. 59.
    Mihalkova, L., & Mooney, R. J. (2007). Bottom-up learning of Markov logic network structure. In Proceedings of the 24th international conference on machine learning (pp. 625–632). New York: ACM. CrossRefGoogle Scholar
  60. 60.
    Moore, D., & Essa, I. (2001). Recognizing multitasked activities using stochastic context-free grammar. In Proceedings of AAAI conference. Google Scholar
  61. 61.
    Muggleton, S. (2002). Learning structure and parameters of stochastic logic programs. In Proceedings of the 12th international conference on inductive logic programming (pp. 198–206). Berlin: Springer. Google Scholar
  62. 62.
    Murphy, K. P. (2002). Dynamic Bayesian networks: representation, inference and learning. Ph.D. thesis, University of California, Berkeley. Google Scholar
  63. 63.
    Naaman, M., Boase, J., & Lai, C.-H. (2010). Is it really about me?: message content in social awareness streams. In CSCW’10: Proceedings of the 2010 ACM conference on computer supported cooperative work (pp. 189–192). New York: ACM. CrossRefGoogle Scholar
  64. 64.
    Natarajan, S., Tadepalli, P., Altendorf, E., Dietterich, T. G., Fern, A., & Restificar, A. (2005). Learning first-order probabilistic models with combining rules. In Proceedings of the 22nd international conference on machine learning (pp. 609–616). New York: ACM. CrossRefGoogle Scholar
  65. 65.
    Natarajan, S., Bui, H. H., Tadepalli, P., Kersting, K., & Wong, W. (2008). Logical hierarchical hidden Markov models for modeling user activities. In Proc. of ILP-08. Google Scholar
  66. 66.
    Paulus, J., Müller, M., & Klapuri, A. (2010). Audio-based music structure analysis. In Proceedings of the international symposium on music information retrieval (pp. 625–636). Google Scholar
  67. 67.
    Pentland, A. S. (2008). Honest signals: how they shape our world. Cambridge: The MIT Press. Google Scholar
  68. 68.
    Poon, H., & Domingos, P. (2006). Sound and efficient inference with probabilistic and deterministic dependencies. In Proceedings of the national conference on artificial intelligence (Vol. 21, p. 458). Menlo Park: AAAI Press. Google Scholar
  69. 69.
    Poon, H., & Domingos, P. (2007). Joint inference in information extraction. In Proceedings of the 22nd national conference on artificial intelligence (Vol. 1, pp. 913–918). Menlo Park: AAAI Press. Google Scholar
  70. 70.
    Poon, H., & Domingos, P. (2008). Joint unsupervised coreference resolution with Markov logic. In Proceedings of the conference on empirical methods in natural language processing (pp. 650–659). Association for Computational Linguistics. Google Scholar
  71. 71.
    Riedel, S. (2008). Improving the accuracy and efficiency of map inference for Markov logic. In Proceedings of the twenty-fourth conference annual conference on uncertainty in artificial intelligence (UAI-08), Corvallis, Oregon (pp. 468–475). New York: AUAI Press. Google Scholar
  72. 72.
    Sadilek, A., & Kautz, H. (2010). Modeling and reasoning about success, failure, and intent of multi-agent activities. In Mobile context-awareness workshop, twelfth ACM international conference on ubiquitous computing. Google Scholar
  73. 73.
    Sadilek, A., & Kautz, H. (2010). Recognizing multi-agent activities from GPS data. In Twenty-fourth AAAI conference on artificial intelligence. Google Scholar
  74. 74.
    Sadilek, A., & Kautz, H. (2011). Location-based reasoning about complex multi-agent behavior. Journal of Artificial Intelligence Research. AI Access Foundation. Google Scholar
  75. 75.
    Sato, T., & Kameya, Y. (2001). Parameter learning of logic programs for symbolic-statistical modeling. Journal of Artificial Intelligence Research. Google Scholar
  76. 76.
    Sato, T., & Kameya, Y. (2008). New advances in logic-based probabilistic modeling by PRISM. In Probabilistic inductive logic programming (pp. 118–155). Berlin: Springer. CrossRefGoogle Scholar
  77. 77.
    Shakarian, P., Subrahmanian, V. S., & Spaino, M. L. (2009). SCARE: A Case Study with Baghdad. In Proceedings of the third international conference on computational cultural dynamics. Menlo Park: AAAI Press. Google Scholar
  78. 78.
    Shen, J. (2009). Activity recognition in desktop environments. Ph.D. Thesis, Oregon State University. Google Scholar
  79. 79.
    Shoenfield, J. R. (1967). Mathematical logic. Reading: Addison-Wesley. MATHGoogle Scholar
  80. 80.
    Singla, P., & Domingos, P. (2005). Discriminative training of Markov logic networks. In Proceedings of the national conference on artificial intelligence (Vol. 20, p. 868). Menlo Park: AAAI Press. Google Scholar
  81. 81.
    Tang, K. P., Lin, J., Hong, J. I., Siewiorek, D. P., & Sadeh, N. (2010). Rethinking location sharing: exploring the implications of social-driven vs. purpose-driven location sharing. In Proceedings of the 12th ACM international conference on ubiquitous computing (pp. 85–94). New York: ACM. CrossRefGoogle Scholar
  82. 82.
    Tran, S., & Davis, L. (2008). Visual event modeling and recognition using Markov logic networks. In Proceedings of the 10th European conference on computer vision. Google Scholar
  83. 83.
    Ullman, T. D., Baker, C., Macindoe, O., Evans, O., Goodman, N., & Tenenbaum, J. (2010). Help or hinder: Bayesian models of social goal inference. In Advances in neural information processing systems (NIPS) (Vol. 22). Google Scholar
  84. 84.
    Vail, D. L. (2008). Conditional random fields for activity recognition. Ph.D. Thesis, Carnegie Mellon University. Google Scholar
  85. 85.
    Vail, D. L., & Veloso, M. M. (2008). Feature selection for activity recognition in multi-robot domains. In Proceedings of AAAI (Vol. 2008). Google Scholar
  86. 86.
    Wang, J., & Domingos, P. (2008). Hybrid Markov logic networks. In Proceedings of the 23rd national conference on artificial intelligence (Vol. 2, pp. 1106–1111). Menlo Park: AAAI Press. Google Scholar
  87. 87.
    Wellner, B., McCallum, A., Peng, F., & Hay, M. (2004). An integrated, conditional model of information extraction and coreference with application to citation matching. In Proceedings of the 20th conference on uncertainty in artificial intelligence (pp. 593–601). Menlo Park: AAAI Press. Google Scholar
  88. 88.
    Wilson, A., Fern, A., Ray, S., & Tadepalli, P. (2008). Learning and transferring roles in multi-agent mdps. In Proceedings of AAAI. Google Scholar
  89. 89.
    Wilson, A., Fern, A., & Tadepalli, P. (2010). Bayesian role discovery for multi-agent reinforcement learning. In Proceedings of the 9th international conference on autonomous agents and multiagent systems (Vol. 1, pp. 1587–1588). International Foundation for Autonomous Agents and Multiagent Systems. Google Scholar
  90. 90.
    Wrobel, S. (1996). First order theory refinement. In Advances in inductive logic programming (pp. 14–33). Amsterdam: IOS Press. Google Scholar
  91. 91.
    Wu, T., Lian, C., & Hsu, J. Y. (2007). Joint recognition of multiple concurrent activities using factorial conditional random fields. In Proc. 22nd conf. on artificial intelligence (AAAI-2007). Google Scholar
  92. 92.
    Yoshikawa, K., Riedel, S., Asahara, M., & Matsumoto, Y. (2009). Jointly identifying temporal relations with Markov logic. In Proceedings of the joint conference of the 47th annual meeting of the ACL and the 4th international joint conference on natural language processing of the AFNLP (Vol. 1, pp. 405–413). Association for Computational Linguistics. Google Scholar

Copyright information

© Springer-Verlag London Limited 2012

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of RochesterRochesterUSA

Personalised recommendations