Skip to main content
SpringerLink
Log in

Context-Based Situation Recognition in Computer Vision Systems

  • Chapter
  • First Online:
Context-Enhanced Information Fusion

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

The availability of visual sensors and the increment of their processing capabilities have led to the development of a new generation of multi-camera systems. This increment has also conveyed new expectations and requirements that cannot be fulfilled by applying traditional fusion techniques. The ultimate objective of computer vision systems is to obtain a description of the observed scenario in terms that are both computable and human-readable, which can be seen as a specific form of situation assessment. Particularly, there is a great interest in human activity recognition in several areas such as surveillance and ambient intelligence. Simple activities can be recognized by applying pattern recognition algorithms on sensor data. However, identification of complex activities requires the development of cognitive capabilities close to human understanding. Several recent proposals combine numerical techniques and a symbolic model that represents context-dependent, background and common-sense knowledge relevant to the task. In this chapter the current challenges in the development of vision-based activity recognition systems are described, and how they can be tackled by exploiting formally represented context knowledge. Along with a review of the related literature, we describe an approach with examples in the areas of ambient intelligence and indoor security. The chapter surveys methods for context management in the literature that use symbolic knowledge models to represent and reason with context. Due to their relevance, we will pay special attention to ontology and logic-based models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    A blob is a set of pixels included in a connected image area. A track is a set of blobs corresponding to a scene object with associated properties: size, position, speed, color, etc. Generally, blobs and tracks are represented by their minimum enclosing rectangles.

  2. 2.

    http://opencv.org/.

  3. 3.

    http://vision.fe.uni-lj.si/cvbase06/.

  4. 4.

    http://www.racer-systems.com/.

  5. 5.

    http://www.cs.rochester.edu/~spark/muri/.

References

  1. C.S. Regazzoni, V. Ramesh, G.L. Foresti, Scanning the issue/technology, special issue on video communications, processing, and understanding for 3rd generation surveillance systems. Proc. IEEE, 89(10), 1355–1367 (2001)

    Google Scholar 

  2. M. Valera, S.A. Velastin, Intelligent distributed surveillance systems: a review. IEEE Proc. Vis. Image Sig. Proc. 152(2), 192–204 (2005)

    Article  Google Scholar 

  3. A. Luis Bustamante, J.M. Molina, M.A. Patricio, A practical approach for active camera coordination based on a fusion-driven multi-agent system. Int. J. Syst. Sci. 45(4), 741–755 (2014)

    Article  Google Scholar 

  4. K. Henricksen, A framework for context-aware pervasive computing applications, Ph.D. Thesis, University of Queensland, 2003

    Google Scholar 

  5. M. Kokar, M. Matheus, K. Baclawski, Ontology-based situation awareness. Inf. Fusion 10(1), 83–98 (2009)

    Article  Google Scholar 

  6. N.A. Bradley, M.D. Dunlop, Towards a multidisciplinary model of context to support context-aware computing. Hum. Comput. Inter. 20, 403–446 (2005)

    Article  Google Scholar 

  7. J. McCarthy, Notes on formalizing context, in Proceedings of the 3rd International Joint Conference on Artificial Intelligence (IJCAI’93) (Chambéry, France, 1993), pp. 555–562

    Google Scholar 

  8. T. Stang, C. Linnhoff-Popien, A context modeling survey, in 1st International Workshop on Advanced Context Modeling, Reasoning and Management, Nottingham, UK, 2004

    Google Scholar 

  9. M. Kandefer, S.C. Shapiro, A categorization of contextual constraints, in Biologically Inspired Cognitive Architectures—Papers from the AAAI Fall Symposium (Menlo Park, USA, 2008), pp. 88–93

    Google Scholar 

  10. J. Gómez-Romero, J. García, J. Kandefer, J. Llinas, J.M. Molina, M.A. Patricio, M. Prentice, S.C. Shapiro, Strategies and techniques for use and exploitation of contextual information in high-level fusion architectures, in Proceedings of the 13th Conference on Information Fusion (Fusion 2010), Edinburgh, UK, 2010

    Google Scholar 

  11. L. Snidaro, J. García, J. Llinas, Context-based information fusion: a survey and discussion. Inf. Fusion 25, 16–31 (2015)

    Article  Google Scholar 

  12. J. Gómez-Romero, J. García, M.A. Patricio, J.M. Molina, J. Llinas, High-level information fusion in visual sensor networks, in Visual Information Processing in Wireless Sensor Networks: Technology, Trends and Applications, eds. by L.-M. Ang, K.P. Seng (IGI Global, 2012), pp. 197–223

    Google Scholar 

  13. A.N. Steinberg, G. Rogova, Situation and context in data fusion and natural language understanding, in Proceedings of the 11th Conference on Information Fusion (Fusion 2008), Cologne, Germany, 2008

    Google Scholar 

  14. C.J. Matheus, M. Kokar, K. Baclawski, J. Letkowski, C. Call, M. Hinman, J. Salerno, D. Boulware, SAWA: an assistant for higher-level fusion and situation awareness, in Proceedings of the SPIE Conference on Multisensor, Multisource Information Fusion (Orlando, USA, 2005), pp. 75–85

    Google Scholar 

  15. B. Neumann, R. Möller, On scene interpretation with Description Logics. Imagine Vis. Comput. 26, 82–101 (2008)

    Article  Google Scholar 

  16. R.N. Carvalho, K.B. Laskey, P.C.G. Costa, PR-OWL 2.0—Bridging the gap to OWL semantics, in Uncertainty Reasoning for the Semantic Web II, ed. by F. Bobillo (Springer, Berlin, 2013), pp. 1–18

    Google Scholar 

  17. L. Snidaro, I. Visentini, K. Bryan, Fusing uncertain knowledge and evidence for maritime situational awareness via Markov logic networks. Inf. Fusion 21, 159–172 (2015)

    Article  Google Scholar 

  18. J. Gómez-Romero, M.A. Serrano, J. García, J.M. Molina, G. Rogova, Context-based multi-level information fusion for harbor surveillance. Inf. Fusion 21, 173–186 (2015)

    Article  Google Scholar 

  19. A.N. Steinberg, C.L. Bowman, Revisions to the JDL data fusion model, in Handbook of Multisensor Data Fusion, eds. by J. Llinas, M. Liggins, D. Hall (CRC Press, 2009), pp. 45–68

    Google Scholar 

  20. C.Y. Chong, S. Mori, K.C. Chang, Distributed multitarget multisensor tracking, in Multitarget-Multisensor Tracking: Advanced Applications, ed. by Y. Bar-Shalom, Vol. 1 (Artech House, 1990), pp. 247–295

    Google Scholar 

  21. K. Chang, C.Y. Chong, Y. Bar-Shalom, Joint probabilistic data association in distributed sensor networks. IEEE Trans. Autom. Control 31(10), 889–897 (1986)

    Article  MATH  Google Scholar 

  22. R. Olfati-Saber, Distributed Kalman filtering for sensor networks, in Proceedings of the 46th Conference in Decision and Control (New Orleans, USA, 2007), pp. 5492–5498

    Google Scholar 

  23. A. Yilmaz, O. Javed, M. Shah, Object tracking: a survey. ACM Comput. Surv. 38, 1–45 (2006)

    Article  Google Scholar 

  24. F. Castanedo, J. Gómez-Romero, M.A. Patricio, J. García, J.M., Molina, Distributed data and information fusion in visual sensor networks, in Distributed Data Fusion for network-centric operations Hall, eds. by D. Hall, M. Liggins, C.-Y. Chong, J. Llinas (CRC Press, 2012), pp. 437–467

    Google Scholar 

  25. J. García, M.A. Patricio, A. Berlanga, J.M. Molina, Fuzzy region assignment for visual tracking. Soft. Comput. 15(9), 1845–1864 (2011)

    Article  Google Scholar 

  26. I. Dotú, M.A. Patricio, A. Berlanga, J. García, J.M. Molina, Discrete optimization algorithms in real-time visual tracking. Appl. Artif. Intell. 23(9), 805–827 (2009)

    Article  Google Scholar 

  27. I. Dotú, M.A. Patricio, A. Berlanga, J. García, J.M. Molina, Boosting video tracking performance by means of Tabu search in intelligent visual surveillance systems. J. Heuristics 17(4), 415–440 (2011)

    Article  Google Scholar 

  28. A. Pinz, H. Bischof, W. Kropatsch, G. Schweighofer, Y. Haxhimusa, A. Opelt, A. Ion, Representations for cognitive vision. ELCVIA Electron. Lett. Comput. Vis. Image Anal. 7(2), 35–61 (2008)

    Google Scholar 

  29. I. Horrocks, P. Patel-Schneider, Reducing OWL entailment to description logic satisfiability. Web Seman. Sci. Serv. Agents World Wide Web 1(4), 345–357 (2004)

    Article  Google Scholar 

  30. B. Motik, U. Sattler, R. Studer, Query answering for OWL-DL with rules. Web Seman. Sci. Serv. Agents World Wide Web 3(1), 41–60 (2005)

    Article  Google Scholar 

  31. C. Elsenbroich, O. Kutz, U. Sattler, A case for abductive reasoning over ontologies, in Proceedings of the OWL Workshop: Experiences and Directions (OWLED ‘06), Athens, USA, 2006

    Google Scholar 

  32. V. Haarslev, K. Hidde, R. Möller, M. Wessel, The RacerPro knowledge representation and reasoning system. Semant. Web J. 3(3), 267–277 (2011)

    Google Scholar 

  33. J. Gómez-Romero, M.A. Patricio, J. García, J.M. Molina, Communication in distributed tracking systems: an ontology-based approach to improve cooperation. Expert Syst. 28(4), 288–305 (2011)

    Article  Google Scholar 

  34. M.A. Serrano, J. Gómez-Romero, M.A. Patricio, J. García, J.M. Molina, Applying the dynamic region connection calculus to exploit geographical knowledge in maritime surveillance, in Proceedings of the 15th International Conference on Information Fusion (Fusion 2012), Singapore, 2012

    Google Scholar 

  35. D. Doermann, D. Mihalcik, Tools and techniques for video performance evaluation, in Proceedings of the 15th International Conference on Pattern Recognition (ICPR ’00), Barcelona, Spain, 2000

    Google Scholar 

  36. M.A. Serrano, J. Gómez-Romero, M.A. Patricio, J. García, J.M. Molina, Topological properties in ontology-based applications, in 11th International Conference on Intelligent Systems Design and Applications (ISDA 2011) (Córdoba, Spain, 2011), pp. 1329–1334

    Google Scholar 

  37. J. Gómez-Romero, M.A. Serrano, M.A. Patricio, J. García, J.M. Molina, Context-based scene recognition from visual data in smart homes: an information fusion approach. Pers. Ubiquit. Comput. 16(7), 835–857 (2012)

    Article  Google Scholar 

  38. J. Gómez-Romero, M.A. Patricio, J. García, J.M. Molina, Ontology-based context representation and reasoning for object tracking and scene interpretation in video. Expert Syst. Appl. 38, 7494–7510 (2011)

    Article  Google Scholar 

  39. M.A. Serrano, J. Gómez-Romero, M.A. Patricio, J. García, J.M. Molina, Ontological representation of light wave camera data to support vision-based Aml. Sensors 12, 12126–12152 (2012)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by projects MINECO TEC2014-57022-C2-2-R, TEC2012-37832-C02-01 and TIN2012-30939.

Author information

Authors and Affiliations

  1. Department of Computer Science and A.I., CITIC-UGR, c/Periodista Rafael Gómez Montero, 2, 18071, Granada, Spain

    Juan Gómez-Romero

  2. Applied Artificial Intelligence Group, Universidad Carlos III de Madrid, c/Avenida Gregorio Peces-Barba Martínez, 22, 28270, Colmenarejo, Spain

    Jesús García, Miguel A. Patricio, Miguel A. Serrano & José M. Molina

Authors
  1. Juan Gómez-Romero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jesús García
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Miguel A. Patricio
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Miguel A. Serrano
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. José M. Molina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jesús García .

Editor information

Editors and Affiliations

  1. Department Mathematics and Computer Science, University of Udine, Udine, Italy

    Lauro Snidaro

  2. University Carlos III, Department of Computer Science and Engg. University Carlos III, Colmenarejo, Spain

    Jesús García

  3. Center Multisource Info. Fusion & Dept, Univ. at Buffalo, of Industrial and Engg Center Multisource Info. Fusion & Dept, Buffalo, New York, USA

    James Llinas

  4. Information Directorate, Air Force Research Laboratory Information Directorate, Rome, New York, USA

    Erik Blasch

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland (outside the USA)

About this chapter

Cite this chapter

Gómez-Romero, J., García, J., Patricio, M.A., Serrano, M.A., Molina, J.M. (2016). Context-Based Situation Recognition in Computer Vision Systems. In: Snidaro, L., García, J., Llinas, J., Blasch, E. (eds) Context-Enhanced Information Fusion. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-28971-7_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-28971-7_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-28969-4

  • Online ISBN: 978-3-319-28971-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation