Abstract
It is often the case that related pieces of information lie in adjacent but different types of data sources. Besides extracting such information from each particular type of source, an important issue raised is how to put together all the pieces of information extracted by each source, or, more generally, what is the optimal way to collectively extract information, considering all media sources together. This chapter presents a machine learning method for extracting complex semantics stemming from multimedia sources. The method is based on transforming the inference problem into a graph expansion problem, expressing graph expansion operators as a combination of elementary ones and optimally seeking elementary graph operators. The latter issue is then reduced to learn a set of soft classifiers, based on features each one corresponding to a unique graph path. The advantages of the method are demonstrated on an athletics web-pages corpus, comprising images and text.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
P. Aarabi and B.V. Dasarathy. Robust speech processing using multi-sensor multi-source information fusion – an overview of the state of the art. Information Fusion, 5(2):77–80, 2004.
A. Goshtasby and S. Nikolov. Image fusion: Advances in the state of the art. Information Fusion, 8(2):114–118, 2007.
N. Friedman, L. Getoor, D. Koller, and A. Pfeffer. Learning probabilistic relational models. In International Joint Conference on Artificial Intelligence, volume 16, pages 1300–1309. Citeseer, 1999.
K. Kersting, L. De Raedt, and T. Raiko. Logical hidden markov models. Journal of Artificial Intelligence Research, 25(1):425–456, 2006.
B.D. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. In International joint conference on artificial intelligence, volume 3, pages 674–679. Citeseer, 1981.
T. Lukasiewicz. Probabilistic description logic programs. International Journal of Approximate Reasoning, 45(2):288–307, 2007.
A.V. Nefian, L. Liang, X. Pi, X. Liu, and K. Murphy. Dynamic Bayesian Networks for Audio-Visual Speech Recognition. EURASIP Journal on Applied Signal Processing, 2002(11): 1274–1288, 2002.
J. Neville and D. Jensen. Relational dependency networks. The Journal of Machine Learning Research, 8:692, 2007.
S.E. Peraldi, A. Kaya, S. Melzer, R. Moller, and M. Wessel. Multimedia interpretation as abduction. In Proc. DL-2007: International Workshop on Description Logics, 2007.
S. Petridis and N. Tsapatsoulis. Semantics Extraction from Multimedia Content: The BOEMIE Architecture. In Proceeding of the first international conference on Semantics and digital Media Technology (SAMT 2006), pages 6–8, 2006.
M. Richardson and P. Domingos. Markov logic networks. Machine Learning, 62(1):107–136, 2006.
S. Rudolph, T. Tserendorj, and P. Hitzler. What Is Approximate Reasoning? In Proceedings of the 2nd International Conference on Web Reasoning and Rule Systems, pages 150–164. Springer, 2008.
U. Straccia. Reasoning within Fuzzy Description Logics. Journal of Artificial Intelligence Research, 14:137–166, 2001.
E. Zavitsanos, G. Paliouras, G.A. Vouros, and S. Petridis. Discovering subsumption hierarchies of ontology concepts from text corpora. In Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence, pages 402–408. IEEE Computer Society Washington, DC, USA, 2007.
Acknowledgements
This study is partly supported by the research projects “BOEMIE, Bootstrapping Ontology Evolution with Multimedia Information Extraction”. FP6-027538/STREP, 2006–2009, http://www.boemie.org and “CASAM, Computer-Aided Semantic Annotation of Multimedia”. ICT-217061/STREP, 2008, http://www.casam-project.eu/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Petridis, S., Perantonis, S.J. (2011). Semantics Extraction From Multimedia Data: An Ontology-Based Machine Learning Approach. In: Cutsuridis, V., Hussain, A., Taylor, J. (eds) Perception-Action Cycle. Springer Series in Cognitive and Neural Systems. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-1452-1_12
Download citation
DOI: https://doi.org/10.1007/978-1-4419-1452-1_12
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4419-1451-4
Online ISBN: 978-1-4419-1452-1
eBook Packages: Biomedical and Life SciencesBiomedical and Life Sciences (R0)