Multimodal Analysis of User-Generated Multimedia Content

  • Rajiv Shah
  • Roger Zimmermann

Part of the Socio-Affective Computing book series (SAC, volume 6)

Table of contents

  1. Front Matter
    Pages i-xxii
  2. Rajiv Shah, Roger Zimmermann
    Pages 1-30
  3. Rajiv Shah, Roger Zimmermann
    Pages 31-57
  4. Rajiv Shah, Roger Zimmermann
    Pages 59-99
  5. Rajiv Shah, Roger Zimmermann
    Pages 101-138
  6. Rajiv Shah, Roger Zimmermann
    Pages 139-171
  7. Rajiv Shah, Roger Zimmermann
    Pages 173-203
  8. Rajiv Shah, Roger Zimmermann
    Pages 205-234
  9. Rajiv Shah, Roger Zimmermann
    Pages 235-260
  10. Back Matter
    Pages 261-263

About this book


This book presents a study of semantics and sentics understanding derived from user-generated multimodal content (UGC). It enables researchers to learn about the ways multimodal analysis of UGC can augment semantics and sentics understanding and it helps in addressing several multimedia analytics problems from social media such as event detection and summarization, tag recommendation and ranking, soundtrack recommendation, lecture video segmentation, and news video uploading.

Readers will discover how the derived knowledge structures from multimodal information are beneficial for efficient multimedia search, retrieval, and recommendation. However, real-world UGC is complex, and extracting the semantics and sentics from only multimedia content is very difficult because suitable concepts may be exhibited in different representations. Moreover, due to the increasing popularity of social media websites and advancements in technology, it is now possible to collect a significant amount of important contextual information (e.g., spatial, temporal, and preferential information). Thus, there is a need to analyze the information of UGC from multiple modalities to address these problems.

A discussion of multimodal analysis is presented followed by studies on how multimodal information is exploited to address problems that have a significant impact on different areas of society (e.g., entertainment, education, and journalism). Specifically, the methods presented exploit the multimedia content (e.g., visual content) and associated contextual information (e.g., geo-, temporal, and other sensory data). The reader is introduced to several knowledge bases and fusion techniques to address these problems.

This work includes future directions for several interesting multimedia analytics problems that have the potential to significantly impact society. The work is aimed at researchers in the multimedia field who would like to pursue research in the area of multimodal analysis of UGC.


Multimodal Analysis Semantics and Sentics Analysis User-Gererated Content Social Media Multimedia

Authors and affiliations

  • Rajiv Shah
    • 1
  • Roger Zimmermann
    • 2
  1. 1.School of ComputingNational University of SingaporeSingaporeSingapore
  2. 2.School of ComputingNational University of SingaporeSingaporeSingapore

Bibliographic information

  • DOI
  • Copyright Information The Editor(s) (if applicable) and The Author(s) 2017
  • Publisher Name Springer, Cham
  • eBook Packages Biomedical and Life Sciences
  • Print ISBN 978-3-319-61806-7
  • Online ISBN 978-3-319-61807-4
  • Series Print ISSN 2509-5706
  • Series Online ISSN 2509-5714
  • Buy this book on publisher's site
Industry Sectors
Health & Hospitals