Advertisement

Multimedia Tools and Applications

, Volume 61, Issue 3, pp 675–696 | Cite as

Discrimination of media moments and media intervals: sticker-based watch-and-comment annotation

  • Cesar A. C. Teixeira
  • Erick L. Melo
  • Giliard B. Freitas
  • Celso A. S. Santos
  • Maria da Graça C. Pimentel
Article

Abstract

In this paper we discuss the problem of how to discriminate moments of interest on videos or live broadcast shows. The primary contribution is a system which allows users to personalize their programs with previously created media stickers—pieces of content that may be temporarily attached to the original video. We present the system’s architecture and implementation, which offer users operators to transparently annotate videos while watching them. We offered a soccer fan the opportunity to add stickers to the video while watching a live match: the user reported both enjoying and being comfortable using the stickers during the match—relevant results even though the experience was not fully representative.

Keywords

Interactive multimedia Interactive video Digital TV Sticker TV NCL Authoring Annotation Watch-and-Comment Interactive TV 

Notes

Acknowledgements

We thank the several agencies which provide funds to our research: FAPESP, CAPES, CNPq, FINEP, and MCT/CTIC. We also thank Rudinei Goularte, Renan G. Cattelan, Luiz F. G. Soares, Dick C. A. Bulterman and Pablo S. César for many suggestions and challenging questions. We also thank the anonymous reviewers for their suggestions and advice.

References

  1. 1.
    ABNT-2007: ABNT NBR 15606-2 Associação Brasileira de Normas Técnicas. Digital Terrestrial Television Standard 06: Data Codification and Transmission Specifications for Digital Broadcasting, Part 2 - GINGA-NCL: XML Application Language for Application Coding. http://www.abnt.org.br/imagens/Normalizacao_TV_Digital/ABNTNBR15606-2_2007Ing_2008.pdf (version 2008)
  2. 2.
    Allen JF, Hayes PJ (1990) Moments and points in an interval-based temporal logic. Comput Intell 5(4):225–238. doi: 10.1111/j.1467-8640.1989.tb00329.x Google Scholar
  3. 3.
    Athanasiadis E, Mitropoulos S (2010) A distributed platform for personalized advertising in digital interactive TV environments. J Syst Softw 83:1453–1469. doi: 10.1016/j.jss.2010.02.040 CrossRefGoogle Scholar
  4. 4.
    Blanco-Fernández Y, Arias JJP, Gil-Solla A, Cabrer MR, Nores ML, Duque JG, Vilas AF, Redondo RPD, Muñoz JB (2008) An MHP framework to provide intelligent personalized recommendations about digital TV contents. Softw Pract Exp 38(9):925–960CrossRefGoogle Scholar
  5. 5.
    Blanco-Fernández Y, Pazos-Arias JJ, Gil-Solla A, Ramos-Cabrer M, López-Nores M (2008) Zaptv: Personalized user-generated content for handheld devices in dvb-h mobile networks. In: Proceedings of the European conference on interactive television (EUROITV’08) (LNCS 5866). Springer, Berlin, pp 193–203. doi: 10.1007/978-3-540-69478-6_26 Google Scholar
  6. 6.
    Bulterman DCA, Jansen AJ, César P, Mullender S, Hyche E, DeMeglio M, Quint J, Kawamura H, Weck D, Pañeda XG, Melendi D, Cruz-Lara S, Hanclik M, Zucker DF, Michel T (2008) Synchronized Multimedia Integration Language (SMIL 3.0). http://www.w3.org/TR/SMIL3/
  7. 7.
    Cattelan RG, Santos FS, Goularte R, Teixeira CAC, Pimentel MdGC (2008) Watch-and-comment as a paradigm toward ubiquitous interactive video editing. ACM Trans Multimed Comput Comm Appl (ACM TOMCCAP) 4(4):1–24CrossRefGoogle Scholar
  8. 8.
    César P, Bulterman DCA, Jansen AJ (2006) The ambulant annotator: empowering viewer-side enrichment of multimedia content. In: Proceedings of the ACM symposium on document engineering (DOCENG’06). ACM, New York, pp 186–187 doi:10.1145/1166160.1166209 CrossRefGoogle Scholar
  9. 9.
    César P, Bulterman DCA, Jansen AJ (2008) Usages of the secondary screen in an interactive television environment: Control, enrich, share, and transfer television content. In: Proceedings of the European conference on interactive television (EUROITV’08) (LNCS 5866), pp 168–177Google Scholar
  10. 10.
    César P, Bulterman DCA, Jansen AJ (2009) Leveraging user impact: an architecture for secondary screens usage in interactive television. Multimedia Syst 15(3):127–142CrossRefGoogle Scholar
  11. 11.
    César P, Bulterman DCA, Jansen AJ, Geerts D, Knoche H, Seager W (2009) Fragment, tag, enrich, and send: Enhancing social sharing of video. ACM Trans Multimed Comput Comm Appl (ACM TOMCCAP) 5(19):1–19:27. doi:10.1145/1556134.1556136 CrossRefGoogle Scholar
  12. 12.
    Chi MC, Yeh CH, Chen MJ (2009) Robust region-of-interest determination based on user attention model through visual rhythm analysis. IEEE Trans Circuits Syst Video Technol 19:1025–1038. doi: 10.1109/TCSVT.2009.2022822 CrossRefGoogle Scholar
  13. 13.
    Coppens T, Trappeniers L, Godon M (2004) AmigoTV: Towards a social TV experience. In: Proceedings of the European conference on interactive television (EUROITV’04)Google Scholar
  14. 14.
    Costa M, Correia N, Guimarães N (2002) Annotations as multiple perspectives of video content. In: Proceedings of the ACM international conference on multimedia (MULTIMEDIA’02), pp 283–286. doi:10.1145/641007.641065
  15. 15.
    Costa RMdR, Moreno MF, Rodrigues RF, Soares LFG (2006) Live editing of hypermedia documents. In: Proceedings of the ACM symposium on document engineering (DOCENG’06). ACM, New York, pp 165–172. doi:10.1145/1166160.1166202 CrossRefGoogle Scholar
  16. 16.
    de Ávila PM, Zorzo SD (2009) Recommender TV - A Personalized TV Guide System Compliant with Ginga. In: Proceedings of the international conference on security and cryptography (SIGMAP’09), part of the international joint conference on e-business and telecommunications (ICETE’09), pp 149–156Google Scholar
  17. 17.
    de Freitas GB, Teixeira CAC (2009) Ubiquitous services in home networks offered through digital TV. In: Proceedings of the ACM symposium on applied computing (SAC’09). ACM, New York, pp 1834–1838. doi:10.1145/1529282.1529691 CrossRefGoogle Scholar
  18. 18.
    De Lucena VF, Filho JEC, Viana NS, Maia O (2009) A home automation proposal built on the ginga digital TV middleware and the OSGi framework. IEEE Trans Consum Electron 55(3):1254–1262CrossRefGoogle Scholar
  19. 19.
    Deigmoeller J, Itagaki T, Stoll G, Just N (2010) An approach to intelligently crop and scale video for broadcast applications. In: Proceedings of the ACM symposium on applied computing (SAC’10). ACM, New York, pp 1911–1918. doi:10.1145/1774088.1774493 Google Scholar
  20. 20.
    Di Massa R, Montagnuolo M, Messina A (2010) Implicit news recommendation based on user interest models and multimodal content analysis. In: Proceedings of the international workshop on automated information extraction in media production (AIEMPro’10). ACM, New York, pp 33–38.doi:10.1145/1877850.1877861 Google Scholar
  21. 21.
    Dimitrova N, Janevski A, Li D, Zimmerman J (2003) Who’s that actor?: the Infosip TV agent. In: Proceedings of the ACM SIGMM workshop on experiential telepresence (ETP’03). ACM, New York, pp 76–79. doi:10.1145/982484.982499 CrossRefGoogle Scholar
  22. 22.
    Drucker SM, Glatzer A, De Mar S, Wong C (2002) Smartskip: consumer level browsing and skipping of digital video content. In: Proceedings of the ACM conference on human factors in computing systems (CHI’02). ACM, New York, pp 219–226. doi:10.1145/503376.503416 Google Scholar
  23. 23.
    Furini M, Geraci F, Montangero M, Pellegrini M (2010) Stimo: Still and moving video storyboard for the web scenario. Multimedia Tools and Applications 46:47–69. doi: 10.1007/s11042-009-0307-7 CrossRefGoogle Scholar
  24. 24.
    Gao Y, Dai QH (2008) Clip based video summarization and ranking. In: Proceedings of the international conference on content-based image and video retrieval (CIVR’08). ACM, New York, pp 135–140. doi:10.1145/1386352.1386375 CrossRefGoogle Scholar
  25. 25.
    Goularte R, Cattelan RG, Camacho-Guerrero JA, Inácio VR Jr, Pimentel MdGC (2004) Interactive multimedia annotations: enriching and extending content. In: Proceedings of the ACM symposium on document engineering (DOCENG’04), pp 84–86. doi:10.1145/1030397.1030414
  26. 26.
    Hickson I (2011) HTML5 a vocabulary and associated APIs for HTML and XHTML—W3C Working Draft 13 January 2011. http://www.w3.org/TR/html5/
  27. 27.
    Hölbling G, Rabl T, Coquil D, Kosch H (2008) Interactive TV services on mobile devices. IEEE Multimed 15(2):72–76. doi: 10.1109/MMUL.2008.34 CrossRefGoogle Scholar
  28. 28.
    Hsu SH, Wen MH, Lin HC, Lee CC, Lee CH (2007) AIMED–a personalized TV recommendation system. In: Proceedings of the European conference on interactive television (EUROITV’07) (LNCS 4417), pp 166–174Google Scholar
  29. 29.
    Huang EM, Harboe G, Tullio J, Novak A, Massey N, Metcalf CJ, Romano G (2009) Of social television comes home: a field study of communication choices and practices in TV-based text and voice chat. In: Proceedings of the ACM international conference on human factors in computing systems (CHI’09), pp 585–594Google Scholar
  30. 30.
    Huet B, Jiten J, Merialdo B (2005) Personalization of hyperlinked video in interactive television. In: Proceedings of the IEEE conference on multimedia and expo, p 4. doi:10.1109/ICME.2005.1521460
  31. 31.
    ISO (1996) Information technology—generic coding of moving pictures and associated audio information: digital storage media command & control. ISO/IEC 13818-6Google Scholar
  32. 32.
    Jansen J, César P, Bulterman DC (2010) A model for editing operations on active temporal multimedia documents. In: Proceedings of the ACM symposium on document engineering (DOCENG’10). ACM, New York, pp 87–96. doi:10.1145/1860559.1860579 Google Scholar
  33. 33.
    Laiola Guimarães R, César P, Bulterman DC (2010) Creating and sharing personalized time-based annotations of videos on the web. In: Proceedings of the ACM symposium on document engineering (DOCENG’10). ACM, New York, pp 27–36. doi:10.1145/1860559.1860567 Google Scholar
  34. 34.
    Liu Z, Gibbon DC, Drucker H, Basso A (2008) Content personalization and adaptation for three-screen services. In: Proceedings of the international conference on content-based image and video retrieval (CIVR ’08). ACM, New York, pp 635–644. doi:10.1145/1386352.1386444 CrossRefGoogle Scholar
  35. 35.
    Liu Z, Zavesky E, Shahraray B, Gibbon D, Basso A (2008) Brief and high-interest video summary generation: evaluating the at&t labs rushes summarizations. In: Proceedings of the ACM TRECVid video summarization workshop (TVS’08). ACM, New York, pp 21–25. doi:10.1145/1463563.1463565 CrossRefGoogle Scholar
  36. 36.
    Lynn SG, Olsen DR Jr, Partridge BG (2009) Time warp football. In: Proceedings of the European conference on interactive television (EUROITV’09). ACM, New York, pp 77–86. doi:10.1145/1542084.1542098 CrossRefGoogle Scholar
  37. 37.
    Miyahara M, Aoki M, Takiguchi T, Ariki Y (2008) Tagging video contents with positive/negative interest based on user’s facial expression. In: Satoh S, Nack F, Etoh M (eds) Advances in multimedia modeling (LNCS 4903). Springer, Berlin, pp 210–219CrossRefGoogle Scholar
  38. 38.
    Motti VG, Fagá R Jr, Catellan RG, Pimentel MdGC, Teixeira CA (2009) Collaborative synchronous video annotation via the watch-and-comment paradigm. In: Proceedings of the European Conference on Interactive Television (EUROITV’09). ACM, New York, pp 67–76. doi:10.1145/1542084.1542097 CrossRefGoogle Scholar
  39. 39.
    Nathan M, Harrison C, Yarosh S, Terveen LG, Stead L, Amento B (2008) CollaboraTV: making television viewing social again. In: Proceedings of the international conference on designing interactive user experiences for TV and video (UXTV’08). ACM, New York, pp 85–94CrossRefGoogle Scholar
  40. 40.
    Nitta N, Takahashi Y, Babaguchi N (2009) Automatic personalized video abstraction for sports videos using metadata. Multimedia Tools and Applications 41(1):1–25CrossRefGoogle Scholar
  41. 41.
    Patel M, Gossweiler R, Sahami M, Blackburn J, Brown D, Knight A (2008) Google TV search: dual-wielding search and discovery in a large-scale product. In: Proceedings of the international conference on designing interactive user experiences for TV and video (UXTV’08). ACM, New York, pp 95–104. doi:10.1145/1453805.1453826 CrossRefGoogle Scholar
  42. 42.
    Petersen MK, Butkus A (2008) Modeling emotional context from latent semantics. In: Proceedings of the international conference on designing interactive user experiences for TV and video (UXTV’08). ACM, New York, pp 63–66. doi:10.1145/1453805.1453819 CrossRefGoogle Scholar
  43. 43.
    Pimentel MdGC, Cattelan RG, Melo EL, Teixeira CA (2008) End-user editing of interactive multimedia documents. In: Proceedings of the ACM symposium on document engineering (DOCENG’08). ACM, New York, pp 298–301. doi:10.1145/1410140.1410204 Google Scholar
  44. 44.
    Pimentel MdGC, Cattelan RG, Freitas G, Melo EL, Teixeira CAC (2009) Watch-and-comment as an approach to collaborative annotate points of interest in video and interactive-TV programs. In: Marcus A, Roibás AC, Sala R (eds) Mobile TV: customizing content and experience. Springer, Berlin, pp 349–366Google Scholar
  45. 45.
    Pimentel MdGC, Cattelan RG, Melo EL, Prado AF, Teixeira CAC (2010) End-user live editing of iTV programmes. Int J Adv Media Comm 4(1):78–103. doi: 10.1504/IJAMC.2010.030007 CrossRefGoogle Scholar
  46. 46.
    Ramos G, Balakrishnan R (2003) Fluid interaction techniques for the control and annotation of digital video. In: Proceedings of the ACM symposium on user interface software and technology (UIST’03), pp 105–114. doi:10.1145/964696.964708
  47. 47.
    Rogge B, Bekaert J, de Walle R (2004) Timing issues in multimedia formats: review of the principles and comparison of existing formats. IEEE Trans Multimedia 6(6):910–924CrossRefGoogle Scholar
  48. 48.
    Schulzrinne H, Rao A, Lanphier R (1998) Real Time Streaming Protocol (RTSP). http://www.ietf.org/rfc/rfc2326.txt
  49. 49.
    Soares LFG, de Souza Filho (2007) Interactive television in Brazil: system software and the digital divide. In: Proceedings of the European conference on interactive television (EUROITV’07) (LNCS 4417). Springer, Berlin, pp 41–44Google Scholar
  50. 50.
    Soares LFG, Rodrigues RF, Moreno MF (2007) Ginga-NCL: the declarative environment of the Brazilian digital TV system. J Braz Comput Soc 12(4):37–46CrossRefGoogle Scholar
  51. 51.
    Soares LF, Rodrigues RF, Cerqueira R, Barbosa SD (2010) Variable and state handling in NCL. Multimedia Tools and Applications 50:465–489. doi: 10.1007/s11042-010-0478-2 CrossRefGoogle Scholar
  52. 52.
    Teixeira CAC, Freitas G, Pimentel MdGC (2010) Distributed discrimination of media moments and media intervals: a watch-and-comment approach. In: Proceedings of the ACM symposium on applied computing (SAC’10), pp 1929–1935Google Scholar
  53. 53.
    Tjondronegoro D, Chen YPP, Pham B (2004) Integrating highlights for more complete sports video summarization. IEEE MultiMed 11:22–37. doi: 10.1109/MMUL.2004.28 CrossRefGoogle Scholar
  54. 54.
    Troncy R, Mannens E, Pfeiffer S, Deursen DV (2010) Media fragments URI 1.0—W3C working draft, 8 December 2010. http://www.w3.org/2008/WebVideo/Fragments/WD-media-fragments-spec/
  55. 55.
    Vandermolen H, Wiegering A, Sommers M, Levine SJ, Franco A (2009) Identifying events of interest within video content. http://www.freepatentsonline.com/7624416.html US Patent(7624416)
  56. 56.
    Vuorimaa P, Bulterman D, César, P (2008) SMIL timesheets 1.0—W3C working draft, 10 January 2008. http://www.w3.org/TR/2008/WD-timesheets-20080110/
  57. 57.
    Wei Y, Bhandarkar SM, Li K (2009) Client-centered multimedia content adaptation. ACM Trans Multimed Comput Comm Appl (ACM TOMCCAP) 5(22):1–22:26. doi:10.1145/1556134.1556139 CrossRefGoogle Scholar
  58. 58.
    Xie L, Sundaram H, Campbell M (2008) Event mining in multimedia streams. Proc IEEE 96: 623–647CrossRefGoogle Scholar
  59. 59.
    Yamamoto M, Nitta N, Babaguchi N (2006) Estimating intervals of interest during TV viewing for automatic personal preference acquisition. In: Zhuang Y, Yang S, Rui Y, He Q (eds) Advances in multimedia information processing (PCM’06) (LNCS 4261). Springer, Berlin, pp 615–623CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Cesar A. C. Teixeira
    • 1
  • Erick L. Melo
    • 1
  • Giliard B. Freitas
    • 1
  • Celso A. S. Santos
    • 2
  • Maria da Graça C. Pimentel
    • 3
  1. 1.Departamento de ComputaçãoUniversidade Federal de São CarlosSão CarlosBrazil
  2. 2.Departamento de Ciências de ComputaçãoUniversidade Federal da BahiaSalvadorBrazil
  3. 3.Departamento de Ciências de Computação, Instituto de Ciências Matemáticas e de ComputaçãoUniversidade de São PauloSão PauloBrazil

Personalised recommendations