Skip to main content
SpringerLink
Log in
Book cover

Visual Content Indexing and Retrieval with Psycho-Visual Models pp 125–144Cite as

Querying Multiple Simultaneous Video Streams with 3D Interest Maps

  • Chapter
  • First Online:

Part of the book series: Multimedia Systems and Applications ((MMSA))

Abstract

With proliferation of mobile devices equipped with cameras and video recording applications, it is now common to observe multiple mobile cameras filming the same scene at an event from a diverse set of view angles. These recorded videos provide a rich set of data for someone to re-experience the event at a later time. Not all the videos recorded, however, show a desirable view. Navigating through a large collection of videos to find a video with a better viewing angle can be time consuming. We propose a query-response interface in which users can intuitively switch to another video with an alternate, better, view, by selecting a 2D region within a video as a query. The system would then response with another video that has a better view of the selected region, maximizing the viewpoint entropy. The key to our system is a lightweight 3D scene structure, also termed 3D interest map. A 3D interest map is naturally an extension of saliency maps in the 3D space since most users film what they find interesting from their respective viewpoints. A user study with more than 35 users shows that our video query system achieves a suitable compromise between accuracy and run-time.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Calvet, L., Gurdjos, P., Charvillat, V.: Camera tracking using concentric circle markers: paradigms and algorithms. In: ICIP, pp. 1361–1364 (2012)

    Google Scholar 

  2. Carlier, A., Calvet, L., Nguyen, D.T.D., Ooi, W.T., Gurdjos, P., Charvillat, V.: 3D interest maps from simultaneous video recordings. In: Proceedings of the 22nd ACM International Conference on Multimedia, MM ’14, pp. 577–586 (2014)

    Google Scholar 

  3. Chandra, S., Chiu, P., Back, M.: Towards portable, high definition multi-camera video capture using smartphone for tele-immersion. In: IEEE International Symposium on Multimedia (2013)

    Google Scholar 

  4. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  5. Dezfuli, N., Huber, J., Olberding, S., Mühlhäuser, M.: CoStream: in-situ co-construction of shared experiences through mobile video sharing during live events. In: CHI Extended Abstracts, pp. 2477–2482 (2012)

    Google Scholar 

  6. Ercan, A.O., Yang, D.B., Gamal, A.E., Guibas, L.J.: Optimal placement and selection of camera network nodes for target localization. In: IEEE DCOSS, pp. 389–404 (2006)

    Google Scholar 

  7. Furukawa, Y., Ponce, J.: Accurate, dense, and robust multiview stereopsis. IEEE Trans. Pattern Anal. Mach. Intell. 32(8), 1362–1376 (2010)

    Article  Google Scholar 

  8. Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004). ISBN: 0521540518

    Book  MATH  Google Scholar 

  9. Kansal, A., Zhao, F.: Location and mobility in a sensor network of mobile phones. In: ACM Network and Operating Systems Support for Digital Audio and Video (2007)

    Google Scholar 

  10. Ke, Y., Hoiem, D., Sukthankar, R.: Computer vision for music identification. In: CVPR (1), pp. 597–604 (2005)

    Google Scholar 

  11. Kim, J.S., Gurdjos, P., Kweon, I.S.: Geometric and algebraic constraints of projected concentric circles and their applications to camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 637–642 (2005)

    Article  Google Scholar 

  12. Lee, H., Tessens, L., Morbée, M., Aghajan, H.K., Philips, W.: Sub-optimal camera selection in practical vision networks through shape approximation. In: ACIVS, pp. 266–277 (2008)

    Google Scholar 

  13. Oliensis, J., Hartley, R.: Iterative extensions of the sturm/triggs algorithm: convergence and nonconvergence. IEEE Trans. Pattern Anal. Mach. Intell. 29(12), 2217–2233 (2007)

    Article  Google Scholar 

  14. Philip Kelly, C.O.C., Kim, C., O’Connor, N.E.: Automatic camera selection for activity monitoring in a multi-camera system for tennis. In: ACM/IEEE International Conference on Distributed Smart Cameras (2009)

    Google Scholar 

  15. Pollefeys, M., Gool, L.J.V., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. Int. J. Comput. Vis. 59(3), 207–232 (2004)

    Article  Google Scholar 

  16. Saini, M.K., Gadde, R., Yan, S., Ooi, W.T.: MoViMash: online mobile video mashup. In: ACM Multimedia, pp. 139–148 (2012)

    Google Scholar 

  17. Saini, M., Venkatagiri, S.P., Ooi, W.T., Chan, M.C.: The Jiku mobile video dataset. In: Proceedings of the Fourth Annual ACM SIGMM Conference on Multimedia Systems, MMSys, Oslo (2013)

    Book  Google Scholar 

  18. Shrestha, P., Barbieri, M., Weda, H., Sekulovski, D.: Synchronization of multiple camera videos using audio-visual features. IEEE Trans. Multimedia 12(1), 79–92 (2010)

    Article  Google Scholar 

  19. Shrestha, P., de With, P.H.N., Weda, H., Barbieri, M., Aarts, E.H.L.: Automatic mashup generation from multiple-camera concert recordings. In: ACM Multimedia, pp. 541–550 (2010)

    Google Scholar 

  20. Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: exploring photo collections in 3D. ACM Trans. Graph. 25(3), 835–846 (2006)

    Article  Google Scholar 

  21. Sturm, P.F.: Algorithms for plane-based pose estimation. In: CVPR, pp. 1706–1711 (2000)

    Google Scholar 

  22. Tanskanen, P., Kolev, K., Meier, L., Camposeco, F., Saurer, O., Pollefeys, M.: Live metric 3D reconstruction on mobile phones. In: ICCV (2013)

    Book  Google Scholar 

  23. Tatler, B.W.: The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. J. Vis. 7(14), 1–17 (2007)

    Article  Google Scholar 

  24. Tessens, L., Morbée, M., Lee, H., Philips, W., Aghajan, H.K.: Principal view determination for camera selection in distributed smart camera networks. In: IEEE ICDSC, pp. 1–10 (2008)

    Google Scholar 

  25. Vázquez, P.P., Feixas, M., Sbert, M., Heidrich, W.: Viewpoint selection using viewpoint entropy. In: Proceedings of VMV 2001, pp. 273–280 (2001)

    Google Scholar 

  26. Wang, C., Shen, H.W.: Information theory in scientific visualization. Entropy 13(1), 254–273 (2011)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IRIT, UMR 5505, Université Toulouse, 2 rue Charles Camichel, 31071, Toulouse, France

    Axel Carlier, Pierre Gurdjos & Vincent Charvillat

  2. Simula Research Laboratory, Martin Linges vei 25, 1364, Fornebu, Norway

    Lilian Calvet

  3. School of Computing, National University of Singapore, Lower Kent Ridge Road, 119077, Singapore

    Wei Tsang Ooi

Authors
  1. Axel Carlier
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lilian Calvet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pierre Gurdjos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vincent Charvillat
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Tsang Ooi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Axel Carlier .

Editor information

Editors and Affiliations

  1. LaBRI UMR 5800, Univ. Bordeaux, CNRS, Bordeaux INP, Univ. Bordeaux, Talence, France

    Jenny Benois-Pineau

  2. LS2N, UMR CNRS 6004, Université de Nantes, Nantes Cedex 3, France

    Patrick Le Callet

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Carlier, A., Calvet, L., Gurdjos, P., Charvillat, V., Ooi, W.T. (2017). Querying Multiple Simultaneous Video Streams with 3D Interest Maps. In: Benois-Pineau, J., Le Callet, P. (eds) Visual Content Indexing and Retrieval with Psycho-Visual Models. Multimedia Systems and Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-57687-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-57687-9_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-57686-2

  • Online ISBN: 978-3-319-57687-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation