Live Mixed Reality Video Production for Educational Presentation

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10850)


By using Microsoft Kinect, virtual objects could be fused with presenter in a real-time video for educational purpose. Unfortunately Kinect’s video quality is not good also it could not overcome the issues of occlusion. In this paper, real-time high quality RGB video from a professional video camera was calibrated with the depth image from Kinect to achieve real-virtual fusion, and a hand shape blue-color mask was attach to the presenter’s hand to realize high quality occlusion between virtual objects and real presenter.


AR TV High-resolution camera Kinect 


  1. 1.
    Bouguet, J.-Y.: Camera Calibration Toolbox for Matlab.
  2. 2.
    Kim, C., Yun, S., Jung, S.-W., Won, C.S.: Color and depth image correspondence for kinect v2. In: Park, J., Chao, H.-C., Arabnia, H., Yen, N.Y. (eds.) Advanced Multimedia and Ubiquitous Engineering. LNEE, vol. 352, pp. 111–116. Springer, Heidelberg (2015). Scholar
  3. 3.
    Zeller, N., Quint, F., Guan, L.: Kinect based 3D scene reconstruction. In: 22nd International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, WSCG 2014-in co-operation with EUROGRAPHICS Association, 2 June 2014–5 June 2014, pp. 73–81 (2015)Google Scholar
  4. 4.
    Yang, R.S., Chan, Y.H., Gong, R., Nguyen, M., Strozzi, A.G., Delmas, P., Gimel’farb, G., Ababou, R.: Multi-Kinect scene reconstruction: calibration and depth inconsistencies. In: 2013 28th International Conference on Image and Vision Computing New Zealand, IVCNZ 2013, 27 November 2013–29 November 2013, pp. 47–52 (2013)Google Scholar
  5. 5.
    Shibo, L., Qing, Z.: A new approach to calibrate range image and color image from kinect. In: 2012 4th International Conference on Intelligent Human-Machine Systems and Cybernetics, IHMSC 2012, 26 August 2012–27 August 2012, pp. 252–255 (2012)Google Scholar
  6. 6.
    Landau, M.J., Choo, B.Y., Beling, P.A.: Simulating Kinect Infrared and Depth Images (2015)Google Scholar
  7. 7.
    Zhang, Z.Y.: Microsoft kinect sensor and its effect. IEEE Multimedia 19, 4–10 (2012)CrossRefGoogle Scholar
  8. 8.
    Khoshelham, K., Elberink, S.O.: Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12, 1437–1454 (2012)CrossRefGoogle Scholar
  9. 9.
    Han, J., Shao, L., Xu, D., Shotton, J.: Enhanced computer vision with microsoft kinect sensor: a review. IEEE Trans. Cybern. 43, 1318–1334 (2013)CrossRefGoogle Scholar
  10. 10.
    Herrera, D.C., Kannala, J., Heikkila, J.: Joint depth and color camera calibration with distortion correction. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2058–2064 (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Communication of Science and TechnologyUniversity of Science and Technology of ChinaHefeiChina

Personalised recommendations