Surface tracking assessment and interaction in texture space
In this paper, we present a novel approach for assessing and interacting with surface tracking algorithms targeting video manipulation in post-production. As tracking inaccuracies are unavoidable, we enable the user to provide small hints to the algorithms instead of correcting erroneous results afterwards. Based on 2D mesh warp-based optical flow estimation, we visualize results and provide tools for user feedback in a consistent reference system, texture space. In this space, accurate tracking results are reflected by static appearance, and errors can easily be spotted as apparent change. A variety of established tools can be utilized to visualize and assess the change between frames. User interaction to improve tracking results becomes more intuitive in texture space, as it can focus on a small region rather than a moving object. We show how established tools can be implemented for interaction in texture space to provide a more intuitive interface allowing more effective and accurate user feedback.
Keywordssurface tracking assessment interaction mesh warp optical flow
This work was partially funded by the German Science Foundation (Grant No. DFG EI524/2-1) and by the European Commission (Grant Nos. FP7-288238 SCENE and H2020-644629 AutoPost).
- Imagineer Systems. mocha Pro. 2016. Available at http://www.imagineersystems.com/products/mochapro.Google Scholar
- Foundry. NUKE. 2016. Available at https://www. foundry.com/products/nuke.Google Scholar
- The Pixelfarm. PFTrack. 2016. Available at http://www.thepixelfarm.co.uk/pftrack/.Google Scholar
- Klose, F.; Ruhl, K.; Lipski, C.; Magnor, M. Flowlab—An interactive tool for editing dense image correspondences. In: Proceedings of the Conference for Visual Media Production, 59–66, 2011.Google Scholar
- Zhang, C.; Price, B.; Cohen, S.; Yang, R. Highquality stereo video matching via user interaction and space–time propagation. In: Proceedings of the International Conference on 3D Vision, 71–78, 2013.Google Scholar
- Re:Vision Effects. Twixtor. 2016. Available at http://revisionfx.com/products/twixtor/.Google Scholar
- Wilkes, L. The role of ocula in stereo post production. Technical Report. The Foundry, 2009.Google Scholar
- Chrysos, G. G.; Antonakos, E.; Zafeiriou, S.; Snape, P. Offline deformable face tracking in arbitrary videos. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 1–9, 2015.Google Scholar
- Rother, C.; Kolmogorov, V.; Blake, A. “GrabCut”: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics Vol. 23, No. 3, 309–314, 2004.Google Scholar
- Doron, Y.; Campbell, N. D. F.; Starck, J.; Kautz, J. User directed multi-view-stereo. In: Computer Vision–ACCV 2014 Workshops. Jawahar, C.; Shan, S. Eds. Springer Cham, 299–313, 2014.Google Scholar
- Bartoli, A.; Zisserman, A. Direct estimation of nonrigid registrations. In: Proceedings of the 15th British Machine Vision Conference, Vol. 2, 899–908, 2004.Google Scholar
- Zhu, J.; Van Gool, L.; Hoi, S. C. H. Unsupervised face alignment by robust nonrigid mapping. In: Proceedings of the IEEE 12th International Conference on Computer Vision, 1265–1272, 2009.Google Scholar
- Hilsmann, A.; Eisert, P. Joint estimation of deformable motion and photometric parameters in single view videos. In: Proceedings of the IEEE 12th International Conference on Computer Vision Workshops, 390–397, 2009.Google Scholar
- Seibold, C.; Hilsmann, A.; Eisert, P. Model-based motion blur estimation for the improvement of motion tracking. Computer Vision and Image Understanding DOI: 10.1016/j.cviu.2017.03.005, 2017.Google Scholar
- Hilsmann, A.; Schneider, D. C.; Eisert, P. Image-based tracking of deformable surfaces. In: Object Tracking. InTech, 245–266, 2011.Google Scholar
- Pilet, J.; Lepetit, V.; Fua, P. Real-time nonrigid surface detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 822–828, 2005.Google Scholar
- Wedel, A.; Cremers, D.; Pock, T.; Bischof, H. Structure-and motion-adaptive regularization for high accuracy optic flow. In: Proceedings of the IEEE 12th International Conference on Computer Vision, 1663–1668, 2009.Google Scholar
- Hollywood Camera Work. Face Capture dataset. 2016. Available at https://www.hollywoodcamerawork.com/tracking-plates.html.Google Scholar
Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.