Abstract
Automatic video editing involving at least the steps of selecting the most valuable footage from points of view of visual quality and the importance of action filmed; and cutting the footage into a brief and coherent visual story that would be interesting to watch is implemented in a purely data-driven manner. Visual semantic and aesthetic features are extracted by the ImageNet-trained convolutional neural network, and the editing controller is trained by an imitation learning algorithm. As a result, at test time the controller shows the signs of observing basic cinematography editing rules learned from the corpus of motion pictures masterpieces.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Uchihachi, S., Foote, J.T., Wilcox, L.: US Patent 6,535,639 B1 (2003)
Podlesnaya, A., Podlesnyy, S.: Deep learning based semantic video indexing and retrieval. In: Proceedings of SAI Intelligent Systems Conference, pp. 359–372. Springer, Cham, 21 September 2016
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. CoRR, arXiv:1409.0575 (2014)
Arev, I., Park, H.S., Sheikh, Y., Hodgins, J., Shamir, A.: Automatic Editing of Footage from Multiple Social Cameras. ACM Trans. Graph. 33(4), 1–11 (2014)
Xin, J., Chi, J., Peng, S., Tian, Y., Ye, C., Li, X.: Deep image aesthetics classification using inception modules and fine-tuning connected layer. In: 8th International Conference on Wireless Communications and Signal Processing (WCSP), Yangzhou, China, 13–15 October (2016)
Merabti, B., Christie, M., Bouatouch, K.: A virtual director using hidden markov models. In: Computer Graphics Forum, Wiley (2015). https://doi.org/10.1111/cgf.12775. Hal-01244643
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. CoRR, arXiv:1409.4842 (2014)
ASC Unveils List of 100 Milestone Films in Cinematography of the 20th Century, 08 January 2019. https://theasc.com/news/asc-unveils-list-of-100-milestone-films-in-cinematography-of-the-20th-century
Ross, S., Gordon, G., Bagnell, J.A.: A reduction of imitation learning and structured prediction to no-regret online learning. In: International Conference on Artificial Intelligence and Statistics (AISTATS) (2011)
Langford, J., Li, L., Strehl, A.: Vowpal wabbit online learning project (2007). http://hunch.net/?p=309
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Podlesnyy, S. (2020). Towards Data-Driven Automatic Video Editing. In: Liu, Y., Wang, L., Zhao, L., Yu, Z. (eds) Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery. ICNC-FSKD 2019. Advances in Intelligent Systems and Computing, vol 1074. Springer, Cham. https://doi.org/10.1007/978-3-030-32456-8_39
Download citation
DOI: https://doi.org/10.1007/978-3-030-32456-8_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32455-1
Online ISBN: 978-3-030-32456-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)