Abstract
Automatic generation of film comic requires solving several challenging problems such as selecting important frames well conveying the whole story, trimming the frames to fit the shape of panels without corrupting the composition of original image and arranging visually pleasing speech balloons without hiding important objects in the panel. We propose a novel approach to the automatic generation of film comic. The key idea is to aggregate eye-tracking data and image features into a computational map, called iMap, for quantitatively measuring the importance of frames in terms of story content and user attention. The transition of iMap in time sequences provides the solution to frame selection. Word balloon arrangement and image trimming are realized as the results of optimizing the energy functions derived from the iMap.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hwang, W.I., Lee, P.J., Chun, B.K., Ryu, D.S., Cho, H.G.: Cinema comics: Cartoon generation from video stream. In: International Conference on Computer Graphics Theory and Applications, pp. 299–304 (2006)
Preuß, J., Loviscach, J.: From movie to comics, informed by the screenplay. In: ACM SIGGRAPH, Poster (2007)
Hong, R., Yuan, X.T., Xu, M., Wang, M., Yan, S., Chua, T.S.: Movie2comics: a feast of multimedia artwork. In: Proceedings of the International Conference on Multimedia, pp. 611–614 (2010)
Toyoura, M., Kunihiro, M., Mao, X.: Film Comic Reflecting Camera-Works. In: Schoeffmann, K., Merialdo, B., Hauptmann, A.G., Ngo, C.-W., Andreopoulos, Y., Breiteneder, C. (eds.) MMM 2012. LNCS, vol. 7131, pp. 406–417. Springer, Heidelberg (2012)
Smith, T.J., Whitwell, M., Lee, J.: Eye movements and pupil dilation during event perception. In: Proceedings of the Eye Tracking Research and Applications Conference (2006)
Ma, Y., Hua, X., Lu, L., Zhang, H.: A generic framework of user attention model and its application in video summarization. IEEE Trans. on Multimedia 7(5), 907–919 (2005)
Li, K., Guo, L., Faraco, C., Zhu, D., Deng, F., Zhang, T., Jiang, X., Zhang, D., Chen, H., Hu, X., Miller, S., Liuothers, T.: Human-centered attention models for video summarization. In: International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, vol. 27. ACM (2010)
You, J., Liu, G., Sun, L., Li, H.: A multiple visual models based perceptive analysis framework for multilevel video summarization. IEEE Transactions on Circuits and Systems for Video Technology 17(3), 335–342 (2007)
Money, A.G., Agius, H.: Video summarisation: A conceptual framework and survey of the state of the art. Journal of Visual Communication and Image Representation 19(2), 121–143 (2008)
Porter, S.V.: Video Segmentation and Indexing using Motion Estimation. PhD thesis, University of Bristol (2004)
Chen, B., Wang, J., Wang, J.: A novel video summarization based on mining the story structure and semantic relations among concept entities. IEEE Transactions on Multimedia 11(2), 295–312 (2009)
Hu, X., Deng, F., Li, K., Zhang, T., Chen, H., Jiang, X., Lv, J., Zhu, D., Faraco, C., Zhang, D., et al.: Bridging low-level features and high-level semantics via fmri brain imaging for video classification. In: Proceedings of the International Conference on Multimedia, pp. 451–460. ACM (2010)
Liu, A., Yang, Z.: Watching, thinking, reacting: A human-centered framework for movie content analysis. International Journal of Digital Content Technology and its Applications 4(5), 23–37 (2010)
Peng, W.-T., Huang, W.-J., Chu, W.-T., Chou, C.-N., Chang, W.-Y., Chang, C.-H., Hung, Y.-P.: A User Experience Model for Home Video Summarization. In: Huet, B., Smeaton, A., Mayer-Patel, K., Avrithis, Y. (eds.) MMM 2009. LNCS, vol. 5371, pp. 484–495. Springer, Heidelberg (2009)
Boreczky, J.S., Girgensohn, A., Golovchinsky, G., Uchihashi, S.: An interactive comic book presentation for exploring video. In: ACM Conference on Computer-Human Interaction (CHI), pp. 185–192 (2000)
Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: Speeded-up robust features (surf). Computer Vision and Image Understanding 110(3) (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sawada, T., Toyoura, M., Mao, X. (2013). Film Comic Generation with Eye Tracking. In: Li, S., et al. Advances in Multimedia Modeling. MMM 2013. Lecture Notes in Computer Science, vol 7732. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35725-1_43
Download citation
DOI: https://doi.org/10.1007/978-3-642-35725-1_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35724-4
Online ISBN: 978-3-642-35725-1
eBook Packages: Computer ScienceComputer Science (R0)