Abstract
In the 3D animation field, the quality of productions is continuously increasing. It is a very active market with a high level of competitiveness where modest companies, in terms of budget, must reach a balance between the resources they can apply and the economic investment in a given production. Consequently, the automation of manual design processes, which are normally highly time-consuming, has become a crucial research topic for 3D animation studios. The work we are describing here presents one of these automatic tools, specifically focused on the synchronization of the speech and the lip movement of the characters, a process that is called lipsync. We have developed a very robust and accurate speech recognition module that together with a knowledge-based system, autonomously provides lipsync results. Additionally, the system has been integrated in the production plan of a 3D animation company, leading to drastic operator time reductions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Parke, F.I.: Computer generated animation of faces. In: Proceedings of the ACM Annual Conference, vol. 1, pp. 451–457. ACM, New York (1972)
Parent, R., King, S., Fujimura, O.: Issues with lip sync animation: can you read my lips? In: Proceedings of Computer Animation 2002, pp. 3–10. IEEE Press, Los Alamitos (2002)
Tang, S., Wee-Chung Liew, A., Yan, H.: Lip-Sync in Human Face Animation Based on Video Analysis and Spline Models. In: Proceedings of the 10th Int. Multimedia Modelling Conference (MMM 2004), pp. 1–7. IEEE Press, Los Alamitos (2004)
Zoric, G., Pandzic, I.S.: Automatic Lip Sync and its Use in the New Multimedia Services for Mobile Devices. In: 8th Int. Conf. on Telecommunications, pp. 353–358. IEEE Press, Los Alamitos (2005)
Rathinavelu, A., Thiagarajan, H., Savithri, S.R.: Evaluation of a computer aided 3D lip sync instructional model using virtual reality objects. In: Proc. 6th Intl Conf. Disability, Virtual Reality & Assoc. Tech., Esbjerg, Denmark, pp. 67–73 (2006)
Lewis, J.: Automated Lip-sync: Background and Techniques. Journal of Visualization and Computer Animation 2, 118–122 (1991)
The CMU Sphinx Group Open Source Speech Recognition Engines, http://cmusphinx.sourceforge.net
Softimage XSI, http://www.softimage.com/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Monroy, J., Bellas, F., Duro, R.J., Lopez, R., Puentes, A., Isaac, J. (2009). Automatic Speech-Lip Synchronization System for 3D Animation. In: Velásquez, J.D., Ríos, S.A., Howlett, R.J., Jain, L.C. (eds) Knowledge-Based and Intelligent Information and Engineering Systems. KES 2009. Lecture Notes in Computer Science(), vol 5711. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04595-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-04595-0_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04594-3
Online ISBN: 978-3-642-04595-0
eBook Packages: Computer ScienceComputer Science (R0)