Abstract
Modeling and describing temporal structure in multimedia signals, which are captured simultaneously by multiple sensors, is important for realizing human machine interaction and motion generation. This paper proposes a method for modeling temporal structure in multimedia signals based on temporal intervals of primitive signal patterns. Using temporal difference between beginning points and the difference between ending points of the intervals, we can explicitly express timing structure; that is, synchronization and mutual dependency among media signals. We applied the model to video signal generation from an audio signal to verify the effectiveness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Allen, J.F.: Maintaining knowledge about temporal interval. Commun. of the ACM 26(11), 832–843 (1983)
Brand, M.: Voice puppetry. In: Proc. SIGGRAPH, pp. 21–28 (1999)
Brand, M., Oliver, N., Pentland, A.: Coupled hidden Markov models for complex action recognition. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 994–999 (1997)
Bregler, C.: Learning and recognizing human dynamics in video sequences. In: Proc. Int. Conference on Computer Vision and Pattern Recognition, pp. 568–574 (1997)
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998)
Kawashima, H., Matsuyama, T.: Multiphase learning for an interval-based hybrid dynamical system. IEICE Trans. Fundamentals E88-A(11), 3022–3035 (2005)
Levinson, S.E.: Continuously variable duration hidden Markov models for automatic speech recognition. Computer Speech and Language 1, 29–45 (1986)
Li, Y., Wang, T., Shum, H.-Y.: Motion texture: A two-level statistical model for character motion synthesis. In: Proc. SIGGRAPH, pp. 465–472 (2002)
McGurk, H., MacDonald, J.: Hearing lips and seeing voices. Nature, 746–748 (1976)
Murphy, K.P.: Hidden semi-Markov models (HSMMs). Informal Notes (2002)
Nefian, A.V., Liang, L., Pi, X., Liu, X., Murphy, K.: Dynamic Bayesian networks for audio-visual speech recognition. EURASIP Journal on Applied Signal Processing 2002(11), 1–15 (2002)
Nishiyama, M., Kawashima, H., Hirayama, T., Matsuyama, T.: Facial expression representation based on timing structures in faces. In: Zhao, W., Gong, S., Tang, X. (eds.) AMFG 2005. LNCS, vol. 3723, pp. 140–154. Springer, Heidelberg (2005)
Ostendorf, M., Digalakis, V., Kimball, O.A.: From HMMs to segment models: A unified view of stochastic modeling for speech recognition. IEEE Trans. Speech and Audio Process 4(5), 360–378 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kawashima, H., Tsutsumi, K., Matsuyama, T. (2006). Modeling Timing Structure in Multimedia Signals. In: Perales, F.J., Fisher, R.B. (eds) Articulated Motion and Deformable Objects. AMDO 2006. Lecture Notes in Computer Science, vol 4069. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11789239_47
Download citation
DOI: https://doi.org/10.1007/11789239_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-36031-5
Online ISBN: 978-3-540-36032-2
eBook Packages: Computer ScienceComputer Science (R0)