Abstract
We address the problem of embedding entities into Euclidean space over time based on co-occurrence data. We extend the CODE model of [1] to a dynamic setting. This leads to a non-standard factored state space model with real-valued hidden parent nodes and discrete observation nodes. We investigate the use of variational approximations applied to the observation model that allow us to formulate the entire dynamic model as a Kalman filter. Applying this model to temporal co-occurrence data yields posterior distributions of entity coordinates in Euclidean space that are updated over time. Initial results on per-year co-occurrences of authors and words in the NIPS corpus and on synthetic data, including videos of dynamic embeddings, seem to indicate that the model results in embeddings of co-occurrence data that are meaningful both temporally and contextually.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Globerson, A., Chechik, G., Pereira, F., Tishby, N.: Euclidean embedding of co-occurrence data. In: Proc. Eighteenth Annual Conf. on Neural Info. Proc. Systems, NIPS (2004)
Sarkar, P., Moore, A.: Dynamic social network analysis using latent space models. In: Proc. Nineteenth Annual Conf. on Neural Info. Proc. Systems, NIPS (2005)
Berry, M., Dumais, S., Letsche, T.: Computational methods for intelligent information access. In: Proceedings of Supercomputing (1995)
Breiger, R.L., Boorman, S.A., Arabie, P.: An algorithm for clustering relational data with applications to social network analysis and comparison with multidimensional scaling. J. of Math. Psych. 12, 328–383 (1975)
Raftery, A.E., Handcock, M.S., Hoff, P.D.: Latent space approaches to social network analysis. J. Amer. Stat. Assoc. 15, 460 (2002)
Ghahramani, Z., Jordan, M.I.: Factorial hidden Markov models. In: Touretzky, D.S., Mozer, M.C., Hasselmo, M.E. (eds.) Proc. Conf. Advances in Neural Information Processing Systems, NIPS, vol. 8, pp. 472–478. MIT Press, Cambridge (1995)
Ghahramani, Z., Hinton, G.E.: Switching state-space models. Technical report, Toronto, Canada (1998)
Kalman, R.: A new approach to linear filtering and prediction problems (1960)
Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K.: An Introduction to Variational Methods for Graphical Methods. In: Machine Learning (1998)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sarkar, P., Siddiqi, S.M., Gordon, G.J. (2007). Approximate Kalman Filters for Embedding Author-Word Co-occurrence Data over Time. In: Airoldi, E., Blei, D.M., Fienberg, S.E., Goldenberg, A., Xing, E.P., Zheng, A.X. (eds) Statistical Network Analysis: Models, Issues, and New Directions. ICML 2006. Lecture Notes in Computer Science, vol 4503. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73133-7_10
Download citation
DOI: https://doi.org/10.1007/978-3-540-73133-7_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73132-0
Online ISBN: 978-3-540-73133-7
eBook Packages: Computer ScienceComputer Science (R0)