Modeling Marked Temporal Point Process Using Multi-relation Structure RNN
- 49 Downloads
Event sequences with marker and timing information are available in a wide range of domains, from machine log in automatic train supervision systems to information cascades in social networks. Given the historical event sequences, predicting what event will happen next and when it will happen can benefit many useful applications, such as maintenance service schedule for mass rapid transit trains and product advertising in social networks. Temporal point process (TPP) is one effective solution to solve the next event prediction problem due to its capability of capturing the temporal dependence among events. The recent recurrent temporal point process (RTPP) methods exploited recurrent neural network (RNN) to get rid of the parametric form assumption in the density functions of TPP. However, most existing RTPP methods focus only on the temporal dependence among events. In this work, we design a novel multi-relation structure RNN model with a hierarchical attention mechanism to capture not only the conventional temporal dependencies but also the explicit multi-relation topology dependencies. We then propose an RTPP algorithm whose density function conditioned on the event sequence embedding learned from our RNN model for cognitively predict the next event marker and time. The experiments show that our proposed MRS-RMTPP outperforms the state-of-the-art baselines in terms of both event marker prediction and event time prediction on three real-world datasets. The capability of capturing both ontology relation structure and temporal structure in the event sequences is of great importance for the next event marker and time prediction.
KeywordsStructure RNN Recurrent temporal point process Multi-layer attention
This study was financially supported in part by the National Research Foundation (NRF), Prime Minister’s Office, Singapore, under its National Cybersecurity R&D Programme (Award No. NRF2014NCR-NCR001-31) and administered by the National Cybersecurity R&D Directorate. This research is also financially supported in part by the National Research Foundation, Prime Ministers Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) program.
Compliance with Ethical Standards
Conflict of Interests
This research was mainly conducted when Hongyun Cai and Vincent W. Zheng worked at Advanced Digital Sciences Center. This research was supported in part by the National Research Foundation (NRF), Prime Ministers Office, Singapore, under its National Cybersecurity R&D Programme (Award No. NRF2014NCR-NCR001-31) and administered by the National Cybersecurity R&D Directorate. It is also supported in part by the National Research Foundation, Prime Ministers Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme.
- 4.Wang J, Zheng VW, Liu Z, Chang KC. Topological recurrent neural network for diffusion prediction. ICDM; 2017.Google Scholar
- 5.Daley DJ, Vere-Jones D. 2008. An introduction to the theory of point processes, vol II. 2nd ed. Probability and its applications (New York), General theory and structure.Google Scholar
- 6.Kingman JFC, Vol. 3. Poisson processes. Oxford: Oxford University Press; 1993.Google Scholar
- 8.Isham V, Westcott M. A self-correcting point process. Adv Appl Probab 1979;37:629–46.Google Scholar
- 11.Du N, Dai H, Trivedi R, Upadhyay U, Gomez-Rodriguez M, Song L. Recurrent marked temporal point processes: embedding event history to vector. KDD; 2016. p. 1555–64.Google Scholar
- 12.Xiao S, Yan J, Yang X, Zha H, Chu SM. Modeling the intensity function of point process via recurrent neural networks. AAAI; 2017. p. 1597–603.Google Scholar
- 13.Wang Y, Shen H, Liu S, Gao J, Cheng X. Cascade dynamics modeling with attention-based recurrent neural network. IJCAI; 2017. p. 2985–91.Google Scholar
- 17.Cho K, van Merriënboer B, Gülçehre Ç, Bahdanau D, Bougares F, Schwenk H, et al. Learning phrase representations using RNN encoder–decoder for statistical machine translation. EMNLP; 2014. p. 1724–34.Google Scholar
- 20.Tai KS, Socher R, Manning CD. Improved semantic representations from tree-structured long short-term memory networks. ACL; 2015. p. 1556–66.Google Scholar
- 23.Shuai B, Zuo Z, Wang B, Wang G. DAG-recurrent neural networks for scene labeling. CVPR; 2016. p. 3620–29.Google Scholar
- 24.Liu Z, Zheng VW, Zhao Z, Zhu F, Chang KC, Wu M, et al. Semantic proximity search on heterogeneous graph by proximity embedding. AAAI; 2017. p. 154–60.Google Scholar
- 25.Liu Z, Zheng VW, Zhao Z, Yang H, Chang KCC, Wu M, et al. Subgraph-augmented path embedding for semantic user search on heterogeneous social network. WWW; 2018.Google Scholar
- 26.Brillinger DR, Guttorp PM, Schoenberg FP. . Point processes, temporal. American Cancer Society; 2013.Google Scholar
- 27.Aalen O, Borgan O, Gjessing H. 2008. Survival and event history analysis: a process point of view statistics for biology and health.Google Scholar
- 28.Mei H, Eisner J. The neural Hawkes process: a neurally self-modulating multivariate point process. NIPS; 2017.Google Scholar
- 29.Song H, Rajan D, Thiagarajan JJ, Spanias A. Attend and diagnose: clinical time series analysis using attention models. AAAI; 2018. p. 4091–98.Google Scholar
- 30.Ma F, Chitta R, Zhou J, You Q, Sun T, Gao J. Dipole: diagnosis prediction in healthcare via attention-based bidirectional recurrent neural networks. KDD; 2017. p. 1903–11.Google Scholar
- 31.Xiao S, Yan J, Yang X, Zha H, Chu SM. Modeling the intensity function of point process via recurrent neural networks. AAAI; 2017. p. 1597–1603.Google Scholar
- 32.Press WH, Teukolsky SA, Vetterling WT, Flannery BP. Numerical recipes in C. Cambridge: Cambridge University Press; 1992.Google Scholar
- 34.Kingma DP, Ba J. 2014. Adam: a method for stochastic optimization. CoRR arXiv:1412.6980.
- 35.Du N, Dai H, Trivedi R, Upadhyay U, Gomez-Rodriguez M, Song L. Recurrent marked temporal point processes: embedding event history to vector. 1555–64; 2016.Google Scholar
- 36.Hodas NO, Lerman K. 2013. The simple rules of social contagion. CoRR arXiv:1308.5015.
- 37.Leskovec J, Backstrom L, Kleinberg J. 2009. Meme-tracking and the dynamics of the news cycle.Google Scholar
- 38.Team TD. 2016. Theano: a python framework for fast computation of mathematical expressions. CoRR arXiv:1605.02688.