Advertisement

Deep Learning for Knowledge-Driven Ontology Stream Prediction

  • Shumin Deng
  • Jeff Z. Pan
  • Jiaoyan Chen
  • Huajun Chen
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 957)

Abstract

Time series prediction with data stream has been widely studied. Current deep learning methods e.g., Long Short-Term Memory (LSTM) perform well in learning feature representations from raw data. However, most of these models can narrowly learn semantic information behind the data. In this paper, we revisit LSTM from the perspective of Semantic Web, where streaming data are represented as ontology sequences. We propose a novel semantic-based neural network (STBNet) that (i) enriches the semantics of data stream with external text, and (ii) exploits the underlying semantics with background knowledge for time series prediction. Previous models mainly rely on numerical representation of values in raw data, while the proposed STBNet model creatively integrates semantic embedding into a hybrid neural network. We develop a new attention mechanism based on similarity among semantic embedding of ontology stream, and then we combine ontology stream and numerical analysis in the deep learning model. Furthermore, we also enrich ontology stream in STBNet, where Convolutional Neural Networks (CNNs) are incorporated in learning lexical representations of words in the text. The experiments show that STBNet outperforms state-of-the-art methods on stock price prediction.

Keywords

Ontology stream Deep learning Time series prediction 

References

  1. 1.
    Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.: DBpedia: a nucleus for a web of open data. In: Aberer, K., et al. (eds.) ASWC/ISWC -2007. LNCS, vol. 4825, pp. 722–735. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-76298-0_52CrossRefGoogle Scholar
  2. 2.
    Baader, F., Brandt, S., Lutz, C.: Pushing the \(\cal{EL}\) envelope (2005)Google Scholar
  3. 3.
    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Computer Science (2014)Google Scholar
  4. 4.
    Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: SIGMOD Conference, pp. 1247–1250 (2008)Google Scholar
  5. 5.
    Bordes, A., Usunier, N., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: International Conference on Neural Information Processing Systems, pp. 2787–2795 (2013)Google Scholar
  6. 6.
    Burel, G., Saif, H., Alani, H.: Semantic wide and deep learning for detecting crisis-information categories on social media. In: d’Amato, C., et al. (eds.) ISWC 2017. LNCS, vol. 10587, pp. 138–155. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-68288-4_9CrossRefGoogle Scholar
  7. 7.
    Chen, J., Lécué, F., Pan, J.Z., Chen, H.: Learning from ontology streams with semantic concept drift. In: IJCAI, pp. 957–963 (2017)Google Scholar
  8. 8.
    Elman, J.L.: Distributed representations, simple recurrent networks, and grammatical structure. Mach. Learn. 7(2–3), 195–225 (1991)Google Scholar
  9. 9.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  10. 10.
    Huang, Z., Stuckenschmidt, H.: Reasoning with multi-version ontologies. In: The Semantic Web - ISWC 2005, International Semantic Web Conference, ISWC 2005, Galway, Ireland, 6–10 November 2005, Proceedings, pp. 398–412 (2005)Google Scholar
  11. 11.
    Ketkar, N.: Stochastic gradient descent. Optimization (2014)Google Scholar
  12. 12.
    Kim, Y.: Convolutional neural networks for sentence classification. Eprint Arxiv (2014)Google Scholar
  13. 13.
    Lécué, F., Pan, J.Z.: Predicting knowledge in an ontology stream. In: IJCAI, pp. 2662–2669 (2013)Google Scholar
  14. 14.
    Lécué, F., Pan, J.Z.: Consistent knowledge discovery from evolving ontologies. In: AAAI, pp. 189–195 (2015)Google Scholar
  15. 15.
    Lin, T., Guo, T., Aberer, K.: Hybrid neural networks for learning the trend in time series. In: Twenty-Sixth International Joint Conference on Artificial Intelligence, pp. 2273–2279 (2017)Google Scholar
  16. 16.
    Qin, Y., Song, D., Chen, H., Cheng, W., Jiang, G., Cottrell, G.: A dual-stage attention-based recurrent neural network for time series prediction, pp. 2627–2633 (2017)Google Scholar
  17. 17.
    Ren, Y., Pan, J.Z.: Optimising ontology stream reasoning with truth maintenance system. In: ACM International Conference on Information and Knowledge Management, pp. 831–836 (2011)Google Scholar
  18. 18.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(6088), 533–536 (1986)CrossRefGoogle Scholar
  19. 19.
    Werbos, P.: Backpropagation through time: what it does and how to do it. Proc. IEEE 78(10), 1550–1560 (1990)CrossRefGoogle Scholar
  20. 20.
    Milne, D., Witten, I.H.: Learning to link with wikipedia. In: ACM Conference on Information and Knowledge Management, pp. 509–518 (2008)Google Scholar
  21. 21.
    Xu, J., Chen, K., Qiu, X., Huang, X.: Knowledge graph representation with jointly structural and textual encoding, pp. 1318–1324 (2016)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Shumin Deng
    • 1
  • Jeff Z. Pan
    • 2
  • Jiaoyan Chen
    • 3
  • Huajun Chen
    • 1
  1. 1.College of Computer Science and TechnologyZhejiang UniversityHangzhouChina
  2. 2.Department of Computing ScienceThe University of AberdeenAberdeenUK
  3. 3.Department of Computer ScienceUniversity of OxfordOxfordUK

Personalised recommendations