Exploiting the Web for Semantic Change Detection

  • Pierpaolo BasileEmail author
  • Barbara McGillivray
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11198)


Detecting significant linguistic shifts in the meaning and usage of words has gained more attention over the last few years. Linguistic shifts are especially prevalent on the Internet, where words’ meaning can change rapidly. In this work, we describe the construction of a large diachronic corpus that relies on the UK Web Archive and we propose a preliminary analysis of semantic change detection exploiting a particular technique called Temporal Random Indexing. Results of the evaluation are promising and give us important insights for further investigations.


Semantic change detection Diachronic analysis of language Time series 



This research was undertaken with the support of the Alan Turing Institute (EPSRC Grant Number EP/N510129/1). The access to the Oxford English Dictionary API was provided by Oxford University Press via a licence for non-commercial research.


  1. 1.
    Bamler, R., Mandt, S.: Dynamic word embeddings. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 380–389. PMLR, International Convention Centre, Sydney, Australia (06–11 Aug 2017),
  2. 2.
    Basile, P., Caputo, A., Luisi, R., Semeraro, G.: Diachronic analysis of the italian language exploiting google ngram (2016)Google Scholar
  3. 3.
    Basile, P., Caputo, A., Semeraro, G.: Analysing word meaning over time by exploiting temporal random indexing. In: Basili, R., Lenci, A., Magnini, B. (eds.) First Italian Conference on Computational Linguistics CLiC-it 2014. Pisa University Press (2014)Google Scholar
  4. 4.
    Basile, P., Caputo, A., Semeraro, G.: Temporal random indexing: a system for analysing word meaning over time. Ital. J. Comput. Linguist. 1(1), 55–68 (2015)Google Scholar
  5. 5.
    Blei, D.M., Lafferty, J.D.: Dynamic topic models. In: ICML, pp. 113–120 (2006)Google Scholar
  6. 6.
    Boussidan, A., Ploux, S.: Using topic salience and connotational drifts to detect candidates to semantic change. In: Proceeding IWCS ’11 Proceedings of the Ninth International Conference on Computational Semantics, pp. 315–319 (2011)Google Scholar
  7. 7.
    Cook, P., Stevenson, S.: Automatically identifying changes in the semantic orientation of words. In: Proceedings of the Seventh conference on International Language Resources and Evaluation, Valletta, Malta (2010)Google Scholar
  8. 8.
    De Saussure, F.: Course in General Linguistics. Open Court, La Salle, Illinois (1983)Google Scholar
  9. 9.
    Dubossarsky, H., Weinshall, D., Grossman, E.: Outta control: laws of semantic change and inherent biases in word representation models. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1136–1145 (2017)Google Scholar
  10. 10.
    Efron, B., Tibshirani, R.J.: An Introduction to the Bootstrap. Chapman and Hall/CRC, Boca Raton (1994)zbMATHGoogle Scholar
  11. 11.
    Frermann, L., Lapata, M.: A bayesian model of diachronic meaning change. Trans. Assoc. Comput. Linguist. 4, 31–45 (2016)Google Scholar
  12. 12.
    Gulordava, K., Baroni, M.: A distributional similarity approach to the detection of semantic change in the Google Books Ngram corpus. In: Proceedings of the EMNLP 2011 Geometrical Models for Natural Language Semantics (GEMS 2011) Workshop, pp. 67–71 (2011).
  13. 13.
    Hamilton, W.L., Leskovec, J., Jurafsky, D.: Diachronic word embeddings reveal statistical laws of semantic change. arXiv preprint arXiv:1605.09096 (2016)
  14. 14.
    Jatowt, A., Duh, K.: A framework for analyzing semantic change of words across time. In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries, pp. 229–238 (2014).
  15. 15.
    JISC, the Internet Archive: Jisc uk web domain dataset (1996–2013) (2013).
  16. 16.
    Jurgens, D., Stevens, K.: event detection in blogs using temporal random indexing. In: Proceedings of the Workshop on Events in Emerging Text Types, pp. 9–16. Association for Computational Linguistics (2009)Google Scholar
  17. 17.
    Kulkarni, V., Al-Rfou, R., Perozzi, B., Skiena, S.: Statistically significant detection of linguistic change. In: Proceedings of the 24th International Conference on World Wide Web, pp. 625–635. ACM (2015)Google Scholar
  18. 18.
    Kutuzov, A., Øvrelid, L., Szymanski, T., Velldal, E.: Diachronic word embeddings and semantic shifts: a survey (2018). arXiv:1806.03537
  19. 19.
    Lin, Y., Michel, J.B., Aiden, E.L., Orwant, J., Brockman, W., Petrov, S.: Syntactic annotations for the google books ngram corpus. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, Jeju, Republic of Korea, 8–14 July 2012, pp. 169–174. Association for Computational Linguistics (2012)Google Scholar
  20. 20.
    Mitchell, J., Lapata, M.: Composition in distributional models of semantics. Cogn. Sci. 34(8), 1388–1429 (2010)CrossRefGoogle Scholar
  21. 21.
    Rudolph, M., Blei, D.: Dynamic embeddings for language evolution. In: Proceedings of the 2018 World Wide Web Conference on World Wide Web (2018)Google Scholar
  22. 22.
    Sahlgren, M.: The word-space model: using distributional analysis to represent syntagmatic and paradigmatic relations between words in high-dimensional vector spaces (2006)Google Scholar
  23. 23.
    Schiitze, H.: Word space. Adv. Neural Inf. Process. Syst. 5, 895–902 (1993)Google Scholar
  24. 24.
    Tang, X.: A State-of-the-Art of Semantic Change Computation. arXiv preprint arXiv:1801.09872 (Cl), 2–37 (2018)
  25. 25.
    Tang, X., Qu, W., Chen, X.: Semantic change computation: a successive approach. World Wide Web-Internet Web Inf. Syst. 19(3), 375–415 (2016). Scholar
  26. 26.
    Taylor, W.A.: Change-Point Analysis: A Powerful New Tool for Detecting Changes. Taylor Enterprises, Inc. (2000)Google Scholar
  27. 27.
    Wijaya, D.T., Yeniterzi, R.: Understanding semantic change of words over centuries. In: Proceedings of the 2011 International Workshop on DETecting and Exploiting Cultural diversiTy on the Social Web - DETECT ’11, p. 35 (2011).,
  28. 28.
    Xu, Y., Kemp, C.: A computational evaluation of two laws of semantic change. Proc. CogSci 2015, 1–6 (2015)Google Scholar
  29. 29.
    Yao, Z., Sun, Y., Ding, W., Rao, N., Xiong, H.: Dynamic Word Embeddings for Evolving Semantic Discovery. Technical report (2017)., arXiv:1703.00607

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of Bari Aldo MoroBariItaly
  2. 2.Modern and Medieval LanguagesUniversity of CambridgeCambridgeUK
  3. 3.The Alan Turing InstituteLondonUK

Personalised recommendations