Abstract
Due to recent technical and scientific advances, we have a wealth of information hidden in unstructured text data such as offline/online narratives, research articles, and clinical reports. To mine these data properly, attributable to their innate ambiguity, a Word Sense Disambiguation (WSD) algorithm can avoid numbers of difficulties in Natural Language Processing (NLP) pipeline. However, considering a large number of ambiguous words in one language or technical domain, we may encounter limiting constraints for proper deployment of existing WSD models. This paper attempts to address the problem of one-classifier-per-one-word WSD algorithms by proposing a single Bidirectional Long Short-Term Memory (BLSTM) network which by considering senses and context sequences works on all ambiguous words collectively. Evaluated on SensEval-3 benchmark, we show the result of our model is comparable with top-performing WSD algorithms. We also discuss how applying additional modifications alleviates the model fault and the need for more training data.
Keywords
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Vickrey, D., Biewald, L., Teyssier, M., Koller, D.: Word-sense disambiguation for machine translation. In: Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (2005)
Hung, J.C., Wang, C.S., Yang, C.Y., Chiu, M.S., Yee, G.: Applying word sense disambiguation to question answering system for e-learning. In: 19th International Conference on Advanced Information Networking and Applications, AINA 2005, vol. 1, pp. 157–162. IEEE (2005)
Pennington, J., Socher, R., Manning, C.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Mihalcea, R., Chklovski, T., Kilgarriff, A.: The Senseval-3 English lexical sample task. In: Proceedings of SENSEVAL-3, The Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text (2004)
Zhong, Z., Ng, H.T.: It makes sense: a wide-coverage word sense disambiguation system for free text. In: Proceedings of the ACL 2010 System Demonstrations, pp. 78–83. Association for Computational Linguistics (2010)
Miller, G.A.: Wordnet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)
Pesaranghader, A., Pesaranghader, A., Mustapha, N.: Word sense disambiguation for biomedical text mining using definition-based semantic relatedness and similarity measures. Int. J. Biosci. Biochem. Bioinform. 4(4), 280 (2014)
Kim, S., Yoon, J.: Link-topic model for biomedical abbreviation disambiguation. J. Biomed. Inf. 53, 367–380 (2015)
Navigli, R.: Word sense disambiguation: a survey. ACM Comput. Surv. (CSUR) 41(2), 10 (2009)
Collobert, R., Weston, J.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, pp. 160–167. ACM (2008)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Chen, X., Liu, Z., Sun, M.: A unified model for word sense representation and disambiguation. In: EMNLP, pp. 1025–1035 (2014)
Navigli, R., Litkowski, K.C., Hargraves, O.: SemEval-2007 task 07: coarse-grained English all-words task. In: Proceedings of the 4th International Workshop on Semantic Evaluations, pp. 30–35. Association for Computational Linguistics (2007)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18(5), 602–610 (2005)
Kågebäck, M., Salomonsson, H.: Word sense disambiguation using a bidirectional LSTM. arXiv preprint arXiv:1606.03568 (2016)
Taghipour, K., Ng, H.T.: Semi-supervised word sense disambiguation using word embeddings in general and specific domains. In: HLT-NAACL, pp. 314–323 (2015)
McInnes, B.T., Pedersen, T.: Evaluating measures of semantic similarity and relatedness to disambiguate terms in biomedical text. J. Biomed. Inform. 46(6), 1116–1124 (2013)
Pedersen, T., Kolhatkar, V.: Wordnet::Senserelate::Allwords: a broad coverage word sense tagger that maximizes semantic relatedness. In: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Demonstration Session. Association for Computational Linguistics, pp. 17–20 (2009)
Hinton, G., Srivastava, N., Swersky, K.: RMSprop: divide the gradient by a running average of its recent magnitude. Neural Networks for Machine Learning, COURSERA Lecture 6e (2012)
Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Iyyer, M., Manjunatha, V., Boyd-Graber, J., Daumé III, H.: Deep unordered composition rivals syntactic methods for text classification. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), vol. 1, pp. 1681–1691 (2015)
Grozea, C.: Finding optimal parameter settings for high performance word sense disambiguation. In: Proceedings of SensEval-3 Workshop (2004)
Strapparava, C., Gliozzo, A., Giuliano, C.: Pattern abstraction and term similarity for word sense disambiguation: IRST at SensEval-3. In: Proceedings of SENSEVAL-3 Third International Workshop on Evaluation of Systems for the Semantic Analysis of Text, pp. 229–234 (2004)
Lee, Y.K., Ng, H.T., Chia, T.K.: Supervised word sense disambiguation with support vector machines and multiple knowledge sources. In: SensEval-3: Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text, pp. 137–140 (2004)
Pesaranghader, A., Matwin, S., Sokolova, M., Beiko, R.G.: simDEF: definition-based semantic similarity measure of gene ontology terms for functional similarity analysis of genes. Bioinformatics 32(9), 1380–1387 (2015)
Pesaranghader, A., Rezaei, A., Pesaranghader, A.: Adapting gloss vector semantic relatedness measure for semantic similarity estimation: an evaluation in the biomedical domain. In: Kim, W., Ding, Y., Kim, H.-G. (eds.) JIST 2013. LNCS, vol. 8388, pp. 129–145. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-06826-8_11
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Pesaranghader, A., Pesaranghader, A., Matwin, S., Sokolova, M. (2018). One Single Deep Bidirectional LSTM Network for Word Sense Disambiguation of Text Data. In: Bagheri, E., Cheung, J. (eds) Advances in Artificial Intelligence. Canadian AI 2018. Lecture Notes in Computer Science(), vol 10832. Springer, Cham. https://doi.org/10.1007/978-3-319-89656-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-89656-4_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-89655-7
Online ISBN: 978-3-319-89656-4
eBook Packages: Computer ScienceComputer Science (R0)