Advertisement

Korean-Optimized Word Representations for Out-of-Vocabulary Problems Caused by Misspelling Using Sub-character Information

  • Seonhghyun KimEmail author
  • Jai-Eun Kim
  • Seokhyun Hawang
  • Berlocher Ivan
  • Seung-Won Yang
Conference paper
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 70)

Abstract

In this paper, we propose Korean-optimized word representations that can better address the out-of-vocabulary (OOV) problem caused by misspelling. This problem is an important issue in many applications based on natural language processing. However, previous models do not fully consider the representations of misspelled OOV words. To overcome this problem, we propose sub-character information obtained from Korean Jamo units and also adopt additional sub-character information to better withstand the misspelling. Finally, experimental results show that our model is about 2.3 times more accurate than the conventional model in case of the misspelled word while still maintaining the semantic relationship of the words.

Keywords

Word embedding Word representation Machine learning Korean Out-of-vocabulary Misspelling Sub character Natural language processing 

Notes

Acknowledgements

This work was supported by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIT) (2013-0-00109, WiseKB: Big data based self-evolving knowledge base and reasoning platform).

References

  1. 1.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space, arXiv preprint arXiv:1301.3781 (2013)
  2. 2.
    Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)Google Scholar
  3. 3.
    Sienčnik, S.K.: Adapting word2vec to named entity recognition. In: Proceedings of the 20th Nordic Conference of Computational Linguistics, Nodalida 2015, May 11–13, 2015, Vilnius, Lithuania, pp. 239–243. Linköping University Electronic Press (2015)Google Scholar
  4. 4.
    Hu, M., Peng, Y., Qiu, X.: Reinforced mnemonic reader for machine comprehension. CoRR, abs/1705.02798 (2017)Google Scholar
  5. 5.
    Wieting, J., Bansal, M., Gimpel, K., Livescu, K.: Charagram: embedding words and sentences via character n-grams, arXiv preprint arXiv:1607.02789 (2016)
  6. 6.
    Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information, arXiv preprint arXiv:1607.04606 (2016)
  7. 7.
    Sampson, G.: Writing Systems. London (1985)Google Scholar
  8. 8.
    Choi, H., Kwon, H., Yoon, A.: Improving recall for context-sensitive spelling correction rules using conditional probability model with dynamic window sizes. J. KIISE 42(5), 629–636 (2015)CrossRefGoogle Scholar
  9. 9.
    Kang, S.-S., Kim, Y.T.: Syllable-based model for the Korean morphology. In: Proceedings of the 15th Conference on Computational Linguistics, vo. 1, pp. 221–226. Association for Computational Linguistics (1994)Google Scholar
  10. 10.
    Stratos, K.: A Sub-character architecture for Korean language processing, arXiv preprint arXiv:1707.06341 (2017)
  11. 11.
    Luong, T., Socher, R., Manning, C.: Better word representations with recursive neural networks for morphology. In: Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pp. 104–113 (2013)Google Scholar
  12. 12.
    Botha, J., Blunsom, P.: Compositional morphology for word representations and language modelling. In: International Conference on Machine Learning, pp. 1899–1907 (2014)Google Scholar
  13. 13.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)Google Scholar
  14. 14.
    Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification, arXiv preprint arXiv:1607.01759 (2016)
  15. 15.
    Joulin, A., Grave, E., Bojanowski, P., Douze, M., Jégou, H., Mikolov, T.: Fasttext. zip: Compressing text classification models, arXiv preprint arXiv:1612.03651 (2016)
  16. 16.
    Finkelstein, L., Gabrilovich, E., Matias, Y., Rivlin, E., Solan, Z., Wolfman, G., Ruppin, E.: Placing search in context: the concept revisited. In: Proceedings of the 10th International Conference on World Wide Web, pp. 406–414. ACM, New York (2001)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Seonhghyun Kim
    • 1
    Email author
  • Jai-Eun Kim
    • 1
  • Seokhyun Hawang
    • 1
  • Berlocher Ivan
    • 1
  • Seung-Won Yang
    • 1
  1. 1.AI LabsSaltlux Inc.SeoulRepublic of Korea

Personalised recommendations