Advertisement

Portuguese POS Tagging Using BLSTM Without Handcrafted Features

  • Rômulo César Costa de Sousa
  • Hélio LopesEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11896)

Abstract

Training state-of-the-art Part-of-speech (POS) taggers traditionally requires many handcraft features and external data. In this paper, we propose a neural network architecture for POS tagging task for both contemporary and historical Portuguese texts. The proposed architecture does not use the two traditional requirements cited above. It uses word embeddings and character embeddings representations combined with a BLSTM layer. We apply the architecture on three Portuguese corpora and obtaining state-of-the-art accuracy of 97.87% on the Mac-Morpho corpus, 97.62% accuracy on the revised Mac-Morpho and 97.36% on Tycho Brahe. We also improve the tagging accuracy for Out of Vocabulary (OOV) words in the Mac-Morpho corpus and in the revised Mac-Morpho.

Keywords

Part-of-speech tagging Deep learning Word embeddings 

References

  1. 1.
    Alam, F., Chowdhury, S.A., Noori, S.R.H.: Bidirectional LSTMs-CRFs networks for bangla POS tagging. In: 2016 19th International Conference on Computer and Information Technology (ICCIT), pp. 377–382. IEEE (2016)Google Scholar
  2. 2.
    Aluísio, S., Pelizzoni, J., Marchi, A.R., de Oliveira, L., Manenti, R., Marquiafável, V.: An account of the challenge of tagging a reference corpus for Brazilian Portuguese. In: Mamede, N.J., Trancoso, I., Baptista, J., das Graças Volpe Nunes, M. (eds.) PROPOR 2003. LNCS (LNAI), vol. 2721, pp. 110–117. Springer, Heidelberg (2003).  https://doi.org/10.1007/3-540-45011-4_17CrossRefzbMATHGoogle Scholar
  3. 3.
    Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017). http://aclweb.org/anthology/Q17-1010CrossRefGoogle Scholar
  4. 4.
    Das, O., Balabantaray, R.C.: Sentiment analysis of movie reviews using POS tags and term frequencies. Int. J. Comput. Appl. 96(25), 36–41 (2014)Google Scholar
  5. 5.
    Fernandes, E.R., Rodrigues, I.M., Milidiú, R.L.: Portuguese part-of-speech tagging with large margin structure learning. In: 2014 Brazilian Conference on Intelligent Systems (BRACIS), pp. 25–30. IEEE (2014)Google Scholar
  6. 6.
    Fonseca, E.R., Rosa, J.L.G., Aluísio, S.M.: Evaluating word embeddings and a revised corpus for part-of-speech tagging in portuguese. J. Braz. Comput. Soc. 21(1), 2 (2015)CrossRefGoogle Scholar
  7. 7.
    Fonseca, E.R., Rosa, J.L.G.: Mac-Morpho revisited: towards robust part-of-speech tagging. In: Proceedings of the 9th Brazilian Symposium in Information and Human Language Technology (2013)Google Scholar
  8. 8.
    Giménez, J., Marquez, L.: SVMtool: a general POS tagger generator based on support vector machines. In: Proceedings of the 4th International Conference on Language Resources and Evaluation. Citeseer (2004)Google Scholar
  9. 9.
    Gimpel, K., et al.: Part-of-speech tagging for twitter: annotation, features, and experiments. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pp. 42–47. Association for Computational Linguistics (2011)Google Scholar
  10. 10.
    Hartmann, N., Fonseca, E., Shulby, C., Treviso, M., Silva, J., Aluísio, S.: Portuguese word embeddings: evaluating on word analogies and natural language tasks. In: Proceedings of the 11th Brazilian Symposium in Information and Human Language Technology, pp. 122–131 (2017)Google Scholar
  11. 11.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  12. 12.
    Jung, S., Lee, C., Hwang, H.: End-to-end Korean part-of-speech tagging using copying mechanism. ACM Trans. Asian Low-Resour. Lang. Inf. Process. (TALLIP) 17(3), 19 (2018)Google Scholar
  13. 13.
    Lai, S., Liu, K., He, S., Zhao, J.: How to generate a good word embedding. IEEE Intell. Syst. 31(6), 5–14 (2016)CrossRefGoogle Scholar
  14. 14.
    Ling, W., Dyer, C., Black, A.W., Trancoso, I.: Two/too simple adaptations of word2vec for syntax problems. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1299–1304. Association for Computational Linguistics (2015).  https://doi.org/10.3115/v1/N15-1142. http://aclweb.org/anthology/N15-1142
  15. 15.
    Ma, J., Liu, H., Huang, D., Sheng, W.: An English part-of-speech tagger for machine translation in business domain. In: 2011 7th International Conference on Natural Language Processing and Knowledge Engineering (NLP-KE), pp. 183–189. IEEE (2011)Google Scholar
  16. 16.
    Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 1064–1074 (2016)Google Scholar
  17. 17.
    Ma, X., Xia, F.: Unsupervised dependency parsing with transferring distribution via parallel guidance and entropy regularization. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 1337–1348 (2014)Google Scholar
  18. 18.
    Makazhanov, A., Yessenbayev, Z.: Character-based feature extraction with LSTM networks for POS-tagging task. In: 2016 IEEE 10th International Conference on Application of Information and Communication Technologies (AICT), pp. 1–5. IEEE (2016)Google Scholar
  19. 19.
    Marujo, W.L.T.L.L., Astudillo, R.F.: Finding function in form: Compositional character models for open vocabulary word representation (2015) Google Scholar
  20. 20.
    Marulli, F., Pota, M., Esposito, M.: A comparison of character and word embeddings in bidirectional LSTMs for POS tagging in Italian. In: De Pietro, G., Gallo, L., Howlett, R.J., Jain, L.C., Vlacic, L. (eds.) KES-IIMSS-18 2018. SIST, vol. 98, pp. 14–23. Springer, Cham (2019).  https://doi.org/10.1007/978-3-319-92231-7_2CrossRefGoogle Scholar
  21. 21.
    Mikolov, T., Chen, K., Corrado, G.S., Dean, J.: Efficient estimation of word representations in vector space. CoRR abs/1301.3781 (2013)Google Scholar
  22. 22.
    Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)Google Scholar
  23. 23.
    Plank, B., Søgaard, A., Goldberg, Y.: Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 412–418 (2016)Google Scholar
  24. 24.
    dos Santos, C.N., Zadrozny, B.: Training state-of-the-art Portuguese POS taggers without handcrafted features. In: Baptista, J., Mamede, N., Candeias, S., Paraboni, I., Pardo, T.A.S., Volpe Nunes, M.G. (eds.) PROPOR 2014. LNCS (LNAI), vol. 8775, pp. 82–93. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-09761-9_8CrossRefGoogle Scholar
  25. 25.
    Temponi, C.N., et al.: O corpus anotado do português histórico: um avanço para as pesquisas em lingüística histórica do português. Revista Virtual de Estudos da Linguagem: ReVEL 2(3), 1 (2004)Google Scholar
  26. 26.
    Wang, P., Qian, Y., Soong, F.K., He, L., Zhao, H.: Part-of-speech tagging with bidirectional long short-term memory recurrent neural network. arXiv preprint arXiv:1510.06168 (2015)
  27. 27.
    Wang, W., Auer, J., Parasuraman, R., Zubarev, I., Brandyberry, D., Harper, M.: A question answering system developed as a project in a natural language processing course. In: Proceedings of the 2000 ANLP/NAACL Workshop on Reading Comprehension Tests as Evaluation for Computer-based Language Understanding Sytems-Volume 6, pp. 28–35. Association for Computational Linguistics (2000)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Rômulo César Costa de Sousa
    • 1
  • Hélio Lopes
    • 1
    Email author
  1. 1.Pontifícia Universidade Católica do Rio de JaneiroRio de JaneiroBrazil

Personalised recommendations