Abstract
Numerals contain rich semantic information in financial documents, and they play significant roles in financial data analysis and financial decision making. This paper proposes a model based on the Bidirectional Encoder Representations from Transformers (BERT) to identify the category and subcategory of a numeral in financial documents. Our model holds the obvious advantages in the fine-grained numeral understanding and achieves good performance in the FinNum task at NTCIR-14. The FinNum task is to classify the numerals in financial tweets into seven categories, and further extend these categories into seventeen subcategories. In our proposed model, we first analyze the obtained financial data from the FinNum task and enhance data for some subcategories by entity replacement. And then, we adopt our fine-tuning BERT model to finish the task. As a supplement, some popular traditional and deep learning models have been selected for comparative experiments, and the experimental results show that our model has achieved the state-of-the-art performances.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dhar, V., Stein, R.M.: FinTech platforms and strategy. Commun. ACM 60(10), 32–35 (2017)
Zhou, Y., Ni, B., Yan, S., Moulin, P., Tian, Q.: Pipelining localized semantic features for fine-grained action recognition. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 481–496. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_32
Karaoglu, S., et al.: Con-text: text detection for fine-grained object classification. IEEE Trans. Image Process. 26, 3965–3980 (2017)
McCallum, A., Nigam, K.: A comparison of event models for Naive Bayes text classification. In: AI-98 Workshop on Learning for Text Categorization, vol. 752, no. 1, pp. 41–48 (1998)
Kim, Y.: Convolutional neural networks for sentence classification. Eprint Arxiv (2014)
Hüsken, M., Stagge, P.: Recurrent neural networks for time series classification. Neurocomputing 50, 223–235 (2003)
Zhou, C., Sun, C., et al.: A C-LSTM neural network for text classification. Comput. Sci. 1(4), 39–44 (2015)
Devlin, J., Chang, M.W., Lee, K., et al.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Chen, C.C., Huang, H.H., Takamura, et al.: Overview of the NTCIR-14 FinNum task: fine-grained numeral understanding in financial social media data. In: Proceedings of the 14th NTCIR Conference on Evaluation of Information Access Technologies (2019)
Zhu, Y., Ryan, K., Richard, S., et al.: Aligning books and movies: towards story-like visual explanations by watching movies and reading books. In: 2015 IEEE International Conference on Computer Vision, pp. 19–27 (2015)
Lee, et al.: BioBERT: pre-trained biomedical language representation model for biomedical text mining. arXiv preprint arXiv:1901.08746 (2019)
Armand, J., Edouard, G., Piotr, B., Douze, M., et al.: FastText.zip: compressing text classification models. arXiv preprint arXiv:1612.03651 (2016)
Moraes, R., Valiati, J.F., Neto, W.P.G.: Document-level sentiment classification: an empirical comparison between SVM and ANN. Expert Syst. Appl. 40(2), 621–633 (2013)
Zhang, X., Zhao, J.B., et al.: Character-level convolutional networks for text classification. In: International Conference on Neural Information Processing Systems (2015)
Lai, S., Xu, L., Liu, K., et al.: Recurrent convolutional neural networks for text classification. In: Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
Liu, Y., et al.: An attention-gated convolutional neural network for sentence classification. CoRR (2018)
Pappas, N., Popescu, B.A.: Multilingual hierarchical attention networks for document classification. arXiv preprint arXiv:1707.00896 (2017)
Zhang, Y., et al.: A text sentiment classification modeling method based on coordinated CNN-LSTM-attention model. Chin. J. Electron. 28(01), 124–130 (2019)
Liu, P.F., Qiu, X., Huang, X.: Adversarial multi-task learning for text classification. arXiv preprint arXiv:1704.05742 (2017)
Peters, M.E., et al.: Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018)
Alec, R., Karthik, N., Tim, S., et al.: Improving language understanding with unsupervised learning. Technical report. OpenAI (2018)
Chen, Q., Zhuo, Z., Wang, W.: BERT for joint intent classification and slot filling. arXiv preprint arXiv:1902.10909 (2019)
Schumaker, R.P., Chen, H.: Textual analysis of stock market prediction using breaking financial news. The AZFin text system (2009)
Suykens, J.A., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)
Vaswani, A., et al.: Attention is all you need. Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Yang, Y.: An evaluation of statistical approaches to text categorization. Inf. Retrieval 1(1–2), 69–90 (1999)
Acknowledgments
The work presented in this paper is partially supported by the Major Projects of National Social Foundation of China under Grant No. 11&ZD189.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, W., Liu, M., Zhang, Y., Xiang, J., Mao, R. (2019). Financial Numeral Classification Model Based on BERT. In: Kato, M., Liu, Y., Kando, N., Clarke, C. (eds) NII Testbeds and Community for Information Access Research. NTCIR 2019. Lecture Notes in Computer Science(), vol 11966. Springer, Cham. https://doi.org/10.1007/978-3-030-36805-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-36805-0_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36804-3
Online ISBN: 978-3-030-36805-0
eBook Packages: Computer ScienceComputer Science (R0)