Abstract
We show that the recently proposed piecewise approximation approach can benefit conditional random fields estimation using the structured perceptron algorithm. We present experiments in noun-phrase chunking task on the CoNLL-2000 corpus. The results show that, compared to standard training, applying the piecewise approach during model estimation may yield not only savings in training time but also improvement in model performance on test set due to added model regularization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Lafferty, J., McCallum, A., Pereira, F.C.N.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning, pp. 282–289 (2001)
Collins, M.: Discriminative training methods for hidden markov models: Theory and Experiments with Perceptron Algorithms. In: Proceedings of the ACL 2002 Conference on Empirical Methods in Natural Language Processing, vol. 10, pp. 1–8. Association for Computational Linguistics (2002)
Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y.: Large margin methods for structured and interdependent output variables. Journal of Machine Learning Research 6, 1453–1484 (2005)
Zhang, Y., Clark, S.: Syntactic processing using the generalized perceptron and beam search. Computational Linguistics 37(1), 105–151 (2011)
Sutton, C., McCallum, A.: Piecewise training for structured prediction. Machine Learning 77(2), 165–194 (2009)
Sontag, D., Meshi, O., Jaakkola, T., Globerson, A.: More data means less inference: A pseudo-max approach to structured learning. In: Lafferty, J., Williams, C.K.I., Shawe-Taylor, J., Zemel, R., Culotta, A. (eds.) Advances in Neural Information Processing Systems 23, pp. 2181–2189 (2010)
Samdani, R., Roth, D.: Efficient decomposed learning for structured prediction. In: Proceedings of the 29th International Conference on Machine Learning (2012)
Srikumar, V., Kundu, G., Roth, D.: On amortizing inference cost for structured prediction. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1114–1124 (2012)
Vapnik, V.: An overview of statistical learning theory. IEEE Transactions on Neural Networks 10, 988–999 (1999)
Freund, Y., Schapire, R.: Large margin classification using the perceptron algorithm. Machine Learning 37(3), 277–296 (1999)
Daumé III, H.: Practical structured learning techniques for natural language processing. PhD thesis, University of Southern California (2006)
Kim, T., Sang, E., Buchholz, S.: Introduction to the conll-2000 shared task: Chunking. In: Proceedings of the 2nd Workshop on Learning Language in Logic and the 4th Conference on Computational Natural Language Learning, vol. 7, pp. 127–132. Association for Computational Linguistics (2000)
Marcus, M., Marcinkiewicz, M., Santorini, B.: Building a large annotated corpus of english: The penn treebank. Computational Linguistics 19(2), 313–330 (1993)
Bishop, C.: Pattern recognition and machine learning, vol. 4. Springer, New York (2006)
Frey, B., MacKay, D.: A revolution: Belief propagation in graphs with cycles. Advances in Neural Information Processing Systems, 479–485 (1998)
Wainwright, M.J., Jaakkola, T.S., Willsky, A.S.: Map estimation via agreement on trees: message-passing and linear programming. IEEE Transactions on Information Theory 51, 3697–3717 (2005)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 1222–1239 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ruokolainen, T. (2012). Applying Piecewise Approximation in Perceptron Training of Conditional Random Fields. In: Hollmén, J., Klawonn, F., Tucker, A. (eds) Advances in Intelligent Data Analysis XI. IDA 2012. Lecture Notes in Computer Science, vol 7619. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34156-4_30
Download citation
DOI: https://doi.org/10.1007/978-3-642-34156-4_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34155-7
Online ISBN: 978-3-642-34156-4
eBook Packages: Computer ScienceComputer Science (R0)