Abstract
We investigate maximum likelihood parameter learning in Conditional Random Fields (CRF) and present an empirical study of pseudo-likelihood (PL) based approximations of the parameter likelihood gradient. We show, as opposed to [1][2], that these parameter learning methods can be improved and evaluate the resulting performance employing different inference techniques. We show that the approximation based on penalized pseudo-likelihood (PPL) in combination with the Maximum A Posteriori (MAP) inference yields results comparable to other state of the art approaches, while providing advantages to formulating parameter learning as a convex optimization problem. Eventually, we demonstrate applicability on the task of detecting man-made structures in natural images.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kumar, S., Hebert, M.: Discriminative random fields: A discriminative framework for contextual interaction in classification. In: Proc. of the 9th IEEE International Conference on Computer Vision, vol. 2, pp. 1150–1157 (2003)
Kumar, S., August, J., Hebert, M.: Exploiting inference for approximate parameter learning in discriminative fields: An empirical study. In: 5th International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition (2005)
Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proc. 18th International Conf. on Machine Learning, pp. 282–289 (2001)
Korč, F., Förstner, W.: Interpreting terrestrial images of urban scenes using Discriminative Random Fields. In: Proc. of the 21st Congress of the International Society for Photogrammetry and Remote Sensing (ISPRS) (July 2008)
Kumar, S., Hebert, M.: Discriminative random fields. International Journal of Computer Vision 68(2), 179–201 (2006)
Vishwanathan, S.V.N., Schraudolph, N.N., Schmidt, M.W., Murphy, K.: Accelerated training of conditional random fields with stochastic gradient methods. In: Cohen, W.W., Moore, A. (eds.) Proc. of the 24th International Conf. on Machine Learning. ACM International Conference Proceeding Series, vol. 148, pp. 969–976. ACM Press, New York (2006)
Sutton, C., McCallum, A.: Piecewise pseudolikelihood for efficient training of conditional random fields. In: Ghahramani, Z. (ed.) International Conference on Machine learning (ICML). ACM International Conference Proceeding Series, vol. 227, pp. 863–870 (2007)
Lee, C.-H., Wang, S., Jiao, F., Schuurmans, D., Greiner, R.: Learning to model spatial dependency: Semi-supervised discriminative random fields. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems 19, pp. 793–800. MIT Press, Cambridge (2007)
Kumar, S., Hebert, M.: Multiclass discriminative fields for parts-based object detection. In: Snowbird Learning Workshop (2004)
Besag, J.: Statistical analysis of non-lattice data. The Statistician 24(3), 179–195 (1975)
Besag, J.: Efficiency of pseudo-likelihood estimation for simple gaussian fields. Biometrika 64, 616–618 (1977)
Geiger, D., Girosi, F.: Parallel and deterministic algorithms from mrfs: Surface reconstruction. IEEE Trans. on Pattern Analysis and Machine Intelligence 13(5), 401–412 (1991)
McCallum, A., Rohanimanesh, K., Sutton, C.: Dynamic conditional random fields for jointly labeling multiple sequences. In: NIPS Workshop on Syntax, Semantics, and Statistics (December 2003)
Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Computation 14(8), 1771–1800 (2002)
Greig, D.M., Porteous, B.T., Seheult, A.H.: Exact maximum a posteriori estimation for binary images. Journal of the Royal Statistical Society 51(2), 271–279 (1989)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. on Pattern Analysis and Machine Intelligence 23(11), 1222–1239 (2001)
Besag, J.: On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society 48(3), 259–302 (1986)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Korč, F., Förstner, W. (2008). Approximate Parameter Learning in Conditional Random Fields: An Empirical Investigation. In: Rigoll, G. (eds) Pattern Recognition. DAGM 2008. Lecture Notes in Computer Science, vol 5096. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69321-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-540-69321-5_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69320-8
Online ISBN: 978-3-540-69321-5
eBook Packages: Computer ScienceComputer Science (R0)