Abstract
Discovering relational structure between input features in sequence labeling models has shown to improve their accuracies in several problem settings. The problem of learning relational structure for sequence labeling can be posed as learning Markov Logic Networks (MLN) for sequence labeling, which we abbreviate as Markov Logic Chains (MLC). This objective in propositional space can be solved efficiently and optimally by a Hierarchical Kernels based approach, referred to as StructRELHKL, which we recently proposed. However, the applicability of StructRELHKL in complex first order settings is non-trivial and challenging. We present the challenges and possibilities for optimally and simultaneously learning the structure as well as parameters of MLCs (as against learning them separately and/or greedily). Here, we look into leveraging the StructRELHKL approach for optimizing the MLC learning steps to the extent possible. To this end, we categorize first order MLC features based on their complexity and show that complex features can be constructed from simpler ones. We define a self-contained class of features called absolute features (\(\mathcal{AF}\)), which can be conjoined to yield complex MLC features. Our approach first generates a set of relevant \(\mathcal{AF}\)s and then makes use of the algorithm for StructRELHKL to learn their optimal conjunctions. We demonstrate the efficiency of our approach by evaluating on a publicly available activity recognition dataset.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Getoor, L., Taskar, B.: Statistical relational learning. MIT Press (2006)
Nienhuys-Cheng, S.H., de Wolf, R.: Foundations of Inductive Logic Programming. Springer-Verlag New York, Inc., Secaucus (1997)
Richardson, M., Domingos, P.: Markov logic networks. Mach. Learn. 62(1-2), 107–136 (2006)
Domingos, P., Kok, S., Poon, H., Richardson, M., Singla, P.: Unifying logical and statistical AI. In: Proceedings of the 21st National Conference on Artificial Intelligence, AAAI 2006, vol. 1, pp. 2–7. AAAI Press (2006)
Muggleton, S., De Raedt, L., Poole, D., Bratko, I., Flach, P., Inoue, K., Srinivasan, A.: ILP turns 20. Mach. Learn. 86(1), 3–23 (2012)
Zhuo, H.H., Yang, Q., Hu, D.H., Li, L.: Learning complex action models with quantifiers and logical implications. Artif. Intell. 174(18), 1540–1569 (2010)
Kok, S., Domingos, P.: Learning the structure of markov logic networks. In: Proceedings of the 22nd International Conference on Machine Learning, ICML 2005, pp. 441–448. ACM, New York (2005)
Biba, M., Ferilli, S., Esposito, F.: Structure learning of markov logic networks through iterated local search. In: Proceedings of the 2008 Conference on ECAI 2008: 18th European Conference on Artificial Intelligence, pp. 361–365. IOS Press, Amsterdam (2008)
Khot, T., Natarajan, S., Kersting, K., Shavlik, J.: Learning markov logic networks via functional gradient boosting. In: Proceedings of the 2011 IEEE 11th International Conference on Data Mining, ICDM 2011, pp. 320–329. IEEE Computer Society, Washington, DC (2011)
Rabiner, L.R.: Readings in speech recognition, pp. 267–296. Morgan Kaufmann Publishers Inc., San Francisco (1990)
Lafferty, J.D., McCallum, A., Pereira, F.C.N.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning, ICML 2001, pp. 282–289. Morgan Kaufmann Publishers Inc., San Francisco (2001)
Forney, G.J.: The viterbi algorithm. Proceedings of IEEE 61(3), 268–278 (1973)
Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support vector machine learning for interdependent and structured output spaces. In: Proceedings of the Twenty-First International Conference on Machine Learning, ICML 2004, pp. 104–111. ACM, New York (2004)
McCallum, A., Li, W.: Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, CONLL 2003, vol. 4, pp. 188–191. Association for Computational Linguistics, Stroudsburg (2003)
Nair, N., Ramakrishnan, G., Krishnaswamy, S.: Enhancing activity recognition in smart homes using feature induction. In: Cuzzocrea, A., Dayal, U. (eds.) DaWaK 2011. LNCS, vol. 6862, pp. 406–418. Springer, Heidelberg (2011)
Nair, N., Saha, A., Ramakrishnan, G., Krishnaswamy, S.: Rule ensemble learning using hierarchical kernels in structured output spaces. In: Twenty-Sixth AAAI Conference on Artificial Intelligence (2012)
Wilson, D.H.: Assistive intelligent environments for automatic health monitoring. PhD Thesis, Carnegie Mellon University (2005)
van Kasteren, T., Noulas, A., Englebienne, G., Kröse, B.: Accurate activity recognition in a home setting. In: Proceedings of the 10th International Conference on Ubiquitous Computing, UbiComp 2008, pp. 1–9. ACM, New York (2008)
Gibson, C., van Kasteren, T., Krose, B.: Monitoring homes with wireless sensor networks. In: Proceedings of the International Med-e-Tel Conference (2008)
McCallum, A.K.: Efficiently inducing features of conditional random fields. In: Proceedings of the Nineteenth Conference Annual Conference on Uncertainty in Artificial Intelligence (2003)
Bach, F.: High-dimensional non-linear variable selection through hierarchical kernel learning. Technical report, INRIA, France (2009)
Gutmann, B., Kersting, K.: TildeCRF: Conditional random fields for logical sequences. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 174–185. Springer, Heidelberg (2006)
Huynh, T.N., Mooney, R.J.: Online max-margin weight learning with markov logic networks. In: Proceedings of the AAAI 2010 Workshop on Statistical Relational AI (Star-AI 2010), Atlanta, GA, pp. 32–37 (July 2010)
Huynh, T.N., Mooney, R.J.: Online structure learning for markov logic networks. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011, Part II. LNCS, vol. 6912, pp. 81–96. Springer, Heidelberg (2011)
Kersting, K., De Raedt, L., Raiko, T.: Logical hidden markov models. Journal of Artificial Intelligence Research 25 (2006)
Kersting, K.: Say em for selecting probabilistic models for logical sequences. In: Proceedings of the Twenty First Conference on Uncertainty in Artificial Intelligence, pp. 300–307. Morgan Kaufmann (2005)
Thon, I.: Don’t fear optimality: Sampling for probabilistic-logic sequence models. In: De Raedt, L. (ed.) ILP 2009. LNCS, vol. 5989, pp. 226–233. Springer, Heidelberg (2010)
Thon, I., Landwehr, N., De Raedt, L.: Stochastic relational processes: Efficient inference and applications. Mach. Learn. 82(2), 239–272 (2011)
Schulte, O., Khosravi, H., Kirkpatrick, A., Man, T., Gao, T., Zhu, Y.: Modelling relational statistics with bayes nets. In: Proceedings of 22nd International Conference on Inductive Logic Programming (ILP 2012). Springer (2012)
Flach, P., Lachiche, N.: 1BC: A first-order bayesian classifier. In: Džeroski, S., Flach, P. (eds.) ILP 1999. LNCS (LNAI), vol. 1634, pp. 92–103. Springer, Heidelberg (1999)
McCreath, E., Sharma, A.: LIME: A system for learning relations. In: Richter, M.M., Smith, C.H., Wiehagen, R., Zeugmann, T. (eds.) ALT 1998. LNCS (LNAI), vol. 1501, pp. 336–374. Springer, Heidelberg (1998)
Jawanpuria, P., Nath, J.S., Ramakrishnan, G.: Efficient rule ensemble learning using hierarchical kernels. In: Getoor, L., Scheffer, T. (eds.) ICML, pp. 161–168. Omnipress (2011)
Dehaspe, L., Toivonen, H.: Discovery of frequent datalog patterns. Data Min. Knowl. Discov. 3(1), 7–36 (1999)
Dehaspe, L., Toironen, H.: Relational data mining, pp. 189–208. Springer-Verlag New York, Inc., New York (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nair, N., Nagesh, A., Ramakrishnan, G. (2013). Probing the Space of Optimal Markov Logic Networks for Sequence Labeling. In: Riguzzi, F., Železný, F. (eds) Inductive Logic Programming. ILP 2012. Lecture Notes in Computer Science(), vol 7842. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38812-5_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-38812-5_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38811-8
Online ISBN: 978-3-642-38812-5
eBook Packages: Computer ScienceComputer Science (R0)