Learning Logic Rules for Scene Interpretation Based on Markov Logic Networks

  • Mai Xu
  • Maria Petrou
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5996)


We propose a novel logic-rule learning approach for the Tower of Knowledge (ToK) architecture, based on Markov logic networks, for scene interpretation. This approach is in the spirit of the recently proposed Markov logic networks of machine learning. Its purpose is to learn the soft-constraint logic rules for labelling the components of a scene. This approach also benefits from the architecture of ToK, in reasoning whether a component in a scene has the right characteristics in order to fulfil the functions a label implies, from the logic point of view. One significant advantage of the proposed approach, rather than the previous versions of ToK, is its automatic logic learning capability such that the manual insertion of logic rules is not necessary. Experiments of building scene interpretation illustrate the promise of this approach.


Training Dataset Soft Constraint Logic Rule Inductive Logic Programming Gradient Ascent 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Neumann, B., Möller, R.: On scene interpretation with description logics. Image and Vision Computing 26(1), 82–101 (2008)CrossRefGoogle Scholar
  2. 2.
    Sudderth, E.B., Torralba, A., William, F.T., Willsky, A.S.: Describing visual scenes using transformed objects and parts. International Journal of Computer Vision 77(1-3), 291–330 (2008)CrossRefGoogle Scholar
  3. 3.
    Thomas, A., Ferrari, V., Leibe, B., Tuytelaars, T., Schiele, B., Gool., L.V.: Towards multi-view object class detection. In: Proceedings of CVPR, pp. 1589–1596 (2006)Google Scholar
  4. 4.
    Heitz, G., Gould, S., Saxena, A., Koller, D.: Cascaded classification models: Combining models for holistic scene understanding. In: Proceedings of NIPS (2008)Google Scholar
  5. 5.
    Fei-fei, L., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: Proceedings of CVPR, pp. 524–531 (2005)Google Scholar
  6. 6.
    Komodakis, N., Tziritas, G., Paragios, N.: Fast approximately optimal solutions for single and dynamic MRFs. In: Proceedings of CVPR, pp. 1–8 (2007)Google Scholar
  7. 7.
    Petrou, M.: Learning in computer vision: some thoughts. In: Rueda, L., Mery, D., Kittler, J. (eds.) CIARP 2007. LNCS, vol. 4756, pp. 1–12. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  8. 8.
    Petrou, M., Xu, M.: The tower of knowledge scheme for learning in computer vision. In: Proceedings of DICTA 2007, pp. 85–91 (2007)Google Scholar
  9. 9.
    Xu, M., Petrou, M.: Recursive tower of knowledge for learning to interpret scenes. In: Proceedings of BMVC (2008)Google Scholar
  10. 10.
    Yakimovsky, Y., Feldman, J.: A semantics-based decision theory region analyzer. In: Proceedings of IJCAI, pp. 580–588 (1973)Google Scholar
  11. 11.
    Ohta, Y.: Knowledge-based interpretation of outdoor natural color scenes. Pitman Publishing (1985)Google Scholar
  12. 12.
    Han, F., Zhu, S.: Bottom-up/top-down image parsing with attribute graph grammar. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(1), 59–73 (2009)CrossRefMathSciNetGoogle Scholar
  13. 13.
    Richardson, M., Domingos, P.: Markov logic networks. Machine Learning 62(1-2), 107–136 (2006)CrossRefGoogle Scholar
  14. 14.
    Mitchell, T.M.: Machine Learning, 1st edn. McGraw-Hill Higher Education, New York (1997)zbMATHGoogle Scholar
  15. 15.
    Quinlan, J.R., Cameron-Jones, M.: Foil: A midterm report. In: Brazdil, P.B. (ed.) ECML 1993. LNCS, vol. 667, pp. 3–20. Springer, Heidelberg (1993)Google Scholar
  16. 16.
    IST06: E-training for interpreting images of man-made scenes,
  17. 17.
    IMPACT: The image processing for automatic cartographic tools project,
  18. 18.
    Dick, A.R., Torr, P.H.S., Cipolla, R.: Modelling and interpretation of architecture from several images. International Journal of Computer Vision 60(2), 111–134 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Mai Xu
    • 1
  • Maria Petrou
    • 1
  1. 1.Electrical and Electronic Engineering DepartmentImperial College LondonLondonUnited Kingdom

Personalised recommendations