Analysis of Building Textures for Reconstructing Partially Occluded Facades

  • Thommen Korah
  • Christopher Rasmussen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5302)


As part of an architectural modeling project, this paper investigates the problem of understanding and manipulating images of buildings. Our primary motivation is to automatically detect and seamlessly remove unwanted foreground elements from urban scenes. Without explicit handling, these objects will appear pasted as artifacts on the model. Recovering the building facade in a video sequence is relatively simple because parallax induces foreground/background depth layers, but here we consider static images only. We develop a series of methods that enable foreground removal from images of buildings or brick walls. The key insight is to use a priori knowledge about grid patterns on building facades that can be modeled as Near Regular Textures (NRT). We describe a Markov Random Field (MRF) model for such textures and introduce a Markov Chain Monte Carlo (MCMC) optimization procedure for discovering them. This simple spatial rule is then used as a starting point for inference of missing windows, facade segmentation, outlier identification, and foreground removal.


Markov Chain Monte Carlo Markov Random Field Reversible Jump Markov Chain Monte Carlo Markov Random Field Model Markov Chain Monte Carlo Iteration 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Debevec, P., Taylor, C., Malik, J.: Modeling and rendering architecture from photographs. In: SIGGRAPH (1996)Google Scholar
  2. 2.
    Teller, S., Antone, M., Bodnar, Z., Bosse, M., Coorg, S., Jethwa, M., Master, N.: Calibrated, registered images of an extended urban area. Int. J. Computer Vision 53, 93–107 (2003)CrossRefGoogle Scholar
  3. 3.
    Dick, A., Torr, P., Cipolla, R.: Modelling and interpretation of architecture from several images. Int. J. Computer Vision 60(2) (November 2004)Google Scholar
  4. 4.
    van den Heuvel, F.: Automation in Architectural Photogrammetry; Line-Photogrammetry for the Reconstruction from Single and Multiple Images. PhD thesis, Delft University of Technology, Delft, The Netherlands (2003)Google Scholar
  5. 5.
    Früh, C., Zakhor, A.: An automated method for large-scale, ground-based city model acquisition. Int. J. Comput. Vision 60(1), 5–24 (2004)CrossRefGoogle Scholar
  6. 6.
    Wilczkowiak, M., Brostow, G., Tordoff, B., Cipolla, R.: Hole filling through photomontage. In: Proc. British Machine Vision Conference (2005)Google Scholar
  7. 7.
    Korah, T., Rasmussen, C.: Spatiotemporal inpainting for recovering texture maps of occluded building facades. IEEE Transactions on Image Processing 16(9), 2262–2271 (2007)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: SIGGRAPH, pp. 417–424 (2000)Google Scholar
  9. 9.
    Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Processing 13(9) (2004)Google Scholar
  10. 10.
    Sun, J., Yuan, L., Jia, J., Shum, H.Y.: Image completion with structure propagation. ACM Transactions on Graphics 24, 861–868 (2005)CrossRefGoogle Scholar
  11. 11.
    Mayer, H., Reznik, S.: Building faade interpretation from image sequences. In: Proc. of the ISPRS Workshop CMRT 2005 - Object Extraction for 3D City Models, Road Databases and Traffic Monitoring - Concepts, Algorithms and Evaluation (2005)Google Scholar
  12. 12.
    Neal, R.M.: Probabilistic inference using markov chain Monte Carlo methods. Technical Report CRG-TR-93-1, University of Toronto (1993)Google Scholar
  13. 13.
    Wonka, P., Wimmer, M., Sillion, F., Ribarsky, W.: Instant architecture. ACM Transactions on Graphics 22, 669–677 (2003)CrossRefGoogle Scholar
  14. 14.
    Alegre, F., Dellaert, F.: A probabilistic approach to the semantic interpretation of building facades. In: International Workshop on Vision Techniques Applied to the Rehabilitation of City Centres (2004)Google Scholar
  15. 15.
    Mueller, P., Zeng, G., Wonka, P., Gool, L.V.: Image-based procedural modeling of facades. In: Proceedings of ACM SIGGRAPH 2007. ACM Press, New York (2007)Google Scholar
  16. 16.
    Schaffalitzky, F., Zisserman, A.: Geometric grouping of repeated elements within images. In: Proceedings of the 9th British Machine Vision Conference, Southampton (1998)Google Scholar
  17. 17.
    Turina, A., Tuytelaars, T., Gool, L.V.: Efficient grouping under perspective skew. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (2001)Google Scholar
  18. 18.
    Liu, Y., Lin, W.C., Hays, J.H.: Near regular texture analysis and manipulation. In: ACM Transactions on Graphics (SIGGRAPH 2004), vol. 23(3), pp. 368–376 (August 2004)Google Scholar
  19. 19.
    Liu, Y., Collins, R., Tsin, Y.: A computational model for periodic pattern perception based on frieze and wallpaper groups. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(3), 354–371 (2004)CrossRefGoogle Scholar
  20. 20.
    Hays, J.H., Leordeanu, M., Efros, A.A., Liu, Y.: Discovering texture regularity as a higher-order correspondence problem. In: 9th European Conference on Computer Vision (May 2006)Google Scholar
  21. 21.
    Ahuja, N., Todorovic, S.: Extracting texels in 2.1d natural textures. In: Proc. IEEE Int. Conf. Computer Vision (ICCV 2007) (2007)Google Scholar
  22. 22.
    Li, S.Z.: Markov random field modeling in computer vision. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  23. 23.
    Hartelius, K., Carstensen, J.M.: Bayesian grid matching. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 162–173 (2003)CrossRefGoogle Scholar
  24. 24.
    Lin, W.C., Liu, Y.: Tracking dynamic near-regular textures under occlusion and rapid movements. In: 9th European Conference on Computer Vision (2006)Google Scholar
  25. 25.
    Matsuyama, T., Miura, S., Nagao, M.: A structural analysis of natural textures by fourier transformation. In: Proc. Int. Conf. Pattern Recognition (1982)Google Scholar
  26. 26.
    Han, F., Zhu, S.C.: Bottom-up/top-down image parsing by attribute graph grammar. In: Proc. of the IEEE International Conference on Computer Vision (ICCV 2005) (2005)Google Scholar
  27. 27.
    Khan, Z., Balch, T., Dellaert, F.: Mcmc-based particle filtering for tracking a variable number of interacting targets. Pattern Analysis and Machine Intelligence 27(11), 1805–1918 (2005)CrossRefGoogle Scholar
  28. 28.
    Hastings, W.: Monte carlo sampling methods using markov chains and their applications. Biometrika 57(1), 97–109 (1970)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Lee, S., Liu, Y.: Psu Near-Regular Texture Database (2007),
  30. 30.
    Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: Proceedings of the British Machine Vision Conference (2002)Google Scholar
  31. 31.
    Rother, C., Kolmogorov, V., Blake, A.: Grabcut - interactive foreground extraction using iterated graph cuts. In: SIGGRAPH (2004)Google Scholar
  32. 32.
    la Torre, F.D., Black, M.: A framework for robust subspace learning. Int. J. Computer Vision 54, 117–142 (2003)CrossRefzbMATHGoogle Scholar
  33. 33.
    Turk, M., Pentland, A.: Face recognition using eigenfaces. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (1991)Google Scholar
  34. 34.
    Xu, L., Yuille, A.: Robust principal component analysis by self-organizing rules based on statistical physics approach. IEEE Transactions on Neural Networks 6(1), 131–143 (1995)CrossRefGoogle Scholar
  35. 35.
    Roweis, S.: EM algorithms for PCA and SPCA. In: Advances in Neural Information Processing Systems (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Thommen Korah
    • 1
  • Christopher Rasmussen
    • 2
  1. 1.HRL LaboratoriesLLC MalibuUSA
  2. 2.University of DelawareNewarkUSA

Personalised recommendations