Abstract
As part of an architectural modeling project, this paper investigates the problem of understanding and manipulating images of buildings. Our primary motivation is to automatically detect and seamlessly remove unwanted foreground elements from urban scenes. Without explicit handling, these objects will appear pasted as artifacts on the model. Recovering the building facade in a video sequence is relatively simple because parallax induces foreground/background depth layers, but here we consider static images only. We develop a series of methods that enable foreground removal from images of buildings or brick walls. The key insight is to use a priori knowledge about grid patterns on building facades that can be modeled as Near Regular Textures (NRT). We describe a Markov Random Field (MRF) model for such textures and introduce a Markov Chain Monte Carlo (MCMC) optimization procedure for discovering them. This simple spatial rule is then used as a starting point for inference of missing windows, facade segmentation, outlier identification, and foreground removal.
Keywords
- Markov Chain Monte Carlo
- Markov Random Field
- Reversible Jump Markov Chain Monte Carlo
- Markov Random Field Model
- Markov Chain Monte Carlo Iteration
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Debevec, P., Taylor, C., Malik, J.: Modeling and rendering architecture from photographs. In: SIGGRAPH (1996)
Teller, S., Antone, M., Bodnar, Z., Bosse, M., Coorg, S., Jethwa, M., Master, N.: Calibrated, registered images of an extended urban area. Int. J. Computer Vision 53, 93–107 (2003)
Dick, A., Torr, P., Cipolla, R.: Modelling and interpretation of architecture from several images. Int. J. Computer Vision 60(2) (November 2004)
van den Heuvel, F.: Automation in Architectural Photogrammetry; Line-Photogrammetry for the Reconstruction from Single and Multiple Images. PhD thesis, Delft University of Technology, Delft, The Netherlands (2003)
Früh, C., Zakhor, A.: An automated method for large-scale, ground-based city model acquisition. Int. J. Comput. Vision 60(1), 5–24 (2004)
Wilczkowiak, M., Brostow, G., Tordoff, B., Cipolla, R.: Hole filling through photomontage. In: Proc. British Machine Vision Conference (2005)
Korah, T., Rasmussen, C.: Spatiotemporal inpainting for recovering texture maps of occluded building facades. IEEE Transactions on Image Processing 16(9), 2262–2271 (2007)
Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: SIGGRAPH, pp. 417–424 (2000)
Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Processing 13(9) (2004)
Sun, J., Yuan, L., Jia, J., Shum, H.Y.: Image completion with structure propagation. ACM Transactions on Graphics 24, 861–868 (2005)
Mayer, H., Reznik, S.: Building faade interpretation from image sequences. In: Proc. of the ISPRS Workshop CMRT 2005 - Object Extraction for 3D City Models, Road Databases and Traffic Monitoring - Concepts, Algorithms and Evaluation (2005)
Neal, R.M.: Probabilistic inference using markov chain Monte Carlo methods. Technical Report CRG-TR-93-1, University of Toronto (1993)
Wonka, P., Wimmer, M., Sillion, F., Ribarsky, W.: Instant architecture. ACM Transactions on Graphics 22, 669–677 (2003)
Alegre, F., Dellaert, F.: A probabilistic approach to the semantic interpretation of building facades. In: International Workshop on Vision Techniques Applied to the Rehabilitation of City Centres (2004)
Mueller, P., Zeng, G., Wonka, P., Gool, L.V.: Image-based procedural modeling of facades. In: Proceedings of ACM SIGGRAPH 2007. ACM Press, New York (2007)
Schaffalitzky, F., Zisserman, A.: Geometric grouping of repeated elements within images. In: Proceedings of the 9th British Machine Vision Conference, Southampton (1998)
Turina, A., Tuytelaars, T., Gool, L.V.: Efficient grouping under perspective skew. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (2001)
Liu, Y., Lin, W.C., Hays, J.H.: Near regular texture analysis and manipulation. In: ACM Transactions on Graphics (SIGGRAPH 2004), vol. 23(3), pp. 368–376 (August 2004)
Liu, Y., Collins, R., Tsin, Y.: A computational model for periodic pattern perception based on frieze and wallpaper groups. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(3), 354–371 (2004)
Hays, J.H., Leordeanu, M., Efros, A.A., Liu, Y.: Discovering texture regularity as a higher-order correspondence problem. In: 9th European Conference on Computer Vision (May 2006)
Ahuja, N., Todorovic, S.: Extracting texels in 2.1d natural textures. In: Proc. IEEE Int. Conf. Computer Vision (ICCV 2007) (2007)
Li, S.Z.: Markov random field modeling in computer vision. Springer, Heidelberg (1995)
Hartelius, K., Carstensen, J.M.: Bayesian grid matching. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 162–173 (2003)
Lin, W.C., Liu, Y.: Tracking dynamic near-regular textures under occlusion and rapid movements. In: 9th European Conference on Computer Vision (2006)
Matsuyama, T., Miura, S., Nagao, M.: A structural analysis of natural textures by fourier transformation. In: Proc. Int. Conf. Pattern Recognition (1982)
Han, F., Zhu, S.C.: Bottom-up/top-down image parsing by attribute graph grammar. In: Proc. of the IEEE International Conference on Computer Vision (ICCV 2005) (2005)
Khan, Z., Balch, T., Dellaert, F.: Mcmc-based particle filtering for tracking a variable number of interacting targets. Pattern Analysis and Machine Intelligence 27(11), 1805–1918 (2005)
Hastings, W.: Monte carlo sampling methods using markov chains and their applications. Biometrika 57(1), 97–109 (1970)
Lee, S., Liu, Y.: Psu Near-Regular Texture Database (2007), http://vivid.cse.psu.edu/texturedb/gallery/
Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: Proceedings of the British Machine Vision Conference (2002)
Rother, C., Kolmogorov, V., Blake, A.: Grabcut - interactive foreground extraction using iterated graph cuts. In: SIGGRAPH (2004)
la Torre, F.D., Black, M.: A framework for robust subspace learning. Int. J. Computer Vision 54, 117–142 (2003)
Turk, M., Pentland, A.: Face recognition using eigenfaces. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (1991)
Xu, L., Yuille, A.: Robust principal component analysis by self-organizing rules based on statistical physics approach. IEEE Transactions on Neural Networks 6(1), 131–143 (1995)
Roweis, S.: EM algorithms for PCA and SPCA. In: Advances in Neural Information Processing Systems (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Korah, T., Rasmussen, C. (2008). Analysis of Building Textures for Reconstructing Partially Occluded Facades. In: Forsyth, D., Torr, P., Zisserman, A. (eds) Computer Vision – ECCV 2008. ECCV 2008. Lecture Notes in Computer Science, vol 5302. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88682-2_28
Download citation
DOI: https://doi.org/10.1007/978-3-540-88682-2_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88681-5
Online ISBN: 978-3-540-88682-2
eBook Packages: Computer ScienceComputer Science (R0)