Abstract
Having precise measurements of an indoor scene is important for several applications - e.g.augmented reality furniture placement - whereas geometric details are only needed up to a certain scale. Depth sensors provide a highly detailed reconstruction but mobile phones are not able to display and manipulate these models in real-time due to the massive amount of data and the lack of computational power. This paper therefore aims to close this gap and provides a simplification of indoor scenes. RGB-D input sequences are exploited to extract wall segments and object candidates. For each input frame, walls, ground plane and ceiling are estimated by plane segments, object candidates are detected using a state-of-the-art object detector. The objects’ correct poses and semantic types are gathered by exploiting a 3D CAD dataset and by introducing a Markov Random Field over time. A vast variety of experiments outline the practicability and low memory consumption of the resulting models on mobile phones and demonstrate the ability of preserving precise 3D measurements based on a variety of real indoor scenes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aubry, M., Maturana, D., Efros, A., Russell, B., Sivic, J.: Seeing 3d chairs: exemplar part-based 2d–3d alignment using a large dataset of CAD models. In: Proc. of CVPR, pp. 3762–3769 (2014)
Bódis-Szomorú, A., Riemenschneider, H., Gool, L.V.: Fast, approximate piecewise-planar modeling based on sparse structure-from-motion and superpixels. In: Proc. of CVPR, pp. 469–476 (2014)
Choi, W., Chao, Y., Pantofaru, C., Savarese, S.: Understanding indoor scenes using 3d geometric phrases. In: Proc. of CVPR, pp. 33–40 (2013)
Edelsbrunner, H., Kirkpatrick, D., Seidel, R.: On the shape of a set of points in the plane. Trans. on Information Theory 29(4), 551–558 (1983)
Felzenszwalb, P., Girshick, R., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. TPAMI 32(9), 1627–1645 (2010)
Felzenszwalb, P., Huttenlocher, D.: Efficient graph-based image segmentation. IJCV 59(2), 167–181 (2004)
Garland, M., Heckbert, P.: Surface simplification using quadric error metrics. In: SIGGRAPH, pp. 209–216 (1997)
Gupta, A., Efros, A.A., Hebert, M.: Blocks world revisited: image understanding using qualitative geometry and mechanics. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 482–496. Springer, Heidelberg (2010)
Hedau, V., Hoiem, D., Forsyth, D.: Recovering the spatial layout of cluttered rooms. In: Proc. of ICCV (2009)
Hermans, A., Floros, G., Leibe, B.: Dense 3d semantic mapping of indoor scenes from RGB-D images. In: Proc. of ICRA, pp. 2631–2638 (2014)
Hödlmoser, M., Mičušík, B., Kampel, M.: Sparse point cloud densification by combining multiple segmentation methods. In: Proc. of 3DV (2013)
Hoiem, D., Efros, A., Hebert, M.: Recovering surface layout from an image. IJCV 75(1) (2007)
Newcombe, R., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A., Kohli, P., Shotton, J., Hodges, S., Fitzgibbon, A.: Kinectfusion: real-time dense surface mapping and tracking. In: Proc. of ISMAR, pp. 127–136 (2011)
Pero, L.D., Bowdish, J., Fried, D., Kermgard, B., Hartley, E., Barnard, K.: Bayesian geometric modeling of indoor scenes. In: Proc. of CVPR (2012)
Rusu, R.B., Bradski, G., Thibaux, R., Hsu, J.: Fast 3d recognition and pose using the viewpoint feature histogram. In: Proc. of IROS, October 2010
Schwing, A., Hazan, T., Pollefeys, M., Urtasun, R.: Efficient structured prediction for 3d indoor scene understanding. In: Proc. of CVPR, pp. 2815–2822 (2012)
Song, S., Xiao, J.: Sliding shapes for 3D object detection in depth images. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part VI. LNCS, vol. 8694, pp. 634–651. Springer, Heidelberg (2014)
Taylor, C., Cowley, A.: Parsing indoor scenes using RGB-D imagery. In: Robotics: Science and Systems (2012)
Valentin, J., Sengupta, S., Warrell, J., Shahrokni, A., Torr, P.: Mesh based semantic modelling for indoor and outdoor scenes. In: CVPR, pp. 2067–2074 (2013)
Zhang, J., Kan, C., Schwing, A., Urtasun, R.: Estimating the 3d layout of indoor scenes and its clutter from depth sensors. In: Proc. of ICCV, pp. 1273–1280 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Hödlmoser, M., Wolf, P., Kampel, M. (2015). Simplifying Indoor Scenes for Real-Time Manipulation on Mobile Devices. In: Azzopardi, G., Petkov, N. (eds) Computer Analysis of Images and Patterns. CAIP 2015. Lecture Notes in Computer Science(), vol 9257. Springer, Cham. https://doi.org/10.1007/978-3-319-23117-4_42
Download citation
DOI: https://doi.org/10.1007/978-3-319-23117-4_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23116-7
Online ISBN: 978-3-319-23117-4
eBook Packages: Computer ScienceComputer Science (R0)