Point sets joint registration and co-segmentation
We present a novel approach of joint registration and co-segmentation for point sets where objects move in different ways. We consider joint registration and co-segmentation as two problems that are heavily entangled with each other; thus, we represent the input point sets as samples from a generative model and bring up with a novel formulation based on Gaussian mixture model. By maximizing the posterior probability of the samples, we gradually recover the latent object models as well as an object-level segmentation and simultaneously align the segmented points to the latent object models. Along with the formulation, we design an interactive tool that helps users intuitively intervene the process to optimize the registration and segmentation results. The experiment results on a group of synthetic and scanned point clouds demonstrate that our method is powerful and effective for joint registration and co-segmentation on point sets of multiple objects.
KeywordsPoint cloud Registration Co-segmentation
We would like to thank YantingLin and Jian Wu. They helped with data preparation for our experiments. We would also like to thank the National Natural Science Foundation for their funding. This study was funded by the National Natural Science Foundation of China under Nos. 61472377, 61632006, and 6133101.
Compliance with ethical standards
Conflict of interest
Siyu Hu declares that he has no conflict of interest. Xuejin Chen has received research Grants from Microsoft and Huawei Technology Co. Ltd. Xuejin Chen had visited Leonidas Guibass Group in Stanford University during February 21 to August 20, 2017. Xin Tong is researcher of Microsoft. He is associate editor of ACM TOG and IEEE TVCG. He is also guest professor of University of Science and Technology of China and Tianjin University.
- 1.Armeni, I., Sener, O., Zamir, A.R., Jiang, H., Brilakis, I., Fischer, M., Savarese, S.: 3D semantic parsing of large-scale indoor spaces. In: IEEE Computer Society, pp. 1534–1543. Los Alamitos, CA (2016). https://doi.org/10.1109/CVPR.2016.170
- 2.Campbell, D., Petersson, L.: Gogma: Globally-optimal Gaussian mixture alignment. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5685–5694 (2016). https://doi.org/10.1109/CVPR.2016.613
- 3.Charles, R.Q., Su, H., Kaichun, M., Guibas, L.J.: Pointnet: deep learning on point sets for 3d classification and segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 77–85 (2017). https://doi.org/10.1109/CVPR.2017.16
- 8.Dema, M.A., Sari-Sarraf, H.: 3D scene generation by learning from examples. In: IEEE International Symposium on Multimedia, pp. 58–64 (2012). https://doi.org/10.1109/ISM.2012.19
- 9.Evangelidis, G.D., Kounades-Bastian, D., Horaud, R., Psarakis, E.Z.: A generative model for the joint registration of multiple point sets. In: ECCV, pp. 109–122 (2014). https://doi.org/10.1007/978-3-319-10584-0_8
- 13.Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., Fitzgibbon, A.: Kinectfusion: real-time 3D reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST ’11, pp. 559–568. ACM, New York (2011). https://doi.org/10.1145/2047196.2047270
- 16.Li, Y., Paluri, M., Rehg, J.M., Dollar, P.: Unsupervised learning of edges. In: CVPR, pp. 1619–1627 (2016). https://doi.org/10.1109/CVPR.2016.179
- 17.Liu, Z., Zhang, Y., Wu, W., Liu, K., Sun, Z.: Model-driven indoor scenes modeling from a single image. In: Proceedings of the 41st Graphics Interface Conference, GI ’15, Halifax, Nova Scotia, Canada, June 3–5, 2015, pp. 25–32. Canadian Information Processing Society, Toronto (2015). http://dl.acm.org/citation.cfm?id=2788896
- 22.Rother, C., Kolmogorov, V., Blake, A.: GrabCut: interactive foreground extraction using iterated graph cuts. In: ACM SIGGRAPH 2004 Papers, pp. 309–314. ACM, New York (2004). https://doi.org/10.1145/1186562.1015720
- 23.Rother, C., Minka, T.P., Blake, A., Kolmogorov, V.: Cosegmentation of image pairs by histogram matching: incorporating a global constraint into MRFS. In: CVPR, pp. 993–1000 (2006). https://doi.org/10.1109/CVPR.2006.91
- 24.Strasdat, H., Newcombe, R.A., Salas-Moreno, R.F., Kelly, P.H., Davison, A.J.: Slam++: simultaneous localisation and mapping at the level of objects. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp. 1352–1359 (2013). https://doi.org/10.1109/CVPR.2013.178
- 25.Taniai, T., Sinha, S.N., Sato, Y.: Joint recovery of dense correspondence and cosegmentation in two images. In: CVPR, pp. 4246–4255 (2016). https://doi.org/10.1109/CVPR.2016.460
- 26.Tombari, F., Stefano, L.D.: Object recognition in 3d scenes with occlusions and clutter by hough voting. In: Fourth Pacific-Rim Symposium on Image and Video Technology, pp. 349–355 (2010). https://doi.org/10.1109/PSIVT.2010.65