Advertisement

Journal of Computer Science and Technology

, Volume 33, Issue 6, pp 1164–1177 | Cite as

A Geometry-Based Point Cloud Reduction Method for Mobile Augmented Reality System

  • Hao-Ren Wang
  • Juan Lei
  • Ao Li
  • Yi-Hong WuEmail author
Regular Paper
  • 45 Downloads

Abstract

In this paper, a geometry-based point cloud reduction method is proposed, and a real-time mobile augmented reality system is explored for applications in urban environments. We formulate a new objective function which combines the point reconstruction errors and constraints on spatial point distribution. Based on this formulation, a mixed integer programming scheme is utilized to solve the points reduction problem. The mobile augmented reality system explored in this paper is composed of the offline and online stages. At the offline stage, we build up the localization database using structure from motion and compress the point cloud by the proposed point cloud reduction method. While at the online stage, we compute the camera pose in real time by combining an image-based localization algorithm and a continuous pose tracking algorithm. Experimental results on benchmark and real data show that compared with the existing methods, this geometry-based point cloud reduction method selects a point cloud subset which helps the image-based localization method to achieve higher success rate. Also, the experiments conducted on a mobile platform show that the reduced point cloud not only reduces the time consuming for initialization and re-initialization, but also makes the memory footprint small, resulting a scalable and real-time mobile augmented reality system.

Keywords

mobile platform augmented reality point cloud reduction structure from motion 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

11390_2018_1879_MOESM1_ESM.pdf (1.3 mb)
ESM 1 (PDF 1337 kb)

References

  1. [1]
    Agarwal S, Snavely N, Simon I, Seitz S M, Szeliski R. Building Rome in a day. In Proc. the 12th IEEE International Conference on Computer Vision, September 2009, pp.72-79.Google Scholar
  2. [2]
    Arth C, Wagner D, Klopschitz M, Irschara A, Schmalstieg D.Wide area localization on mobile phones. In Proc. the 8th IEEE International Symposium on Mixed and Augmented Reality, October 2009, pp.73-82.Google Scholar
  3. [3]
    Arth C, Mulloni A, Schmalstieg D. Exploiting sensors on mobile phones to improve wide-area localization. In Proc. the 21st International Conference on Pattern Recognition, November 2012, pp.2152-2156.Google Scholar
  4. [4]
    Middelberg S, Sattler T, Untzelmann O, Kobbelt L. Scalable 6-DOF localization on mobile devices. In Proc. the 13th European Conference on Computer Vision, September 2014, pp.268-283.Google Scholar
  5. [5]
    Jeong Y, Bok Y, Kim J S, Kweon I S. Complementation of cameras and lasers for accurate 6D SLAM: From correspondences to bundle adjustment. In Proc. the 2011 IEEE International Conference on Robotics and Automation, May 2011, pp.3581-3588.Google Scholar
  6. [6]
    Miraldo P, Araujo H. Pose estimation for non-central cameras using planes. Journal of Intelligent & Robotic Systems, 2015, 80(3/4): 595-608.Google Scholar
  7. [7]
    Miraldo P, Araujo H, Gon¸calves N. Pose estimation for general cameras using lines. IEEE Transactions on Cybernetics, 2015, 45(10): 2156-2164.Google Scholar
  8. [8]
    Liu H, Zhang G, Bao H. Robust keyframe-based monocular SLAM for augmented reality. In Proc. the 2016 IEEE International Symposium on Mixed and Augmented Reality, September 2016.Google Scholar
  9. [9]
    Sattler T, Leibe B, Kobbelt L. Fast image-based localization using direct 2D-to-3D matching. In Proc. the 2011 IEEE International Conference on Computer Vision, November 2011, pp.667-674.Google Scholar
  10. [10]
    Sattler T, Weyand T, Leibe B, Kobbelt L. Image retrieval for image-based localization revisited. In Proc. the 2012 British Machine Vision Conference, Sept. 2012.Google Scholar
  11. [11]
    Choudhary S, Indelman V, Christensen H I, Dellaert F. Information-based reduced landmark SLAM. In Proc. the 2015 IEEE International Conference on Robotics and Automation, May 2015, pp.4620-4627.Google Scholar
  12. [12]
    Torii A, Arandjelović R, Sivic J, Okutomi M, Pajdla T. 24/7 place recognition by view synthesis. In Proc. the 2015 IEEE Conference on Computer Vision and Pattern Recognition, June 2015, pp.1808-1817.Google Scholar
  13. [13]
    Zeisl B, Sattler T, Pollefeys M. Camera pose voting for large-scale image-based localization. In Proc. the 2015 IEEE International Conference on Computer Vision, December 2015, pp.2704-2712.Google Scholar
  14. [14]
    Feng Y, Fan L, Wu Y. Fast localization in large-scale environments using supervised indexing of binary features. IEEE Transactions on Image Processing, 2016, 25(1): 343-358.Google Scholar
  15. [15]
    Campbell D, Petersson L, Kneip L, Li H. Globally-optimal inlier set maximisation for simultaneous camera pose and feature correspondence. In Proc. the 2017 IEEE International Conference on Computer Vision, October 2017.Google Scholar
  16. [16]
    Yang J, Li H, Campbell D, Jia Y. Go-ICP: A globally optimal solution to 3D ICP point-set registration. IEEE Transactions on Pattern Analysis and Machine Intelligence, November 2016, 38(11): 2241-2254.Google Scholar
  17. [17]
    Li Y, Snavely N, Huttenlocher D P. Location recognition using prioritized feature matching. In Proc. the 11th European Conference on Computer Vision, September 2010, pp.791-804.Google Scholar
  18. [18]
    Irschara A, Zach C, Frahm J M, Bischof H. From structurefrom-motion point clouds to fast location recognition. In Proc. the 2009 IEEE Conference on Computer Vision and Pattern Recognition, June 2009, pp.2599-2606.Google Scholar
  19. [19]
    Dong Z, Zhang G, Jia J, Bao H. Efficient keyframe-based real-time camera tracking. Computer Vision and Image Understanding, 2014, 118: 97-110.Google Scholar
  20. [20]
    Cao S, Snavely N. Minimal scene descriptions from structure from motion models. In Proc. the 2014 IEEE Conference on Computer Vision and Pattern Recognition, June 2014, pp.461-468.Google Scholar
  21. [21]
    Park H S, Wang Y, Nurvitadhi E, Hoe J C, Sheikh Y, Chen M. 3D point cloud reduction using mixed-integer quadratic programming. In Proc. the 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops, June 2013, pp.229-236.Google Scholar
  22. [22]
    Leutenegger S, Chli M, Siegwart R. BRISK: Binary robust invariant scalable keypoints. In Proc. the 2011 IEEE International Conference on Computer Vision, November 2011, pp.2548-2555.Google Scholar
  23. [23]
    Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91-110.Google Scholar
  24. [24]
    Rosten E, Drummond T. Machine learning for high-speed corner detection. In Proc. the 9th European Conference on Computer Vision, May 2006, pp.430-443.Google Scholar
  25. [25]
    Gionis A, Indyk P, Motwani R. Similarity search in high dimensions via hashing. In Proc. the 25th International Conference on Very Large Data Bases, September 1999, pp.518-529.Google Scholar
  26. [26]
    Chum O, Matas J. Matching with PROSAC — Progressive sample consensus. In Proc. the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Volume 1, June 2005, pp.220-226.Google Scholar
  27. [27]
    Lepetit V, Moreno-Noguer F, Fua P. EPnP: An accurate O(n) solution to the PnP problem. International Journal of Computer Vision, 2009, 81(2): 155-166.Google Scholar
  28. [28]
    Takacs G, Chandrasekhar V, Tsai S, Chen D, Grzeszczuk R, Girod B. Unified real-time tracking and recognition with rotation-invariant fast features. In Proc. the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 2010, pp.934-941.Google Scholar
  29. [29]
    Stewart C V. Robust parameter estimation in computer vision. SIAM Review, 1999, 41(3): 513-537.Google Scholar
  30. [30]
    Hartley R, Zisserman A. Multiple View Geometry in Computer Vision (2nd edition). Cambridge University Press, 2004.Google Scholar
  31. [31]
    Fischler M A, Bolles R C. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 1981, 24(6): 381-395.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of SciencesBeijingChina
  2. 2.University of Chinese Academy of SciencesBeijingChina

Personalised recommendations