Advertisement

The World Is Changing: Finding Changes on the Street

  • Kuan-Ting Chen
  • Fu-En Wang
  • Juan-Ting Lin
  • Fu-Hsiang Chan
  • Min SunEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10116)

Abstract

We propose to find changes in the constantly changing world, given visual observations at street-level. In particular, we identify “long-term” changes between Google Street View images and dashcam videos captured at different months or even years. This is a challenging task, since (1) dashcam frames are not localized in world coordinate, and (2) there are many changes introduced by moving objects. We propose a robust sequence alignment method to align dashcam sequence to Street View images. Our method outperforms a strong baseline method [1] by \(12\%\) mean Average Precision (AP). We also propose a novel change detection method designed to detect long-term changes. Our change detection method (\({13.54}\%\)) outperforms a baseline method without handling car interior and moving objects (\({11.70}\%\)) by \(1.84\%\) (relatively \(13.6\%\)) in mean AP. In a controlled experiment, given manually aligned high quality Street View images, our change detection method achieves a significantly better mean AP (\(45.57\%\)).

Keywords

Reference Image Average Precision Reconstruction Error Query Image Confidence Score 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgement

We thank Industrial Technology Research Institute (ITRI) project grants and MOST 103-2218-E-007-025 and MOST 104-3115-E-007-005 in Taiwan for their support.

References

  1. 1.
    Vaca-Castano, G., Zamir, A., Shah, M.: City scale geo-spatial trajectory estimation of a moving camera. In: CVPR (2012)Google Scholar
  2. 2.
    Badino, H., Huber, D., Kanade, T.: Real-time topometric localization. In: ICRA (2012)Google Scholar
  3. 3.
    Kim, J., Liu, C., Sha, F., Grauman, K.: Deformable spatial pyramid matching for fast dense correspondences. In: CVPR (2013)Google Scholar
  4. 4.
    Robertson, D., Cipolla, R.: An image-based system for urban navigation. In: BMVC (2004)Google Scholar
  5. 5.
    Zhang, W., Kosecka, J.: Image based localization in urban environments. In: 3DPVT (2006)Google Scholar
  6. 6.
    Schindler, G., Brown, M., Szeliski, R.: City-scale location recognition. In: CVPR (2007)Google Scholar
  7. 7.
    Hays, J., Efros, A.A.: Im2gps: estimating geographic information from a single image. In: CVPR (2008)Google Scholar
  8. 8.
    Zamir, A.R., Shah, M.: Accurate image localization based on google maps street view. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 255–268. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15561-1_19 CrossRefGoogle Scholar
  9. 9.
    Cao, S., Snavely, N.: Graph-based discriminative learning for location recognition. In: CVPR (2013)Google Scholar
  10. 10.
    Bettadapura, V., Essa, I., Pantofaru, C.: Egocentric field-of-view localization using first-person point-of-view devices. In: WACV (2015)Google Scholar
  11. 11.
    Irschara, A., Zach, C., Frahm, J., Bischof, H.: From structure-from-motion point clouds to fast location recognition. In: CVPR (2009)Google Scholar
  12. 12.
    Li, Y., Snavely, N., Huttenlocher, D.P.: Location recognition using prioritized feature matching. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 791–804. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15552-9_57 CrossRefGoogle Scholar
  13. 13.
    Sattler, T., Leibe, B., Kobbelt, L.: Fast image-based localization using direct 2d-to-3d matching. In: ICCV (2011)Google Scholar
  14. 14.
    Li, Y., Snavely, N., Huttenlocher, D., Fua, P.: Worldwide pose estimation using 3D point clouds. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7572, pp. 15–29. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33718-5_2 CrossRefGoogle Scholar
  15. 15.
    Taneja, A., Ballan, L., Pollefeys, M.: Never get lost again: vision based navigation using streetview images. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9007, pp. 99–114. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-16814-2_7 Google Scholar
  16. 16.
    Lategahn, H., Schreiber, M., Ziegler, J., Stiller, C.: Urban localization with camera and inertial measurement unit. In: Intelligent Vehicles Symposium (IV) (2013)Google Scholar
  17. 17.
    Floros, G., van der Zander, B., Leibe, B.: OpenStreetSLAM: global vehicle localization using openstreetmaps. In: ICRA (2013)Google Scholar
  18. 18.
    Brubaker, M., Geiger, A., Urtasun, R.: Lost! leveraging the crowd for probabilistic visual self-localization. In: CVPR (2013)Google Scholar
  19. 19.
    Radke, R., Andra, S., Al-Kofahi, O., Roysam, B.: Image change detection algorithms: a systematic survey. TIP 14, 294–307 (2005)MathSciNetGoogle Scholar
  20. 20.
    Pollard, T., Mundy, J.: Change detection in a 3-d world. In: CVPR (2007)Google Scholar
  21. 21.
    Taneja, A., Ballan, L., Pollefeys, M.: City-scale change detection in cadastral 3d models using images. In: CVPR (2013)Google Scholar
  22. 22.
    Matzen, K., Snavely, N.: Scene chronology. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 615–630. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10584-0_40 Google Scholar
  23. 23.
    Wu, C.: Towards linear-time incremental structure from motion. In: 3DV (2013)Google Scholar
  24. 24.
    Wu, C.: Visualsfm: a visual structure from motion system. http://ccwu.me/vsfm/
  25. 25.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)CrossRefGoogle Scholar
  26. 26.
    Jegou, H., Perronnin, F., Douze, M., Snchez, J., Perez, P., Schmid, C.: Aggregating local image descriptors into compact codes. TPAMI 34, 1704–1716 (2011)CrossRefGoogle Scholar
  27. 27.
    Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., Torr, P.: Conditional random fields as recurrent neural networks. In: ICCV (2015)Google Scholar
  28. 28.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR (2015)Google Scholar
  29. 29.
    Zitnick, C.L., Dollár, P.: Edge boxes: locating object proposals from edges. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 391–405. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10602-1_26 Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Kuan-Ting Chen
    • 1
  • Fu-En Wang
    • 1
  • Juan-Ting Lin
    • 1
  • Fu-Hsiang Chan
    • 1
  • Min Sun
    • 1
    Email author
  1. 1.Department of Electrical EngineeringNational Tsing Hua UniversityHsinchuTaiwan

Personalised recommendations