Evolution of a Tracking System

  • Sebastian Lieberknecht
  • Quintus Stierstorfer
  • Georg Kuschk
  • Daniel Ulbricht
  • Marion Langer
  • Selim Benhimane


This chapter describes the evolution of a feature-based tracking system developed by metaio. One of the reasons that started the development of the system was the first tracking contest at the International Symposium of Mixed and Augmented Reality (ISMAR) in 2008, which was designed to fairly evaluate different tracking systems. We present the toolchain we conceived to solve common problems like referencing to another coordinate system or creating a map of the environment from photos; we also describe the principles of our tracking method which, in contrast to the methods of all other contestants, was robust enough to use exactly the same parameters for all scenarios of the tracking contest held within the German research project AVILUS1 but at the same time was the most accurate. The ultimate goal of development is its integration into an end consumer product.


Tracking System Augmented Reality Extend Kalman Filter Global Coordinate System Bundle Adjustment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



Our thank goes to Daniel Pustka and Gudrun Klinker who conceived and organized the ISMAR 2008 and 2009 tracking contests as well as Björn Schwerdfeger who together with Gudrun Klinker mainly organized the AVILUS Tracking contest. Futhermore, we are thankful to Harald Wuest, Mark Fiala, Peter Keitler and Sudeep Sundaram for disclosing their technology and for valuable discussions about their view on the tracking contests.

This work was partially supported by BMBF grant Avilus / 01 IM08001 P.


  1. 1.
    Aicon 3d systems gmbh., accessed 2010-12-01.
  2. 2.
    Faro europe gmbh & co. kg., accessed 2010-12-01.
  3. 3.
    Instantvision, subproject of instantlabs., accessed 2010-12-01.
  4. 4.
    The rawseeds project., accessed 2010-12-01.
  5. 5.
    Ronald Azuma. A survey of augmented reality. Presence, 6(4):355–385, 1997.Google Scholar
  6. 6.
    Simon Baker and Iain Matthews. Lucas-kanade 20 years on: A unifying framework. IJCV, 56(3):221–255, 2004.CrossRefGoogle Scholar
  7. 7.
    Simon Baker, Daniel Scharstein, J. P. Lewis, Stefan Roth, Michael J. Black, and Richard Szeliski. A database and evaluation methodology for optical flow. In ICCV, 2007.Google Scholar
  8. 8.
    Gabriele Bleser and Didier Stricker. Advanced tracking through efficient image processing and visual-inertial sensor fusion. Computers and Graphics, 33(1):59–72, 2009.CrossRefGoogle Scholar
  9. 9.
    Gary Bradski. The OpenCV Library. Dr. Dobb’s Journal of Software Tools, 2000.Google Scholar
  10. 10.
    Robert Castle, Georg Klein, and David Murray. Video-rate localization in multiple maps for wearable augmented reality. In Proc 12th IEEE Int Symp on Wearable Computers, Pittsburgh PA, Sept 28 - Oct 1, 2008, pages 15–22, 2008.Google Scholar
  11. 11.
    Andrew J. Davison, Ian D. Reid, Nicholas D. Molton, and Olivier Stasse. MonoSLAM: Real-time single camera SLAM. PAMI, 26(6):1052–1067, 2007.CrossRefGoogle Scholar
  12. 12.
    Daniel DeMenthon and Larry S. David. Model-based object pose in 25 lines of code. IJCV, 15:123–141, 1995.CrossRefGoogle Scholar
  13. 13.
    Mark Fiala. Designing highly reliable fiducial markers. PAMI, 32(7):1317–1324, 2010.MathSciNetCrossRefGoogle Scholar
  14. 14.
    Andrew P. Gee, Denis Chekhlov, Walterio Mayol, and Andrew Calway. Discovering planes and collapsing the state space in visual slam. In BMVC, 2007.Google Scholar
  15. 15.
    Richard I. Hartley and Andrew Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, ISBN: 0521540518, second edition, 2004.Google Scholar
  16. 16.
    Hirokazu Kato and Mark Billinghurst. Marker tracking and hmd calibration for a video-based augmented reality conferencing system. In Proceedings of the 2nd International Workshop on Augmented Reality (IWAR 99), San Francisco, USA, October 1999.Google Scholar
  17. 17.
    Georg Klein and David Murray. Parallel tracking and mapping for small AR workspaces. In ISMAR, 2007.Google Scholar
  18. 18.
    Marion Langer and Selim Benhimane. An interactive vision-based 3d reconstruction workflow for industrial ar applications. In ISMAR 2010, Workshop: Augmented Reality Super Models, 2010.Google Scholar
  19. 19.
    Vincent Lepetit and Pascal Fua. Keypoint Recognition using Randomized Trees. PAMI, 28(9):1465–1479, 2006.CrossRefGoogle Scholar
  20. 20.
    Sebastian Lieberknecht, Selim Benhimane, Peter Meier, and Nassir Navab. A dataset and evaluation methodology for template-based tracking algorithms. In ISMAR, 2009.Google Scholar
  21. 21.
    David G. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60:91–110, 2004.CrossRefGoogle Scholar
  22. 22.
    Bruce D. Lucas and Takeo Kanade. An iterative image registration technique with an application to stereo vision (ijcai). In Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI ’81), pages 674–679, April 1981.Google Scholar
  23. 23.
    Katharina Pentenrieder, Peter Meier, and Gudrun Klinker. Analysis of tracking accuracy for single-camera square-marker-based tracking. In Proc. Dritter Workshop Virtuelle und Erweiterte Realitt der GI-Fachgruppe VR/AR, 2006.Google Scholar
  24. 24.
    Daniel Pustka, Manuel Huber, Christian Waechter, Florian Echtler, Peter Keitler, Gudrun Klinker, Joseph Newman, and Dieter Schmalstieg. Ubitrack: Automatic Configuration of Pervasive Sensor Networks for Augmented Reality. to appear in IEEE Pervasive Computing, 2010.Google Scholar
  25. 25.
    Björn Schwerdtfeger. AVILUS tracking contest 2010 – Abschlussbericht. Technical report, TUM, 2010.Google Scholar
  26. 26.
    Jianbo Shi and Carlo Tomasi. Good features to track. In CVPR, 1994.Google Scholar
  27. 27.
    Shinji Umeyama. Least-squares estimation of transformation parameters between two point patterns. PAMI, 13(4):376–380, 1991.CrossRefGoogle Scholar
  28. 28.
    Zhengyou Zhang. A flexible new technique for camera calibration. PAMI, 22(11):1330–1334, 2000.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Sebastian Lieberknecht
    • 1
  • Quintus Stierstorfer
    • 1
  • Georg Kuschk
    • 1
  • Daniel Ulbricht
    • 1
  • Marion Langer
    • 1
  • Selim Benhimane
    • 1
  1. 1.Research, metaio GmbHMunichGermany

Personalised recommendations