Abstract
Selecting which algorithms should be used by a mobile robot computer vision system is a decision that is usually made a priori by the system developer, based on past experience and intuition, not systematically taking into account information that can be found in the images and in the visual process itself to learn which algorithm should be used, in execution time. This paper presents a method that uses Reinforcement Learning to decide which algorithm should be used to recognize objects seen by a mobile robot in an indoor environment, based on simple attributes extracted on-line from the images, such as mean intensity and intensity deviation. Two state-of-the-art object recognition algorithms can be selected: the constellation method proposed by Lowe together with its interest point detector and descriptor, the Scale-Invariant Feature Transform and Nistér and Stewénius Vocabulary Tree approach. A set of empirical evaluations was conducted using a household mobile robots image database, and results obtained shows that the approach adopted here is very promising.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bhanu, B., Peng, J.: Adaptive integrated image segmentation and object recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 30(4), 427–441 (2000)
Borotschnig, H., Paletta, L., Prantl, M., Pinz, A.: Appearance-based active object recognition. Image and Vision Computing 18(9), 715–728 (1999)
Bradski, G.: The OpenCV Library. Dr Dobb’s Journal of Software Tools, 120–125 (2000)
Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)
Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Sebe, N., Lew, M., Huang, T.S. (eds.) ECCV/HCI 2004. LNCS, vol. 3058, pp. 1–22. Springer, Heidelberg (2004)
Darrell, T.: Reinforcement learning of active recognition behaviors, Interval research technical report 1997-045. In: Advances in Neural Information Processing Systems, NIPS 1995, vol. 8, pp. 858–864. MIT Press, Cambridge (1995), http://www.interval.com/papers/1997-045 , portions of this paper previously appeared
Darrell, T., Pentland, A.: Active gesture recognition using learned visual attention. In: Touretzky, D.S., Mozer, M.C., Hasselmo, M.E. (eds.) Advances in Neural Information Processing Systems, vol. 8, pp. 858–864. MIT Press, Cambridge (1996a)
Darrell, T., Pentland, A.: Active gesture recognition using partially observable markov decision processes. In: Proceedings of the 13th International Conference on Pattern Recognition, vol. 3, pp. 984–988 (1996b), doi:10.1109/ICPR.1996.547315
Draper, B.A., Bins, J., Baek, K.: ADORE: Adaptive object recognition. In: Christensen, H.I. (ed.) ICVS 1999. LNCS, vol. 1542, pp. 522–537. Springer, Heidelberg (1998)
Fogel, I., Sagi, D.: Gabor filters as texture discriminator. Biological Cybernetics 61(2), 103–113 (1989), http://dx.doi.org/10.1007/BF00204594
Haralick, R.M., Sternberg, S.R., Zhuang, X.: Image analysis using mathematical morphology. IEEE Trans. Pattern Anal. Mach. Intell. 9(4), 532–550 (1987)
Hossain, I., Liu, J., You, J.: Tropical cyclone pattern recognition for intensity and forecasting analysis from satellite imagery. In: IEEE International Conference on Systems, Man, and Cybernetics, IEEE SMC 1999 Conference Proceedings 1999, vol. 6, pp. 851–856 (1999), doi:10.1109/ICSMC.1999.816663
Lowe, D.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)
Minut, S., Mahadevan, S.: A reinforcement learning model of selective visual attention. In: AGENTS 2001: Proceedings of the fifth international conference on Autonomous agents, pp. 457–464. ACM, New York (2001), http://doi.acm.org/10.1145/375735.376414
Mitchell, T.: Machine Learning. McGraw Hill, New York (1997)
Nistér, D., Stewénius, H.: Scalable recognition with a vocabulary tree. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2161–2168 (2006)
Paletta, L., Pinz, A.: Active object recognition by view integration and reinforcement learning. Robotics and Autonomous Systems 31(1-2), 71–86 (2000)
Paletta, L., Prantl, M., Pinz, A.: Reinforcement learning of 3-d object recognition from appearance. In: CONALD 1998: Proceedings of the 1998 Conference on Automated Learning and Discovery (1998)
Paletta, L., Fritz, G., Seifert, C.: Q-learning of sequential attention for visual object recognition from informative local descriptors. In: ICML 2005: Proceedings of the 22nd International Conference on Machine Learning, pp. 649–656. ACM, New York (2005), http://doi.acm.org/10.1145/1102351.1102433
Paris, S., Kornprobst, P., Tumblin, J., Durand, F.: A gentle introduction to bilateral filtering and its applications. In: SIGGRAPH 2007: ACM SIGGRAPH 2007 courses, p. 1. ACM, New York (2007), http://doi.acm.org/10.1145/1281500.1281602
Peng, J., Bhanu, B.: Closed-loop object recognition using reinforcement learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(2), 139–154 (1998a)
Peng, J., Bhanu, B.: Delayed reinforcement learning for adaptive image segmentation and feature extraction. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 28(3), 482–488 (1998b)
Peng, J., Williams, R.J.: Incremental multi-step q-learning. Mach. Learn. 22(1-3), 283–290 (1996), http://dx.doi.org/10.1007/BF00114731
Ramisa, A., Vasudevan, S., Scaramuzza, D., López de Mántaras, R., Siegwart, R.: A tale of two object recognition methods for mobile robots. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds.) ICVS 2008. LNCS, vol. 5008, pp. 353–362. Springer, Heidelberg (2008)
Rummery, G.A., Niranjan, M.: On-line Q-learning using connectionist systems. Tech. Rep. CUED/F-INFENG/TR 166, Cambridge University Engineering Department (1994)
Sahba, F., Tizhoosh, H.R., Salama, M.M.M.A.: A reinforcement agent for object segmentation in ultrasound images. Expert Syst. Appl. 35(3), 772–780 (2008), http://dx.doi.org/10.1016/j.eswa.2007.07.057
Shokri, M., Tizhoosh, H.R.: Using reinforcement learning for image thresholding. In: IEEE CCECE 2003 Canadian Conference on Electrical and Computer Engineering, vol. 2, pp. 1231–1234 (2003), doi:10.1109/CCECE.2003.1226121
Shokri, M., Tizhoosh, H.R.: Q(lambda)-based image thresholding. In: CRV 2004: Proceedings of the 1st Canadian Conference on Computer and Robot Vision, pp. 504–508. IEEE Computer Society Press, Los Alamitos (2004), http://doi.ieeecomputersociety.org/10.1109/CCCRV.2004.1301490
Shokri, M., Tizhoosh, H.R.: A reinforcement agent for threshold fusion. Appl. Soft Comput. 8(1), 174–181 (2008)
Sivic, J., Schaffalitzky, F., Zisserman, A.: Object Level Grouping for Video Shots. International Journal of Computer Vision 67(2), 189–210 (2006)
Sutton, R., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Taylor, G.W.: Reinforcement Learning for Parameter Control of Image-Based Applications. MSc Thesis, University of Waterloo, Ontario, Canada (2004)
Tizhoosh, H., Taylor, G.: Reinforced contrast adaptation. International Journal of Image and Graphics 6(3), 377–392 (2006)
Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: ICCV 1998: Proceedings of the Sixth International Conference on Computer Vision, p. 839. IEEE Computer Society, Washington (1998)
Watkins, C.J.C.H.: Learning from Delayed Rewards. PhD Thesis, University of Cambridge (1989)
Weldon, T.P., Higgins, W.E., Dunn, D.F.: Efficient gabor filter design for texture segmentation. Pattern Recognition 29(12), 2005–2015 (1996), http://dx.doi.org/10.1016/S0031-32039600047-7
Whitehead, S.D., Ballard, D.H.: Learning to perceive and act by trial and error. Mach. Learn. 7(1), 45–83 (1991), http://dx.doi.org/10.1023/A:1022619109594
Wolf, C., Jolion, J.M.: Extraction and recognition of artificial text in multimedia documents. Pattern Anal. Appl. 6(4), 309–326 (2003), http://dx.doi.org/10.1007/s10044-003-0197-7
Yin, P.Y.: Maximum entropy-based optimal threshold selection using deterministic reinforcement learning with controlled randomization. Signal Process. 82(7), 993–1006 (2002), http://dx.doi.org/10.1016/S0165-16840200203-7
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Bianchi, R.A.C., Ramisa, A., de Mántaras, R.L. (2010). Automatic Selection of Object Recognition Methods Using Reinforcement Learning. In: Koronacki, J., Raś, Z.W., Wierzchoń, S.T., Kacprzyk, J. (eds) Advances in Machine Learning I. Studies in Computational Intelligence, vol 262. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05177-7_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-05177-7_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-05176-0
Online ISBN: 978-3-642-05177-7
eBook Packages: EngineeringEngineering (R0)