General Framework

  • Alejandro Héctor Toselli
  • Enrique Vidal
  • Francisco Casacuberta


Lately, the paradigm for Pattern Recognition (PR) systems design is shifting from the concept of full-automation to systems where the decision process is conditioned by human feedback. This shift is motivated by the fact that full automation often proves elusive, or unnatural in many applications where technology is expected to assist rather than replace the human agents.

This chapter examines the challenges and research opportunities entailed by placing PR within the human-interaction framework; namely: (a) taking direct advantage of the feedback information provided by the user in each interaction step to improve raw performance; (b) acknowledging the inherent multimodality of interaction to improve overall system behavior and usability and (c) using the feedback-derived data to tune the system to the user behavior and the specific task considered, by means of adaptive learning techniques.

One of the most influential factors for the rapid development of PR technology in the last few decades is the nowadays commonly adopted assessment paradigm based on labeled training and testing corpora. This chapter includes a discussion about simple but realistic “user models” or interaction protocols and assessment criteria which allow the successful labeled corpus-based assessment paradigm to be applied also in the interactive scenario.

This chapter also provides an introduction to general approaches available to solve the underlying interactive search problems on the basis of existing methods to solve the corresponding non-interactive counterparts and an overview of modern machine learning approaches which can be useful in the interactive framework.


Machine Translation Confidence Measure Interaction Protocol Active Learning Strategy Interaction Step 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Andrieu, C., de Freitas, N., Doucet, A., & Jordan, M. I. (2003). An introduction to mcmc for machine learning. Machine Learning, 50(1–2), 5–43. MATHCrossRefGoogle Scholar
  2. 2.
    Berger, A. L., Pietra, S. A. D., & Pietra, V. J. D. (1996). A maximum entropy approach to natural language processing. Computational Linguistics, 22, 39–71. Google Scholar
  3. 3.
    Bertolami, R., Zimmermann, M., & Bunke, H. (2006). Rejection strategies for offline handwritten text recognition. Pattern Recognition Letters, 27, 2005–2012. CrossRefGoogle Scholar
  4. 4.
    Canny, J. (2006). The future of human-computer interaction. ACM Queue, 4(6), 24–32. CrossRefGoogle Scholar
  5. 5.
    Cappé, O., & Moulines, E. (2009). Online EM algorithm for latent data models. Journal of the Royal Statistical Society Series B, 71(3), 593–613. MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Casacuberta, F., & Higuera, C. D. L. (2000). Computational complexity of problems on probabilistic grammars and transducers. In ICGI’00: Proceedings of the 5th international colloquium on grammatical inference (pp. 15–24), London, UK. Berlin: Springer. Google Scholar
  7. 7.
    Chapelle, O., Schölkopf, B., & Zien, A. (2006). Semi-supervised learning. Cambridge: MIT Press. Google Scholar
  8. 8.
    Christmas, W. J., Kittler, J., & Petrou, M. (1995). Structural matching in computer vision using probabilistic relaxation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17, 749–764. CrossRefGoogle Scholar
  9. 9.
    Cohn, D., Atlas, L., & Ladner, R. (1994). Improving generalization with active learning. Machine Learning, 15(2), 201–221. Google Scholar
  10. 10.
    Crammer, K., Dekel, O., Keshet, J., Shalev-Shwartz, S., & Singer, Y. (2006). Online passive–aggressive algorithms. Journal of Machine Learning Research, 7, 551–585. MathSciNetMATHGoogle Scholar
  11. 11.
    Dasgupta, S. (2005). Coarse sample complexity bounds for active learning. In Neural information processing systems. Google Scholar
  12. 12.
    Dasgupta, S. (2009). The two faces of active learning. In Discovery science (p. 35). CrossRefGoogle Scholar
  13. 13.
    Dasgupta, S., Hsu, D., & Monteleoni, C. (2008). A general agnostic active learning algorithm. In J. C. Platt, D. Koller, Y. Singer & S. Roweis (Eds.), Advances in neural information processing systems (Vol. 20, pp. 353–360). Cambridge: MIT Press. Google Scholar
  14. 14.
    De Bra, P., Kobsa, A., & Chin, D. E. (2010). Lecture notes in computer science: Vol. 6075. User modeling, adaptation, and personalization. Proceedings of the 18th international conference UMAP 2010. CrossRefGoogle Scholar
  15. 15.
    Dechter, R., & Pearl, J. (1985). Generalized best-first search strategies and the optimality of A . Journal of the ACM, 32, 505–536. MathSciNetMATHCrossRefGoogle Scholar
  16. 16.
    Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society, Series B, 39, 1–38. MathSciNetMATHGoogle Scholar
  17. 17.
    Duda, R. O., & Hart, P. E. (1973). Pattern classification and scene analysis. New York: Wiley. MATHGoogle Scholar
  18. 18.
    Fischer, G. (2001). User modeling in human–computer interaction. User Modeling and User-Adapted Interaction, 11, 65–86. MATHCrossRefGoogle Scholar
  19. 19.
    Freund, Y., Seung, H. S., Shamir, E., & Tishby, N. (1995). Selective sampling using the query by committee algorithm. In Machine learning (pp. 133–168). Google Scholar
  20. 20.
    Frey, B. J., & Hinton, G. E. (1999). Variational learning in nonlinear Gaussian belief networks. Neural Computation, 11, 193–213. CrossRefGoogle Scholar
  21. 21.
    Geman, S., & Geman, D. (1987). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. In Readings in computer vision: issues, problems, principles, and paradigms (pp. 564–584). San Mateo: Morgan Kaufmann. Google Scholar
  22. 22.
    González-Rubio, J., Ortiz-Martínez, D., & Casacuberta, F. (2010). Balancing user effort and translation error in interactive machine translation via confidence measures. In Proceedings of the 48th annual meeting of the association for computational linguistics (ACL10) (pp. 173–177). Google Scholar
  23. 23.
    Groena, F. C. A., tenKateb, T. K., Smeuldersc, A. W. M., & Youngd, I. T. (1989). Human chromosome classification based on local band descriptors. Pattern Recognition Letters, 9(3), 211–222. CrossRefGoogle Scholar
  24. 24.
    Hanneke, S. (2009). Theoretical foundations of active learning. Machine Learning Department, School of Computer Science, Carnegie Mellon University, Pittsburgh, USA. Advisors: Blum, A., Dasgupta, S., Wasserman, L., & Xing, E. P. Google Scholar
  25. 25.
    Hazan, E., & Seshadhri, C. (2009). Efficient learning algorithms for changing environments. In Proceedings of the 26th annual international conference on machine learning. Google Scholar
  26. 26.
    Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14, 1771–1800. MATHCrossRefGoogle Scholar
  27. 27.
    Jaksch, T., Ortner, R., & Auer, P. (2010). Near-optimal regret bounds for reinforcement learning. Journal of Machine Learning Research, 99, 1563–1600. MathSciNetGoogle Scholar
  28. 28.
    Jaimes, A., & Sebe, N. (2006). Multimodal human–computer interaction: A survey. Computer Vision and Image Understanding, 108(1–2), 116–134. Special Issue on Vision for Human-Computer Interaction. Google Scholar
  29. 29.
    Jarvis, R. A. (1974). An interactive minicomputer laboratory for graphics, image processing, and pattern recognition. Computer, 7(10), 49–60. MathSciNetCrossRefGoogle Scholar
  30. 30.
    Jelinek, F. (1998). Statistical methods for speech recognition. Cambridge: MIT Press. Google Scholar
  31. 31.
    Jiménez, V. M., & Marzal, A. (1999). Computing the k shortest paths: a new algorithm and an experimental comparison. In J. S. Viter & C. D. Zaraliagis (Eds.), Lecture notes in computer science: Vol. 1668. Algorithm engineering (pp. 15–29). Berlin: Springer. CrossRefGoogle Scholar
  32. 32.
    Kittler, J., & Illingworth, J. (1986). Relaxation labelling algorithms-a review. Image and Vision Computing, 3, 206–216. CrossRefGoogle Scholar
  33. 33.
    Martin, S. C., Ney, H., & Hamacher, C. (2000). Maximum entropy language modeling and the smoothing problem. IEEE Transactions on Speech and Audio Processing, 8(5), 626–632. CrossRefGoogle Scholar
  34. 34.
    Martínez, C., García, H., & Juan, A. (2003). Chromosome classification using continuous hidden Markov models. In LNCS. Pattern recognition and image analysis (pp. 494–501). Berlin: Springer. CrossRefGoogle Scholar
  35. 35.
    Martínez, C., Juan, A., & Casacuberta, F. (2007). Iterative contextual recurrent classification of chromosomes. Neural Processing Letters, 26(3), 159–175. CrossRefGoogle Scholar
  36. 36.
    Neal, R. M., & Hinton, G. E. (1998). A view of the em algorithm that justifies incremental, sparse, and other variants. In Learning in graphical models (pp. 355–368). Dordrecht: Kluwer Academic. CrossRefGoogle Scholar
  37. 37.
    Och, F. J., & Ney, H. (2002). Discriminative training and maximum entropy models for statistical machine translation. In Proc. of ACL (pp. 295–302). Google Scholar
  38. 38.
    Pastor, M., Toselli, A. H., & Vidal, E. (2005). Writing speed normalization for on-line handwritten text recognition. In Proc. of the eighth international conference on document analysis and recognition (ICDAR ’05) (pp. 1131–1135), Seoul, Korea. CrossRefGoogle Scholar
  39. 39.
    Pearl, J. (1984). Heuristics: intelligent search strategies for computer problem solving. Reading: Addison-Wesley. Google Scholar
  40. 40.
    Pearl, J. (1988). Probabilistic reasoning in intelligent systems: networks of plausible inference. San Mateo: Morgan Kaufmann. Google Scholar
  41. 41.
    Plamondon, R., & Srihari, S. N. (2000). On-line and off-line handwriting recognition: a comprehensive survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1), 63–84. CrossRefGoogle Scholar
  42. 42.
    Rabiner, L. (1989). A tutorial of hidden Markov models and selected application in speech recognition. Proceedings of the IEEE, 77, 257–286. CrossRefGoogle Scholar
  43. 43.
    Ritter, G., Gallegos, M. T., & Gaggermeier, K. (1995). Automatic context-sensitive karyotyping of human chromosomes based on elliptically symmetric statistical distributions. Pattern Recognition, 28(6), 823–831. CrossRefGoogle Scholar
  44. 44.
    Rosenfeld, A., Hummel, R. A., & Zucker, S. W. (1976). Scene labeling by relaxation operations. IEEE Transactions on Systems, Man, and Cybernetics, 6(6), 420–433. MathSciNetMATHCrossRefGoogle Scholar
  45. 45.
    Schröck, E., du Manoir, S., Veldman, T., Schoell, B., Wienberg, J. W., Ferguson-Smith, M. A., Ning, Y., Ledbetter, D. H., Bar-Am, I., Soenksen, D., Garini, Y., & Ried, T. (1996). Multicolor spectral karyotyping of human chromosomes. Science, 273(5274), 494–497. CrossRefGoogle Scholar
  46. 46.
    Serrano, N., Sanchis, A., & Juan, A. (2010). Balancing error and supervision effort in interactive-predictive handwriting recognition. In Proceedings of the international conference on intelligent user interfaces (IUI 2010) (pp. 373–376). Google Scholar
  47. 47.
    Shalev-shwartz, S., & Tewari, A. (2008). Efficient bandit algorithms for online multiclass prediction sham m. kakade. In Proc. of the 25th international conference on machine learning. Google Scholar
  48. 48.
    Toselli, A. H., Pastor, M., & Vidal, E. (2007). On-line handwriting recognition system for Tamil handwritten characters. In Lecture notes in computer science: Vol. 4477. 3rd Iberian conference on pattern recognition and image analysis (pp. 370–377), Girona (Spain). Berlin: Springer. CrossRefGoogle Scholar
  49. 49.
    Ueffing, N., & Ney, H. (2007). Word-level confidence estimation for machine translation. Computational Linguistics, 33(1), 9–40. MATHCrossRefGoogle Scholar
  50. 50.
    Viterbi, A. (1967). Error bounds for convolutional codes and an asymptotically optimal decoding algorithm. IEEE Transactions on Information Theory, 13, 260–269. MATHCrossRefGoogle Scholar
  51. 51.
    Wessel, F., Schlüter, R., Macherey, K., & Ney, H. (2001). Confidence measures for large vocabulary continuous speech recognition. IEEE Transactions on Speech and Audio Processing, 9(3), 288–298. CrossRefGoogle Scholar
  52. 52.
    Zaphiris, P., & Ang, C. S. (2008). Cross-disciplinary advances in human computer interaction: user modeling, social computing, and adaptive interfaces. Advances in technology and human interaction book series. Information Science Reference. Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Alejandro Héctor Toselli
    • Enrique Vidal
      • Francisco Casacuberta

        There are no affiliations available

        Personalised recommendations