Quality & Quantity

, Volume 49, Issue 1, pp 339–347 | Cite as

The process of basic training, applied training, maintaining the performance of an observer

  • José Luis Losada
  • Rumen Manolov


In the field of observational methodology the observer is obviously a central figure, and close attention should be paid to the process through which he or she acquires, applies, and maintains the skills required. Basic training in how to apply the operational definitions of categories and the rules for coding, coupled with the opportunity to use the observation instrument in real-life situations, can have a positive effect in terms of the degree of agreement achieved when one evaluates intra- and inter-observer reliability. Several authors, including Arias et al. (Apunts, 4:40–45, 2009) and Medina and Delgado (Motricidad: Revista de Ciencias de la Actividad Física y del Deporte, 5:69–86, 1999) , have put forward proposals for the process of basic and applied training in this context. Reid and De Master (ORI Res Bull, 12:1–13, 1972) focuses on the observer’s performance and how to maintain the acquired skills, it being argued that periodic checks are needed after initial training because an observer may, over time, become less reliable due to the inherent complexity of category systems. The purpose of this subsequent training is to maintain acceptable levels of observer reliability. Various strategies can be used to this end, including providing feedback about those categories associated with a good reliability index, or offering re-training in how to apply those that yield lower indices. The aim of this study is to develop a performance-based index that is capable of assessing an observer’s ability to produce reliable observations in conjunction with other observers.


Observational methodology Reliability Observation instrument  Basic training Applied training  Maintenance 



This study forms part of the research project Observación de la interacción en deporte y actividad física: Avances técnicos y metodológicos en registros automatizados cualitativos-cuantitativos [Observing interactions in sport and physical activity: Technical and methodological advances in automated qualitative/quantitative registers], which is funded by the Spanish Secretary of State for Research, Development and Innovation and the Ministry of Economy and Competitiveness [DEP2012-32124] over the three-year period 2012–2015.


  1. Anguera, M.T.: Observational typology. Quality & quantity. Eur. Am. J. Methodol. 13(6), 449–484 (1979)Google Scholar
  2. Anguera, M.T.: La observación. In: Moreno Rosset, C. (ed.) Evaluación psicológica. Concepto, proceso y aplicación en las áreas del desarrollo y de la inteligencia, 271308. Sanz y Torres, Madrid (2003)Google Scholar
  3. Anguera, M.T., Blanco, A., Losada, J.L., Sánchez-Algarra, P.: Análisis de la competencia en la selección de observadores. Metodología de las Ciencias del Comportamiento 1(1), 95–114 (1999)Google Scholar
  4. Anguera, M.T.: Posibilidades y relevancia de la observación sistemática por el profesional de la psicología. Papeles del Psicólogo 31(1), 122–130 (2010)Google Scholar
  5. Arias, J.L., Argudo, F., Alonso, J.I.: El proceso de formación de observadores y la obtención de la fiabilidad en metodología observacional para analizar la dinámica de juego en minibásquet. Apunts 4, 40–45 (2009)Google Scholar
  6. Bakeman, R., Quera, V., Gnisci, A.: Observer agreement for timed-event sequential data: a comparison of time-based and event-based algorithms. Behav. Res. Methods 41(1), 137–147 (2009)CrossRefGoogle Scholar
  7. Bradley, J.V.: Robustness? Br. J. Math. Stat. Psychol. 31(2), 144–152 (1978)CrossRefGoogle Scholar
  8. Callahan, C.D., Barisa, M.T.: Statistical process control and rehabilitation outcome: the single-subject design reconsidered. Rehabil. Psychol. 50(1), 24–33 (2005)CrossRefGoogle Scholar
  9. Chacón, S., Holgado, P., Losada, J.L.: Using generalizability theory to assess the validity of evaluation process. Qual. Quant. 3, 315–329 (2006)Google Scholar
  10. Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20(1), 37–46 (1960)CrossRefGoogle Scholar
  11. Gast, D.L., Spriggs, A.D.: Visual analysis of graphic data. In: Gast, D.L. (ed.) Single Subject Research Methodology in Behavioral Sciences, pp. 199–233. Routledge, London (2009)Google Scholar
  12. Hantula, D.A.: Disciplined decision making in an interdisciplinary environment: some implications for clinical applications of statistical process control. J. Appl. Behav. Anal. 28(3), 371–377 (1995)CrossRefGoogle Scholar
  13. Heyns, R., Zander, A.F.: Observación de la conducta de grupo. In: Festinger, L., Katz, D. (eds.) Los métodos de investigación en las ciencias sociales, pp. 251–289. Paidós, Buenos Aires (1972)Google Scholar
  14. Manly, B.F.J.: Randomization, Bootstrap and Monte Carlo Methods in Biology. Chapman & Hall, London (2007)Google Scholar
  15. Mawhinney, T.: Total quality management and organizational behavior management: an integration for continual improvement. J. Appl. Behav. Anal. 25(3), 525–543 (1992)CrossRefGoogle Scholar
  16. Medina, J., Delgado, M.: Metodología de entrenamiento de observadores para investigaciones sobre E. F. y deporte en las que se utilice como método la observación. Motricidad: Revista de Ciencias de la Actividad Física y del Deporte 5, 69–86 (1999)Google Scholar
  17. Reid, J.B., De Master, B.: The efficacy of the spotcheck procedure in maintaining the reliability of data collected by observers in quasi-natural settings: two pilot studies. ORI Res. Bull. 12(8), 1–13 (1972)Google Scholar
  18. Remmert, H.: Analysis of group-tactical offensive behavior in elite basketball on the basis of a process orientated model. Eur. J. Sport Sci. 3(3), 1–12 (2003)CrossRefGoogle Scholar
  19. Sánchez-Algarra, P., Anguera, M.T.: Qualitative/quantitative integration in the inductive observational study of interactive behaviour: impact of recording and coding predominating perspectives. Qual. Quant. Int. J. Methodol. 47(2), 1237–1257 (2013)CrossRefGoogle Scholar
  20. Sweeney, M.A., Cottle, W.C.: Nonverbal acuity: a comparison of counselors and noncounselors. J. Couns. Psychol. 23, 394–397 (1976)CrossRefGoogle Scholar
  21. Tukey, J.W.: Exploratory Data Analysis. Addison-Wesley, London (1977)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  1. 1.University of BarcelonaBarcelonaSpain

Personalised recommendations