Abstract
In the field of observational methodology the observer is obviously a central figure, and close attention should be paid to the process through which he or she acquires, applies, and maintains the skills required. Basic training in how to apply the operational definitions of categories and the rules for coding, coupled with the opportunity to use the observation instrument in real-life situations, can have a positive effect in terms of the degree of agreement achieved when one evaluates intra- and inter-observer reliability. Several authors, including Arias et al. (Apunts, 4:40–45, 2009) and Medina and Delgado (Motricidad: Revista de Ciencias de la Actividad Física y del Deporte, 5:69–86, 1999) , have put forward proposals for the process of basic and applied training in this context. Reid and De Master (ORI Res Bull, 12:1–13, 1972) focuses on the observer’s performance and how to maintain the acquired skills, it being argued that periodic checks are needed after initial training because an observer may, over time, become less reliable due to the inherent complexity of category systems. The purpose of this subsequent training is to maintain acceptable levels of observer reliability. Various strategies can be used to this end, including providing feedback about those categories associated with a good reliability index, or offering re-training in how to apply those that yield lower indices. The aim of this study is to develop a performance-based index that is capable of assessing an observer’s ability to produce reliable observations in conjunction with other observers.
Similar content being viewed by others
References
Anguera, M.T.: Observational typology. Quality & quantity. Eur. Am. J. Methodol. 13(6), 449–484 (1979)
Anguera, M.T.: La observación. In: Moreno Rosset, C. (ed.) Evaluación psicológica. Concepto, proceso y aplicación en las áreas del desarrollo y de la inteligencia, 271308. Sanz y Torres, Madrid (2003)
Anguera, M.T., Blanco, A., Losada, J.L., Sánchez-Algarra, P.: Análisis de la competencia en la selección de observadores. Metodología de las Ciencias del Comportamiento 1(1), 95–114 (1999)
Anguera, M.T.: Posibilidades y relevancia de la observación sistemática por el profesional de la psicología. Papeles del Psicólogo 31(1), 122–130 (2010)
Arias, J.L., Argudo, F., Alonso, J.I.: El proceso de formación de observadores y la obtención de la fiabilidad en metodología observacional para analizar la dinámica de juego en minibásquet. Apunts 4, 40–45 (2009)
Bakeman, R., Quera, V., Gnisci, A.: Observer agreement for timed-event sequential data: a comparison of time-based and event-based algorithms. Behav. Res. Methods 41(1), 137–147 (2009)
Bradley, J.V.: Robustness? Br. J. Math. Stat. Psychol. 31(2), 144–152 (1978)
Callahan, C.D., Barisa, M.T.: Statistical process control and rehabilitation outcome: the single-subject design reconsidered. Rehabil. Psychol. 50(1), 24–33 (2005)
Chacón, S., Holgado, P., Losada, J.L.: Using generalizability theory to assess the validity of evaluation process. Qual. Quant. 3, 315–329 (2006)
Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20(1), 37–46 (1960)
Gast, D.L., Spriggs, A.D.: Visual analysis of graphic data. In: Gast, D.L. (ed.) Single Subject Research Methodology in Behavioral Sciences, pp. 199–233. Routledge, London (2009)
Hantula, D.A.: Disciplined decision making in an interdisciplinary environment: some implications for clinical applications of statistical process control. J. Appl. Behav. Anal. 28(3), 371–377 (1995)
Heyns, R., Zander, A.F.: Observación de la conducta de grupo. In: Festinger, L., Katz, D. (eds.) Los métodos de investigación en las ciencias sociales, pp. 251–289. Paidós, Buenos Aires (1972)
Manly, B.F.J.: Randomization, Bootstrap and Monte Carlo Methods in Biology. Chapman & Hall, London (2007)
Mawhinney, T.: Total quality management and organizational behavior management: an integration for continual improvement. J. Appl. Behav. Anal. 25(3), 525–543 (1992)
Medina, J., Delgado, M.: Metodología de entrenamiento de observadores para investigaciones sobre E. F. y deporte en las que se utilice como método la observación. Motricidad: Revista de Ciencias de la Actividad Física y del Deporte 5, 69–86 (1999)
Reid, J.B., De Master, B.: The efficacy of the spotcheck procedure in maintaining the reliability of data collected by observers in quasi-natural settings: two pilot studies. ORI Res. Bull. 12(8), 1–13 (1972)
Remmert, H.: Analysis of group-tactical offensive behavior in elite basketball on the basis of a process orientated model. Eur. J. Sport Sci. 3(3), 1–12 (2003)
Sánchez-Algarra, P., Anguera, M.T.: Qualitative/quantitative integration in the inductive observational study of interactive behaviour: impact of recording and coding predominating perspectives. Qual. Quant. Int. J. Methodol. 47(2), 1237–1257 (2013)
Sweeney, M.A., Cottle, W.C.: Nonverbal acuity: a comparison of counselors and noncounselors. J. Couns. Psychol. 23, 394–397 (1976)
Tukey, J.W.: Exploratory Data Analysis. Addison-Wesley, London (1977)
Acknowledgments
This study forms part of the research project Observación de la interacción en deporte y actividad física: Avances técnicos y metodológicos en registros automatizados cualitativos-cuantitativos [Observing interactions in sport and physical activity: Technical and methodological advances in automated qualitative/quantitative registers], which is funded by the Spanish Secretary of State for Research, Development and Innovation and the Ministry of Economy and Competitiveness [DEP2012-32124] over the three-year period 2012–2015.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Losada, J.L., Manolov, R. The process of basic training, applied training, maintaining the performance of an observer. Qual Quant 49, 339–347 (2015). https://doi.org/10.1007/s11135-014-9989-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11135-014-9989-7