Abstract
Recent years have seen leading medical educationalists repeatedly call for a paradigm shift in the way we view, value and use subjectivity in assessment. The argument is that subjective expert raters generally bring desired quality, not just noise, to performance evaluations. While several reviews document the psychometric qualities of the Multiple Mini-Interview (MMI), we currently lack qualitative studies examining what we can learn from MMI raters’ subjectivity. The present qualitative study therefore investigates rater subjectivity or taste in MMI selection interview. Taste (Bourdieu 1984) is a practical sense, which makes it possible at a pre-reflective level to apply ‘invisible’ or ‘tacit’ categories of perception for distinguishing between good and bad. The study draws on data from explorative in-depth interviews with 12 purposefully selected MMI raters. We find that MMI raters spontaneously applied subjective criteria—their taste—enabling them to assess the candidates’ interpersonal attributes and to predict the candidates’ potential. In addition, MMI raters seemed to share a taste for certain qualities in the candidates (e.g. reflectivity, resilience, empathy, contact, alikeness, ‘the good colleague’); hence, taste may be the result of an ongoing enculturation in medical education and healthcare systems. This study suggests that taste is an inevitable condition in the assessment of students’ performance. The MMI set-up should therefore make room for MMI raters’ taste and their connoisseurship, i.e. their ability to taste, to improve the quality of their assessment of medical school candidates.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Albanese, M. A., Snow, M. H., Skochelak, S. E., Huggett, K. N., & Farrell, P. M. (2003). Assessing personal qualities in medical school admissions. Academic Medicine, 78(3), 313–321.
Andreassen, P., Pedersen, K., Jensen, R. D., Møller, J. E., Carlsen, C. G. & O’Neill, L. (2016). Optagelsessamtaler på medicinstudiet ved Aarhus Universitet—En pilottest af et Multiple Mini Interview (MMI). Retrieved from Aarhus University: http://www.e-pages.dk/aarhusuniversitet/1608/.
Axelson, R. D., & Kreiter, C. D. (2009). Rater and occasion impacts on the reliability of pre-admission assessments. Medical Education, 43(12), 1198–1202.
Batalden, P., Leach, D., Swing, S., Dreyfus, H., & Dreyfus, S. (2002). General competencies and accreditation in graduate medical education. Health Affairs, 21, 103–111.
Bourdieu, P. (1984). Distinction. A social critique of the judgement of taste. London, Melbourne and Henley: Routledge & Kegan Paul.
Bourdieu, P. (1998). Practical reason: On the theory of action. Cambridge: Polity Press.
Bourdieu, P. (2000). Pascalian meditations. Stanford: Stanford University Press.
Brennan, R. L. (2001). Generalizability theory. New York: Springer.
Carlisle, C. (2014). On habit. New York: Routledge.
Charmaz, K. (2000). Grounded theory: Objectivist and constructivist methods. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 509–536). Beverly Hills: Sage Publications.
Charmaz, K. (2006). Constructing grounded theory. London: Sage Publications.
Christensen, M. K. (2009). An eye for talent: Talent identification and practical sense of top-level soccer coaches. Sociology of Sport Journal, 26, 365–382.
Crossley, N. (2013). Interactions, juxtapositions, and tastes: Conceptualizing “relations” in relational sociology. In C. Powell & F. Dépelteau (Eds.), Conceptualizing relational sociology (pp. 123–144). New York: Plagrave Macmillan.
Cunnington, J. P. W., Neville, A. J., & Norman, G. R. (1996). The risks of thoroughness: Reliability and validity of global ratings and checklists in an OSCE. Advances in Health Sciences Education, 1(3), 227–233.
Dowell, J., Lynch, B., Till, H., Kumwenda, B., & Husbands, A. (2012). The multiple mini-interview in the U.K. context: 3 years of experience at Dundee. Medical Teacher, 34(4), 297–304.
Dreyfus, S. E. (2004). The five-stage model of adult skill acquisition. Bulletin of Science, Technology & Society, 24, 177–181.
Dreyfus, H., & Dreyfus, S. (1986). Mind over machine. The power of human intuitive experience in the era of the computer. New York: The Free Press.
Eva, K. W., Reiter, H. I., Rosenfeld, J., & Norman, G. R. (2004a). The relationship between interviewers’ characteristics and ratings assigned during a multiple mini-interview. Academic Medicine, 79(6), 602–609. doi:10.1097/00001888-200406000-00021.
Eva, K. W., Rosenfeld, J., Reiter, H. I., & Norman, G. R. (2004b). An admissions OSCE: The multiple mini-interview. Medical Education, 38(3), 314–326. doi:10.1046/j.1365-2923.2004.01776.x.
Gafni, N., Moshinsky, A., Eisenberg, O., Zeigler, D., & Ziv, A. (2012). Reliability estimates: Behavioural stations and questionnaires in medical school admissions. Medical Education, 46(3), 277–288.
Garro, L. C., & Mattingly, C. (2000). Narrative as construct and construction. In C. Mattingly & L. C. Garro (Eds.), Narrative and the cultural construction of illness and healing (pp. 1–49). Berkeley: University of California Press.
Govaerts, M., & van der Vleuten, C. P. (2013). Validity in work-based assessment: expanding our horizons. Medical Education, 47(12), 1164–1174.
Harris, S., & Owen, C. (2007). Discerning quality: Using the multiple mini-interview in student selection for the Australian National University Medical School. Medical Education, 41(3), 234–241.
Hodges, B. (2013). Assessment in the post-psychometric era: Learning to love the subjective and collective. Medical Teacher, 35(7), 564–568.
Hofmeister, M., Lockyer, J., & Crutcher, R. (2009). The multiple mini-interview for selection of international medical graduates into family medicine residency education. Medical Education, 43(6), 573–579.
Hogg, M. A., & Vaughan, G. (2005). Social Psychology (4th ed.). Essex: Pearson Eduation Limited.
Knorr, M., & Hissbach, J. (2014). Multiple mini-interviews: same concept, different approaches. Medical Education, 48(12), 1157–1175.
Kumar, K., Roberts, C., Rothnie, I., du Fresne, C., & Walton, M. (2009). Experiences of the multiple mini-interview: A qualitative analysis. Medical Education, 43(4), 360–367. doi:10.1111/j.1365-2923.2009.03291.x.
Kvale, S., & Brinkmann, S. (2009). InterViews: Learning the craft of qualitative research interviewing (2nd ed.). Los Angeles: SAGE Publications.
Lemay, J. F., Lockyer, J. M., Collin, V. T., & Brownell, K. W. (2007). Assessment of non-cognitive traits through the admissions multiple mini-interview. Medical Education, 41(6), 573–579.
Malterud, K., Siersma, V. K., & Guassora, A. D. (2015). Sample size in qualititative interview studies. Qualitative Health Research, 26(13), 1753–1760.
Miles, M. B., Huberman, A. M., & Saldaña, J. (2014). Qualitative data analysis. A methods sourcebook (3rd ed.). Los Angeles: SAGE Publications.
Onyon, C., Wall, D., & Goodyear, H. M. (2009). Reliability of multistation interviews in selection of junior doctors for specialty training. Medical Teacher, 31(7), 665–667.
Patterson, F., Knight, A., Dowell, J., Nicholson, S., Cousans, F., & Cleland, J. (2016). How effective are selection methods in medical education? A systematic review. Medical Education, 50(1), 36–60.
Pau, A., Jeevaratnam, K., Chen, Y. S., Fall, A. A., Khoo, C., & Nadarajah, V. D. (2013). The multiple mini-interview (MMI) for student selection in health professions training—a systematic review. Medical Teacher, 35(12), 1027–1041. doi:10.3109/0142159x.2013.829912.
Penã, A. (2010). The Dreyfus model of clinical problem-solving skills acquisition: A critical perspective. Medical Education Online. doi:10.3402/meo.v15i0.48461-11.
Polanyi, M. (1958). Personal knowledge. London: Routledge and Kegan Paul.
Rees, E. L., Hawarden, A. W., Dent, G., Hays, R., Bates, J., & Hassell, A. B. (2016). Evidence regarding the utility of multiple mini-interview (MMI) for selection to undergraduate health programs: A BEME systematic review: BEME Guide No. 37. Medical Teacher, 38(5), 443–455.
Roberts, C., Rothnie, I., Zoanetti, N., & Crossley, J. (2010). Should candidate scores be adjusted for interviewer stringency or leniency in the multiple mini-interview? Medical Education, 44(7), 690–698. doi:10.1111/j.1365-2923.2010.03689.x.
Salvatori, P. (2001). Reliability and validity of admissions tolls to select students for the health professions. Advances in Health Sciences Education: Theory and Practice, 6(2), 159–175.
Sebok, S. S., Luu, K., & Klinger, D. A. (2014). Psychometric properties of the multiple mini-interview used for medical admissions: Findings from generalisability and Rasch analyses. Advances in Health Science Education: Theory and Practice, 19(1), 71–84.
Stevens, D. P. (2002). Finding safety in medical education. Quality & Safety in Health Care, 11, 109–110.
van der Vleuten, C. P. M. (2014). When I say …context specificity. Medical Education, 48(3), 234–235.
van der Vleuten, C. P., Schuwirth, L. W., Driessen, E. W., Dijkstra, J., Tigelaar, D., Baartman, L. K., et al. (2012). A model for programmatic assessment fit for purpose. Medical Teacher, 34(3), 205–214.
Acknowledgements
The authors wish to extend their gratitude to the 12 MMI raters who participated in the interviews and kindly shared their experiences.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Christensen, M.K., Lykkegaard, E., Lund, O. et al. Qualitative analysis of MMI raters’ scorings of medical school candidates: A matter of taste?. Adv in Health Sci Educ 23, 289–310 (2018). https://doi.org/10.1007/s10459-017-9794-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-017-9794-x