Advances in Health Sciences Education

, Volume 23, Issue 2, pp 289–310 | Cite as

Qualitative analysis of MMI raters’ scorings of medical school candidates: A matter of taste?

  • Mette K. Christensen
  • Eva Lykkegaard
  • Ole Lund
  • Lotte D. O’Neill


Recent years have seen leading medical educationalists repeatedly call for a paradigm shift in the way we view, value and use subjectivity in assessment. The argument is that subjective expert raters generally bring desired quality, not just noise, to performance evaluations. While several reviews document the psychometric qualities of the Multiple Mini-Interview (MMI), we currently lack qualitative studies examining what we can learn from MMI raters’ subjectivity. The present qualitative study therefore investigates rater subjectivity or taste in MMI selection interview. Taste (Bourdieu 1984) is a practical sense, which makes it possible at a pre-reflective level to apply ‘invisible’ or ‘tacit’ categories of perception for distinguishing between good and bad. The study draws on data from explorative in-depth interviews with 12 purposefully selected MMI raters. We find that MMI raters spontaneously applied subjective criteria—their taste—enabling them to assess the candidates’ interpersonal attributes and to predict the candidates’ potential. In addition, MMI raters seemed to share a taste for certain qualities in the candidates (e.g. reflectivity, resilience, empathy, contact, alikeness, ‘the good colleague’); hence, taste may be the result of an ongoing enculturation in medical education and healthcare systems. This study suggests that taste is an inevitable condition in the assessment of students’ performance. The MMI set-up should therefore make room for MMI raters’ taste and their connoisseurship, i.e. their ability to taste, to improve the quality of their assessment of medical school candidates.


Admission Assessment Bourdieu Medical school candidates MMI rater’s scorings Multiple Mini-Interview Qualitative study Selection Subjectivity Taste 



The authors wish to extend their gratitude to the 12 MMI raters who participated in the interviews and kindly shared their experiences.


  1. Albanese, M. A., Snow, M. H., Skochelak, S. E., Huggett, K. N., & Farrell, P. M. (2003). Assessing personal qualities in medical school admissions. Academic Medicine, 78(3), 313–321.CrossRefGoogle Scholar
  2. Andreassen, P., Pedersen, K., Jensen, R. D., Møller, J. E., Carlsen, C. G. & O’Neill, L. (2016). Optagelsessamtaler på medicinstudiet ved Aarhus Universitet—En pilottest af et Multiple Mini Interview (MMI). Retrieved from Aarhus University:
  3. Axelson, R. D., & Kreiter, C. D. (2009). Rater and occasion impacts on the reliability of pre-admission assessments. Medical Education, 43(12), 1198–1202.CrossRefGoogle Scholar
  4. Batalden, P., Leach, D., Swing, S., Dreyfus, H., & Dreyfus, S. (2002). General competencies and accreditation in graduate medical education. Health Affairs, 21, 103–111.CrossRefGoogle Scholar
  5. Bourdieu, P. (1984). Distinction. A social critique of the judgement of taste. London, Melbourne and Henley: Routledge & Kegan Paul.Google Scholar
  6. Bourdieu, P. (1998). Practical reason: On the theory of action. Cambridge: Polity Press.Google Scholar
  7. Bourdieu, P. (2000). Pascalian meditations. Stanford: Stanford University Press.Google Scholar
  8. Brennan, R. L. (2001). Generalizability theory. New York: Springer.CrossRefGoogle Scholar
  9. Carlisle, C. (2014). On habit. New York: Routledge.Google Scholar
  10. Charmaz, K. (2000). Grounded theory: Objectivist and constructivist methods. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 509–536). Beverly Hills: Sage Publications.Google Scholar
  11. Charmaz, K. (2006). Constructing grounded theory. London: Sage Publications.Google Scholar
  12. Christensen, M. K. (2009). An eye for talent: Talent identification and practical sense of top-level soccer coaches. Sociology of Sport Journal, 26, 365–382.CrossRefGoogle Scholar
  13. Crossley, N. (2013). Interactions, juxtapositions, and tastes: Conceptualizing “relations” in relational sociology. In C. Powell & F. Dépelteau (Eds.), Conceptualizing relational sociology (pp. 123–144). New York: Plagrave Macmillan.CrossRefGoogle Scholar
  14. Cunnington, J. P. W., Neville, A. J., & Norman, G. R. (1996). The risks of thoroughness: Reliability and validity of global ratings and checklists in an OSCE. Advances in Health Sciences Education, 1(3), 227–233.CrossRefGoogle Scholar
  15. Dowell, J., Lynch, B., Till, H., Kumwenda, B., & Husbands, A. (2012). The multiple mini-interview in the U.K. context: 3 years of experience at Dundee. Medical Teacher, 34(4), 297–304.CrossRefGoogle Scholar
  16. Dreyfus, S. E. (2004). The five-stage model of adult skill acquisition. Bulletin of Science, Technology & Society, 24, 177–181.CrossRefGoogle Scholar
  17. Dreyfus, H., & Dreyfus, S. (1986). Mind over machine. The power of human intuitive experience in the era of the computer. New York: The Free Press.Google Scholar
  18. Eva, K. W., Reiter, H. I., Rosenfeld, J., & Norman, G. R. (2004a). The relationship between interviewers’ characteristics and ratings assigned during a multiple mini-interview. Academic Medicine, 79(6), 602–609. doi: 10.1097/00001888-200406000-00021.CrossRefGoogle Scholar
  19. Eva, K. W., Rosenfeld, J., Reiter, H. I., & Norman, G. R. (2004b). An admissions OSCE: The multiple mini-interview. Medical Education, 38(3), 314–326. doi: 10.1046/j.1365-2923.2004.01776.x.CrossRefGoogle Scholar
  20. Gafni, N., Moshinsky, A., Eisenberg, O., Zeigler, D., & Ziv, A. (2012). Reliability estimates: Behavioural stations and questionnaires in medical school admissions. Medical Education, 46(3), 277–288.CrossRefGoogle Scholar
  21. Garro, L. C., & Mattingly, C. (2000). Narrative as construct and construction. In C. Mattingly & L. C. Garro (Eds.), Narrative and the cultural construction of illness and healing (pp. 1–49). Berkeley: University of California Press.Google Scholar
  22. Govaerts, M., & van der Vleuten, C. P. (2013). Validity in work-based assessment: expanding our horizons. Medical Education, 47(12), 1164–1174.CrossRefGoogle Scholar
  23. Harris, S., & Owen, C. (2007). Discerning quality: Using the multiple mini-interview in student selection for the Australian National University Medical School. Medical Education, 41(3), 234–241.CrossRefGoogle Scholar
  24. Hodges, B. (2013). Assessment in the post-psychometric era: Learning to love the subjective and collective. Medical Teacher, 35(7), 564–568.CrossRefGoogle Scholar
  25. Hofmeister, M., Lockyer, J., & Crutcher, R. (2009). The multiple mini-interview for selection of international medical graduates into family medicine residency education. Medical Education, 43(6), 573–579.CrossRefGoogle Scholar
  26. Hogg, M. A., & Vaughan, G. (2005). Social Psychology (4th ed.). Essex: Pearson Eduation Limited.Google Scholar
  27. Knorr, M., & Hissbach, J. (2014). Multiple mini-interviews: same concept, different approaches. Medical Education, 48(12), 1157–1175.CrossRefGoogle Scholar
  28. Kumar, K., Roberts, C., Rothnie, I., du Fresne, C., & Walton, M. (2009). Experiences of the multiple mini-interview: A qualitative analysis. Medical Education, 43(4), 360–367. doi: 10.1111/j.1365-2923.2009.03291.x.CrossRefGoogle Scholar
  29. Kvale, S., & Brinkmann, S. (2009). InterViews: Learning the craft of qualitative research interviewing (2nd ed.). Los Angeles: SAGE Publications.Google Scholar
  30. Lemay, J. F., Lockyer, J. M., Collin, V. T., & Brownell, K. W. (2007). Assessment of non-cognitive traits through the admissions multiple mini-interview. Medical Education, 41(6), 573–579.CrossRefGoogle Scholar
  31. Malterud, K., Siersma, V. K., & Guassora, A. D. (2015). Sample size in qualititative interview studies. Qualitative Health Research, 26(13), 1753–1760.CrossRefGoogle Scholar
  32. Miles, M. B., Huberman, A. M., & Saldaña, J. (2014). Qualitative data analysis. A methods sourcebook (3rd ed.). Los Angeles: SAGE Publications.Google Scholar
  33. Onyon, C., Wall, D., & Goodyear, H. M. (2009). Reliability of multistation interviews in selection of junior doctors for specialty training. Medical Teacher, 31(7), 665–667.CrossRefGoogle Scholar
  34. Patterson, F., Knight, A., Dowell, J., Nicholson, S., Cousans, F., & Cleland, J. (2016). How effective are selection methods in medical education? A systematic review. Medical Education, 50(1), 36–60.CrossRefGoogle Scholar
  35. Pau, A., Jeevaratnam, K., Chen, Y. S., Fall, A. A., Khoo, C., & Nadarajah, V. D. (2013). The multiple mini-interview (MMI) for student selection in health professions training—a systematic review. Medical Teacher, 35(12), 1027–1041. doi: 10.3109/0142159x.2013.829912.CrossRefGoogle Scholar
  36. Penã, A. (2010). The Dreyfus model of clinical problem-solving skills acquisition: A critical perspective. Medical Education Online. doi: 10.3402/meo.v15i0.48461-11.Google Scholar
  37. Polanyi, M. (1958). Personal knowledge. London: Routledge and Kegan Paul.Google Scholar
  38. Rees, E. L., Hawarden, A. W., Dent, G., Hays, R., Bates, J., & Hassell, A. B. (2016). Evidence regarding the utility of multiple mini-interview (MMI) for selection to undergraduate health programs: A BEME systematic review: BEME Guide No. 37. Medical Teacher, 38(5), 443–455.CrossRefGoogle Scholar
  39. Roberts, C., Rothnie, I., Zoanetti, N., & Crossley, J. (2010). Should candidate scores be adjusted for interviewer stringency or leniency in the multiple mini-interview? Medical Education, 44(7), 690–698. doi: 10.1111/j.1365-2923.2010.03689.x.CrossRefGoogle Scholar
  40. Salvatori, P. (2001). Reliability and validity of admissions tolls to select students for the health professions. Advances in Health Sciences Education: Theory and Practice, 6(2), 159–175.CrossRefGoogle Scholar
  41. Sebok, S. S., Luu, K., & Klinger, D. A. (2014). Psychometric properties of the multiple mini-interview used for medical admissions: Findings from generalisability and Rasch analyses. Advances in Health Science Education: Theory and Practice, 19(1), 71–84.CrossRefGoogle Scholar
  42. Stevens, D. P. (2002). Finding safety in medical education. Quality & Safety in Health Care, 11, 109–110.CrossRefGoogle Scholar
  43. van der Vleuten, C. P. M. (2014). When I say …context specificity. Medical Education, 48(3), 234–235.CrossRefGoogle Scholar
  44. van der Vleuten, C. P., Schuwirth, L. W., Driessen, E. W., Dijkstra, J., Tigelaar, D., Baartman, L. K., et al. (2012). A model for programmatic assessment fit for purpose. Medical Teacher, 34(3), 205–214.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2017

Authors and Affiliations

  • Mette K. Christensen
    • 1
  • Eva Lykkegaard
    • 1
  • Ole Lund
    • 1
  • Lotte D. O’Neill
    • 2
  1. 1.Centre for Health Sciences EducationAarhus UniversityAarhusDenmark
  2. 2.SDU Centre for Teaching and LearningUniversity of Southern DenmarkOdenseDenmark

Personalised recommendations