Advances in Health Sciences Education

, Volume 23, Issue 2, pp 311–321 | Cite as

Response distortion on personality tests in applicants: comparing high-stakes to low-stakes medical settings

  • Jeromy Anglim
  • Stefan Bozic
  • Jonathon Little
  • Filip Lievens
Article

Abstract

The current study examined the degree to which applicants applying for medical internships distort their responses to personality tests and assessed whether this response distortion led to reduced predictive validity. The applicant sample (n = 530) completed the NEO Personality Inventory whilst applying for one of 60 positions as first-year post-graduate medical interns. Predictive validity was assessed using university grades, averaged over the entire medical degree. Applicant responses for the Big Five (i.e., neuroticism, extraversion, openness, conscientiousness, and agreeableness) and 30 facets of personality were compared to a range of normative samples where personality was measured in standard research settings including medical students, role model physicians, current interns, and standard young-adult test norms. Applicants had substantially higher scores on conscientiousness, openness, agreeableness, and extraversion and lower scores on neuroticism with an average absolute standardized difference of 1.03, when averaged over the normative samples. While current interns, medical students, and especially role model physicians do show a more socially desirable personality profile than standard test norms, applicants provided responses that were substantially more socially desirable. Of the Big Five, conscientiousness was the strongest predictor of academic performance in both applicants (r = .11) and medical students (r = .21). Findings suggest that applicants engage in substantial response distortion, and that the predictive validity of personality is modest and may be reduced in an applicant setting.

Keywords

Personality traits Academic performance Medical students Five Factor Model Medical student selection 

Supplementary material

10459_2017_9796_MOESM1_ESM.docx (493 kb)
Supplementary material 1 (DOCX 493 kb)

References

  1. Albanese, M. A., Snow, M. H., Skochelak, S. E., Huggett, K. N., & Farrell, P. M. (2003). Assessing personal qualities in medical school admissions. Academic Medicine, 78(3), 313–321.CrossRefGoogle Scholar
  2. Anglim, J., & Grant, S. L. (2016). Predicting psychological and subjective well-being from personality: Incremental prediction from 30 facets over the big 5. Journal of Happiness Studies, 17, 59–80.CrossRefGoogle Scholar
  3. Anglim, J., Morse, G., De Vries, R. E., Maccann, C., & Marty, A. (2017). Comparing job applicants to non-applicants using an item-level bifactor model on the HEXACO Personality Inventory. European Journal of Personality. doi: 10.1002/per.2120.Google Scholar
  4. Birkeland, S. A., Manson, T. M., Kisamore, J. L., Brannick, M. T., & Smith, M. A. (2006). A meta-analytic investigation of job applicant faking on personality measures. International Journal of Selection and Assessment, 14(4), 317–335.CrossRefGoogle Scholar
  5. Bore, M., Munro, D., & Powis, D. (2009). A comprehensive model for the selection of medical students. Medical Teacher, 31(12), 1066–1072.CrossRefGoogle Scholar
  6. Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.CrossRefGoogle Scholar
  7. Costa, P. T., & Mccrae, R. R. (1992). Neo PI-R professional manual: Odessa. FL: Psychological Assessment Resources.Google Scholar
  8. De Fruyt, F., De Bolle, M., Mccrae, R. R., Terracciano, A., & Costa, P. T. (2009). Assessing the universal structure of personality in early adolescence: The NEO-PI-R and NEO-PI-3 in 24 cultures. Assessment, 16(3), 301–311.CrossRefGoogle Scholar
  9. De Leng, W. E., Stegers-Jager, K. M., Husbands, A., Dowell, J. S., Born, M. P., & Themmen, A. P. N. (2017). Scoring method of a situational judgment test: Influence on internal consistency reliability, adverse impact and correlation with personality? Advances in Health Sciences Education, 22, 243–265.CrossRefGoogle Scholar
  10. De Vries, A., De Vries, R. E., & Born, M. P. (2011). Broad versus narrow traits: Conscientiousness and Honesty–Humility as predictors of academic criteria. European Journal of Personality, 25(5), 336–348.CrossRefGoogle Scholar
  11. Doherty, E. M., & Nugent, E. (2011). Personality factors and medical training: A review of the literature. Medical Education, 45(2), 132–140.CrossRefGoogle Scholar
  12. Eva, K. W., & Macala, C. (2014). Multiple mini-interview test characteristics: ‘Tis better to ask candidates to recall than to imagine. Medical Education, 48(6), 604–613.CrossRefGoogle Scholar
  13. Eva, K. W., Reiter, H. I., Trinh, K., Wasi, P., Rosenfeld, J., & Norman, G. R. (2009). Predictive validity of the multiple mini-interview for selecting medical trainees. Medical Education, 43(8), 767–775.CrossRefGoogle Scholar
  14. Eva, K. W., Rosenfeld, J., Reiter, H. I., & Norman, G. R. (2004). An admissions OSCE: The multiple mini-interview. Medical Education, 38(3), 314–326.CrossRefGoogle Scholar
  15. Ferguson, E., Mcmanus, I., James, D., O’hehir, F., & Sanders, A. (2003). Pilot study of the roles of personality, references, and personal statements in relation to performance over the five years of a medical degreeCommentary: How to derive causes from correlations in educational studies. BMJ, 326(7386), 429–432.CrossRefGoogle Scholar
  16. Gray, E. K., & Watson, D. (2002). General and specific traits of personality and their relation to sleep and academic performance. Journal of Personality, 70(2), 177–206.CrossRefGoogle Scholar
  17. Griffin, B., Harding, D. W., Wilson, I. G., & Yeomans, N. D. (2008). Does practice make perfect? The effect of coaching and retesting on selection tests used for admission to an Australian medical school. Medical Journal of Australia, 189(5), 270–273.Google Scholar
  18. Griffin, B., Hesketh, B., & Grayson, D. (2004). Applicants faking good: Evidence of item bias in the NEO PI-R. Personality and Individual Differences, 36(7), 1545–1558.CrossRefGoogle Scholar
  19. Griffin, B., & Wilson, I. G. (2012a). Associations between the big five personality factors and multiple mini-interviews. Advances in Health Sciences Education, 17(3), 377–388.CrossRefGoogle Scholar
  20. Griffin, B., & Wilson, I. G. (2012b). Faking good: Self-enhancement in medical school applicants. Medical Education, 46(5), 485–490.CrossRefGoogle Scholar
  21. Haight, S. J., Chibnall, J. T., Schindler, D. L., & Slavin, S. J. (2012). Associations of medical student personality and health/wellness characteristics with their medical school performance across the curriculum. Academic Medicine, 87(4), 476–485.CrossRefGoogle Scholar
  22. Hobfoll, S., Anson, O., & Antonovsky, A. (1982). Personality factors as predictors of medical student performance. Medical Education, 16(5), 251–258.CrossRefGoogle Scholar
  23. Hojat, M., Michalec, B., Jon Veloski, J., & Tykocinski, M. L. (2015). Can empathy, other personality attributes, and level of positive social influence in medical school identify potential leaders in medicine? Academic Medicine, 90(4), 505–510.CrossRefGoogle Scholar
  24. Hojat, M., Nasca, T. J., Magee, M., Feeney, K., Pascual, R., Urbano, F., et al. (1999). A comparison of the personality profiles of internal medicine residents, physician role models, and the general population. Academic Medicine, 74(12), 1327–1333.CrossRefGoogle Scholar
  25. Horwood, S., Anglim, J., & Tooley, G. (2015). Type D personality and the Five-Factor Model: A facet-level analysis. Personality and Individual Differences, 83, 50–54.CrossRefGoogle Scholar
  26. Jerant, A., Griffin, E., Rainwater, J., Henderson, M., Sousa, F., Bertakis, K. D., et al. (2012). Does applicant personality influence multiple mini-interview performance and medical school acceptance offers? Academic Medicine, 87(9), 1250–1259.CrossRefGoogle Scholar
  27. Knights, J. A., & Kennedy, B. J. (2007). Medical school selection: Impact of dysfunctional tendencies on academic performance. Medical Education, 41(4), 362–368.CrossRefGoogle Scholar
  28. Kulasegaram, K., Reiter, H. I., Wiesner, W., Hackett, R. D., & Norman, G. R. (2010). Non-association between NEO-5 personality tests and multiple mini-interview. Advances in Health Sciences Education, 15(3), 415–423.CrossRefGoogle Scholar
  29. Libbrecht, N., Lievens, F., Carette, B., & Côté, S. (2014). Emotional intelligence predicts success in medical school. Emotion, 14(1), 64–73.CrossRefGoogle Scholar
  30. Lievens, F. (2013). Adjusting medical school admission: Assessing interpersonal skills using situational judgement tests. Medical Education, 47(2), 182–189.CrossRefGoogle Scholar
  31. Lievens, F., Coetsier, P., De Fruyt, F., & De Maeseneer, J. (2002). Medical students’ personality characteristics and academic performance: A five-factor model perspective. Medical Education, 36(11), 1050–1056.CrossRefGoogle Scholar
  32. Lievens, F., Ones, D. S., & Dilchert, S. (2009). Personality scale validities increase throughout medical school. Journal of Applied Psychology, 94(6), 1514–1535.CrossRefGoogle Scholar
  33. Mackenzie, R. K., Dowell, J., Ayansina, D., & Cleland, J. A. (2017). Do personality traits assessed on medical school admission predict exit performance? A UK-wide longitudinal cohort study. Advances in Health Sciences Education, 22, 365–385.CrossRefGoogle Scholar
  34. Marshall, M. B., De Fruyt, F., Rolland, J.-P., & Bagby, R. M. (2005). Socially desirable responding and the factorial stability of the NEO PI-R. Psychological Assessment, 17(3), 379–384.CrossRefGoogle Scholar
  35. Mccrae, R. R., Costa, J., Paul, T., & Martin, T. A. (2005). The NEO-PI-3: A more readable revised NEO personality inventory. Journal of Personality Assessment, 84(3), 261–270.CrossRefGoogle Scholar
  36. Mclarnon, M. J., Rothstein, M. G., Goffin, R. D., Rieder, M. J., Poole, A., Krajewski, H. T., et al. (2017). How important is personality in the selection of medical school students? Personality and Individual Differences, 104, 442–447.CrossRefGoogle Scholar
  37. Mcmanus, I., Keeling, A., & Paice, E. (2004). Stress, burnout and doctors’ attitudes to work are determined by personality and learning style: A twelve year longitudinal study of UK medical graduates. BMC Medicine, 2, 29.CrossRefGoogle Scholar
  38. Mesmer-Magnus, J., & Viswesvaran, C. (2006). Assessing response distortion in personality tests: A review of research designs and analytic strategies. In M. H. Peterson & R. L. Griffith (Eds.), A closer examination of applicant faking behavior (pp. 85–113). Charlotte: Information Age Publishing.Google Scholar
  39. Morgeson, F. P., Campion, M. A., Dipboye, R. L., Hollenbeck, J. R., Murphy, K., & Schmitt, N. (2007). Reconsidering the use of personality tests in personnel selection contexts. Personnel Psychology, 60(3), 683–729.CrossRefGoogle Scholar
  40. Musson, D. M. (2009). Personality and medical education. Medical Education, 43(5), 395–397.CrossRefGoogle Scholar
  41. Olkin, I., & Finn, J. D. (1995). Correlations redux. Psychological Bulletin, 118(1), 155–164.CrossRefGoogle Scholar
  42. Oswald, F. L., & Hough, L. M. (2008). Personality testing and industrial–organizational psychology: A productive exchange and some future directions. Industrial and Organizational Psychology: Perspectives on Science and Practice, 1(3), 323–332.CrossRefGoogle Scholar
  43. Patterson, F., Ashworth, V., Zibarras, L., Coan, P., Kerrin, M., & O’neill, P. (2012). Evaluations of situational judgement tests to assess non-academic attributes in selection. Medical Education, 46(9), 850–868.CrossRefGoogle Scholar
  44. Patterson, F., Baron, H., Carr, V., Plint, S., & Lane, P. (2009). Evaluation of three short-listing methodologies for selection into postgraduate training in general practice. Medical Education, 43(1), 50–57.CrossRefGoogle Scholar
  45. Patterson, F., Knight, A., Dowell, J., Nicholson, S., Cousans, F., & Cleland, J. (2016). How effective are selection methods in medical education? A systematic review. Medical Education, 50(1), 36–60.CrossRefGoogle Scholar
  46. Paunonen, S. V., & Jackson, D. N. (2000). What is beyond the big five? Plenty! Journal of Personality, 68(5), 821–835.CrossRefGoogle Scholar
  47. Peng, R., Khaw, H., & Edariah, A. (1995). Personality and performance of preclinical medical students. Medical Education, 29(4), 283–288.CrossRefGoogle Scholar
  48. Pohl, C. A., Hojat, M., & Arnold, L. (2011). Peer nominations as related to academic attainment, empathy, personality, and specialty interest. Academic Medicine, 86(6), 747–751.CrossRefGoogle Scholar
  49. Poropat, A. E. (2009). A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin, 135(2), 322–338.CrossRefGoogle Scholar
  50. Rothstein, M. G., & Goffin, R. D. (2006). The use of personality measures in personnel selection: What does current research support? Human Resource Management Review, 16(2), 155–180.CrossRefGoogle Scholar
  51. Shen, H., & Comrey, A. L. (1997). Predicting medical students’ academic performances by their cognitive abilities and personality characteristics. Academic Medicine, 72(9), 781–786.CrossRefGoogle Scholar
  52. Song, Y., & Shi, M. (2017). Associations between empathy and big five personality traits among Chinese undergraduate medical students. PLoS ONE, 12(2), e0171665.CrossRefGoogle Scholar
  53. Tyssen, R., Dolatowski, F. C., Røvik, J. O., Thorkildsen, R. F., Ekeberg, Ø., Hem, E., et al. (2007). Personality traits and types predict medical school stress: A six-year longitudinal and nationwide study. Medical Education, 41(8), 781–787.CrossRefGoogle Scholar
  54. Woo, S. E., Jin, J., & Lebreton, J. M. (2015). Specificity matters: Criterion-related validity of contextualized and facet measures of conscientiousness in predicting college student performance. Journal of Personality Assessment, 97(3), 301–309.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2017

Authors and Affiliations

  1. 1.School of PsychologyDeakin UniversityGeelongAustralia
  2. 2.Abbotsford, MelbourneAustralia
  3. 3.Department of Personnel Management, Work and Organizational PsychologyGhent UniversityGhentBelgium

Personalised recommendations