SVT Meets PVT: Development and Initial Validation of the Inventory of Problems – Memory (IOP-M)

Abstract

A growing literature indicates that to evaluate the credibility of a clinical presentation it would be optimal to rely on multiple sources of information, and use both symptom validity tests (SVTs) and performance validity tests (PVTs) whenever possible. In this paper, we present the development and initial validation of a PVT module designed to be used in combination with a free-standing SVT. Named Inventory of Problems – Memory (IOP-M), this new PVT module is given to the examinee immediately after completing the Inventory of Problems – 29 (IOP-29). It consists of a 34-item, two-alternative, forced-choice, implicit recognition test. Results from 360 nonclinical volunteers – 192 instructed to respond honestly (honest controls) and 168 instructed to feign mental illness (experimental simulators) – suggest that the IOP-M has the potential to yield incremental validity over using the IOP-29 alone. In fact, a series of hierarchical logistic regressions using group as criterion variable (0 = honest control; 1 = experimental simulator) and the IOP-29 and IOP-M as predictors showed that the models including both measures significantly improved classification accuracy over those including the IOP-29 only, Δχ2 ≥ 19.1, p < .01. When considering the optimal cut scores for each measure, only 6 of the 168 simulators (i.e., less than 4%) passed both the IOP-29 and IOP-M, and only 3 of the 192 honest responders (i.e., less than 2%) failed both. A closer examination of false positive classifications, however, revealed that the IOP-M could be prone to false positive errors in examinees with moderate to severe cognitive impairment.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

Notes

  1. 1.

    Accordingly, the IOP-Memory test may be referred to as the IOP-Memory module or IOP-M module.

  2. 2.

    Because homoscedasticity could not be assumed, the Welch–Satterthwaite method was used to adjust degrees of freedom.

  3. 3.

    Because homoscedasticity could not be assumed, the Welch–Satterthwaite method was used to adjust degrees of freedom.

  4. 4.

    Severity levels for the MoCA were retrieved from the official MoCA website at https://www.mocatest.org/faq/. However, because research for these severity ranges has not been established yet, these characterizations should be considered “tentative.”

References

  1. Abeare, C. A., Messa, I., Zuccato, B. G., Merker, B., & Erdodi, L. A. (2018). Prevalence of invalid performance on baseline testing for sport-related concussion by age and validity indicator. JAMA Neurology, 75(6), 697–703. https://doi.org/10.1001/jamaneurol.2018.0031.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Abeare, C. A., Messa, I., Whitfield, C., Zuccato, B., Casey, J., & Erdodi, L. (2019). Performance validity in collegiate football athletes at baseline neurocognitive testing. Journal of Head Trauma Rehabilitation, 34(4), 20–31. https://doi.org/10.1097/HTR.0000000000000451.

    Article  Google Scholar 

  3. An, K. Y., Kaploun, K., Erdodi, L. A., & Abeare, C. A. (2017). Performance validity in undergraduate research participants: A comparison of failure rates across tests and cutoffs. The Clinical Neuropsychologist, 31(1), 193–206. https://doi.org/10.1080/13854046.2016.1217046.

    Article  PubMed  Google Scholar 

  4. Bigler, E. D. (2012). Symptom validity testing, effort and neuropsychological assessment. Journal of the International Neuropsychological Society, 18, 632–642. https://doi.org/10.1017/S1355617712000252.

    Article  PubMed  Google Scholar 

  5. Bigler, E. D. (2015). Neuroimaging as a biomarker in symptom validity and performance validity testing. Brain Imaging and Behavior, 9(3), 421–444. https://doi.org/10.1007/s11682-015-9409-1.

    Article  PubMed  Google Scholar 

  6. Boone, K. B. (2013). Clinical practice of forensic neuropsychology. New York, NY: Guilford.

    Google Scholar 

  7. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  8. Dandachi-FitzGerald, B., Ponds, R. W., Peters, M. J., & Merckelbach, H. (2011). Cognitive underperformance and symptom over-reporting in a mixed psychiatric sample. The Clinical Neuropsychologist, 25(5), 812–828. https://doi.org/10.1080/13854046.2011.583280.

    Article  PubMed  Google Scholar 

  9. Davis, J. J. (2014). Further consideration of advanced clinical solutions word choice: Comparison to the recognition memory test – Words and classification accuracy on a clinical sample. The Clinical Neuropsychologist, 28(8), 1278–1294. https://doi.org/10.1080/13854046.2014.975844.

    Article  PubMed  Google Scholar 

  10. Elhai, J. D., Gray, M. J., Kashdan, T. B., & Franklin, L. C. (2005). Which instruments are most commonly used to assess traumatic event exposure and posttraumatic effects? Asurvey of traumatic stress professionals. Journal of Traumatic Stress, 18, 541–545.

    PubMed  Google Scholar 

  11. Erdodi, L. A. (2019). Aggregating validity indicators: The salience of domain specificity and the indeterminate range in multivariate models of performance validity assessment. Applied Neuropsychology: Adult, 26(2), 155–172. https://doi.org/10.1080/23279095.2017.1384925.

    Article  Google Scholar 

  12. Erdodi, L. A., Abeare, C. A., Lichtenstein, J. D., Tyson, B., T., Kucharski, B., Zuccato, B. G., & Roth, R. M. (2017). WAIS-IV processing speed scores as measures of non-credible responding – The third generation of embedded performance validity indicators. Psychological Assessment, 29(2), 148–157. doi: https://doi.org/10.1037/pas0000319.

  13. Erdodi, L. A., Kirsch, N. L., Lajiness-O’Neill, R., Vingilis, E., & Medoff, B. (2014). Comparing the recognition memory test and the word choice test in a mixed clinical sample: Are they equivalent? Psychological Injury and Law, 7(3), 255–263. https://doi.org/10.1007/s12207-014-9197-8.

    Article  Google Scholar 

  14. Erdodi, L. A., Kirsch, N. L., Sabelli, A. G., & Abeare, C. A. (2018). The grooved pegboard test as a validity indicator – A study on psychogenic interference as a confound in performance validity research. Psychological Injury and Law, 11(4), 307–324. https://doi.org/10.1007/s12207-018-9337-7.

    Article  Google Scholar 

  15. Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). Mini-mental state: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12(3), 189–198.

    PubMed  Google Scholar 

  16. Fox, K. A. & Vincent, J.P. (2020). Types of malingering in PTSD: Evidence from a psychological injury paradigm. Psychological Injury and Law, 13, 90–104. https://doi.org/10.1007/s12207-019-09367-5 .

    Article  Google Scholar 

  17. Gervais, R. O., Ben-Porath, Y. S., Wygant, D. B., & Green, P. (2007). Development and validation of a response Bias scale (RBS) for the MMPI-2. Assessment, 14, 196–208.

    PubMed  Google Scholar 

  18. Giromini, L., Barbosa, F., Coga, G., Azeredo, A., Viglione, D. J., & Zennaro, A. (2019b). Using the inventory of problems – 29 (IOP-29) with the Test of Memory Malingering (TOMM) in symptom validity assessment: A study with a Portuguese sample of experimental feigners. In Using the inventory of problems – 29 (IOP-29) with the test of memory malingering (TOMM) in symptom validity assessment: A study with a Portuguese sample of experimental feigners. Applied Neuropsychology: Adult, [Epub ahead of print]. https://doi.org/10.1080/23279095.2019.1570929.

    Google Scholar 

  19. Giromini, L., Carfora Lettieri, S., Zizolfi, S., Zizolfi, D., Viglione, D. J., Brusadelli, E., Perfetti, B., di Carlo, D. A., & Zennaro, A. (2019). Beyond rare-symptoms endorsement: A clinical comparison simulation study using the Minnesota multiphasic personality Inventory-2 (MMPI-2) with the inventory of Problems-29 (IOP-29). Psychological Injury and Law, [Epub ahead of print], 12, 212–224. https://doi.org/10.1007/s12207-019-09357-7.

    Article  Google Scholar 

  20. Giromini, L., Viglione, D. J., Pignolo, C., & Zennaro, A. (2018). A clinical comparison, simulation study testing the validity of SIMS and IOP-29 with an Italian sample. Psychological Injury and Law, 11(4), 340–350.

    Google Scholar 

  21. Giromini, L., Viglione, D. J., Pignolo, C., & Zennaro, A. (2019). An inventory of problems – 29 (IOP-29) sensitivity study investigating feigning of four different symptom presentations via malingering experimental paradigm. Journal of Personality Assessment, [Epub ahead of print], 1–10. https://doi.org/10.1080/00223891.2019.1566914.

  22. Green, P. (2003). Green’s word memory test. Edmonton, Canada: Green’s Publishing.

    Google Scholar 

  23. Green, P., Allen, L. M., & Astner, K. (1996). The word memory test: A user’s guide to the oral and computer-administered forms, US version 1.1. Durham, NC: CogniSyst.

    Google Scholar 

  24. Green, R. L. (1991). MMPI-2/MMPI: An interpretative manual. Boston, MA: Allyn & Bacon.

    Google Scholar 

  25. Jones, A. (2013). Test of memory malingering: Cutoff scores for psychometrically defined malingering groups in a military sample. The Clinical Neuropsychologist, 27(6), 1043–1059.

    PubMed  Google Scholar 

  26. Jones, A. (2016). Cutoff scores for MMPI-2 and MMPI-2-RF cognitive-somatic validity scales for psychometrically defined malingering groups in a military sample. Archives of Clinical Neuropsychology, 31, 786–801.

    PubMed  Google Scholar 

  27. Ilgunaite, G., Giromini, L., Bosi, J., Viglione, D. J., & Zennaro, A. (2020). A clinical comparison simulation study using the Inventory of Problems-29 (IOP-29) with the Center for Epidemiologic Studies Depression Scale (CES-D) in Lithuania. In A clinical comparison simulation study using the inventory of Problems-29 (IOP-29) with the Center for Epidemiologic Studies Depression Scale (CES-D) in Lithuania. Applied Neuropsychology: Adult, [Epub ahead of print]. https://doi.org/10.1080/23279095.2020.1725518.

    Google Scholar 

  28. Larrabee, G. J. (2008). Aggregation across multiple indicators improves the detection of malingering: Relationship to likelihood ratios. The Clinical Neuropsychologist, 22, 666–679. https://doi.org/10.1080/13854040701494987.

    Article  PubMed  Google Scholar 

  29. Larrabee, G. J. (2012). Performance validity and symptom validity in neuropsychological assessment. Journal of International Neuropsychological Society, 18(4), 625–630.

    Google Scholar 

  30. MacAllister, W. S., Vasserman, M., & Armstrong, K. (2019). Are we documenting performance validity testing in pediatric neuropsychological assessments? A brief report. Child Neuropsychology, 25, 1–8.

    Google Scholar 

  31. Martin, P. K., Schroeder, R. W., & Odland, A. P. (2015). Neuropsychologists’ validity testing beliefs and practices: A survey on north American professionals. The Clinical Neuropsychologist, 29(6), 741–776.

    PubMed  Google Scholar 

  32. Mass, R., Haasen, C., & Wolf, K. (2000). The Eppendorf schizophrenia inventory (ESI). Development and evaluation of a questionnaire for assessment of characteristic self-perception of cognitive dysfunctions by schizophrenic patients. Nervenarzt, 71, 885–892.

    PubMed  Google Scholar 

  33. Mass, R., Haasen, C., & Borgart, E. (2005). Abnormal subjective experiences of schizophrenia: Evaluation of the eppendorf schizophrenia inventory. Psychiatry Research, 135(2), 91–101. https://doi.org/10.1016/j.psychres.2004.08.011

  34. Merten, T., Merckelbach, H., Giger, P., & Stevens, A. (2016). The self-report symptom inventory (SRSI): A new instrument for the assessment of distorted symptom endorsement. Psychological Injury and Law, 9(2), 102–111.

    Google Scholar 

  35. Morey, L. C. (2007). Personality assessment inventory (PAI). Professional manual (2nd ed.). Odessa, FL: Psychological Assessment Resources.

    Google Scholar 

  36. Nasreddine, Z. S., Phillips, N. A., Bedirian, V., Charbonneau, S., Whitehead, V., Collin, I., Cummings, J. L., & Chertkow, H. (2005). The Montreal cognitive assessment, MoCA: A brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 53, 695–699.

    PubMed  Google Scholar 

  37. Nelson, N., Sweet, J., & Heilbronner, R. (2007). Examination of the new MMPI-2 response Bias scale (Gervais): Relationship with the MMPI-2 validity scales. Journal of Clinical and Experimental Neuropsychology, 29, 67–72.

    PubMed  Google Scholar 

  38. Nguyen, C. T., Green, D., & Barr, W. B. (2015). Evaluation of the MMPI-2-RF for detecting over-reported symptoms in a civil forensic and disability setting. The Clinical Neuropsychologist, 29(2), 255–271.

    PubMed  Google Scholar 

  39. Pearson (2009). Advanced clinical solutions for the WAIS-IV and WMS-IV – Technical manual. San Antonio, TX: Author.

  40. Rabin, L., Spadaccini, A., Brodale, D., Charcape, M., & Barr, B. (2014). Utilization rates of computerized tests and test batteries among clinical neuropsychologists in the U.S. and Canada. Professional Psychology: Research and Practice, 45, 368–377.

    Google Scholar 

  41. Radloff, L. S. (1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385–401. https://doi.org/10.1177/014662167700100306.

    Article  Google Scholar 

  42. Rai, J., An, K. Y., Charles, J., Ali, S., & Erdodi, L. A. (2019). Introducing a forced choice recognition trial to the Rey complex figure test. Psychology & Neuroscience, 12(4), 451–472. https://doi.org/10.1037/pne0000175.

    Article  Google Scholar 

  43. Rai, J. K., & Erdodi, L. A. (2019). Impact of criterion measures on the classification accuracy of TOMM-1. In Impact of criterion measures on the classification accuracy of TOMM-1. Applied Neuropsychology: Adult, [Epub ahead of print]. https://doi.org/10.1080/23279095.2019.1613994.

    Google Scholar 

  44. Rey, A. (1941). L’examen psychologique dans les cas d’ence’phalopathie traumatique. Archives de Psychologie, 28, 286–340.

    Google Scholar 

  45. Rogers, R., & Bender, D. (2018). Clinical assessment of malingering and deception. New York, NY: Guilford Press.

    Google Scholar 

  46. Rogers, R., Sewell, K. W., Martin, M. A., & Vitacco, M. J. (2003). Detection of feigned mental disorders: A meta-analysis of the MMPI-2 and malingering. Assessment, 10(2), 160–177.

    PubMed  Google Scholar 

  47. Roma, P., Giromini, L., Burla, F., Ferracuti, S., Viglione, D. J., & Mazza, C. (2020). Ecological validity of the inventory of Problems-29 (IOP-29): An Italian study of court-ordered, psychological injury evaluations using the structured inventory of malingered symptomatology (SIMS) as criterion variable. Psychological Injury and Law, 13, 57–65.

    Google Scholar 

  48. Schroeder, R. W., Martin, P. K., Heindrichs, R. J., & Baade, L. E. (2019). Research methods in performance validity testing studies: Criterion grouping approach impacts study outcomes. The Clinical Neuropsychologist, 33(3), 466–477.

    PubMed  Google Scholar 

  49. Slick, D. J., Sherman, E. M., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. The Clinical Neuropsychologist, 13(4), 545–561.

    PubMed  Google Scholar 

  50. Smith, G. P., & Burger, G. K. (1997). Detection of malingering: Validation of the structured inventory of malingered symptomatology (SIMS). Journal of the American Academy on Psychiatry and Law, 25, 180–183.

    Google Scholar 

  51. Tarachow, S. (1947). The syndrome of inhibition. Psychiatric Quarterly, 21(2), 233–252.

    Google Scholar 

  52. Tombaugh, T. N. (1996). Test of memory malingering (TOMM). New York, USA: Multi Health Systems.

    Google Scholar 

  53. van Dyke, S. A., Millis, S. R., Axelrod, B. N., & Hanks, R. A. (2013). Assessing effort: Differentiating performance and symptom validity. The Clinical Neuropsychologist, 27(8), 1234–1246.

    PubMed  Google Scholar 

  54. Viglione, D. J., & Giromini, L. (2020). Inventory of problems–29: Professional manual. Columbus, OH: IOP-Test, LLC.

    Google Scholar 

  55. Viglione, D. J., Giromini, L., & Landis, P. (2017). The development of the inventory of problems–29: A brief self-administered measure for discriminating bona fide from feigned psychiatric and cognitive complaints. Journal of Personality Assessment, 99(5), 534–544.

    PubMed  Google Scholar 

  56. Viglione, D. J., Wright, D., Dizon, N. T., Moynihan, J. E., DuPuis, S., & Pizitz, T. D. (2001). Evading detection on the MMPI–2: Does caution produce more realistic patterns of responding? Assessment, 8, 237–250.

    PubMed  Google Scholar 

  57. Wechsler, D. (1997). Wechsler Adult Intelligence Scale (3rd ed.). San Antonio: The Psychological Corporation.

    Google Scholar 

  58. Weiss, D. S. (2004). The impact of event scale-revised. In J. P. Wilson & T. M. Keane (Eds.), Assessing psychological trauma and PTSD: A practitioner’s handbook (2nd ed., pp. 168–189). New York: Guilford Press.

    Google Scholar 

  59. Weiss, D. S., & Marmar, C. R. (1997). The impact of event scale-revised. In J. P. Wilson & T. M. Keane (Eds.), Assessing psychological trauma and PTSD: A Practitioner’s handbook (pp. 399–411). New York: Guilford Press.

    Google Scholar 

  60. Westmacott, R., & Moscovitch, M. (2003). The contribution of autobiographical significance to semantic memory. Memory & Cognition, 31(5), 761–774.

    Google Scholar 

  61. Whiteside, D. M., Hunt, I., Choate, A., Caraher, K., & Basso, M. R. (2020). Stratified performance on the test of memory malingering (TOMM) is associated with differential responding on the personality assessment inventory (PAI). Journal of Clinical and Experimental Neuropsychology, 42(2), 131–141. https://doi.org/10.1080/13803395.2019.1695749.

    Article  PubMed  Google Scholar 

  62. Widows, M. R., & Smith, G. P. (2005). SIMS-structured inventory of malingered symptomatology. Professional manual. Lutz, FL: Psychological Assessment Resources.

    Google Scholar 

  63. Wilcox, R. R. (2016). Comparing dependent robust correlations. British Journal of Mathematical and Statistical Psychology, 69, 215–224.

    PubMed  Google Scholar 

  64. Young, G. (2019). The cry for help in psychological injury and law: Concepts and review. Psychological Injury and Law, 12, 225–237.

    Google Scholar 

Download references

Acknowledgments

We thank Cecilia Gherghi, Lucia Lorenzetti and Mariachiara Didonna for their help in the data collection.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Luciano Giromini.

Ethics declarations

Conflict of Interest

Luciano Giromini and Donald J. Viglione declare that they own a share in the corporate (LLC) that possesses the rights to Inventory of Problems instruments (both IOP-29 and IOP-M). The other authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Giromini, L., Viglione, D.J., Zennaro, A. et al. SVT Meets PVT: Development and Initial Validation of the Inventory of Problems – Memory (IOP-M). Psychol. Inj. and Law (2020). https://doi.org/10.1007/s12207-020-09385-8

Download citation

Keywords

  • Inventory of problems
  • IOP-29
  • Malingering
  • Memory
  • IOP-M