Individualised or Standardised Outcome Measures: A Co-habitation?
Mental health outcome measurement is conflicted between two different schools of thought which underlie the division between standardised (nomothetic) and individualised or patient-generated (idiographic) measures. The underpinning philosophies of both approaches have very different starting points in terms of how we understand the world. And yet the strengths of both may contribute something useful for patients and mental health services. We suggest a convergence of approaches with new thinking on options for co-habitation.
KeywordsOutcome measurement Patient-generated measure Standardized measure Epistomology
There is a divide at the heart of mental health outcome measurement. The divide is driven by epistemology, the branch of philosophy concerned with the theory of knowledge, how we know things, the nature of knowledge and its limitations.
The German philosopher Immanuel Kant, in his Critique of Pure Reason (1781), described two distinct forms of knowledge. Firstly, there was the knowledge of the natural sciences, seeking underlying laws of nature through generalisations derived from objective data. Secondly, there was the knowledge of the humanities, in search of an understanding of the unique, the specific and the individual. The distance between the two was not merely a simple divide between science and art. At the end of the nineteenth century, the neo-Kantian philosopher Wilhelm Windelband used the terms ‘nomothetic’ and ‘idiographic’ to propose a distinction between two types of knowledge, the one being about ‘the general’, the other about ‘the particular’, respectively. He maintained that in spite of the dominance of nomothetic knowledge in science, science was not exclusively nomothetic and that some science depended on idiographic knowledge. Similarly, idiographic knowledge also required access to nomothetic methods. His argument was for philosophical and scientific paradigms to move towards a more holistic understanding of human and social phenomena.
The American psychologist Allport is generally credited with introducing the terms, ‘nomothetic’ and ‘idiographic’, into the discipline of psychology. The ideas were developed in his theories of personality: ‘Personality: A psychological interpretation’, first published in 1937, updated in 1961 (Allport 1937, 1961). He emphasised individuality in the understanding of personality. And yet, within the context of complete uniqueness, individuals shared certain common features. In his words, ‘The psychology of personality is not exclusively nomothetic, nor exclusively idiographic. It seeks an equilibrium between the two extremes.’ (Allport 1961; Lundh 2015).
Two further schools of thought underpin our understanding of the terms, ‘nomothetic’ and ‘idiographic’. The nomothetic approach is based on the epistemology of ‘logical positivism’. Wittgenstein is often cited as the father of logical positivism. After developing his philosophical output, he worked in 1941–1942 as a lineament mixer in the pharmacy at Guy’s Hospital (where two of us are based, MK, MA). He asserted that only those things which are verifiable have meaning, or at least cognitively meaningful. Measurement was king (although he probably would not have expressed it in these terms). In contrast, idiographic understanding is based on constructivist or postmodern epistemology which is about ‘meaning-making’, or ‘sense-making’, the way in which the mind constructs meaning from the observed reality of the world. The postmodern world view values idealism, relativism and constructivism. Postmodern thought is often described as ‘anti-theoretical’ and in the context of psychology is seen as shifting the focus away from population-based psychological theory, toward the perception of each patient as a unique entity.
Outcome Measurement: The Divide
There appears to be a yawning chasm between nomothetic and idiographic approaches to outcome measurement. Both are underpinned by opposing philosophical standpoints and the divide appears unbridgeable. They are represented by objective and subjective, rationalism and empiricism, theory and anti-theory, quantitative and qualitative.
Traditional nomothetic outcome measures consist of validated and reliable items with established population norms. In contrast, idiographic instruments are individualised, their central point of reference being the individual patient, not the general population. Nomothetic measures are ideally suited to establish baseline ‘severity’ and define diagnostic cut-points. Idiographic measures tell a story; they generate qualitative data. They also generate quantitative data but in doing so, encounter problems about the meaning and validity of data derived from an individualised instrument. Baseline idiographic data have personal validity but cannot be referenced against population norms. However, quantitative data do provide one key strength of the idiographic instrument—it is strongly responsive to change after a therapeutic intervention (Lacasse et al. 1999). Thus idiographic change, by measuring items of personal significance to the patient, is likely to exceed nomothetic change, which generates change scores for items of importance to populations but not necessarily of importance to the individual and the uniqueness of their distress.
Nomothetic instruments share theoretical assumptions in clearer ways than idiographic measures, most evident in construct validity testing as proposed by Cronbach and Meehl (1955). In this classical view of psychological testing, measures are valid when correlation values confirm linkages between theoretically linked constructs (‘nomological networks’). Conversely, there is a lack of theory to guide construct validity testing in idiographic instruments which may hamper their wider acceptance.
Outcome Measurement: A Convergence
Theory emphasises the differences between outcome measures. In practice, differences are less stark. Baseline (severity) scores of both types of instrument, when administered to the same population, correlate moderately or strongly (Ashworth et al. 2005). So too do change scores (ibid), with the caveat, as noted, that changes scores are, in general, greater with idiographic instruments. Most idiographic instruments seek to compare themselves with ‘gold-standard’ nomothetic instruments during the course of validity and reliability testing. Some might consider that to be an unfair comparison, because nomothetic metrics are rarely established by testing in the other direction, against idiographic metrics. On the other hand, like-for-like comparison may disadvantage nomothetic instruments, since items of personal significance to the patient may not feature on the nomothetic instrument. Some might argue that instead, patient generated idiographic instruments should be validated against real-life sense checking, a concept termed ‘ecological validity’.
Several recent studies have reported on distress captured by idiographic instruments but ‘missed’ by commonly used nomothetic instruments. In one study within an addiction and general psychiatric setting, over two-thirds of themes identified on the idiographic instrument PSYCHLOPS (‘Psychological Outcome Profiles’) did not feature in nomothetic comparators (Sales et al. 2017). The authors concluded that conventional measures might both capture and miss the patient story (ibid). Common themes not captured by the nomothetic approach include relationship, employment and sexual problems (ibid).
Given the strengths of both types of instruments, is there an argument that they may be complementary? Certainly, both patients and therapists display a preference for the idiographic (Jensen-Doss et al. 2018; Ashworth et al. 2009; Black 2013; Godfrey et al. 2019).
Recent developments suggest convergence. CORE-OM (‘Clinical Outcomes Routine Evaluation—Outcome Measure), a 34-item nomothetic measure, has developed an individualised approach to items, termed ‘TRIMS’ (Tracking Responses to Items in Measures) (Cross et al. 2014). Standardised items are selected enabling patient and therapist to focus on individual item-level change rather than on total mean instrument-level change. This approach effectively individualises the analysis of baseline and change scores. Adapting and testing other nomothetic measures to an individualised approach opens up a whole new potential area of psychometric research.
Similarly, the move toward ‘Feedback Informed Therapy’ (FIT) encourages discussion between therapist and patient about the meaning of nomothetic-derived changed scores, taking nomothetic feedback into idiographic (individualised) territory (Reese et al. 2009). Again, this is fertile ground for research as FIT has not been tested for idiographic instruments.
Some difficulties have hampered convergence. Although patient-generated and patient-focussed, idiographic instruments come with certain limitations: they have been slow to make the transition to electronic modes, many cannot be self-completed requiring therapist-completion and many are more time consuming to complete than nomothetic counterparts. In consequence, nomothetic instruments, their potential insights notwithstanding, have tended to be used as add-ons to nomothetic measures.
Outcome Measurement: A Cohabitation
Combinations of nomothetic and idiographic instruments appear to be a natural progression based on the strengths of both approaches, with known omissions of each complemented by inclusion of the other. The ‘Individualised Patient-Progress System’ (IPPS) was developed in Portugal and incorporated CORE-OM with two idiographic measures: Patient Questionnaire (PQ) and Helpful Aspects of Therapy (HAT) (Sales et al. 2014). It was embedded in the philosophy of practice-based research and was the first patient progress measure to combine both types of instrument and then administer the combination through an electronic format to facilitate continuous progress measurement. CORE-OM and PSYCHLOPS share the same domains (problem-function-wellbeing), representing both perspectives applied to each domain; a collaborative was recently launched to develop a single electronic platform for these measures and explore the feasibility of cohabitation (http://www.psychlops.org.uk). Few other combinations have been rigorously tested and this represents a research opportunity. However, the direction of travel is clear. The move is toward a meeting of the epistemologies and the enhancement of patient involvement in outcomes (Alves et al. 2013).
Logical positivism and constructivism have operated both as driving forces and dividing forces for the development of mental health outcome measures. In spite of the empirical complementarity of nomothetic and idiographic instruments, utilisation in practice and the volume of research outputs still favour the nomothetic approach to measurement. If we are to shift the locus of measurement away from professionally determined measures toward a more patient-centred approach, capturing outcomes of importance to patients, the equilibrium between both instrument types needs to tilt further toward idiographic measures. Perhaps our quest is not so modern after all. Maybe, the assertion of Alport (1961) could be adapted to our future development of outcome measurement itself, such that it, ‘…….is not exclusively nomothetic, nor exclusively idiographic…(and)… seek an equilibrium between the two extremes’.
This Point of View article was not funded. The authors have not received any funding for their research on mental health outcome measures.
Compliance with Ethical Standards
Conflict of interest
Daniel Guerra declares that he has no conflict of interest. Mark Ashworth and Maria Kordowicz declare that they were part of the research group that developed the mental health outcome measure, PSYCHLOPS. They continue to run the official PSYCHLOPS website. They were in receipt of PSYCHLOPS user-fees until 2012 which part-funded PSYCHLOPS validation studies. Since 2012, PSYCHLOPS has been freely available and the authors have not received any fees for the use of PSYCHLOPS. No authors have received any other source of funding for research work related to PSYCHLOPS.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
- Allport, G. W. (1937). Personality: A psychological interpretation. New York: Holt, Rinehart & Winston.Google Scholar
- Allport, G. W. (1961). Pattern and growth in personality. New York: Holt, Rinehart & Winston.Google Scholar
- Ashworth, M., Evans, C., & Clement, S. (2009). Measuring psychological outcomes after cognitive behaviour therapy in primary care: A comparison between a new patient-generated measure “PSYCHLOPS” (Psychological Outcome Profiles) and “HADS” (Hospital Anxiety and Depression Scale). Journal of Mental Health, 18(2), 169–177.CrossRefGoogle Scholar
- Ashworth, M., Robinson, S., Godfrey, E., Shepherd, M., Evans, C., Seed, P., Parmentier, H., & Tylee, A. (2005). Measuring mental health outcomes in primary care: The psychometric properties of a new patient-generated outcome measure, PSYCHLOPS (Psychological Outcome Profiles). Primary Care Mental Health, 3(4), 261–270.Google Scholar
- Cross, S., Mellor-Clark, J., & Macdonald, J. (2014). Tracking responses to items in measures as a means of increasing therapeutic engagement in clients: A complementary clinical approach to tracking outcomes. Clinical Psychology & Psychotherapy, 22(6), 698–707. https://doi.org/10.1002/cpp.1929.CrossRefGoogle Scholar
- Jensen-Doss, A., Smith, A. M., Becker-Haimes, E. M., Ringle, V. M., Walsh, L. M., Nanda, M., Walsh, S. L., Maxwell, C. A., & Lyon, A. R. (2018). Individualized progress measures are more acceptable to clinicians than standardized measures: Results of a national survey. Administration and Policy in Mental Health and Mental Health Services Research, 45(3), 392–403. https://doi.org/10.1007/s10488-017-0833-y.CrossRefGoogle Scholar
- Lacasse, Y., Wong, E., & Guyatt, G. (1999). Individualising questionnaires. In C. R. B. Joyce, C. A. O’Boyle & H. McGee (Eds.), Individual quality of life. Approaches to conceptualisation and assessment (pp. 87–103). Amsterdam: Harwood Academic Publishers.Google Scholar
- Sales, C. M. D., Alves, P., Evans, C., & Elliott, R., On behalf of IPHA Group (2014). The individualised patient-progress system (IPPS): A decade of international collaborative networking. Counselling and Psychotherapy Research, 14(3), 181-191. https://doi.org/10.1080/14733145.2014.929417.CrossRefGoogle Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.