Advertisement

Journal of Well-Being Assessment

, Volume 2, Issue 1, pp 57–73 | Cite as

Psychometric Properties of the Mental Health Continuum – Short Form in a Psychiatric Sample

  • Alexandra L. Silverman
  • Marie Forgeard
  • Courtney Beard
  • Thröstur Björgvinsson
Original Research
  • 99 Downloads

Abstract

The Mental Health Continuum – Short Form (MHC-SF) is a well-established measure that assesses general well-being and three well-being components: emotional, social, and psychological. However, its psychometric properties have never been investigated in a psychiatric sample. We examined the psychometric properties of the MHC-SF, including factor structure, convergent validity, and sensitivity to change, in 768 patients attending a psychiatric partial hospital program. Patients completed the MHC-SF as well as self-report measures of depression, and motivation and pleasure at admission and discharge from the program. Results revealed that a Bifactor Exploratory Structural Equation Modeling (ESEM) model better fit the data than competing models (Confirmatory Factor Analysis, ESEM, and Bifactor models). This model supported the existence of a general well-being factor, but provided limited evidence for the existence of emotional, social, or psychological well-being factors. The MHC-SF negatively correlated with a measure of depression and positively correlated with a measure of motivation and pleasure, suggesting good convergent validity. General well-being increased significantly from pre- to post-treatment. Results support the use of the MHC-SF to reliably measure general well-being in a psychiatric sample.

Keywords

Well-being Mental health Psychopathology Bifactor ESEM 

1 Introduction

The scientific study of well-being has rapidly expanded in recent decades, as researchers have extended the definition of this construct beyond the absence of mental illness to also include the presence of positive psychological experiences (Seligman and Csikszentmihalyi 2000). Although lay theories may assume that mental health and mental illness are opposites, well-being researchers have instead proposed that they are two correlated but distinct constructs (Keyes 2005). For example, many individuals who do not meet criteria for a psychological disorder may still not function effectively or judge their life to be fulfilling (e.g., Wang et al. 2005). Conversely, individuals with diagnosable mental illness may report experiencing high levels of well-being in some or most areas of life, even as they face specific psychological difficulties. Understanding positive psychological experiences in individuals experiencing psychological disorders is important in several regards. First, documenting the presence and nature of well-being in clinical samples can help provide a more accurate and comprehensive picture of psychological functioning in individuals with mental illness. Second, assessing positive psychological experiences can provide important information for treatment, by not only tracking problems, but also by tracking what is going well, and what strengths can be of use during the recovery process (Duckworth et al. 2005; Rapp 1998; Wood and Tarrier 2010).

One widely used brief measure of well-being is the 14-item Mental Health Continuum – Short Form (MHC-SF; Keyes et al. 2008). The MHC-SF is grounded in Keyes’ (2007) multidimensional approach to well-being that emphasizes the distinction between mental health and mental illness, and further proposes three different dimensions of mental health: emotional, social, and psychological well-being. Derived from the longer (39 item) version of the questionnaire (MHC-LF; Keyes 2002), the MHC-SF assesses both hedonic (affective, e.g., feeling good) and eudaimonic (meaningful, e.g., doing good; Deci and Ryan 2008; Ryff and Singer 2006) components of well-being. Emotional well-being is measured using three items that assess an individuals’ happiness, interest in life, and satisfaction with life (Keyes et al. 2008). Social well-being is measured using five items that assess the individual’s perceptions about their ability to contribute to society, their sense of belonging to a community, their beliefs about the goodness of society itself, their beliefs about the goodness of the people within society, and their impressions of whether the way society works makes sense (Keyes et al. 2008). Finally, psychological well-being is measured using six items that assess an individual’s perceptions of their self in relation to self-acceptance, positive relations with others, autonomy, environmental mastery, purpose in life, and personal growth (Ryff 1989).

The MHC-SF has demonstrated adequate convergent validity with other measures of emotional, social, psychological, and general well-being in general population samples (e.g., Hides et al. 2016; Keyes et al. 2008; Lamers et al. 2011; Petrillo et al. 2015). Furthermore, compared to other measures of well-being, the MHC-SF shows a weaker correlation with measures of mental illness (Hides et al. 2016; Keyes et al. 2008; Lamers et al. 2011; Petrillo et al. 2015), demonstrating that mental health and mental illness are two related, but distinct constructs. The MHC-SF however only seems to have modest test-retest reliability, with higher reliability coefficients for the total score than the three subscales scores (e.g., Lamers et al. 2011, 2012; Petrillo et al. 2015). The MHC-SF has been successfully translated into numerous languages and has demonstrated strong psychometric properties across cultural contexts in nonclinical samples in the Netherlands (Westerhof and Keyes 2010), Italy (Petrillo et al. 2015), South Africa (Keyes et al. 2008), and China (Guo et al. 2015).

Three main analytic approaches have been used to investigate the factor structure of the MHC-SF in past studies: Confirmatory Factor Analysis (CFA), Exploratory Structural Equation Modeling (ESEM), and Bifactor Analysis. First, studies using CFA have for the most part supported the three-factor structure consisting of emotional well-being, social well-being, and psychological well-being consistent with Keyes’ (2007) model. These results have been found in general population samples of adults (Joshanloo et al. 2013; Karaś et al. 2014; Keyes et al. 2008; Lamers et al. 2011; Petrillo et al. 2015), as well as in adolescents and children (De Carvalho et al. 2016; Guo et al. 2015). In these studies, a correlated three-factor model yielded a better fit than competing one-factor (general well-being) and two-factor (emotional well-being and combined social well-being/psychological well-being) models.

Second, other researchers have recently suggested that more advanced methods are needed to accurately characterize the dimensions of well-being captured by the MHC-SF. Indicators forming a latent variable are by nature imperfect and include not only random measurement error, but also systematic error reflecting associations with other constructs, especially when constructs overlap (Morin et al. 2016). In the case of the MHC-SF, items hypothesized to load on the social well-being factor (i.e., the “target” factor) may also to some degree be systematically associated with the emotional well-being and psychological well-being factors (i.e., the “nontarget” factors). In other words, cross-loadings on nontarget factors may not be equal to zero. Traditional CFA uses restrictive assumptions by requiring that indicators (items) only load on their constructs of interest, leading to worse fit and overestimation of factor correlations (Marsh et al. 2014). A statistical approach that can be used to address this issue is ESEM, a procedure that allows nontarget factor loadings not to be constrained to zero, but specifies that these should be as close as possible to zero (Asparouhov and Muthén 2009). ESEM has been shown to produce better fit (supporting the existence of three distinct emotional, social, and psychological well-being factors) and more accurate factor correlations compared to CFA when applied to data from the MHC-SF (Joshanloo 2016a; Joshanloo and Jovanović 2016; Joshanloo and Lamers 2016; Joshanloo et al. 2016).

A third set of concerns have inspired researchers to adopt methods going beyond traditional CFA. Multidimensional constructs such as well-being often capture both global and specific constructs, and independent cluster models do not allow researchers to model their coexistence (Morin et al. 2016). In contrast, Bifactor models allow researchers to simultaneously specify both one “general factor” (well-being) and separate orthogonal “group factors” calculated from residuals (i.e., variance not explained by the general factor) for different content domains (emotional well-being, social well-being, and psychological well-being; Reise et al. 2013b). A Bifactor structure has been investigated for a wide range of constructs that can similarly be characterized by both general and specific factors, such as intelligence (Gignac and Watkins 2013), personality (e.g., Chen et al. 2012), or psychopathology (e.g., Lahey et al. 2012). Examining whether a Bifactor structure is appropriate for the MHC-SF is warranted given the widespread use of total scores on this measure (suggesting that despite the 3-factor model, users of the measure still conceptualize scores as forming one main latent variable; De Bruin and Du Plessis 2015). Empirical studies have confirmed that Bifactor models fit data from the MHC-SF better than traditional CFA models (De Bruin and Du Plessis 2015; Hides et al. 2016; Jovanović 2015), supporting the notion that this measure captures a global well-being construct. Interestingly, results also suggested that after controlling for the general well-being factor, items on the social and psychological well-being subscales did not have robust loadings on their specific factors, calling into question the use of subscale scores to measure distinct aspects of well-being (De Bruin and Du Plessis 2015; Hides et al. 2016; Jovanović 2015).

Previous research therefore suggests that ESEM and Bifactor modeling both independently provide more informative approaches to examining the factor structure of the MHC-SF than traditional CFA. A combined Bifactor ESEM approach provides an optimal approach to model responses on the MHC-SF by concurrently allowing items to load on all factors (with nontarget loadings as close to zero as possible) and to load on both general and specific factors (Morin et al. 2016). This approach has already been found to be superior to other statistical techniques (e.g., CFA, ESEM, Bifactor models) to study the multidimensional nature of a variety of constructs including self-concept, motivation, and interpersonal styles, among others (Litalien et al. 2017; Morin et al. 2016; Sánchez-Oliva et al. 2017; Stenling et al. 2015). Bifactor ESEM is therefore a versatile and useful statistical approach to examine potential multidimensional constructs. To our knowledge, two studies have previously tested the factor structure of MHC-SF using Bifactor ESEM (Longo et al. 2017; Schutte and Wissing 2017). Schutte and Wissing (2017) examined the factor structure of the MHC-SF using the English, Afrikaan, and Setswana versions of the scale in three separate nonclinical samples using Bifactor ESEM. Longo et al. (2017) tested the factor structure of the MHC-SF using CFA, Bifactor CFA, ESEM, and Bifactor ESEM in a combined sample of participants from the Netherlands, Poland, Portugal and Serbia. Results from both studies revealed that the Bifactor ESEM model had a better fit with the general well-being scale than competing models (Longo et al. 2017; Schutte and Wissing 2017) indicating the usefulness of the Bifactor ESEM approach for examining the factor structure of the MHC-SF.

1.1 The Present Study

The present study examined the psychometric properties of the MHC-SF in a psychiatric sample. It is important to assess well-being, in addition to symptom severity, in psychiatric populations in order to obtain a more comprehensive understanding of psychological functioning in individuals with mental illness. More specifically, it is critical to assess the degree to which well-being is (or is not) impacted by psychiatric symptoms. Such information may be able to inform intervention strategies to address both psychopathology and well-being.

The MHC-SF may constitute a useful measure of well-being for use in psychiatric settings because of its ability to provide a well-rounded picture of well-being while remaining brief (Hone et al. 2014). Although the MHC-SF has demonstrated strong psychometric properties in undergraduate university and epidemiological samples (e.g., De Bruin and Du Plessis 2015; Hides et al. 2016; Lamers et al. 2011), its psychometric properties have never been examined in a psychiatric sample. Patients treated in mental health settings have unique characteristics, such as higher symptom severity and diagnostic comorbidity, and lower overall functioning compared to general population samples. Therefore, it is crucial to examine the MHC-SF’s psychometric properties specifically in a psychiatric sample in order to determine its utility as a measure of well-being in clinical settings.

Our study tested a number of specific hypotheses. First, we expected that the Bifactor ESEM model (one general well-being factor, three group factors consisting of emotional well-being, social well-being, and psychological well-being) would result in a better fit for the data and more unbiased estimates than traditional CFA, ESEM, or Bifactor models. Unlike these approaches, Bifactor ESEM can simultaneously consider and handle two sources of psychometric multidimensionality - hierarchical and conceptual overlap between groups of items (Howard et al. 2016). If this overlap is not modeled in analyses (e.g., traditional CFA does not model the presence of general vs. specific factors, and does not allow items to load on nontarget factors), these unmodeled relationships will express themselves through other model parameters that are estimated. This results in biased estimates, by for example inflating factor correlations in CFA, inflating factor correlations and/or cross-loadings in ESEM, or inflating loadings on the general factor in Bifactor modeling (Howard et al. 2016; Morin et al. 2016; Sánchez-Oliva et al. 2017). Our hypothesis is supported by results of previous work (cited above) that similarly found Bifactor ESEM to provide the best modeling approach for potentially multidimensional constructs, including well-being (Longo et al. 2017; Schutte and Wissing 2017). We also expected the general well-being factor to be reliable; in contrast, we did not expect this to be true for subscale factors given that previous studies using Bifactor modeling have found these to have insufficient reliability once a general factor is modeled (De Bruin and Du Plessis 2015; Hides et al. 2016; Jovanović 2015; Schutte and Wissing 2017).

Second, we expected the MHC-SF to demonstrate adequate convergent validity via a positive correlation with a related measure of motivation and pleasure and a negative correlation with a measure of depression severity. Third, given the demand for brief assessments for use in clinical settings, we tested the adequacy of the MHC-SF as a transdiagnostic treatment outcome measure by examining its sensitivity to changes over the course of treatment. We expected that patients’ well-being would increase significantly from pre- to post-treatment, providing preliminary evidence for sensitivity to change.

2 Method

2.1 Participants and Treatment Setting

Participants included 768 patients receiving treatment at a partial hospital program in a psychiatric hospital in New England from December of 2015 to March of 2017 who provided informed consent for their clinical data to be used for research purposes. The partial hospital program treats adults (18 years and older) with a broad range of psychiatric disorders (primarily mood, anxiety, personality, and psychotic disorders). Approximately half of patients are referred from the community by outpatient treatment providers and half transition from higher levels of care such as inpatient hospitalizations to the partial hospital program. Participants were predominantly in young to middle adulthood (M = 34.54, SD = 14.02), approximately half female (52.5%, male = 45.3%, non-binary = 2.2%), White (86%), and approximately half were unemployed (see Table 1). The most common primary diagnosis (assigned by a program psychiatrist) was Major Depressive Disorder, followed by Bipolar Disorder (I or II), anxiety disorders, and psychotic disorders.
Table 1

Demographic and clinical characteristics (N = 768)

Demographic characteristics

N

%

Gender

 Female

403

52.5

 Male

348

45.3

 Non-Binary

17

2.2

Race

 White

659

85.8

 Asian

44

5.7

 Black

11

1.4

 Multiracial

28

3.7

 Did not specify or Other

26

3.4

Ethnicity

 Non-Latinx

730

95.1

 Latinx

38

4.9

Education

 Some high school or less

8

1,0

 High school/GED

64

8.3

 Some college

280

36.5

 4-year college graduate

409

53.2

 Did not specify

7

0.9

Employment

 Unemployed

382

49.7

 Employed

379

49.4

 Missing

7

0.9

Marital Status

 Never Married

458

59.6

 Separated, divorced, or widowed

94

12.3

 Married or living with partner

213

27.8

 Missing

3

0.4

Psychiatric hospitalization in the past 6 months?

 Yes

401

52.2

 No

367

47.8

Age (M, SD)

35.54

14.02

GED General Equivalency Diploma

The program focuses on acquisition and practice of skills drawn from Cognitive-Behavioral Therapy (Beck et al. 1979) and Dialectical Behavior Therapy (Linehan 1993). Treatment consists primarily of group and individual therapy provided by psychologists, social workers, nurses, postdoctoral and graduate level psychology trainees, and mental health counselors. Patients attend up to five 50-min groups each day, five days per week. Patients also meet with a psychiatrist for pharmacotherapy and a clinical team manager for case management (for a more complete description of the partial hospital program see Björgvinsson et al. 2014). The average duration of treatment in the sample for this study was 12.02 (SD = 4.74) days (including non-treatment days on weekends and holidays).

As part of standard clinical care at the partial hospital program, patients completed self-reported measures to track progress. We obtained a de-identified dataset from the partial hospital program to evaluate the MHC-SF in this population. Thus, measures of convergent validity were selected from what was available in this database during the time period that MHC-SF data was collected. Sample sizes for some analyses varied based upon when various measures were introduced to the clinical battery. All measures had adequate internal reliability (Cronbach’s α provided for the present sample).

2.2 Measures

The Mental Health Continuum – Short Form (MHC-SF; Keyes et al. 2008)

The MHC-SF is a 14-item self-report measure assessing positive psychological experiences pertaining to three dimensions of well-being: emotional (e.g., “happy”), social (e.g., “that you have something important to contribute to society”), and psychological (e.g., “that your life has a sense of direction or meaning to it”). In the current study, it was necessary to slightly modify the MHC-SF for use in a clinical setting. Similar to other outcomes measures used in acute clinical settings (e.g., Beard and Björgvinsson 2014), we used a shorter time frame (past week, instead of past month) in order to assess the measure’s sensitivity to change. We also used a 5-point Likert scale to approximate frequency of experiences (1: never to 5: always), instead of the typical 6-point scale describing number of instances such as “once or twice,” or “about 2 or 3 times a week”. This was done to accommodate the modified timeframe and keep answer options consistent with other measures to facilitate participants’ experience answering multiple self-report questionnaires in a row. MHC-SF total scores are calculated by adding the total number of points for all 14 items with higher scores signifying higher levels of well-being. A validity check item was embedded within the measure to verify that participants were paying attention to the questions (“Please select “often” for this item”). We excluded 44 participants from analysis due to incorrect responses to this item. In this sample 768 participants completed the MHC-SF at admission (M = 39.67, SD = 10.93), and 333 participants completed it at discharge (M = 45.45, SD = 9.65).

Patient Health Questionnaire-9 (PHQ-9; Kroenke and Spitzer 2002)

The PHQ-9 is a nine item self-report measure used to assess severity of depressive symptoms (e.g., “little interest or pleasure in doing things”) over the past two weeks that was administered at admission and discharge (Cronbach’s α = .86) using a 4-point Likert-type scale (0 to 3). A total PHQ-9 score is calculated by adding the points for all nine items, with higher scores indicating higher depression severity. In this sample, 545 participants completed the PHQ-9 at admission (M = 13.99, SD = 6.55), and 419 participants completed it at discharge (M = 9.47, SD = 5.69). The PHQ-9 has demonstrated strong psychometric properties, including good internal consistency and construct validity, in similar psychiatric populations (Beard et al. 2016). Similar to previous psychometric studies of the MHC-SF that used depression symptom severity measures for convergent validity analyses (e.g., Hides et al. 2016; Petrillo et al. 2015), we selected the PHQ-9 as a measure of convergent validity.

Motivation and Pleasure Scale-Self Report (MAP-SR; Llerena et al. 2013)

The MAP-SR is a 15-item self-report measure of past (previous week) and anticipated (next few weeks) motivation and pleasure related to social, recreational, and work domains that was administered at admission and discharge. The measure uses a 5-point Likert-type scale (0 to 4; Cronbach’s α = .89) to assess social pleasure (3 items) recreational/work pleasure (3 items), feelings/motivations about close caring relationships (3 items), and recreational/work pleasure (6 items). A total score is calculated by adding the scores from all 15 items, with higher scores indicating higher levels of motivation and pleasure. The MAP-SR was selected for convergent validity analyses because it measures a construct that closely relates to general well-being. The MAP-SR was added to the battery of measures administered to patients as part of routine clinical monitoring seven months after the MHC-SF; thus, convergent validity analyses using the MAP-SR include a subset of the total sample (n = 384 at admission, M = 30.58, SD = 11.58; n = 331 at discharge, M = 36.57, SD = 10.35).

Demographic items

We collected the following demographic information at admission to the program: gender, age, race, ethnicity, education, employment status, marital status, and psychiatric hospitalization history (see Table 1).

2.3 Procedure

Participants completed a computerized battery of self-report questionnaires as part of routine clinical monitoring upon admission to and discharge from the program. Measures were administered using REDCap (Research Electronic Data Capture), a secure, web-based application designed to support data collection for research studies (Harris et al. 2009). The admission assessment included the MHC-SF, PHQ-9, and MAP-SR. Additionally, we examined sensitivity to change during treatment in a subsample of participants who completed the MHC-SF again at discharge (n = 329).

Participants did not complete the MHC-SF at discharge for the following reasons: measure not in battery at time of discharge (n = 288), patient symptoms too acute to complete discharge assessment (n = 144), patient did not attend program after admission (n = 20), or patient prematurely ended treatment or was hospitalized before completing the discharge assessment (n = 124). We excluded five additional participants from analyses due to suspect answers on the MHC-SF (e.g., participant reported answering all items the same way to save time) as noted by a trained research coordinator.

2.4 Data Analytic Strategy

We assessed the factor structure of the MHC-SF in this clinical sample by comparing four structural equation models examined in Mplus 8 (Muthén and Muthén 2017; see Fig. 1). We conducted these analyses using responses provided by participants at admission to the program (n = 768). Analyses implemented full-information maximum likelihood (FIML) estimation with robust standard errors, which adjusted results for missing data (0.28% of values). Model 1 tested an independent cluster model using CFA, with three specific correlated latent variables (emotional well-being, social well-being, psychological well-being). Model 2 tested this factor structure again using ESEM (with target oblique rotation) to assess whether allowing all items to also load on nontarget factors (with loadings close to zero) would improve model fit. Model 3 tested a Bifactor model in which all items loaded both on a general well-being factor, as well as their hypothesized group factor. Model 4 tested this Bifactor structure again using ESEM (with target orthogonal rotation) to assess whether allowing nontarget crossloadings would improve model fit. In both Models 3 and 4, factors were not allowed to correlate with each other following principles of Bifactor modeling. We followed existing recommendations to assess model fit using multiple indices given that sample size has an important effect on the significance of chi-square tests (Bentler and Bonett 1980; Kenny and McCoach 2003; Kline 2005). First, we examined chi-square values for each model and compared models using Satorra-Bentler scaled chi-square tests. Second, we examined absolute values of competing models using the Standardized Root Mean Squared Residual (summary of the average covariance residual: SRMR = or < .08 indicates acceptable fit; Hu and Bentler 1999), the Root Mean Square Error of Approximation (estimate of the “misfit” of the model based on a noncentral index: RMSEA < .05 indicates close fit, between .05 and .08 indicates reasonable fit, and > .10 indicates poor fit; Browne and Cudeck 1993), and the Comparative Fit Index (another noncentral index of fit: CFI = or > .90 indicates adequate fit; Hu and Bentler 1999). Third we examined relative change on the Akaike Information Criteria (BIC) and the sample-size adjusted Bayesian Information Criterion (BIC) across models, parsimony-adjusted indices which take into account model complexity (models with lower AIC/BIC values are preferred; (Dziak et al. 2017).
Fig. 1

Models tested. [E = Emotional Well-Being; S = Social Well-Being; P = Psychological Well-Being; G = General Well Being]

Based on the final model retained, we examined each factor using the following procedures. First, we examined both the significance (p < .05) and the size of standardized β loadings for items hypothesized to load on specific factors. We also assessed which items loaded on which factors, including whether any items loaded on multiple factors. Next, we computed two sets of omega reliability coefficients (McDonald 1970, 1999) to assess the reliability of the general and group factors using Watkins’ (2017) software omega. First, omega (ω) refers to variance in the total score attributable to both general and specific constructs. For subscales, this coefficient (ωs) refers to variance in a subscale score attributable to both general and specific constructs. Second, and in contrast, omega hierarchical (ωh) refers to variance in the total score attributable to only the general construct (removing specific variance explained by specific constructs). For subscales, this coefficient (ωhs) refers to variance in a subscale score attributable to only the specific construct (removing variance explained by the general construct; Cho 2016; Reise et al. 2013a; Rodriguez et al. 2016).

Next, we assessed the convergent validity of the MHC-SF by examining its correlation with a measure of depression severity (PHQ-9) and a measure of motivation and pleasure (MAP-SR). Due to the limited space in our assessment battery, we did not have specific measures of convergent validity for the MHC-SF’s hypothesized subscale scores. Finally, we examined the MHC-SF’s sensitivity to change by testing pre- to post-treatment changes using a repeated-measures ANOVA, and the magnitude of the corresponding effect size (partial η2 and Cohen’s d).

3 Results

3.1 Psychometric Properties

We compared four models using multiple fit indices to determine which factor structure and procedures would result in the best fit for the MHC-SF data in this clinical sample (see Table 2 for all fit statistics). Examination of indices suggested that the fit for Model 1 (CFA) was acceptable; the fit for Model 2 (ESEM) and Model 3 (Bifactor) was good; the fit for Model 4 (Bifactor ESEM) was excellent. All fit indices were optimized in Model 4 (lower chi-square values, RMSEA, SRMR, and AIC, as well as higher CFI value). Satorra-Bentler squared chi-squared tests confirmed that Model 4 provided a better fit for the data than all other models (all ps < .001). Thus, we retained this model for further examination.
Table 2

Fit Indices for All Models of the Mental Health Continuum – Short Form

Model

 

χ2

df

RMSEA

RMSEA 90% CI

CFI

SRMR

AIC

SABIC

Model 1

CFA

593.924***

74

.096***

.089, .103

.895

.059

26,747.953

26,814.028

Model 2

ESEM

197.606***

52

.060*

.052, .069

.971

.025

26,303.746

26,402.124

Model 3

Bifactor

195.297***

63

.052

.044, .061

.973

.028

26,276.084

26,358.311

Model 4

Bifactor ESEM

81.614***

41

.036

.024, .047

.992

.013

26,185.717

26,300.247

CFA Confirmatory Factor Analysis, ESEM Exploratory Structural Equation Modeling, RMSEA Root Mean Square Error of Approximation, CI Confidence Interval, CFI Comparative Fit Index, SRMR Standardized Root Mean Square Residual, AIC Akaike’s Information Criterion, SABIC Sample Size Adjusted Bayesian Information Criterion; * p < .05, *** p < .001

Standardized path estimates (see Table 3) indicated that all items loaded significantly on the general well-being factor (all ps < .001, βs = .44 to .86). The three emotional well-being items loaded on their hypothesized group factor (all ps < .001 and βs = .36 to .63). Only three of the five social well-being items loaded on their hypothesized group factor (for these three items, ps < .001 and βs = .58–.62). Finally, only two of the six psychological well-being items positively loaded on their hypothesized group factor (for these two items, ps < .05 and βs = .26 to .54). Three items did not load significantly on this factor, and one item loaded significantly but negatively. As explained above, the general and three specific factors were not allowed to correlate following principles of Bifactor modeling.
Table 3

Standardized factor loadings for the bifactor ESEM model of the MHC-SF as well as omega reliability coefficients for the general and group factors

 

General WB

Emotional WB

Social WB

Psychological WB

Item 1

.638 ***

.629 ***

−.061**

.059**

Item 2

.774 ***

.356 ***

−.024

.052

Item 3

.750 ***

.388 ***

−.041

−.080*

Item 4

.798 ***

−.043

−.059

−.148***

Item 5

.734 ***

−.011

.011

.125***

Item 6

.603 ***

−.004

.575 ***

−.114**

Item 7

.469 ***

−.037

.584 ***

.099**

Item 8

.441 ***

−.068*

.618 ***

.069

Item 9

.777 ***

.043

.038

.084

Item 10

.686 ***

.096*

.025

−.002

Item 11

.609 ***

.063*

.088**

.536 ***

Item 12

.663 ***

.004

−.005

.087

Item 13

.606 ***

−.062

−.019

.258 ***

Item 14

.858 ***

−.031

.003

−.098 *

Omega Classical (ω and ωs)

 All hypothesized items

.95

.90

.86

.88

 Items with significant loadings only

.95

.90

.82

.70

Omega Hierarchical (ωh and ωhs)

 All hypothesized items

.89

.26

.21

.04

 Items with significant loadings only

.89

.26

.48

.21

ESEM Exploratory Structural Equation Modeling, WB Well-being; * p < .05, **p < .01, ***p < .001; Bolded factor loadings - Items hypothesized to load on factors

Reliability coefficients for the total score indicated that 95% of its variance was explained by both general and specific constructs (ω= .95), and 89% was explained by general well-being alone (ωh = .89).1 Following the guidelines of Rodriguez et al. (2016), we compared ω with ωh and found that almost all (94%) of the reliable variance in total MHC-SF scores (.89/ .95 = .94) could be attributed to the general well-being factor. In contrast, only 6% (.95–.89 = .06) of the reliable variance could be attributed to the multidimensionality associated with group factors. Only 5% corresponded to random error. Thus, raw total scores derived from the MHC-SF can best be characterized as unidimensional reflections of general well-being.

Reliability coefficients for subscales indicated that 90% of variance in the emotional well-being subscale was explained by both general and emotional well-being (ωs-emotional = .90), 86% of variance in the social well-being subscale was explained by both general and social well-being (ωs-social = .86), and 88% of the variance in the psychological well-being subscale was explained by both general and psychological well-being (ωs-psychological = .88). Subscale reliability however decreased substantially once we partitioned out variance for the general factor, as indicated by low ωhs coefficients (ωhs-emotional = .26, ωhs-social = .21, ωhs-psychological = .04). Similar findings emerged when we re-calculated coefficients to only include the three items that significantly loaded on the social well-being subscale (ωs-social = .82, ωhs-social = .48) and the two items that significantly loaded on the psychological well-being subscale (ωs-psychological = .70, ωhs-psychological = .21).

As expected, higher MHC-SF scores correlated positively with a measure of motivation and pleasure, MAP-SR scores, r = .78, p < .001 (n = 380), and negatively with a measure of depression symptoms, PHQ-9, r = −.67, p < .001 (n = 754).

3.2 Sensitivity to Change

Overall, participants experienced a significant increase in general well-being over the course of treatment, F (1, 328) = 122.61, p < .001, with a medium effect size, partial η2 = .27, Cohen’s d = .50 (for interpretation of effect sizes see Cohen 1988; Mpre = 40.30, SDpre = 10.73, Mpost = 45.43, SDpost = 9.58, n = 329). Changes in total MHC-SF scores were significantly related to changes in both MAP-SR (r = .68, p < .001, n = 298) and PHQ-9 scores (r = −.39, p < .001, n = 321) in the expected directions.

4 Discussion

The current study was the first to examine the psychometric properties of the MHC-SF in a psychiatric sample. In line with previous research using Bifactor ESEM, results supported the superiority of the Bifactor ESEM model with a general well-being factor (Longo et al. 2017; Schutte and Wissing 2017). By addressing limitations inherent to traditional CFA, ESEM, and Bifactor models, Bifactor ESEM provided the best fit for the MHC-SF and should be recommended for use in future projects examining the factor structure of this measure. The strong loadings of all fourteen items on the general factor and high reliability coefficients supported the existence of a unitary construct of general well-being that is reliably measured by the MHC-SF total score. The three subscale scores were also reliable (as indicated by ωs coefficients); however, they explained little variance beyond that explained by the general factor (as indicated by ωhs coefficients). Thus, using subscale scores provides limited distinct information from the total score. Researchers interested in capturing distinct facets of well-being need to use appropriate techniques (e.g., use Bifactor modeling) to do so. Our findings suggest that the use of subscale scores may not be warranted, replicating results found in previous investigations using Bifactor Analysis (De Bruin and Du Plessis 2015; Hides et al. 2016; Jovanović 2015).

Of note, two out of five hypothesized items did not load on the social well-being factor, and four out of six hypothesized items did not load on the psychological well-being factor. With regard to the social well-being subscale, the two items (items 4 and 5) that specifically assessed individuals’ feelings about themselves in relation to society (e.g., “That you had something important to contribute to society”) did not load on their hypothesized group factor. In contrast, the three items (items 6, 7, and 8) that assessed individuals’ feelings about society in general (e.g., “That our society is a good place, or is becoming a better place, for all people”) loaded on their intended social well-being factor. These findings suggest that in this clinical sample, the respondents distinguished between their beliefs about the goodness of society in general, and their self-reported ability to contribute to the goodness of society. One possible explanation for the loadings of items 4 and 5 that has been previously mentioned (Joshanloo 2016b), is that the belief that one has something important to contribute to society (item 4) and the feeling of belonging to one’s society (item 5) are associated with one’s belief that they also possess a number of valuable psychological capabilities (e.g., a favorable view of one’s abilities and positive relationships with others). Therefore, one’s beliefs about their contribution to society and sense of belonging in a community may be more closely related with more individual psychological aspects of well-being (Joshanloo 2016b). Schutte and Wissing (2017) similarly highlighted the more individually-focused aspects of items 4 and 5 by explaining that social and psychological well-being exist on a continuum that ranges from internal (individual experiences and functioning) to external (interpersonal relationships and beliefs about the world outside) well-being. Items 4 and 5 have a more personal tone (e.g., assessing internal, individual experiences) in comparison to items 6, 7, and 8, which assess more external aspects of well-being (e.g., beliefs about the external world; Schutte and Wissing 2017). These differences may provide insight into why items 4 and 5 did not load on the hypothesized social well-being factor.

With regard to the psychological well-being subscale, the two items that loaded on their hypothesized group factor included item 11, which assesses positive relationships with others, and item 13, which assesses feelings of autonomy and confidence. Four items did not load significantly and positively on this factor (items 9, 10, 12, and 14). These items assess self-acceptance (liking oneself), environmental mastery (feeling able to manage daily life responsibilities), personal growth (having had to cope and grow from challenges), and purpose in life (having a sense of meaning or direction). These results suggest that these positive psychological experiences were more closely related to general well-being in this sample. One previous research study reporting similar factor loadings of the psychological well-being items (Joshanloo 2016b) hypothesized that one’s sense of competence in managing the external environment, as well as a positive attitude toward the self capture how individuals feel about themselves and their environments, and are therefore strongly related to higher levels of positive affect and lower levels of negative affect. Therefore, both items 9 (self-acceptance) and 10 (environmental mastery) have eudaimonic as well as hedonic aspects to them, which may explain why they did not load on the hypothesized psychological well-being factor in this study. Joshanloo’s (2016b) explanation may also apply to items 12 and 14 which potentially have eudaimonic as well as hedonic aspects to them, and did not load on their expected psychological well-being group factor.

Results from this study and others call into question the use of specific subscale scores from the MHC-SF. This had led other researchers to propose that this scale should only be used to assess general well-being, through its total score (Hides et al. 2016; Jovanović 2015). Other assessments may be necessary to effectively and distinctly measure other aspects of well-being, such as combining scores from different scales to maximize content coverage (e.g., Gallagher et al. 2009).

As expected, the MHC-SF demonstrated good convergent validity, providing support for the soundness of the MHC-SF total score, and its potential utility as a measure of general well-being. Correlations with measures of motivation/pleasure and depressive symptoms were moderate to large, and comparable to prior findings (Hides et al. 2016; Keyes et al. 2008; Lamers et al. 2012; Petrillo et al. 2015). Of note, the magnitude of the correlation between mental health (i.e., depression) and well-being was greater than previously found in nonclinical samples using various measures of mental illness (previous rs ranging from −0.22 to −0.59; Hides et al. 2016; Keyes et al. 2008; Lamers et al. 2012; Petrillo et al. 2015). It is possible that well-being and mental illness are strongly related in this acute psychiatric sample because information pertaining to immediate psychological difficulties is likely very salient to individuals when responding to all questionnaires.

Finally, we found that participants’ total scores on the MHC-SF increased significantly from admission to discharge from the partial hospital program, indicating that patient well-being improved over the course of treatment. Increases in general well-being were significantly associated with increases in motivation/pleasure, as well as decreases in depressive symptoms. Further research is needed to assess factors associated with stability and variability in MHC-SF scores (including the effects of psychological treatment on well-being). Although these initial findings suggest that the MHC-SF may be a useful transdiagnostic treatment outcome measure, it is important to note that the MHC-SF has demonstrated modest test-retest reliability in other studies (Lamers et al. 2011, 2012; Petrillo et al. 2015), which could potentially influence the measure’s sensitivity to change.

4.1 Limitations and Future Directions

The current study has several strengths. First, this is the first study to answer several important questions related to the psychometric properties of the MHC-SF in a psychiatric sample. Second, this study tested four competing models, including Bifactor Analysis and Bifactor ESEM, to evaluate the best psychometric approach for this instrument. However, the present findings must also be interpreted in the context of the study’s limitations. First, although our sample was diverse in clinical presentation and most demographic variables, it lacked ethnic and racial diversity, which limits the generalizability of the current findings. Second, we were limited in the number of convergent validity measures we could use in this study to minimize burden on our participants. Consequently, the present study used measures chosen because of their transdiagnostic relevance (that would give us information about processes related to, but distinct from well-being) rather than other measures of well-being. Third, the present study used data collected from self-report questionnaires that participants completed as part of routine clinical monitoring. Consistent with how clinical data are used in clinical practice, sharing answers allowed clinical providers to monitor patients’ progress. However, this also introduces the potential for response bias related to patients’ desire to appear better (or worse). Finally, the present study did not include a control group. Therefore, we are unable to determine whether improvements in well-being from pre- to post-treatment are due to the treatment intervention, or due to another factor such as time or regression to the mean.

Despite these limitations, the present study extends well-being research by examining the psychometric properties of the MHC-SF in a psychiatric sample. Findings from the present study suggest that in clinical practice, the MHC-SF total score can be used to measure a patient’s general well-being. Results from the present study also showed that participants experienced significant increases in general well-being over the course of treatment. However, in line with previous findings, the three subscale scores (hypothesized to measure emotional, social, and psychological well-being) may not provide distinct information beyond that provided by the total score. Researchers interested in measuring these unique facets need to use appropriate techniques to remove the variance explained by general well-being. Future research should replicate and extend these findings to examine the usefulness of the MHC-SF and other well-being assessments as treatment outcome measures.

Footnotes

  1. 1.

    We calculated omega coefficients for the total score as detailed by Rodriguez et al. (2016), where λ corresponds to factor loadings and (1-h2) to error variances for each item. Similar calculations apply for omega coefficients for subscales (see Rodriguez et al. 2016).

    \( {\displaystyle \begin{array}{c}\omega =\frac{{\left(\sum {\lambda}_{general}\right)}^2+{\left(\sum {\lambda}_{emotional}\right)}^2+{\left(\sum {\lambda}_{social}\right)}^2+{\left(\sum {\lambda}_{phsycho\mathit{\log} ical}\right)}^2}{{\left(\sum {\lambda}_{general}\right)}^2+{\left(\sum {\lambda}_{emotional}\right)}^2+{\left(\sum {\lambda}_{social}\right)}^2+{\left(\sum {\lambda}_{phsycho\mathit{\log} ical}\right)}^2+\sum \left(1-{h}^2\right)}=\frac{88.47+1.89+3.07+0.75}{88.47+1.89+3.07+0.75+5.26}=.95\\ {}{\omega}_h=\frac{{\left(\sum {\lambda}_{general}\right)}^2}{{\left(\sum {\lambda}_{general}\right)}^2+{\left(\sum {\lambda}_{emotional}\right)}^2+{\left(\sum {\lambda}_{social}\right)}^2+{\left(\sum {\lambda}_{phsycho\mathit{\log} ical}\right)}^2+\sum \left(1-{h}^2\right)}=\frac{88.47}{88.47+1.89+3.07+0.75+5.26}=.89\end{array}} \)

Notes

Compliance with ethical standards

Conflict of Interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

References

  1. Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 16, 397–438.  https://doi.org/10.1080/10705510903008204.CrossRefGoogle Scholar
  2. Beck, A. T., Rush, A. J., Shaw, B. F., & Emery, G. (1979). Cognitive Therapy of Depression. New York: Guilford Press.Google Scholar
  3. Beard, C. & Björgvinsson, T. (2014). Beyond generalized anxiety disorder: Psychometric properties of the GAD-7 in a heterogeneous sample. Journal of Anxiety Disorders, 28, 547–552.  https://doi.org/10.1016/j.janxdis.2014.06.002.
  4. Beard, C., Hsu, K. J., Rifkin, L. S., Busch, A. B., & Björgvinsson, T. (2016). Validation of the PHQ-9 in a psychiatric sample. Journal of Affective Disorders, 193, 267-273.  https://doi.org/10.1016/j.jad.2015.12.075.
  5. Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606.  https://doi.org/10.1037/0033-2909.88.3.588.CrossRefGoogle Scholar
  6. Björgvinsson, T., Kertz, S., Bigda-Peyton, J., Rosmarin, D. H., Aderka, I., & Neuhaus, E. (2014). Effectiveness of cognitive behavior therapy for severe mood disorders in an acute psychiatric naturalistic setting: A benchmarking study. Cognitive Behaviour Therapy, 43, 209-220.  https://doi.org/10.1080/16506073.2014.901988.
  7. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing Structural Equation Models (pp. 136–162). Newbury Park: Sage Publications.Google Scholar
  8. Chen, F. F., Hayes, A., Carver, C. S., Laurenceau, J. P., & Zhang, Z. (2012). Modeling general and specific variance in multifaceted constructs: a comparison of the bifactor model to other approaches. Journal of Personality, 80, 219–251.  https://doi.org/10.1111/j.1467-6494.2011.00739.x.CrossRefGoogle Scholar
  9. Cho, E. (2016). Making reliability reliable: a systematic approach to reliability coefficients. Organizational Research Methods, 19, 651–682.CrossRefGoogle Scholar
  10. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. New York: Routledge Academic.Google Scholar
  11. De Bruin, G. P., & Du Plessis, G. A. (2015). Bifactor analysis of the Mental Health Continuum-Short Form (MHC-SF). Psychological Reports, 116, 438–446.  https://doi.org/10.2466/03.02.PR0.116k20w6.CrossRefGoogle Scholar
  12. De Carvalho, J. S., Pereira, N. S., Pinto, A. M., & Marôco, J. (2016). Psychometric properties of the Mental Health Continuum-Short Form: a study of Portuguese speaking children/youths. Journal of Child and Family Studies, 25, 2141–2154.  https://doi.org/10.1007/s10826-016-0396-7.CrossRefGoogle Scholar
  13. Deci, E. L., & Ryan, R. M. (2008). Hedonia, eudaimonia, and well-being: an introduction. Journal of Happiness Studies, 9, 1–11.  https://doi.org/10.1007/s10902-006-9018-1.CrossRefGoogle Scholar
  14. Duckworth, A. L., Steen, T. A., & Seligman, M. E. (2005). Positive psychology in clinical practice. Annual Review of Clinical Psychology, 1, 629–651.  https://doi.org/10.1146/annurev.clinpsy.1.102803.144154.CrossRefGoogle Scholar
  15. Dziak, J. J., Coffman, D. L., Lanza, S. T., & Li, R. (2017). Sensitivity and specificity of information criteria. Peer Journal Preprints, 5, e1103v3.  https://doi.org/10.7287/peerj.preprints.1103v3.Google Scholar
  16. Gallagher, M. W., Lopez, S. J., & Preacher, K. J. (2009). The hierarchical structure of well-being. Journal of Personality, 77, 1025–1050.  https://doi.org/10.1111/j.1467-6494.2009.00573.x.CrossRefGoogle Scholar
  17. Gignac, G. E., & Watkins, M. W. (2013). Bifactor modeling and the estimation of model-based reliability in the WAIS-IV. Multivariate Behavioral Research, 48, 639–662.  https://doi.org/10.1080/00273171.2013.804398.CrossRefGoogle Scholar
  18. Guo, C., Tomson, G., Guo, J., Li, X., Keller, C., & Söderqvist, F. (2015). Psychometric evaluation of the Mental Health Continuum-Short Form (MHC-SF) in Chinese adolescents–A methodological study. Health and Quality of Life Outcomes, 13, 198–206.  https://doi.org/10.1186/s12955-015-0394-2.CrossRefGoogle Scholar
  19. Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009). Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42, 377–381.  https://doi.org/10.1016/j.jbi.2008.08.010.CrossRefGoogle Scholar
  20. Hides, L., Quinn, C., Stoyanov, S., Cockshaw, W., Mitchell, T., & Kavanagh, D. J. (2016). Is the mental wellbeing of young Australians best represented by a single, multidimensional or bifactor model? Psychiatry Research, 241, 1–7.  https://doi.org/10.1016/j.psychres.2016.04.077.CrossRefGoogle Scholar
  21. Hone, L. C., Jarden, A., Schofield, G., & Duncan, S. (2014). Measuring flourishing: the impact of operational definitions on the prevalence of high levels of wellbeing. International Journal of Wellbeing, 4, 62–90.  https://doi.org/10.5502/ijw.v4i1.4.CrossRefGoogle Scholar
  22. Howard, J. L., Gagné, M., Morin, A. J. S., & Forest, J. (2016). Using bifactor exploratory structural equation modeling to test for a continuum structure of motivation. Journal of Management, 1–27. doi:  https://doi.org/10.1177/0149206316645653.
  23. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55.  https://doi.org/10.1080/10705519909540118.CrossRefGoogle Scholar
  24. Joshanloo, M. (2016a). A new look at the factor structure of the MHC-SF in Iran and the United States using exploratory structural equation modeling. Journal of Clinical Psychology, 72, 701–713.  https://doi.org/10.1002/jclp.22287.CrossRefGoogle Scholar
  25. Joshanloo, M. (2016b). Revisiting the empirical distinction between hedonic and eudaimonic aspects of well-being using exploratory structural equation modeling. Journal of Happiness Studies, 17, 2023–2036.  https://doi.org/10.1007/s10902-015-9683-z.CrossRefGoogle Scholar
  26. Joshanloo, M., & Jovanović, V. (2016). The factor structure of the Mental Health Continuum-Short Form (MHC-SF) in Serbia: an evaluation using exploratory structural equation modeling. Journal of Mental Health, 1–6. doi:  https://doi.org/10.1080/09638237.2016.1222058.
  27. Joshanloo, M., & Lamers, S. M. A. (2016). Reinvestigation of the factor structure of the MHC-SF in the Netherlands: contributions of exploratory structural equation modeling. Personality and Individual Differences, 97, 8–12.  https://doi.org/10.1016/j.paid.2016.02.089.CrossRefGoogle Scholar
  28. Joshanloo, M., Wissing, M. P., Khumalo, I. P., & Lamers, S. M. A. (2013). Measurement invariance of the Mental Health Continuum-Short Form (MHC-SF) across three cultural groups. Personality and Individual Differences, 55, 755–759.  https://doi.org/10.1016/j.paid.2013.06.002.CrossRefGoogle Scholar
  29. Joshanloo, M., Jose, P. E., & Kielpikowski, M. (2016). The value of exploratory structural equation modeling in identifying factor overlap in the Mental Health Continuum-Short Form (MHC-SF): a study with a New Zealand sample. Journal of Happiness Studies, 1–14. doi:  https://doi.org/10.1007/s10902-016-9767-4.
  30. Jovanović, V. (2015). Structural validity of the Mental Health Continuum-Short Form: the bifactor model of emotional, social and psychological well-being. Personality and Individual Differences, 75, 154–159.  https://doi.org/10.1016/j.paid.2014.11.026.CrossRefGoogle Scholar
  31. Karaś, D., Cieciuch, J., & Keyes, C. L. M. (2014). The Polish adaptation of the Mental Health Continuum-Short Form (MHC-SF). Personality and Individual Differences, 69, 104–109.  https://doi.org/10.1016/j.paid.2014.05.011.CrossRefGoogle Scholar
  32. Kenny, D. A., & McCoach, D. B. (2003). Effect of the number of variables on measures of fit in structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 10, 333–351.  https://doi.org/10.1207/S15328007SEM1003_1.CrossRefGoogle Scholar
  33. Keyes, C. L. (2002). The mental health continuum: from languishing to flourishing in life. Journal of Health and Social Behavior, 43, 207–222.CrossRefGoogle Scholar
  34. Keyes, C. L. (2005). Mental illness and/or mental health? Investigating axioms of the complete state model of health. Journal of Consulting and Clinical Psychology, 73, 539–548.  https://doi.org/10.1037/0022-006X.73.3.539.CrossRefGoogle Scholar
  35. Keyes, C. L. (2007). Promoting and protecting mental health as flourishing: a complementary strategy for improving national mental health. American Psychologist, 62, 95–108.  https://doi.org/10.1037/0003-066X.62.2.95.CrossRefGoogle Scholar
  36. Keyes, C. L., Wissing, M., Potgieter, J. P., Temane, M., Kruger, A., & van Rooy, S. (2008). Evaluation of the Mental Health Continuum-Short Form (MHC-SF) in Setswana- speaking South Africans. Clinical Psychology & Psychotherapy, 15, 181–192.  https://doi.org/10.1002/cpp.572.CrossRefGoogle Scholar
  37. Kline, R. B. (2005). Principles and Practice of Structural Equation Modeling (2nd ed.). New York: Guilford Press.Google Scholar
  38. Kroenke, K., & Spitzer, R. L. (2002). The PHQ-9: a new depression diagnostic and severity measure. Psychiatric Annals, 32, 509–515.  https://doi.org/10.3928/0048-5713-20020901-06.CrossRefGoogle Scholar
  39. Lahey, B. B., Applegate, B., Hakes, J. K., Zald, D. H., Hariri, A. R., & Rathouz, P. J. (2012). Is there a general factor of prevalent psychopathology during adulthood? Journal of Abnormal Psychology, 121, 971–977.  https://doi.org/10.1037/a0028355.CrossRefGoogle Scholar
  40. Lamers, S. M. A., Westerhof, G. J., Bohlmeijer, E. T., ten Klooster, P. M., & Keyes, C. L. M. (2011). Evaluating the psychometric properties of the Mental Health Continuum-Short Form (MHC-SF). Journal of Clinical Psychology, 67, 99–110.  https://doi.org/10.1002/jclp.20741.CrossRefGoogle Scholar
  41. Lamers, S. M. A., Glas, C. A. W., Westerhof, G. J., & Bohlmeijer, E. T. (2012). Longitudinal evaluation of the Mental Health Continuum-Short Form (MHC-SF). European Journal of Psychological Assessment, 28, 290–296.  https://doi.org/10.1027/1015-5759/a000109.CrossRefGoogle Scholar
  42. Linehan, M. (1993). Cognitive-behavioral Treatment of Borderline Personality Disorder. New York, NY: Guilford Press.Google Scholar
  43. Litalien, D., Morin, A. J., Gagné, M., Vallerand, R. J., Losier, G. F., & Ryan, R. M. (2017). Evidence of a continuum structure of academic self-determination: a two-study test using a bifactor-ESEM representation of academic motivation. Contemporary Educational Psychology, 51, 67–82.  https://doi.org/10.1016/j.cedpsych.2017.06.010.CrossRefGoogle Scholar
  44. Llerena, K., Park, S. G., McCarthy, J. M., Couture, S. M., Bennett, M. E., & Blanchard, J. J. (2013). The Motivation and Pleasure Scale-Self-Report (MAP-SR): reliability and validity of a self-report measure of negative symptoms. Comprehensive Psychiatry, 54, 568–574.  https://doi.org/10.1016/j.comppsych.2012.12.001.CrossRefGoogle Scholar
  45. Longo, Y., Jovanović, V., Sampaio de Carvalho, J., & Karaś, D. (2017). The general factor of well-being: Multinational evidence using bifactor ESEM on the Mental Health Continuum–Short Form. Assessment, Advanced online publication. doi:  https://doi.org/10.1177/1073191117748394.
  46. Marsh, H. W., Morin, A. J. S., Parker, P. D., & Kaur, G. (2014). Exploratory structural equation modeling: an integration of the best features of exploratory and confirmatory factor analysis. Annual Review of Clinical Psychology, 10, 85–110.  https://doi.org/10.1146/annurev-clinpsy-032813-153700.CrossRefGoogle Scholar
  47. McDonald, R. P. (1970). The theoretical foundations of principal factor analysis, canonical factor analysis, and alpha factor analysis. British Journal of Mathematical and Statistical Psychology, 23, 1–21.  https://doi.org/10.1111/j.2044-8317.1970.tb00432.x.CrossRefGoogle Scholar
  48. McDonald, R. P. (1999). Test Theory: a Unified Treatment. Mahwah: Lawrence Erlbaum Associates.Google Scholar
  49. Morin, A. J., Arens, A. K., & Marsh, H. W. (2016). A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Structural Equation Modeling: A Multidisciplinary Journal, 23, 116–139.  https://doi.org/10.1080/10705511.2014.961800.CrossRefGoogle Scholar
  50. Muthén, L. K., & Muthén, B. O. (2017). Mplus statistical modeling software: release 8.0. Los Angeles: Muthén & Muthén.Google Scholar
  51. Petrillo, G., Capone, V., Caso, D., & Keyes, C. L. (2015). The Mental Health Continuum-Short Form (MHC–SF) as a measure of well-being in the Italian context. Social Indicators Research, 121, 291–312.  https://doi.org/10.1007/s11205-014-0629-3.CrossRefGoogle Scholar
  52. Rapp, C. A. (1998). The strengths model: case management with people suffering from severe and persistent mental illness. New York: Oxford University Press.Google Scholar
  53. Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013a). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95, 129–140.CrossRefGoogle Scholar
  54. Reise, S. P., Scheines, R., Widaman, K. F., & Haviland, M. G. (2013b). Multidimensionality and structural coefficient bias in structural equation modeling: a bifactor perspective. Educational and Psychological Measurement, 73, 5–26.  https://doi.org/10.1177/0013164412449831.CrossRefGoogle Scholar
  55. Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Evaluating bifactor models: calculating and interpreting statistical indices. Psychological Methods, 21, 137–150.CrossRefGoogle Scholar
  56. Ryff, C. D. (1989). Happiness is everything, or is it? Explorations on the meaning of psychological well-being. Journal of Personality and Social Psychology, 57, 1069–1081.  https://doi.org/10.1037/0022-3514.57.6.1069.CrossRefGoogle Scholar
  57. Ryff, C. D., & Singer, B. H. (2006). Know thyself and become what you are: a eudaimonic approach to psychological well-being. Journal of Happiness Studies, 9, 13–39.  https://doi.org/10.1007/s10902-006-9019-0.CrossRefGoogle Scholar
  58. Sánchez-Oliva, D., Morin, A. J., Teixeira, P. J., Carraça, E. V., Palmeira, A. L., & Silva, M. N. (2017). A bifactor exploratory structural equation modeling representation of the structure of the basic psychological needs at work scale. Journal of Vocational Behavior, 98, 173–187.  https://doi.org/10.1016/j.jvb.2016.12.001.CrossRefGoogle Scholar
  59. Schutte, L., & Wissing, M. P. (2017). Clarifying the factor structure of the Mental Health Continuum Short Form in three languages: a bifactor exploratory structural equation modeling approach. Society and Mental Health, 142–158. doi:  https://doi.org/10.1177/2156869317707793.
  60. Seligman, M. E. P., & Csikszentmihalyi, M. (2000). Special issue: positive psychology. American Psychologist, 55, 5–14.  https://doi.org/10.1007/978-94-017-9088-8_18.CrossRefGoogle Scholar
  61. Stenling, A., Ivarsson, A., Hassmén, P., & Lindwall, M. (2015). Using bifactor exploratory structural equation modeling to examine global and specific factors in measures of sports coaches’ interpersonal styles. Frontiers in Psychology, 6, 1–12.  https://doi.org/10.3389/fpsyg.2015.01303.CrossRefGoogle Scholar
  62. Wang, P. S., Lane, M., Olfson, M., Pincus, H. A., Wells, K. B., & Kessler, R. C. (2005). Twelve- month use of mental health services in the United States: results from the National Comorbidity Survey Replication. Archives of General Psychiatry, 62, 629–640.  https://doi.org/10.1001/archpsyc.62.6.629.CrossRefGoogle Scholar
  63. Watkins, M. W. (2017). The reliability of multidimensional neuropsychological measures: from alpha to omega. The Clinical Neuropsychologist, 31, 1113–1126.CrossRefGoogle Scholar
  64. Westerhof, G. J., & Keyes, C. L. M. (2010). Mental illness and mental health: the two continua model across the lifespan. Journal of Adult Development, 17, 110–119.  https://doi.org/10.1007/s10804-009-9082-y.CrossRefGoogle Scholar
  65. Wood, A. M., & Tarrier, N. (2010). Positive clinical psychology: a new vision and strategy for integrated research and practice. Clinical Psychology Review, 30, 819–829.  https://doi.org/10.1016/j.cpr.2010.06.003.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Alexandra L. Silverman
    • 1
  • Marie Forgeard
    • 1
    • 2
  • Courtney Beard
    • 1
    • 2
  • Thröstur Björgvinsson
    • 1
    • 2
  1. 1.Behavioral Health Partial ProgramMcLean HospitalBelmontUSA
  2. 2.Harvard Medical SchoolBostonUSA

Personalised recommendations