Advertisement

BMC Pediatrics

, 19:108 | Cite as

A screening tool for psychological difficulties in children aged 6 to 36 months: cross-cultural validation in Kenya, Cambodia and Uganda

  • Fabienne NackersEmail author
  • Thomas Roederer
  • Caroline Marquer
  • Scholastic Ashaba
  • Samuel Maling
  • Juliet Mwanga-Amumpaire
  • Sothara Muny
  • Chea Sokeo
  • Vireak Shom
  • Maria Palha
  • Pauline Lefèbvre
  • Beatrice W. Kirubi
  • Grace Kamidigo
  • Bruno Falissard
  • Marie-Rose Moro
  • Rebecca F. Grais
Open Access
Research article
Part of the following topical collections:
  1. Behavior and development

Abstract

Background

In low-resource settings, the lack of mental health professionals and cross-culturally validated screening instruments complicates mental health care delivery. This is especially the case for very young children. Here, we aimed to develop and cross-culturally validate a simple and rapid tool, the PSYCa 6–36, that can be administered by non-professionals to screen for psychological difficulties among children aged six to 36 months.

Methods

A primary validation of the PSYCa 6–36 was conducted in Kenya (n = 319 children aged 6 to 36 months; 2014), followed by additional validations in Kenya (n = 215; 2014) Cambodia (n = 189; 2015) and Uganda (n = 182; 2016). After informed consent, trained interviewers administered the PSYCa 6–36 to caregivers participating in the study. We assessed the psychometric properties of the PSYCa 6–36 and external validity was assessed by comparing the results of the PSYCa 6–36 against a clinical global impression severity [CGIS] score rated by an independent psychologist after a structured clinical interview with each participant.

Results

The PSYCa 6–36 showed satisfactory psychometric properties (Cronbach’s alpha > 0.60 in Uganda and > 0.70 in Kenya and Cambodia), temporal stability (intra-class correlation coefficient [ICC] > 0.8), and inter-rater reliability (ICC from 0.6 in Uganda to 0.8 in Kenya). Psychologists identified psychological difficulties (CGIS score > 1) in 11 children (5.1%) in Kenya, 13 children (8.7%) in Cambodia and 15 (10.5%) in Uganda, with an area under the receiver operating characteristic curve of 0.65 in Uganda and 0.80 in Kenya and Cambodia.

Conclusions

The PSYCa 6–36 allowed for rapid screening of psychological difficulties among children aged 6 to 36 months among the populations studied. Use of the tool also increased awareness of children’s psychological difficulties and the importance of early recognition to prevent long-term consequences. The PSYCa 6–36 would benefit from further use and validation studies in popula`tions with higher prevalence of psychological difficulties.

Keywords

Mental health Psychology Screening Validation Preschool children Low-income population Kenya Cambodia Uganda 

Abbreviations

95%CI

95% confidence interval

AUC

Area under the curve (AUC)

CGIS score

Clinical global impression severity score

CPP

Committee for the Protection of Persons (Ile de France XI)

DC: 0–3R

Diagnostic Classification of Mental Health and Developmental Disorders of Infancy and Early Childhood, Revised Edition

ICC

Intra-class correlation coefficient

KEMRI

Kenyan Medical Research Institute

MSF

Médecins Sans Frontières

MUST

Mbarara University of Science and Technology

NECHR

National Ethics Committee for Health Research

PIR-GAS

Parent-Infant Relationship Global Assessment Scale

PSYCa 6–36

Screening tool for psychological difficulties among children aged six to 36 months

ROC curve

Receiver Operating Characteristic curve

UNCST

Uganda National Council for Science and Technology

Background

Despite the lack and heterogeneity of existing prevalence data, the burden of mental health problems in children and adolescents is estimated to be as high as 10–20% worldwide [1]. The largest proportion of this burden is located in low-resource countries, where up to half of the population is younger than 15 years [1]. In these countries, childhood psychological difficulties often remain undetected and thus untreated [2], limiting children’s full developmental potential and increasing the risk of later mental health difficulties [1]. In particular, infant and toddler’s mental health is often very low on the list of priorities [3]. In low-resource countries, the provision of mental health care is hampered by the lack of qualified personnel and limited access to health services [4] combined with stigma and poor awareness of psychological difficulties in young children [5, 6]. The absence of easy-to-use and cross-culturally adapted tools to assess mental health in young children further complicates disease burden estimation [7, 8, 9, 10] and the delivery of care [7]. Existing screening tools for children younger than three years may focus on specific disorders or symptoms [11, 12]; necessitate a long administration time [13, 14, 15, 16, 17, 18]; require highly-trained administrators [19]; and/or have not been cross-culturally validated in low-resource countries [20, 21]. Validating instruments to assess psychological difficulties in young children living in low-resource countries can provide an important tool to identify those in need. Building on the methods used for the cross-cultural validation of a screening tool designed for children aged three to six years [22, 23], we aimed to develop and to cross-culturally validate a screening tool for psychological difficulties among children aged six to 36 months.

Methods

Development of the PSYCa 6–36

As a first step in the development of the PSYCa 6–36, an expert panel based on consensus was convened prior to the start of the study. The panel was comprised of eleven experts in the mental health of infants and young children and transcultural psychopathology from France, Senegal, Canada, USA, and Norway. They were asked to individually list the twelve most important items to screen for psychological difficulties in children aged six to 36 months. Responses were compiled by consensus, aiming for a maximum 20 items, or statements, related to emotions and behaviour that would require little (maximum 10%) or no adaptation when used among different populations. The resulting composition of the PSYCa 6–36 is presented in the Table 1. The tool is completed by the caregiver through an interviewer, with the aid of a guideline (Additional file 1), who reads each item. The caregiver is asked to respond to each item considering the previous month and responding “no or not at all”, “sometimes or occasionally”, “often or frequently”. The interviewer rates each item (0, 1 or 2) accordingly and, at the end of administration, computes a total score ranging from zero to 40, with higher scores indicating greater psychological distress and a need of further mental health assessment. An answer is expected for each item and, when necessary, prompted. However, the rating of a specific item can remain missing if the caregiver does not know or does not want to answer. Examples illustrating each item are included in the guideline for the interviewer.
Table 1

English version of the PSYCa 6–36

Study setting

The study took place from August 2014 to January 2015 in Mathare, a major urban slum with high level of poverty and violence in Nairobi (Kenya) where Médecins Sans Frontières (MSF) was providing psychological and medical care to victims of sexual violence and to patients diagnosed with multidrug resistant tuberculosis (TB). The study in Mathare was followed by two additional validations. The first took place from July to September 2015 in Kampong Cham, a quiet urban district of Kampong Cham Province (Cambodia) where MSF was providing TB diagnosis, treatment and social support. The second took place from July to August 2016 in Mbarara municipality, the second-largest town of Uganda where Epicentre, a research organisation created by MSF, has been conducting clinical research for over twenty years in collaboration with the Mbarara University of Sciences and Technology (MUST) and the Mbarara Regional Referral Hospital (MRRH). All settings were in low resource but stable environments; none had been affected by a recent acute traumatic event such as a natural disaster or a conflict. MSF and Epicentre facilitated the management, reference and follow-up of children in need of mental health treatment or other relevant medical evaluation and care.

Translation

Two professional translators fluent in local language (Swahili in Kenya; Khmer in Cambodia; Runyankore in Uganda) and English translated independently the PSYCa 6–36. After reconciliation of the two translations by a mental health professional, the relevance, semantic equivalence and formulation of each item was assessed through discussions with national health professionals, psychosocial workers and groups of caregivers [24]. The resulting translation was back-translated into English. Final translations are presented in the Additional file 1.

Procedures, population and data collection

Two or three national interviewers were recruited, fluent in the local language and English, in all sites. Children aged six to 36 months accompanied by their main caregiver (child-caregiver dyad) and permanently living in the local community were eligible for participation. Caregivers could be the mother, the father or an adult caring for the child on a regular basis. Children with apparent development retardation or motor disability were not excluded. Exclusion criteria included a previously diagnosed mental health disorder or visible signs of severe mental health disorders. Eligible dyads were selected in the community, starting from the house nearest from a starting point (randomly selected spatial point in Kenya; house randomly selected from a census list in Cambodia; centre of the village in Uganda). Other dyads were recruited by proximity with the objective to include five to eight dyads per day with a maximum of ten per starting point. If several children aged six to 36 months lived in the same house, one was selected at random. Two series of dyads were recruited in Kenya and one series in Cambodia and Uganda.

All children were assessed at home by an interviewer trained to use the PSYCa 6–36. A subsample of children were assessed twice with PSYCa 6–36, 24 hours apart, in the same location, either by a same interviewer to assess the tool’s temporal stability or by different interviewers to assess the tool’s inter-rater reliability. A subsample of children were assessed by a clinical psychologist, blind to results of the PSYCa 6–36. In Kenya, one psychologist worked under the daily supervision of a child psychiatrist experienced in transcultural psychology. In Cambodia and Uganda, a national and an international psychologist assessed most of the children together and otherwise discussed their clinical evaluations. The psychologists were trained by a child psychiatrist to conduct a comprehensive structured mental health examination in young children, through observation and a structured interview with the child’s caregiver. They were also trained to use two additional tools: the Parent-Infant Relationship Global Assessment Scale (PIR-GAS) from the Diagnostic Classification of Mental Health and Developmental Disorders of Infancy and Early Childhood (Revised Edition; DC: 0–3R) [25]; and a seven-point Clinical Global Impression Severity (CGIS) scale assessing the patient’s current symptom(s) severity. The rating of the CGIS scale was considered as the gold standard to assess external validity, with a score higher than one identifying the presence of psychological difficulties.

Data analysis

Data were double entered in EpiData 3.1 (EpiData, Odense, Denmark) and analysed using Stata (version 13, College Station TX, USA). The total score was calculated as the sum of the individual score for all 20 items. If more than 5 item scores were missing, the total score was not calculated. In case of one to five missing item score(s), the total score was calculated as the sum of the individual item scores and then imputed taking in account the proportion of missing items. Scores were compared between groups using the Kruskal-Wallis test and sensitivity analyses were conducted excluding children with imputed score.

Internal consistency was assessed using Cronbach’s alpha [26] and the inter-rater and temporal stability using the intra-class correlation coefficient (ICC) [27]. Unidimensionality of the instrument was described and different dimensional structures were explored using Catell’s Scree-test [28] and factor analysis with orthogonal varimax. The external validity of the tool, in comparison with the gold standard was assessed using the Spearman’s rho correlation coefficient and using Receiver Operating Characteristic (ROC) curves that plotted the sensitivity against 1–specificity for all PSYCa 6–36 cut-off points to differentiate children with CGIS score of > 1 versus 1. The area under the curve (AUC) were computed with 95% confidence intervals (95% CI), an AUC of 0.5 indicating no discriminating ability, while an AUC of 1.0 indicates perfect discrimination ability.

Sample size

For the primary validation in Kenya, we aimed to recruit a first series of at least 300 children [29] to estimate a Cronbach’s alpha coefficient with a 95% confidence interval [95%CI] semi-amplitude of 0.05. Of this series, 50 children were assessed twice to estimate the inter-rater reliability and 50 children to assess the temporal stability. In addition, we aimed to recruit a second series of at least 200 children to assess of the external validity. For the subsequent validations conducted in Uganda and Cambodia, a sample of at least 141 children was needed to assess external validity (assuming an AUC under the ROC curve against the CGIS scale of 0.9, with α at 0.05, a power of 0.8, and a standard error ratio between negative and positive results of 0.33), with 20 additional children to assess the inter-rater reliability and 20 children to assess the temporal stability.

Ethical considerations and consent to participate

Ethical clearance was obtained from the French National Committee for the Protection of Persons (CPP Ile de France XI), the Ethics Review Committee of the Kenyan Medical Research Institute (KEMRI), the Cambodian National Ethics Committee for Health Research (NECHR), the Research Ethics Committee of the Mbarara University of Science and Technology (MUST-REC), and, the Uganda National Council for Science and Technology (UNCST). All participants’ caregivers provided written informed consent before participation. Children in need of psychological or medical care according to the psychologist were offered referral to previously identified professionals for further clinical assessment and, when possible, free treatment.

Results

In Kenya, 319 children were included in the first series (including 64 assessed twice for the inter-rater reliability and 56 assessed twice for temporal stability) and 215 in the second series. In Cambodia, 148 children were included to assess the external validity; 20 for inter-rater reliability and 21 for temporal stability. In Uganda, 142 children were included to assess external validity; 20 for inter-rater reliability and 20 for temporal stability. None of the children assessed for eligibility presented a previously diagnosed mental health disorder or visible signs of a severe mental health disorder. Participant characteristics are presented in Table 2. Median age of the children included was between 17 and 20 months. Across the three study settings, 19 children had an apparent development retardation or motor disability.
Table 2

Participant characteristics, PSYCa 6–36 cross-cultural validation study, Kenya, Cambodia, Uganda

Socio-demographic

Kenya 1 (n = 319)

Kenya 2 (n = 215)

Cambodia (n = 148)

Uganda (n = 142)

n

%

n

%

n

%

n

%

Age of the child (months)

 6–11

77

24.1

58

27.0

26

17.6

31

21.8

 12–17

66

20.7

58

27.0

31

21.0

45

31.7

 18–23

55

17.2

36

16.7

34

23.0

17

12.0

 24–29

70

21.9

28

13.0

26

17.6

24

16.9

 30–36

51

16.0

35

16.3

31

21.0

25

17.6

Sex of the child

 Boy

158

49.5

123

57.2

85

57.4

77

54.2

Caregiver-child relation

 Mother

299

93.7

209

97.2

103

69.6

136

95.8

 Father

11

3.5

1

0.5

5

3.4

2

1.4

 Grandmother

3

1.0

4

1.9

33

22.3

2

1.4

 Other (mostly aunt or grandfather)

6

1.9

1

0.5

7

4.7

2

1.4

Household size (including the child and the caregiver)

 2 or 3 persons

99

31.0

83

38.6

14

9.5

57

40.1

 4 or 5 persons

149

46.7

98

45.6

58

39.2

57

40.1

 6 to 13 persons

71

22.3

34

15.8

75

51.4

28

19.8

Number of children < 5 years (including the child)

 1

222

69.6

152

70.7

37

25.0

78

54.9

 2

92

28.8

54

25.1

54

36.5

53

37.3

 3 to 8

5

1.6

9

4.2

57

38.5

11

7.8

Alive siblings from same mother living in same Household

 0

125

39.2

91

42.3

61

41.2

55

38.7

 1

95

29.8

69

32.1

47

31.8

35

24.6

 2

54

16.9

33

15.4

25

16.9

27

19.1

 3 to 5

45

14.1

22

10.2

15

10.1

25

17.6

Parents with which the child usually lives

 Both mother and father

262

82.1

182

84.7

138

93.2

105

73.9

 Mother only

51

16.0

29

13.5

4

2.7

33

23.2

 Father only

2

0.6

0

0

2

1.4

1

0.7

 None

4

1.3

4

1.9

4

2.7

3

2.1

Born ≥1 month preterm

25

7.8

12

5.6

5

3.4

12

8.4

Apparent development retardation or motor disability

9

2.8

1

0.5

3

2.0

6

4.2

Child currently Breastfeeding

 Yes

180

56.4

140

65.1

43

29.1

72

50.7

 No

139

43.6

75

34.9

52

35.8

70

49.3

 Baby bottle

/

/

/

/

53

35.1

/

/

Child can walk

 Not yet

105

32.9

80

37.2

33

22.3

39

27.6

Since birth, child witness/victim of stressful/violent eventa

41

12.9

44

20.5

24

16.2

13

9.1

aEvents reported: Domestic violence (n = 48), Fire/burnt (n = 9), Accident/injury (n = 11), Fighting (n = 38), other (n = 16)

Due to missing values, 305 (95.6%) PSYCa 6–36 were completed in the first series of Kenya and 145 (98.0%) in Cambodia. There were no missing values in the second series in Kenya or Uganda. The scoring distributions of each item are presented in the Additional file 2. The median total score was a bit lower in the first series in Kenya and in Uganda (Table 3) and there was no evidence for a score difference according to age and sex. The PSYCa 6–36 was administered in a median time less than 15 min (Table 4).
Table 3

Total PSYCA 6-36 score for all children, by socio-demographic and clinical characteristics, cross-cultural validation study, Kenya, Cambodia, Uganda

 

Kenya 1

Kenya 2

Cambodia

Uganda

 

n

Median

IQR*

 

n

Median

IQR*

 

n

Median

IQR*

 

n

Median

IQR*

 

All children

319

4a

2–8

 

215

6

3–9

 

148

5b

3–8

 

142

4

2–8

 
 

Min; Max

0; 21

  

Min; Max

0; 21

  

Min; Max

0; 21

  

Min; Max

0; 17

 
 

Mean (SD)

5.36

(4.19)

 

Mean (SD)

6.70

(4.71)

 

Mean (SD)

6.06

(3.83)

 

Mean (SD)

4.99

(3.43)

 

n

Median

IQR*

p**

n

Median

IQR*

p**

n

Median

IQR*

p**

n

Median

IQR*

p**

6-12 m

77

4

2–6

0.15

58

7

3–10

0.15

34

7

3–9

0.51

34

4

3–7

0.84

12-24 m

121

5

3–9

 

94

7

4–10

 

58

5

3–8

 

60

5

2.5–7.5

 

24-36 m

121

4

2–7

 

63

5

2–8

 

56

5

3.5–8

 

48

5

2–8

 

Boys

158

5

3–8

0.70

123

6

3–10

0.62

85

5

3–7

0.41

77

5

3–8

0.12

Girls

160

4

2–8

 

92

6

3–9

 

63

6

4–9

 

65

4

2–7

 

CGIS = 1

    

204

6

3–9

< 0.001

135

5.5

3–8

< 0.001

127

4

2–7

0.02

CGIS> 1

    

11

11

8–14

 

13

11

8–16

 

15

7

3–9

 

PIRGAS <81c/<91d

 

184

6

3–9

< 0.001

120

5

3–7

0.004

96

4

2–8

0.38

PIRGAS 81c/91d-100

 

31

10

7–14

 

25

8

5–12

 

46

5

3–7

 

SD Standard deviation; * IQR Interquartile range; ** Kruskal-Wallis test; a One child with > 25% items missing is excluded from the analysis; 13 children had an incomplete score. Similar results were obtained when excluding children with imputed score (sensitivity analysis); b Three children had an incomplete score due to unknown answers. Similar results were obtained when excluding children with imputed score (sensitivity analysis); c in Cambodia and Uganda; d in Kenya

Table 4

Psychometric properties of the PSYCa 6–36, PSYCa 6–36 cross-cultural validation study, Kenya, Cambodia, Uganda

a including only 305 complete PSYCa 6-36.

b including only 145 complete PSYCa 6-36.

Internal consistency and reliability

The overall Cronbach’s alpha coefficients were ≥ 0.70 [30, 31], except in Uganda (≥ 0.60) (Table 4). The inter-rater ICC on the total score ranged from 0.63 (Uganda) to 0.83 (in Kenya) and the ICC for temporal stability was ≥0.80 in the three settings.

External validity

Psychologists identified difficulties (CGIS score > 1) in 11 (5.1%) children in Kenya, 13 (8.7%) children in Cambodia and 15 (10.5%) in Uganda (Table 5). The distributions of the CGIS and PIR-GAS scores are presented in Table 5. The median PSYCA 6–36 score was higher among children with a CGIS score > 1, and, in Kenya and Cambodia, among children with a lower PIRGAS score (Table 3). The frequency of positive responses per item of the PSYCa 6–36 according to the CGIS score of the children is presented in the Additional file 3. The Spearman’s rho indicated a weak correlation between the final tool and CGIS score (Table 4). The sensitivity and specificity of various PSYCa 6–36 cut-off points to differentiate children with CGIS score of > 1 versus 1 are presented in the Table 6 and the ROC curves in Fig. 1. The area under the ROC curve, measuring the ability of the PSYCa 6–36 to differentiate children with CGIS score of > 1 versus 1, was 0.80 in Kenya and Cambodia but lower in Uganda (Table 4 and Fig. 1). A cut-off point between eight and eleven maximizes the sensitivity and specificity in Kenya and Cambodia but a cut-off point of five is needed to ensure a sensitivity of at least 70% in Uganda. Accounting for the frequency of CGIS score higher than one in the different settings, a cut-off point of eight would identify a third to a fifth (73/215 = 34.0% in Kenya; 35/148 = 23.6% in Cambodia; 29/142 = 20.4% in Uganda) of the total population as falsely positive.
Table 5

Clinical evaluation, PSYCa 6–36 cross-cultural validation study, Kenya, Cambodia, Uganda

 

Kenya 2 (n = 215)

Cambodia (n = 148)

Uganda (n = 142)

n

%

n

%

n

%

CGIS score

 1: Normal, not at all ill

204

94.9

135

91.2

127

89.5

 2: Borderline mentally ill

4

1.9

11

7.4

12

8.4

 3: Mildly ill

5

2.3

1

0.7

3

2.1

 4: Moderately ill

0

0

1

0.7

0

0

 5: Markedly ill

1

0.5

0

0

0

0

 6: Severely ill

1

0.5

0

0

0

0

 7: Extremely ill

0

0

0

0

0

0

PIRGAS scores

 91–100: Well adapted

184

85.6

0

0.0

5

3.6

 81–90: Adapted

29

13.5

122

82.4

41

29.3

 71–80: Perturbed

2

0.9

20

13.5

65

46.4

 61–70: Significantly perturbed

0

0

6

4.1

15

10.7

 51–60: Distressed

0

0

0

0

10

7.2

 41–50: Disturbed

0

0

0

0

3

2.1

 31–40: Disordered

0

0

0

0

1

0.7

a2 missing values b 3 missing values

Table 6

Sensitivity and Specificity of various PSYCA 6–36 score cut-off points using CGIS score (> 1 versus 1) as gold standard

PSYCa 6–36 cut-off

CGIS

“Not Case”

PSYCa 6–36 “Not Case”

CGIS

“Case”

PSYCa 6–36 “Not Case”

CGIS

“Not Case”

PSYCa 6–36 “Case”

CGIS

“Case”

PSYCa 6–36 “Case”

Sensitivity

Specificity

Correctly Classified

LR+

LR-

Positive predictive value

Negative predictive value

 

n

n

n

n

%

%

%

  

%

%

Kenya 2 (n = 215)

 ≥ 5

81

1

123

10

90.9

39.7

42.3

1.51

0.23

7.5

98.8

 ≥ 6

92

1

112

10

90.9

45.1

47.4

1.66

0.20

8.2

98.9

 ≥ 7

111

1

93

10

90.9

54.4

56.3

1.99

0.17

9.7

99.1

 ≥ 8

131

2

73

9

81.8

64.2

65.1

2.29

0.28

11.0

98.5

 ≥ 9

146

3

58

8

72.7

71.6

71.6

2.56

0.38

12.1

98.0

 ≥ 10

160

3

44

8

72.7

78.4

78.1

3.37

0.35

15.4

98.2

 ≥ 11

172

4

32

7

63.6

84.3

83.3

4.06

0.43

17.9

97.7

Cambodia (n = 148)

 ≥ 5

56

2

79

11

84.6

41.5

45.3

1.45

0.37

12.2

96.6

 ≥ 6

73

3

62

10

76.9

54.1

56.1

1.67

0.43

13.9

96.1

 ≥ 7

88

3

47

10

76.9

65.2

66.2

2.21

0.35

17.5

96.7

 ≥ 8

100

3

35

10

76.9

74.1

74.3

2.97

0.31

22.2

97.1

 ≥ 9

109

4

26

9

69.2

80.7

79.7

3.59

0.38

25.7

96.5

 ≥ 10

120

4

15

9

69.2

88.9

87.2

6.23

0.35

37.5

96.8

 ≥ 11

125

5

10

8

61.5

92.6

89.9

8.31

0.42

44.4

96.2

Uganda (n = 142)

 ≥ 5

68

4

59

11

73.3

53.5

55.6

1.58

0.49

15.7

94.4

 ≥ 6

81

7

46

8

53.3

63.8

62.7

1.47

0.73

14.8

92,0

 ≥ 7

89

7

38

8

53.3

70.1

68.3

1.78

0.66

17.4

92.7

 ≥ 8

98

8

29

7

46.7

77.2

73.9

2.04

0.69

19.4

92.5

 ≥ 9

110

9

17

6

40.0

86.6

81.7

2.98

0.69

26.1

92.4

 ≥ 10

119

12

8

3

20.0

93.7

85.9

3.17

0.85

27.3

90.8

 ≥ 11

123

12

4

3

20.0

96.8

88.7

6.35

0.82

42.9

91.1

Fig. 1

ROC curves of the PSYCA 6–36 score compared with the CGIS score. (Upper: Kenya; Middle: Cambodia; Lower, Uganda)

Factor analysis and dimensionality

The visual exploration of the eigenvalues plot (Cattell’s scree test; Fig. 2) suggests a strong uni-dimensionality in Kenya and Cambodia (one meaningful factor explaining 17 and 18% of the variance) and up to seven factors explaining 61% of the variance in Uganda.
Fig. 2

Scree plots of eigenvalues, PSYCa 6–36 cross cultural validation study. (Upper: Kenya; Middle: Cambodia; Lower, Uganda)

Discussion

We report the results of a cross-cultural validation study of a new instrument for screening children aged six to 36 months for psychological difficulties. More than 800 children with their caregivers were included across three low-resources settings.

Infant and child psychopathology measurements are challenging, notably due to the rapid motor, cognitive and emotional development in the first three years of life [32, 33]. Considering this complexity, the inherent limitation of quantitative measures to capture human behaviours and emotions [7], as well as the uniqueness and recent development of the PSYCa 6–36, this screening tool showed satisfactory psychometric properties and the ability to classify children with or without psychological difficulties as closely as the CGIS score. The performance of the PSYCa 6–36 was similar in Cambodia and Kenya, two very different cultural and linguistic contexts. This highlights the cross-cultural aspect of the PSYCa 6–36. The performance was lower in Uganda, which might result from actual differences across study populations but also from translation and adaptation flaws [34, 35]. The interviewer guidelines were more frequently used in Uganda than in the two other contexts. A limited comprehension of the items or instructions by the Ugandan participants cannot be excluded. In addition, there were study implementation challenges in Uganda, especially a divergence in the judgments of the psychologists which might have led to suboptimal standardization and clinical assessment. There may also have been administration differences between interviewers. This highlights field constraints and that, despite the ease of use of the PSYCa 6–36, proper training is mandatory.

The PSYCa 6–36 was developed with support of experts in child and transcultural psychopathology and translated by specialists in the local languages and cultural contexts. Translation procedures may not have fully achieved content and sematic equivalence but overall, the PSYCa 6–36 appeared well understood by the participants considering the low frequency (less than 5%) of missing answers. However, some caregivers might have rated some items without full understanding of their meaning or wording, as suggested by some low individual ICC in the test-retest reliability analysis. Also, about 10% of caregivers refused participation and interviewers informally reported that some caregivers felt uncomfortable with the use of quantitative questionnaire and with talking about “abnormal child behaviours” in their household. A lack of awareness of child psychology and the stigma surrounding mental health that affects all populations [4] might have influenced the caregivers’ willingness to disclose information about children’s difficulties. Because of such stigma, caregivers might have provided socially acceptable, consequently biased, answers. A qualitative evaluation might have strengthened the results of this study by shedding light on the caregiver’s perception and acceptance of the use of a questionnaire about child psychology in the different cultures.

For infants and toddlers, direct observation and evaluation of a child interacting with their caregiver in their natural environment remains the best option for mental health assessment [36]. We used the CGIS score assessed by a trained psychologist to assess external validity. The cross-cultural validity of childhood diagnostic criteria in mental health remains debated [9, 37, 38, 39]. Although Kenyan, Ugandan and Cambodian psychologists performing the assessment likely limited misinterpretation of possible expressions of mental health disorders or symptoms that may be culture-dependent [9, 37, 38, 39]. Further, because of their limited experience in young children’s mental health, they were trained by a child psychiatrist before the start of the study and then worked either under the close supervision or in tandem with a psychiatrist or psychologist experienced in cross-cultural and young child psychology. The clinical assessment was also reinforced by the use of the PIR-GAS scale, although not validated for low-resource settings. Despite these precautions, we recognize the limitation of the comparison with the CGIS might have biased [40] and, possibly underestimated the real PSYCa 6–36 performance.

In Western settings, the prevalence of socio-emotional and behavioural difficulties has been reported to range from 7 to 24% in children aged one to three years [3, 7, 41, 42] but there are data gaps for low-resources countries [1, 4, 8]. A systematic review of prevalence studies of child and adolescent mental health (age range 5 to 16 years) in Sub-Saharan African communities estimated that 14.3% of children had psychopathological difficulties, and 9.5% among studies of which measurement relied on a diagnostic interview [10]. In our study, the psychologists identified fewer children with a CGIS higher than one than expected. Children were included only in the presence of their caregiver and the study was conducted during working hours, thereby likely biasing the study sample towards children at lower risk. More vulnerable children, such as those living in households without a caring adult or left alone during the day, or street children were not included. Also, caregivers who refused participation may be caring for more vulnerable children. Another explanation might be that children living in these difficult environments and exposed to poverty and chronic adversity develop stronger coping mechanisms [7], protecting them against psychological difficulties or limiting the expression of psychological difficulties. This is particularly likely when children remain under the stable protection of their caregiver or other close relatives [7]. Nevertheless, the PSYCa 6–36 would benefit from further use and validation in populations with higher prevalence of psychological difficulties, notably in children having recently faced an acute traumatic event such as migration, conflict, or natural disaster.

In Kenya and Cambodia, the cut-off point maximizing the sensitivity and specificity of the PSYCa 6–36 to differentiate children with CGIS score of > 1 lies between eight and eleven but it is lower in Uganda. Hence, a cut-off point of eight appears an optimal compromise but it should remain flexible to favour sensitivity or specificity according to the expected burden of psychological difficulties and available health services of each specific setting. A cut-off point of eight would identify a substantial proportion of the population as falsely positive, possibly overloading mental health professionals with unnecessary referrals. A higher cut-off would better limit referral to children in need of further clinical evaluation. The definite choice of the cut-off requires subsequent documentation and analysis in populations with higher prevalence of mental health difficulties such as migrants, refugees or internally displaced children, children living in conflict situations or in the aftermath of a natural disaster, or sick children. Further investigation is also needed among specific age groups, such children below one year of age.

In the three study settings, follow-up care was offered by the psychologists and counsellors focusing on the reinforcement of the caregiver-child relationship. Although the child psychiatrists in Kenya and Uganda ensured access to specialized care, such care was limited in Cambodia, being only available in the capital city, a few hours drive from Kampong Cham. It is important to note that, although follow-up care was free of charge, psychologists needed to build trust through repeated home visits to ensure referred children were cared for appropriately. Reducing stigma, misperceptions, and increasing awareness of child psychology among the community and health professionals remain a challenge to support community screening efforts and subsequent access to mental health care [43, 44, 45, 46]. The PSYCa 6–36 can also be means to raise awareness of child psychology among the population and of the importance of early recognition to limit long term and developmental consequences.

Conclusions

The PSYCa 6–36 allowed for rapid screening of psychological difficulties among children aged six to 36 months among the studies populations. Use of the tool also increased awareness of children’s psychological difficulties and the importance of early recognition to prevent long-term consequences. The PSYCa 6–36 would benefit from further use and validation studies in populations with higher prevalence of psychological difficulties.

Notes

Acknowledgments

We sincerely thank the communities who took part in this study as well as all the field study teams in Mathare, Kampong Cham and Mbarara. We also thank our colleagues from Epicentre (Paris and Mbarara) and from MSF-OCP (Paris, Nairobi, Kampong Cham, and Phnom Penh), especially Vastine Tayebwa, Emilie Sépulchre and Caroline Ponvert. We are grateful to the committee of international experts who supported the development of the PSYCa 6-36.

Funding

Funding for this study was provided by Médecins Sans Frontières - Operational Centre Paris. Funding covered all stages of the study: implementation, conduct, data collection and analysis and publication costs. Epicentre receives core funding from Médecins Sans Frontières.

Availability of data and materials

The data set supporting the conclusions of this article is available on request, in accordance with the data sharing policy of Médecins Sans Frontières (MSF) (Karunakara U, PLoS Med 2013). The MSF data sharing policy ensures that data will be available upon request to interested researchers while addressing all security, legal, and ethical concerns. All readers may contact Ms. Aminata Ndiaye (aminata.ndiaye@epicentre.msf.org) to request the data.

Authors’ contributions

PSYCa 6-36 tool development: CM, BF, MRM; Conceived and designed the experiments: FN, TR, CM, BF, MRM, RFG; Performed the experiments: FN, TR, SA, SM1, JMA, SM2, CS, VS, MP, PL, BWK, GK; Analysed the data: TR; Interpretation of the results: FN, TR, BF, MRM; Wrote the manuscript: FN; Revision of manuscript: FN, TR, CM, SA, SM1, JMA, SM2, CS, VS, MP, PL, BWK, GK, BF, MRM, RFG; Approval of the final manuscript: FN, TR, CM, SA, SM1, JMA, SM2, CS, VS, MP, PL, BWK, GK, BF, MRM, RFG. All authors have read and approved the manuscript.

Ethics approval and consent to participate

Ethical clearance was obtained from the French National Committee for the Protection of Persons (CPP Ile de France XI), the Ethics Review Committee of the Kenyan Medical Research Institute (KEMRI), the Cambodian National Ethics Committee for Health Research (NECHR), the Research Ethics Committee of the Mbarara University of Science and Technology (MUST-REC), and, the Uganda National Council for Science and Technology (UNCST). All participants’ caregivers provided written informed consent before participation.

Consent for publication

Not applicable.

Competing interests

The authors declare they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary material

12887_2019_1461_MOESM1_ESM.pdf (3.6 mb)
Additional file 1: PSYCa 6–36 and its guidelines for a standardized administration in English, French, Runyankore, Swahili, and Khmer (PDF 3666 kb)
12887_2019_1461_MOESM2_ESM.docx (30 kb)
Additional file 2: Score distribution, missing value, use of the example and Cronbach’s alpha per item of the PSYCa 6–36, cross cultural validation study, Kenya, Cambodia, Uganda. (DOCX 29 kb)
12887_2019_1461_MOESM3_ESM.pdf (618 kb)
Additional file 3: Frequency of positive responses (Sometimes/occasionally; Often/frequently) per item of the PSYCa 6–36 according to the CGIS score of the children (> 1 versus 1), cross cultural validation study, Kenya, Cambodia, Uganda. (PDF 617 kb)

References

  1. 1.
    Kieling C, Baker-Henningham H, Belfer M, Conti G, Ertem I, Omigbodun O, et al. Child and adolescent mental health worldwide: evidence for action. Lancet. 2011;378:1515–25.CrossRefGoogle Scholar
  2. 2.
    Baubet T, Taïeb O, Pradère J, Moro MR. Traumatismes psychiques dans la première enfance. Sémiologie, diagnostic et traitement. EMC-Psychiatrie. 2004;1:15–22.CrossRefGoogle Scholar
  3. 3.
    Skovgaard AM, Houmann T, Christiansen E, Landorph S, Jorgensen T, Olsen EM, et al. The prevalence of mental health problems in children 1(1/2) years of age - the Copenhagen child cohort 2000. J Child Psychol Psychiatry. 2007;48:62–70.CrossRefGoogle Scholar
  4. 4.
    Saxena S, Thornicroft G, Knapp M, Whiteford H. Resources for mental health: scarcity, inequity, and inefficiency. Lancet. 2007;370:878–89.CrossRefGoogle Scholar
  5. 5.
    Crabb J, Stewart RC, Kokota D, Masson N, Chabunya S, Krishnadas R. Attitudes towards mental illness in Malawi: a cross-sectional survey. BMC Public Health. 2012;12:541.CrossRefGoogle Scholar
  6. 6.
    Kapungwe A, Cooper S, Mwanza J, Mwape L, Sikwese A, Kakuma R, et al. Mental illness--stigma and discrimination in Zambia. Afr J Psychiatry (Johannesbg ). 2010;13:192–203.Google Scholar
  7. 7.
    Atilola O. Cross-cultural child and adolescent psychiatry research in developing countries. Glob Ment Health (Camb ). 2015;2:e5.CrossRefGoogle Scholar
  8. 8.
    Belfer ML. Child and adolescent mental disorders: the magnitude of the problem across the globe. J Child Psychol Psychiatry. 2008;49:226–36.CrossRefGoogle Scholar
  9. 9.
    Canino G, Alegria M. Psychiatric diagnosis - is it universal or relative to culture? J Child Psychol Psychiatry. 2008;49:237–50.CrossRefGoogle Scholar
  10. 10.
    Cortina MA, Sodha A, Fazel M, Ramchandani PG. Prevalence of child mental health problems in sub-Saharan Africa: a systematic review. Arch Pediatr Adolesc Med. 2012;166:276–81.CrossRefGoogle Scholar
  11. 11.
    Burns GL, Patterson DR. Factor structure of the Eyberg child behavior inventory: a parent rating scale of oppositional defiant behavior toward adults, inattentive behavior, and conduct problem behavior. J Clin Child Psychol. 2000;29:569–77.CrossRefGoogle Scholar
  12. 12.
    Robins DL, Casagrande K, Barton M, Chen CM, Dumont-Mathieu T, Fein D. Validation of the modified checklist for autism in toddlers, revised with follow-up (M-CHAT-R/F). Pediatrics. 2014;133:37–45.CrossRefGoogle Scholar
  13. 13.
    Achenbach TM, Rescorla L. Manual for the ASEBA preschool forms & profiles. Burlignton: University of Vermont, Research Center for Children, Youth, and Families; 2000.Google Scholar
  14. 14.
    Carter AS, Briggs-Gowan MJ, Jones SM, Little TD. The infant-toddler social and emotional assessment (ITSEA): factor structure, reliability, and validity. J Abnorm Child Psychol. 2003;31:495–514.CrossRefGoogle Scholar
  15. 15.
    Fullard W, McDevitt SC, Carey WB. Assessing temperament in one- to three-year-old children. J Pediatr Psychol. 1984;9:205–17.CrossRefGoogle Scholar
  16. 16.
    Goldsmith HH. Studying temperament via construction of the toddler behavior assessment questionnaire. Child Dev. 1996;67:218–35.CrossRefGoogle Scholar
  17. 17.
    Putnam SP, Gartstein MA, Rothbart MK. Measurement of fine-grained aspects of toddler temperament: the early childhood behavior questionnaire. Infant Behav Dev. 2006;29:386–401.CrossRefGoogle Scholar
  18. 18.
    Robinson EA, Eyberg SM, Ross W. Inventory of child problem behaviors: the standardization of an inventory of child conduct problem behaviors. J Clin Child Psychol. 1980;9:22–9.CrossRefGoogle Scholar
  19. 19.
    Mouton-Simien P, McCain AP, Kelley ML. The development of the toddler behavior screening inventory. J Abnorm Child Psychol. 1997;25:59–64.CrossRefGoogle Scholar
  20. 20.
    Briggs-Gowan MJ, Carter AS, Irwin JR, Wachtel K, Cicchetti DV. The brief infant-toddler social and emotional assessment: screening for social-emotional problems and delays in competence. J Pediatr Psychol. 2004;29:143–55.CrossRefGoogle Scholar
  21. 21.
    DeGangi GA, Poisson S, Sickel RZ, Wiener AS. The Infant-Toddler Symptom Checklist. Tucson: Therapy Skill Builders; 1995.Google Scholar
  22. 22.
    Marquer C, Barry C, Mouchenik Y, Hustache S, Djibo DM, Manzo ML, et al. A rapid screening tool for psychological distress in children 3-6years old: results of a validation study. BMC Psychiatry. 2012;12:170.CrossRefGoogle Scholar
  23. 23.
    Marquer C, Barry C, Mouchenik Y, Djibo DM, Manzo ML, Trujillo Maza EM, et al. Screening for psychological difficulties in young children in crisis: complementary cross-cultural validation. Int Health. 2015;7:438–46.CrossRefGoogle Scholar
  24. 24.
    Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health. 2005;8:94–104.CrossRefGoogle Scholar
  25. 25.
    Zero To Three/National Center for Infants. Diagnostic classification of mental health and developmental disorders of infancy and early childhood: DC: 0–3R. Washington, DC: Zero To Three/National Center for Infants; 2005.Google Scholar
  26. 26.
    Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334.CrossRefGoogle Scholar
  27. 27.
    Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–8.CrossRefGoogle Scholar
  28. 28.
    Cattell RB. The scree test for the number of factors. Multivariate Behav Res. 1966;1:245–76.CrossRefGoogle Scholar
  29. 29.
    Rouquette A, Falissard B. Sample size requirements for the internal validation of psychiatric scales. Int J Methods Psychiatr Res. 2011;20:235–49.CrossRefGoogle Scholar
  30. 30.
    Lance C, Butts M, Michels L. The sources of four commonly reported cutoff criteria what did they really say? Organ Res Methods. 2006;9:202–20.CrossRefGoogle Scholar
  31. 31.
    Nunnally J, Bernstein I. Psychometric theory. 3rd ed. New-York: McGraw-Hill; 1994.Google Scholar
  32. 32.
    Skovgaard AM, Houmann T, Landorph SL, Christiansen E. Assessment and classification of psychopathology in epidemiological research of children 0-3 years of age: a review of the literature. Eur Child Adolesc Psychiatry. 2004;13:337–46.CrossRefGoogle Scholar
  33. 33.
    Zeanah CH, Boris NW, Scheeringa MS. Psychopathology in infancy. J Child Psychol Psychiatry. 1997;38:81–99.CrossRefGoogle Scholar
  34. 34.
    Cha ES, Kim KH, Erlen JA. Translation of scales in cross-cultural research: issues and techniques. J Adv Nurs. 2007;58:386–95.CrossRefGoogle Scholar
  35. 35.
    Weeks A, Swerissen H, Belfrage J. Issues, challenges, and solutions in translating study instruments. Eval Rev. 2007;31:153–65.CrossRefGoogle Scholar
  36. 36.
    BICK E. Notes on infant observation in psycho-analytic training. Int J Psychoanal. 1964;45:558–66.PubMedGoogle Scholar
  37. 37.
    Alegria M, McGuire T. Rethinking a universal framework in the psychiatric symptom-disorder relationship. J Health Soc Behav. 2003;44:257–74.CrossRefGoogle Scholar
  38. 38.
    Angold A, Costello EJ. Nosology and measurement in child and adolescent psychiatry. J Child Psychol Psychiatry. 2009;50:9–15.CrossRefGoogle Scholar
  39. 39.
    Hackett R, Hackett L. Child psychiatry across cultures. Int Rev Psychiatry. 2009;11:225–35.CrossRefGoogle Scholar
  40. 40.
    Reitsma JB, Rutjes AW, Khan KS, Coomarasamy A, Bossuyt PM. A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. J Clin Epidemiol. 2009;62:797–806.CrossRefGoogle Scholar
  41. 41.
    Carter AS, Briggs-Gowan MJ, Davis NO. Assessment of young children's social-emotional development and psychopathology: recent advances and recommendations for practice. J Child Psychol Psychiatry. 2004;45:109–34.CrossRefGoogle Scholar
  42. 42.
    Egger HL, Angold A. Common emotional and behavioral disorders in preschool children: presentation, nosology, and epidemiology. J Child Psychol Psychiatry. 2006;47:313–37.CrossRefGoogle Scholar
  43. 43.
    Jorm AF. Mental health literacy. Public knowledge and beliefs about mental disorders. Br J Psychiatry. 2000;177:396–401.CrossRefGoogle Scholar
  44. 44.
    Patel V, Flisher AJ, Nikapota A, Malhotra S. Promoting child and adolescent mental health in low and middle income countries. J Child Psychol Psychiatry. 2008;49:313–34.CrossRefGoogle Scholar
  45. 45.
    Sartorius N. Stigma and mental health. Lancet. 2007;370:810–1.CrossRefGoogle Scholar
  46. 46.
    Volpe U, Mihai A, Jordanova V, Sartorius N. The pathways to mental healthcare worldwide: a systematic review. Curr Opin Psychiatry. 2015;28:299–306.CrossRefGoogle Scholar

Copyright information

© The Author(s). 2019

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors and Affiliations

  • Fabienne Nackers
    • 1
    Email author
  • Thomas Roederer
    • 1
  • Caroline Marquer
    • 1
  • Scholastic Ashaba
    • 2
  • Samuel Maling
    • 2
  • Juliet Mwanga-Amumpaire
    • 3
    • 4
  • Sothara Muny
    • 5
  • Chea Sokeo
    • 6
  • Vireak Shom
    • 6
  • Maria Palha
    • 6
  • Pauline Lefèbvre
    • 6
  • Beatrice W. Kirubi
    • 7
  • Grace Kamidigo
    • 7
  • Bruno Falissard
    • 8
  • Marie-Rose Moro
    • 8
    • 9
    • 10
  • Rebecca F. Grais
    • 1
  1. 1.EpicentreParisFrance
  2. 2.Department of Psychiatry, Faculty of MedicineMbarara University of Science and TechnologyMbararaUganda
  3. 3.Epicentre, Mbarara Research CentreMbararaUganda
  4. 4.Mbarara University of Science and TechnologyMbararaUganda
  5. 5.Medicine DepartmentPreah Kossamak Hospital, Ministry of HealthPhnom PenCambodia
  6. 6.Médecins Sans FrontièresPhnom PenCambodia
  7. 7.Médecins Sans FrontièresNairobiKenya
  8. 8.Centre de recherche en épidémiologie et santé des populations (CESP)/ Institut national de la santé et de la recherche médicale (INSERM) U1018, Maison de SolennParisFrance
  9. 9.Médecins Sans FrontièresParisFrance
  10. 10.Université Paris Descartes, Sorbonne Paris Cité, Hôpital Cochin, Assistance Publique Hôpitaux de ParisParisFrance

Personalised recommendations