Introduction

The treatment of cleft lip and/or cleft palate (CL/P) consists of reconstructive plastic surgery, and when necessary, treatment with a speech and language pathologist (SLP); an ear, nose and throat specialist; orthodontist; and psychologist or psychotherapist. Most CL/P-centres have standardized treatment protocols for treatment indication and timing. However, different countries and health care centres have different health care resources. Access to some specialities is therefore limited or non-existing. In particular, access to a psychologist is limited at many centres. In these cases, evaluation of psychological aspects of care by other clinicians with an easily administered test can be helpful so that the limited resources are used in the best way possible. There has, however, been a lack of a standardized protocol for detection of and indication for psychological treatment [1]. A plausible reason for this is the non-existence of a widely accepted patient-reported outcome (PRO) measure to detect possible distress experienced by patients.

PRO results in patients with CL/P and clinician/objective rating

Earlier research has demonstrated that the association is weak between professional or laymen rating and patient satisfaction with nasal appearance or QoL [2, 3], speech [4, 5] or the level of self-reported social adjustment, satisfaction with appearance or cleft type [6,7,8]. A surgeon’s judgement of a result is not always synonymous with patient assessment [9].

Patient satisfaction with hearing, appearance and speech and self-reported psychological functioning

A systematic review of effects of CL/P on psychosocial health reported that (although conflicting findings exist) patients with CL/P are in general satisfied with their appearance and speech. Overall adjustment and psychological functioning also appear reasonably good [10]. The same author, however, in a further study highlighted that self-reported psychosocial functioning in patients with CL/P was worse than in non-cleft controls, when analysed in separate domains of depressive symptoms, behavioural problems and satisfaction with appearance and speech [11]. Further studies have suggested that self-reported psychosocial function significantly correlates with satisfaction with appearance [7, 12,13,14] and speech [4, 15,16,17]. Hearing difficulties have also been associated with social impairment [17]. Qualitative studies with people with CL/P have identified that one major theme regarding psychosocial well-being was the satisfaction with the appearance of the face [18,19,20].

Challenges in using PRO

An obstacle associated with PRO analysis in research regarding clefts is the lack of a widely accepted measurement instrument. In a review of current PRO QOL studies, it was found that most questionnaires used for studying PRO had been used in one study only [1]. Another review of patient based outcome revealed that almost all disease-specific measures were ad hoc measures produced for a particular study. Neither did they meet scientific standards of reliability, validity and responsiveness [21, 22]. Study results are therefore difficult to compare, to draw conclusions from and to build upon [17].

Opinions from professionals on implementing PRO

Earlier studies on implementation of PRO in aesthetic plastic surgery, oncology and orthopaedic surgery have described challenges such as lack of time, higher costs, having to change clinic infrastructure, having to sift through large amounts of data and difficulty in accessing data in electrical medical records [23,24,25]. Health care professionals also reported that patients were frustrated if they had taken time to fill out PROMs but the health care professional did not have time to review the data or address their concerns [24]. In general, however, health care professionals felt that PRO-data was valuable to them, made them aware of additional symptoms and patients satisfaction and gave them a starting point to talk about difficult-to-discuss symptoms [23, 24].

PRO measure—CHASQ

The PRO measure used in this study was CHASQ. It is a modified version of the Satisfaction with Appearance questionnaire (SWA) (Cleft Psychology Clinical Excellence Network [26]). SWA was designed by the Cleft Psychology Special Interest Group, Craniofacial Society of Great Britain and Ireland specifically for patients with facial disfigurement [27]. SWA and CHASQ have satisfactory internal validity, construct validity and overall adequate psychometric properties and have been used in clinical research in Europe [2, 3, 6, 7, 14, 28,29,30].

CHASQ consists of nine items regarding features typically affected by a cleft (factor 1 loading items) and six items regarding features not typically affected by a cleft (factor 2 loading items). Therefore, CHASQ produces two scores, sum of factor 1 loading items (total features 1) and sum of factor 2 loading items (total features 2). The score for each item ranges from zero to ten points. In this study, these two scores will be presented separately. An example of an item from the CHASQ is presented in Fig. 1. Norms have been calculated on 867 cleft patients from the UK to aid interpretation of the scores from CHASQ. Scores in the bottom 15% indicate lower satisfaction than expected, and scores in the bottom 5% indicate very much lower satisfaction than expected (Cleft Psychology Clinical Excellence Network [26]). For the total of features 1, the 15th percentile cut-off is approximately 50 points and the 5th percentile cut-off is approximately 34 points. For the total features 2, the 15th percentile cut-off is approximately 44 points and the 5th percentile cut-off is approximately 36 points.

Fig. 1
figure 1

An example of an item from the CHASQ

Full size image

Summary

It is of concern that often no routine psychosocial assessment is carried out, that patients rarely discuss psychosocial issues and that patient and clinician judgement of appearance do not correlate. These conditions mean that often a decision about treatment is made by a clinician based only on an assumption of the patient opinion. In addition, the decision is taken without information concerning the psychosocial situation of the patient, which for some patients is the most difficult part of having a cleft [31]. Therefore, it is essential for a cleft team to include assessments of PRO and psychosocial outcome into standard practice of care [32]. To enable cleft teams to learn as much as possible from a wide variety of experience, it is desirable for cleft teams to use an internationally accepted outcome measure [1, 7, 17, 33, 34].

The objective of this study was to:

  1. 1.

    Translate CHASQ into the following languages: Bulgarian, Estonian, Greek, Latvian, Macedonian, Romanian, Serbian and Swedish

  2. 2.

    Investigate whether scores were comparable across countries

  3. 3.

    Investigate how clinicians experienced implementation of the instrument

Method

Study design

The design of the study was decided within the European Cooperation for Science and Technology (COST) Action IS1210, Appearance Matters - Tackling the physical and psychosocial consequences of dissatisfaction with appearance [35]. The first author gained approval from the local Board of Research Ethics for the design of the study and co-authors followed the principles outlined in the Declaration of Helsinki.

Translation

Via the platform COST Action Appearance Matters, delegates from participating cleft care centres in Bulgaria, Estonia, Cyprus, Latvia, Macedonia, Romania, Serbia and Sweden translated the CHASQ. The translation was carried out according to MAPI guidelines for linguistic validation of a patient reported outcome measure (Mapi Research Institute [36]). The guidelines include translation by two individuals, separately, into the target language and reconciliation of the two versions into one draft. Then, backward translation to the source language is completed by a third individual. Comparison of the backward translation with the original version in the source language is carried out. Finally, patient testing with 5 patients in each language is performed before reaching the final version of the target language questionnaire. During the translation process, cultural differences in expressions about appearance were discussed.

Data collection and analysis of quantitative data

CHASQ was implemented on patients at routine visits to clinics. Data was coded locally before statistical analysis was performed on the joint international population. All calculations were made with IBM SPSS Statistics 22.0. Differences in CHASQ scores between boys and girls were calculated with Mann–Whitney U test. Differences in CHASQ scores between countries and diagnoses were calculated using Kruskal–Wallis test. Spearman’s test was used to calculate correlations between non-parametric data. Covariance was analysed with ranked ANCOVA since the residuals in non-ranked ANCOVA were not normally distributed. Differences in distribution of diagnosis between different countries were investigated with descriptive statistics.

Data collection and analysis of qualitative data

Sixteen health care professionals from all countries, except Cyprus, involved in the collection of CHASQ data participated in a focus group discussion regarding their participation in the Cost Action Appearance Matters including the strengths and challenges in the use of CHASQ. The professionals working with patients with clefts represented in the discussion were surgeons and orthodontists. The discussion was tape recorded, and the data was analysed in accordance with the principles of thematic analysis [37] by co-authors FZ and NS. In this process, themes in the focus group discussion were identified and presented to illustrate the experiences related to the studied phenomenon. Quotes are presented with respective themes to illustrate the participant’s experience. Themes and quotes were checked with the professionals who participated in the study.

Results

Quantitative results

Translated versions of CHASQ are available for further research and clinical use in the languages mentioned above. CHASQ in the source language, English, and all the translated versions can be found in the Appendix. The total number of patients participating was 220. The score of total features 1 ranged from six to 90 points (mean 65 points, 90 points was the highest possible score). High scores indicated high satisfaction. Kruskal–Wallis test for differences between groups of non-parametric data showed a statistically significant difference between countries of the score of total features 1 (p = 0.001). Kruskal–Wallis test also showed that the distribution of age was statistically different between countries (p = 0.001). Difference in the score of total features 1 between countries remained significant after testing for covariance with age (p = 0.001). Hence, the difference in the score of total features 1 between countries was not solely explained by differences in age of the patients from different countries (see Table 1).

Table 1 Total features 1 and 2 by country
Full size table

Higher age correlated significantly with a lower score of total features 1 in Spearman’s test (p = 0.001, r = − 0.219). In total, 48 patients (22%) scored 50 points or less on total features 1. This indicates lower satisfaction than expected according to British norms of CHASQ (Cleft Psychology Clinical Excellence Network [26]). All countries had patients with scores in this range. Twelve patients (6%) scored 34 points or less, which indicates very much lower satisfaction than expected. Six out of the eight countries had patients with scores in this range.

The score of total features 1 was not statistically different between males and females in Mann–Whitney U test (p = 0,431). The score of total features 1 was significantly different between different groups of cleft type in Kruskal–Wallis test (p = 0.001). The same test also showed that the distribution of age was not statistically different between groups of diagnosis (p = 0.289). The mean of the score of total features 1 for patients with cleft palate only was 75 points, for cleft lip with or without cleft alveolus this score was 69 points and for cleft lip and palate it was 60 points (see Fig. 2 and Table 2).

Fig. 2
figure 2

Boxplot of total features 1 by cleft type

Full size image
Table 2 Total features 1 and 2 by cleft type
Full size table

The score of total features 2 ranged from 18 to 60 points (mean 54 points, 60 points was the highest possible score). High scores indicated high satisfaction. Kruskal–Wallis test for differences between groups of non-parametric data showed a statistically significant difference between countries of the score of total features 2 (p = 0.007). The difference in the score of total features 2 between countries remained significant after testing for covariance with age (p = 0.031). Hence, the difference in the score of total features 2 between countries was not solely explained by differences in age of the patients from different countries (see Table 1).

Higher age correlated significantly with a lower score of total features 2 (p = 0.002, r = − 0.208) in Spearman’s test. In total, 23 patients (11%) scored 44 points or less on total features 2. This indicates lower satisfaction than expected according to British norms of CHASQ (Cleft Psychology Clinical Excellence Network [26]). All countries had patients with scores in this range. Four patients (2%) scored 36 points or less, which indicates very much lower satisfaction than expected. Four out of the eight countries had patients with scores in this range.

The score of total features 2 was not statistically different between males and females in Mann–Whitney U test (p = 0,976). The score of total features 2 was not significantly different depending on cleft type in Kruskal–Wallis test (p = 0.562) (see Table 2).

Qualitative results

Themes covered in the focus group discussion were Clinical utility, Broadening the role as caregiver and Implementation in practice.

Clinical utility

Many health care professionals using the CHASQ in their clinics reported that it helped them gain patient information and informed on treatment decisions. It was used during consultation to spark conversation about patient subjective satisfaction and suggest alternative pathways of referral.

“[The CHASQ] is a filter, done by everyone in the team, not just the psychologist. We [use it to] make higher, medium and low risk groups… the cases [high risk cases] are directed to the psychologist.” (Surgeon)

In some centres, however, limited resources or limited systems of referral inhibited the benefit of CHASQ implementation.

“It’s very important if we figure out from the CHASQ that there is a problem, we have a support person to refer [to]. When it’s just staying on a questionnaire, then we have a problem, patients cannot get real help, so you should have some sort of support system.” (Orthodontist)

Broadening the role as caregiver

Some health care professionals were already aware and caring for the psychosocial needs of their patients. Others, however, reflected that working with CHASQ and COST Action Appearance Matters had made them more aware of this aspect of care. It had also given them a new perspective on their role in the cleft team.

“After being in this project, I started to see my role as a surgeon to be more global, to see the patient from a different way, see the patient, not only from the surgical part. That helps me to use [the CHASQ] for me.” (Surgeon)

Implementation in practice

Both strengths and challenges were expressed in relation to CHASQ implementation. CHASQ was perceived as short and easy to use which was perceived as a strength of the questionnaire.

“The CHASQ is really easy and fast. At our centre everyone is keen [to use psychosocial measures] but then say “Ok, we have to do this, but we need to take something else away” because they already work 100% with it. And therefore the CHASQ I think is a lot easier to implement than another big instrument. There is a small amount of time and we don’t get more resources because we want to do this.” (Surgeon)

Issues related to limited time and resources were expressed. Even though the questionnaire is short and relatively easy to explain to patients, this process and discussions surrounding it were perceived as a burden. This was especially clear when this process was not prepared for in the time allotted for the consultation.

“Extra time is needed for the explanation for the questionnaire.” (Orthodontist)

Discussion

Both quantitative and qualitative research methods were used to investigate the aims of this study. This mixed methods research (MMR) approach and parallel mixed design combine the results from quantitative and qualitative data to answer the same basic research question that is not easily investigated by any of the methods alone [38]. The two perspectives are needed in drawing conclusions about how fit the instrument is for use, both locally and internationally. Both aspects are important since there is a need for larger cross-cultural studies [17], and such studies are easier to carry out if an instrument is well accepted and perceived as clinically useful [9].

The translation into 8 languages and utilization of the same questionnaire facilitated valuable international cooperation and comparison in this study and future studies. Some issues with the cultural adaptation were encountered. Clinicians experienced different levels of comfort in presenting questions about appearance to their patients. For patients to rate their own appearance seemed to be a sensitive subject in general in some countries more than others. This issue is not easily tackled since a change of the questionnaire, changing or taking out sensitive questions, would change the score of the questionnaire and counteract the aim of international comparison with the same instrument. Other translation processes of PRO-instruments for patients with a cleft have encountered similar issues [39]. The patient testing in the linguistic translation process of CHASQ, however, did not reveal any problems with or upset feelings about of any the items in the instrument. All items were kept in all different languages in CHASQ. CHASQ has also been translated to Vietnamese through another study related to the COST Action Appearance Matters [40].

Results show some variation in CHASQ scores between countries. Tests showed a statistically significant difference in patient scores between different countries despite taking age into account. It is not clear how much of these variations were influenced by differences in quality of the end result of cleft care. There is a risk that linguistic and cultural differences in translation could have influenced the results although translation followed MAPI guidelines (Mapi Research Institute [36]). In spite of different levels of satisfaction between countries, there were patients in all countries who were less satisfied than expected and who should be identified for further investigation or treatment. As also seen in the focus group discussion, clinicians in many different countries expressed the usefulness of the CHASQ for assessing patients. CHASQ can consequently be a useful instrument for collection of PRO on satisfaction with hearing, appearance and speech (Cleft Psychology Clinical Excellence Network [26]).

The CHASQ was perceived as useful, short and easy to use by the clinicians involved in the study. This is favourable since the simpler the instrument is to use, the more clinicians will be able to fit it in their work routine [9]. Consequently, more data from numerous sources will be available for research. The introduction of CHASQ into clinical routine also seemed to enhance the process of thinking and treating patients in a holistic fashion.

There were no differences between the scores of CHASQ of girls and boys in this study. This is in contrast to earlier studies [2, 29, 30, 41]. There were however statistical differences between cleft types. The more visible the cleft, the less satisfied the participants reported themselves to be. This is in line with some earlier research [7, 29, 41].

An earlier study found that clinicians may avoid asking patients about their psychosocial health or suggesting treatment in an effort not to stigmatize the patient [42]. Some clinicians found some items in the CHASQ ethically questionable; this concern was however not reflected in the patient testing phase of the linguistic and cultural translations of the instrument. On the contrary, patients with various visible anomalies have in previous research expressed the desire for access to an appropriately trained professional to support them in their efforts to deal positively with their appearance-related concerns [43]. Also, parents have been found to be hesitant in asking their children about psychosocial issues such as bullying, being afraid that it would make the matter worse [19]. The barrier to ask about psychosocial health in both health care professionals and parents of children and teenagers with cleft make it even more important to support the process of investigating psychosocial well-being. CHASQ can be used as such support as an icebreaker. The use of a PROM as a starting point on difficult-to-discuss topics has been described in earlier research [24].

Surgical techniques are evolving, and aesthetic and speech results are improving. The objective of the treatment, however, also includes high quality of life for patients [44]. As stated by Richman 1983 “…most of those with concerns regarding facial appearance continue to expect or hope for facial surgery, creating a continued dissatisfaction with self. This interpretation stresses the need for early planning and consultation regarding potential facial surgeries. It suggests the importance of careful and accurate information regarding whether further facial surgery will occur and, if so, what the cosmetic expectations should be.” [12] In treating dissatisfaction with appearance, CHASQ may be used during consultation to spark conversation between clinicians, patients and families about patient subjective satisfaction, realistic expectations from surgery and also to suggest alternative pathways of referral. A clear alternative pathway of referral instead of surgery to treat disappointment in appearance could be useful for this group of patients. Results from the focus group discussion support an earlier statement that a questionnaire that is perceived as clinically useful is more likely to be used in the clinic as well as in studies and trials [9]. As can be seen in the results of this study, CHASQ has already been used to initiate discussion and inform on treatment decisions.

Limitations of this study have been identified. The sampling of patients varied between countries. Some countries sampled patients from regular follow-ups at certain age intervals. Other countries only had appointments with patients in the selected age group who needed further treatment and therefore sampled this patient group for the study. These differences in sampling were hypothesized to result in lower scores from countries that sampled patients who needed further treatment [41]. Results did, however, not support this hypothesis. Varying treatment protocol regarding pre surgical orthodontic treatment and surgery was used in the different countries. Comparison between countries was not matched with regard to treatment protocol. Local treatment protocols are presented in Table 3. Further, no objective comparison of patients was performed to validate the differences in score hypothetically derived from differences in end results.

Table 3 Treatment protocol used by participating countries for included participants
Full size table

Conclusion

The CHASQ has been translated into 8 languages and can be used in these languages for clinical work and research regarding patient reported outcomes in hearing, appearance and speech. This study showed different levels of satisfaction between countries. There were patients in all countries who were less satisfied than expected and who should be identified for further investigation or treatment. In the focus group discussion in this study, clinicians in many different countries expressed the usefulness of the CHASQ in their clinical work. It was perceived as useful, short and easy to use. CHASQ can be a useful instrument for collection of patient reported outcomes on satisfaction with hearing, appearance and speech.