Introduction

The importance of consumer and carer involvement in mental health research is well established [1,2,3,4,5,6,7,8]. In Australia, the Statement on Consumer and Community Participation in Health and Medical Research outlined the importance of consumers and the community playing an active role in health and medical research [9]. Growing acceptance of the mental health recovery movement creates a solid foundation for consumers and carers to move beyond tokenistic or advisory modes of participation and into meaningful and effective involvement [10].

Internationally, these efforts have increased the consumer and carer voice in priority-setting [7]. For example, the James Lind Alliance initiative (based in the United Kingdom) and the Roadmap for Mental Health Research in Europe project have included consumers, carers and other stakeholders in large-scale priority-setting exercises for research into specific disorders and mental health more generally [11, 12]. However, published examples of consumers and carers defining the research agenda and actively participating in research in Australia are limited [10]. Although the research community uses a number of priority-setting methods, consumer and carer perspectives remain largely absent in mental health research priority-setting [3]. Given their unique and valuable perspectives in identifying areas and issues of emerging importance [3], or those that have otherwise been overlooked [2], consumers and carers have much to contribute with significant capacity for leadership in this space [13].

ACACIA – The Australian Capital Territory (ACT) Consumer and Carer Mental Health Research Unit – at The Australian National University (ANU), was established in response to the need for consumer- and carer-led collaborative research. ACACIA aims to enable consumers and carers to take an active role in relevant, high quality mental health research [14]. The Unit is led and staffed by researchers with lived experience as a consumer or carer who facilitate the engagement of consumers and carers from the community in the work of the Unit, and bridge the often difficult gap between academia and mental health communities [6].

A key objective of ACACIA is the collaborative development of a research agenda to address consumer- and carer-identified issues, such as service gaps in the ACT and Australia more broadly. This paper reports the findings from two consecutive studies to address this objective, namely a face-to-face discussion forum to develop topics and set initial priorities (Study 1) and an online survey to extend and update priorities (Study 2).

The ethical aspects of the research were approved by the ANU Human Research Ethics Committee (protocol number 2013/388). All participants provided written or online informed consent.

Study 1: discussion forum

Method

Participants

Participants who self-identified as consumers and/or carers were invited to participate, recruited via advertisements distributed through the mailing lists of ACT consumer and carer networks. Advertisements were also emailed to ACT members of the Depression and Anxiety Consumer Research Unit Register, a database of people who have expressed an interest in research conducted at the ANU Centre for Mental Health Research (CMHR).

Procedure

A face-to-face discussion forum was held in November 2013. Consistent with ACACIA’s commitment to consumer and carer leadership, all those involved in running the forum had lived experience of mental health issues. A local well-known consumer advocate facilitated overall proceedings, and small group discussions were facilitated by ACACIA staff (n = 2) or ACACIA Consumer and Carer Advisory Group members (n = 2). Small groups of 6–8 participants selected their own seats at one of four tables, each of which had a note-taker and a facilitator present. Facilitators assisted groups to express and formulate their ideas into topic areas that could be researched.

The detailed protocol for the discussion forum is provided in Additional file 1. The forum priority-setting comprised three parts, as follows: (1) identifying broad research areas within discussion groups, (2) refining the ideas into specific topics and questions, enabling participants to participate in the early stages of a modified thematic analysis [15], and (3) prioritising the research topics. Researchers and Advisory Group members collated the topics and questions into broad thematic areas by consensus, using an inductive approach [16]; themes were displayed on flipcharts in the forum venue for the prioritisation. Priorities were identified using a “dot-mocracy” [17] process. Each participant was provided with five coloured adhesive dots. The colour of the dot indicated which group – consumer, carer, or people who identified as both (hereafter called consumer/carers) – the participant self-identified with. Each dot represented one vote for a topic or thematic area. Participants were free to distribute their dots across as many or few topics as they wished.

Analysis

As described above, inductive thematic analyses [15, 16] were an integral part of the forum proceedings. Thematic analysis methods [13] were adapted to facilitate the development of topics for future research. Instead of developing descriptive codes, forum participants developed and refined research topics from the content of their discussions. These topics were collated and assigned to themes by the consumer and carer researchers, assisted by Advisory Group members. Results comprise the number of votes each topic or thematic area received from consumers, carers and consumer/carers. Further thematic analyses were conducted by researchers after the forum to produce a final list of research themes. The findings from the forum were circulated to participants for feedback. No further refinements were made.

Results

Participants

A total of 25 people (17 female, 8 male) with lived experience as a consumer and/or carer attended the forum. Of these participants, 14 who identified as consumers, five as carers and five consumer/carers participated in the priority-setting exercise (one consumer participant left before the exercise was conducted). No further demographic information was collected from forum participants.

Research priorities

Seventy-nine topics for research in 14 broad thematic areas were developed by participants and collated for prioritisation at the forum. The themes were services; treatment; medication; health professionals; comorbidity and physical health; justice; consumer and carer involvement; stigma; experiences of care; carers, family and friends; National Disability Insurance Scheme (NDIS); language and communication; peer to peer; and legislation. Additionally, a list of individual ‘other’ topics (e.g. recovery and fulfilling potential, employment) was compiled. The available 120 votes (70 from consumers, 25 from carers and 25 from consumer/carers) were broadly distributed across 59 of the 79 topics (Additional file 2).

Research on trauma and service delivery was highly rated: ‘the integration of trauma-informed care into service delivery’ was considered important by all three participant groups and received the highest overall number of votes (n = 7), and ‘is care traumatising’, received 5 votes (3 from consumers and 2 from carers). ‘Peer-led services’ and ‘recovery and fulfilling potential’ were also considered important, each receiving six votes.

There were some differences in the focus of research priorities between consumers, carers and consumer/carers. Whilst consumer/carer votes were distributed widely across topics, the votes for research into ‘peer-led services’ and ‘recovery and fulfilling potential’ were primarily from consumers, and no participants identifying solely as a carer voted for these topics. Consumers were the only group to vote for research into the effects of ‘stigma’, ‘human rights legislation’ and ‘learned helplessness in response to experience of services’. Carers focused on research related to ‘interaction with health professionals’ and the ‘care experience’, including the influence of ‘privacy’ on care, as well as the effects of ‘drug and alcohol use’.

Study 2: priority-setting survey

Method

To update the research agenda developed in 2013 and extend participation nationally, a second study of research priority-setting, comprising an online survey, was undertaken in 2017. The methods used were based on a previous survey investigating priorities for depression and bipolar disorder research conducted by the lead author [2].

Participants

To recruit participants for the online survey, advertisement targets used for the discussion forum were expanded to include both national and state-based health consumer and carer organisations. Advertisements were also disseminated through the Lived Experience Research register (a database of consumers and carers who have expressed interest in participating in ACACIA research), the CMHR website and the CMHR social media accounts. Recruitment flyers were distributed at community events during National Mental Health Week. Survey participants were required to be at least 18 years old and live in Australia.

Procedure

The survey was conducted over 8 weeks in September and October 2017. Survey items were developed from the research topics formulated by participants at the forum, specifically taking care to preserve original wording. Additional items that were not well-represented by the original topics presented to forum participants were developed from the detailed notes on the small group discussions in Study 1. Eighty-seven items were included in the survey, each representing a single topic. Items were presented within the 14 broad thematic areas that were identified by researchers at the forum. Demographic information (age, sex, identification as a consumer and/or carer, state/territory of residence) was collected at the beginning of the survey. A copy of the survey is provided in Additional file 3.

Survey participants were asked to rate the priority of each research topic on a 5-point Likert scale (1 = Very low priority to 5 = Very high priority). All items an individual participant rated as of very high priority (5) were collated by the survey software. Participants were then asked to rate all items they designated as high priority in order of relative priority using a ‘drag and drop’ process. To encourage variation in an individual’s ratings and create a manageable list to rank, participants were informed about the ranking process before commencing the survey, and reminded at the top of each page that anything rated as ‘very high priority’ would appear in the ranking list. However, feedback provided in the comments section indicated that the ranking process proved too difficult for most participants, raising serious concerns about the validity of the relative rankings. Results are therefore restricted to analysis of the ratings data as described in the next section.

Participants were also invited to suggest new research topics or provide comments in open-ended questions at the end of each theme and after the ranking exercise.

Analysis

Ratings

Due to the large number of survey items and relatively low number of participants, inferential statistics were not used to compare ratings between groups. A descriptive analysis was performed, examining the distribution of ratings and comparing the content of the highest-rated items. All quantitative analyses were performed in IBM SPSS Statistics 25.

Missing data for each item ranged from 1.4% to 20% and progressively increased across the survey. The order of survey items was not randomised and items presented later in the survey appear to have been impacted by participant fatigue. Due to the pattern of missing data, participants were included in descriptive analyses if they had completed any of the rating items. No participants or variables were excluded from the analysis based on missingness.

The distribution of responses to individual rating items were visually examined and a majority of items were found to be negatively skewed. Based on this observation, participant ratings were dichotomised [18] as follows: important (combining very high (5) and high (4) priority ratings) versus all other (combining moderate (3), low (2) and very low (1) priority ratings). For each participant group (consumers, carers and consumer/carers), the percentage of participants whose ratings were in the ‘important’ category for each item was calculated. To address missingness and drop-out across the survey, a valid percentage was calculated based on the number of participants who completed an item, not the total number of participants. Items were then ranked within participant groups in descending order of the percentage of ‘important’ ratings. Tied rankings were assigned the mean rank [19]. The distribution of ‘important’ ratings across items and groups was examined for consensus on clear ‘top’ priorities or cut-off points for the highest priorities for research.

Open-ended responses

Qualitative analyses of the survey comment data were conducted by one author (ARM) and managed using QSR International’s NVivo 11 Software. Participants’ open-ended responses were examined using a framework analysis approach [20]. The coding framework was developed from the key areas for research identified in the descriptive quantitative analysis. Carer participants provided written responses more frequently than consumer and consumer/carer participants; thus, the framework analysis findings may favour carer priorities and concerns. Qualitative findings are incorporated with quantitative findings to provide additional detail.

Results

Participants

Table 1 presents the demographic data for survey participants. A total of 70 consumers and/or carers participated in the online survey, including 37 consumers, 12 carers and 21 consumer/carers. The mean age of participants was 46.3 years (SD 15.8, range 21–83). All Australian states and territories were represented in the sample, although the majority of participants were from the ACT, New South Wales and Victoria.

Table 1 Survey demographic data
Full size table

Research priorities

Table 2 presents the percentage of ‘important’ ratings for each item by participant group. Items were ranked within participant groups based on relative percentage (i.e. 1 = highest percentage of ‘important’ ratings; 87 = lowest percentage). No consensus on clear ‘top’ priorities for research was observed. Most items were rated as important by at least 50% of consumers (81 items), carers (60 items) and consumer/carers (68 items). A small number of items were rated as important by all, or almost all, participants within a group. However, the percentage of ‘important’ ratings gradually decreased across items (Table 2) with no obvious cut-off point for top research priorities.

Table 2 Research priority ratings for consumers, carers and consumer/carers
Full size table

Participants’ open-ended responses further emphasised the ongoing need for research and evaluation across a broad range of topics, reflecting on the extent of perceived problems with the system.

“I hate to say this but the list [of research priorities] touches directly on most of our carers and consumers involved in the mental health system. Your list shows just how far we have to go to have a first class mental health system…” (Carer 1)

In the absence of clear top priorities, to develop initial key areas for future research, a pragmatic decision was taken to examine items rated as important by approximately 80% or more of participants within a group. Across groups, this demonstrated a focus on research topics related to the delivery of services. Participants rated topics about the quality of services, problems with services (including reach, stigma and trauma) and how services impact on consumers and carers as priority areas for future research. The NDIS (ndis.gov.au) was rated as an important topic for all groups. The NDIS is a national Australian programme that was trialled in a number of local sites from 2013, and began a national implementation process in 2016. The programme provides government-funded flexible packages of care to Australians with permanent and significant disability, including psychosocial disability. NDIS-related research topics focused on the impact of the ‘definition of psychosocial disability’ and ‘supports for people who fall outside the scope’ of the scheme.

Open-ended responses also had a substantial focus on the problems with services. Participants shared concerns about under-resourced, under-staffed services that could be difficult to access, poorly coordinated and poorly implemented.

“Where I live … it is really difficult to get a care plan in either the hospital or community setting as the staff do not see the value and feel like they haven't got enough time to do things including writing and reading case notes.” (Consumer/Carer 1)

Concerns were also raised about the mental health training of health professionals, particularly General Practitioners, whether the current approaches of health professionals to diagnosis and care were appropriate, and gaps in services, including gaps created by the implementation of the NDIS.

Alongside the focus on problems with services, some participants also highlighted the negative impacts of services on consumers and carers. Participants commented on the negative consequences of compulsory treatment and inadequate, inappropriate or absent services.

“I was also treated in quite a paternalistic and demeaning way by [the crisis response] team, hospital staff and other staff around these issues as I was not a family member. The feeling of seeing someone you care for being treated in a completely degrading manner but also realising that no one cares about your voice is very horrible.” (Carer 2)

Consumer and carer involvement was also a priority for research, with interest in ‘how involvement works in practice’ and ‘how to expand who is involved’ rated highly, particularly by consumers. Comments highlighted the role of active consumer and carer involvement in improving the quality of services and the education of mental health professionals (including General Practitioners and psychologists). Participants were interested in determining how to encourage diversity in the consumers and carers involved in shaping services, policy and legislation.

“Lived experience will really matter to shaping better services, and making sure it's a good cross section of people from various socio-economic backgrounds, genders, ethnicity etc.” (Carer 2)

Other topics frequently rated as important across the three groups included the ‘over-representation of mental illness in the justice system’, ‘social inclusion’ and ‘employment’. A small number of participants commented on these topics, emphasising the importance of ensuring people have access to basic needs, including accommodation, vocational activities and social support services, and that they received appropriate mental health care if detained in the justice system.

Despite an overall focus on delivery of services, there were some group differences on the specific topics of interest. Consumers frequently rated the implementation and consumer experiences of ‘peer services’ as important topics for research, with four of their top topics on peer-to-peer services. Carers most frequently prioritised ‘carer support services’, the ‘reach’ of services and ‘continuous care and support for suicide’. By contrast, consumer/carer participants focused on the organisation of care, including ‘care coordination’, ‘care planning’ and ‘transparency of services’. They also prioritised ‘carer recovery’.

Discussion

The current study identified a broad range of topics for future mental health research, reflecting the extent of the perceived gaps in the Australian mental health sector. Although there were some group differences and no clear priorities emerged in either research study, many topics amongst the highest-rated were in the area of service organisation and delivery, particularly related to trauma- and recovery-oriented care and peer leadership. The importance of lived experience of mental health issues (as a consumer and/or carer) for the development and evaluation of services and policy was also a focus, particularly for survey participants, and across both studies there was interest in research into recovery for consumers and carers.

This research both confirms and extends the outcomes of previous consumer and carer mental health research priority-setting exercises. Many research priorities developed by participants in Study 1, such as support for transitions between services, medication and alternative treatments [7, 21], stigma [8, 21], support for carers [21], and communication with health professionals [21] are consistent with prior research conducted in other countries. The topic areas of focus, including recovery [21], service delivery [21] and the active involvement of consumers have also been found previously [8]. The findings are similarly consistent with previous Australian research priority-setting work, in which consumers prioritised research into medication and treatment, effective coping strategies/recovery and quality services [2, 3]. However, the topics developed and prioritised across both studies in the current project reflect specific contemporary issues in these areas. For example, many participants were aware of international evidence for mental health peer work and the spread of peer services in Australia; therefore, their suggested topics were focused on implementation and evaluation of best practice in the Australian context, including peer leadership in service delivery. The strong interest by participants in the effect of the recently introduced NDIS, particularly regarding how it affects service availability for those ineligible for the scheme, is also a critical area of concern in the Australian system, although the system-level analyses of interest may also have relevance for other countries in Europe and North America that have introduced similar broad-ranging changes to the financing and organisation of care for people with long-term disabilities [22].

The contemporary and nuanced views on research priorities demonstrated by consumers and carers, along with the changes in emphasis that occurred in the 4 years between studies, highlight the importance of regular updates to priorities that are not confined to rating pre-existing researcher-developed topics. For example, in Study 1 of the current research, topics focused on trauma were discussed widely across groups and rated highly by both consumers and carers, albeit by a small number of votes overall. Trauma-informed care refers to mental health care which acknowledges the impact of previous psychosocial violence, abuse and trauma on the consumer and is sensitive to these experiences [23]. However, trauma-related service delivery was not amongst the highest priority topics in the national survey conducted four years after the first study. This may reflect differences in focus across communities. For example, there may have been a pre-existing emphasis on trauma-informed care within the local ACT consumer network from which participants were recruited for Study 1. Alternatively, this may reflect a recent broader shift in the focus of both service delivery and the consumer movement given that the impact of previous trauma on mental ill health has become increasingly well recognised over time [24].

Although participants across both studies embraced the opportunity to develop research topics, the majority of topics were negatively skewed as participants rated most topics as ‘high priority’ overall. Participant comments in both studies suggested that this reflected the view that everything was equally important. They found it challenging to prioritise some topics over others, commenting that it was difficult to choose between numerous competing priorities. However, it is also possible that a different choice of response style (e.g. forced choice of a limited number; consensus method) may have produced different results. These methods were discussed, but considered less desirable by the ACACIA Advisory Group. Difficulty selecting priorities is an observation also reported in several previous research priority-setting projects [2, 3, 21]. Like the present project, these projects all enabled participants to independently select, rank or vote for personal research priorities [2, 3, 21]. In contrast, methods that require a small group of participants to make a collective decision about priorities, such as the James Lind Alliance approach, have produced ranked lists of priorities [7, 12]. This discrepancy is an important consideration for future priority-setting exercises and suggests that there are a diverse range of research topics of high importance to consumers and carers. This apparent lack of consensus presents a challenge for researchers seeking guidance on where to start in implementing a consumer- and carer-led research agenda. However, within ACACIA, consumers and carers also provided a solution to this dilemma – they suggested that the breadth of priorities reflects the equal importance of all of the topics developed, and research into any of them will address an important gap in the system [14]. Consumers and carers have identified what they need mental health research to address; it is now up to researchers to work in partnership with consumers and carers to implement the agenda.

Limitations

A potential limitation of this project is that participation in a research process involving researchers and/or mental health professionals can be intimidating for consumer and carer participants and may thereby distort the findings from a priority-setting exercise [8]. However, we consider this to be unlikely in the current research. Forum participants commented on the respectful atmosphere within the forum that contributed to a sense that it was truly consumer- and carer-led, or as one participant commented in an evaluation form “…acknowledging our capacity to be researchers, not just be researched”. Participants also remarked in their evaluation forms that the opportunity to discuss their ideas and experiences with other consumers and carers was highly valued.

Due to the large number of items employed, the modest participant sample size in the survey study was not sufficient to conduct inferential statistical comparisons between the consumer, carer and consumer/carer priority ratings. However, the sample size was comparable to previous priority-setting projects with similar methodologies [7, 8, 21]. Additionally, there were a number of characteristics of the sample that may preclude the generalisability of the results, namely (1) the sample self-selected to participate, meaning that the survey may represent the views of those already active in the consumer and carer advocacy spheres, (2) the sample was predominantly female (74%), (3) had an uneven geographical distribution, and (4) 50% of participants fell into the consumer-only category. Recruitment of a diverse sample in future studies may be facilitated by visiting relevant community spaces and engaging with targeted community groups, effectively taking the research to the participants [7, 14]. Drop-out across the survey, potentially due to participant fatigue, reduced the information available about research topics presented late in the survey (Additional file 2). Randomisation of item order may effectively manage this issue in future research.

Although the survey study was developed from the findings of the qualitative study, the research project was not explicitly planned as a two-stage process. The second study to attempt to set clearer priorities and update the agenda using a survey was discussed with the ACACIA Advisory Group on multiple occasions following Study 1, with mixed support [14]. The decision to conduct Study 2 and develop the survey directly from the forum findings was driven by the Advisory Group in 2017, who felt the agenda needed to be updated but the original topics preserved. Developing a survey directly from qualitative work has been a highly successful process in past research conducted by the authors [2]; however, the 4-year gap between studies for the current project and the more specific nature of many of the items compared with previous research may have affected the ease of interpretation of items due to changes in the mental health system in the intervening time.

In both studies, comments indicated that participants found it difficult to prioritise topics as they perceived most to be equally important or as connected to each other. This precluded meaningful analysis of the ranking data and limited the conclusions that could be drawn about clear priorities. Future studies should consider alternative methods to improve consensus; however, it is important that decisions on methods are reached collaboratively with consumers and carers to ensure they are acceptable and avoid reaching misleading conclusions as a result of forced choice.

Conclusion

Consumers and carers consider many topics important targets for research, suggesting a strong understanding of the Australian mental health system and its failures as discussed in a recent commentary [25]. Consistent with this, in the current study, topics focused on services, particularly the organisation and delivery of care, were seen by consumers and carers as some of the most important on which to focus. People with a lived experience of mental health issues are ideally placed to identify inadequacies in the mental health care system, and have a strong desire for active involvement in addressing these challenges. There is a need to engage in an ongoing research partnership with consumers and carers to ensure that their views are at the forefront of research, and to regularly update research agendas to ensure they are responsive to current consumer and carer priorities. Taking this collaborative approach will move mental health internationally to the forefront of co-designed and co-delivered health systems.