A core outcome set for clinical trials of Chinese medicine for hyperlipidemia: a study protocol for a systematic review and a Delphi survey
Hyperlipidemia, defined as elevated lipid levels, is the primary and major risk factor for atherosclerotic cardiovascular disease. Several studies have evaluated the effects of Chinese medicine (CM) on hyperlipidemia. However, due to the varied designs and methods of these studies, data synthesis has been difficult, restricting the practical value of the findings. Developing a core outcome set (COS) could solve these methodological concerns. In this paper, we report a protocol to develop a COS for CM clinical trials for hyperlipidemia (COS-CM-Hyperlipidemia).
The development of COS-CM-Hyperlipidemia will include four stages: (1) a systematic review to identify potential important outcomes—a study advisory group, composed of core stakeholders of hyperlipidemia, will be set up afterwards to evaluate the identified outcomes and a candidate outcome set will be developed accordingly; (2) a panel of experts will be invited to conduct a three-round Delphi survey, so that the experts’ opinions on the importance of outcomes for treating hyperlipidemia with CM will be collected; (3) a consensus meeting with clinicians, patients, and other key stakeholders will be conducted to finalize the items and definitions; and (4) COS-CM-Hyperlipidemia will be promoted and updated.
The development of this COS will improve the design and operation of CM trials on hyperlipidemia, keeping them in compliance with international standards, as well as the comparability and utility of their results.
The Core Outcome Measures in Effectiveness Trials Initiative (COMET): http://www.comet-initiative.org/studies/details/983. Registered on 25 April 2017.
KeywordsHyperlipidemia Core outcome set Chinese medicine Systematic review Delphi
Core Outcome Measures in Effectiveness Trials
Core outcome set
Core outcome set of CM clinical trials for hyperlipidemia
Guangdong Provincial Hospital of Chinese Medicine
Low-density lipoprotein cholesterol
Hyperlipidemia, or elevated lipid levels, presents the symptoms of elevated total cholesterol (TC), low-density lipoprotein cholesterol (LDL-C), triglyceride (TG) levels, and mixed TC and TG . It is a primary and major risk factor for atherosclerotic cardiovascular diseases, as well as a prerequisite for them [2, 3, 4, 5]. In the United States, > 100 million adults have reported elevated TC and hyperlipidemia is the 11th leading cause for direct health expenditures ($37 billion annually) . In China, the prevalence of hyperlipidemia is 34.0% and there is a significantly higher prevalence in men than women (41.9% and 32.5%, respectively) .
LDL-C has been recommended as the primary target of lipid-lowering interventions to reduce the incidence of atherosclerotic cardiovascular disease by the mainstream clinical guidelines [3, 4, 8]. In addition to conventional pharmaceutical therapies such as atorvastatin, Chinese medicine (CM) has abundant resources for lipid-lowering interventions. In CM theory, pattern (also called syndrome) is a diagnostic conclusion based on pathological changes in a disease at a certain stage. It includes features such as the nature of disease, cause, location, and development trends . CM physicians treat patients according to syndrome differentiation, so it is important in clinical trials to measure syndrome change so as to embody the characteristics of CM .
Many studies have reported CM treatments that are effective against hyperlipidemia. For example, a meta-analysis indicated that the effect of Xuefuzhuyu decoction on hyperlipidemia was better than that in the control group . Another systematic review showed that Yinchenwuling powder was more effective at decreasing the levels of TC and TG, while increasing the level of high-density lipoprotein cholesterol without serious adverse effects . A Cochrane systematic review also found that Xuezhikang was the most commonly used herbal formula investigated for hypercholesterolemia; a significant effect on decreasing TC was shown in favor of Xuezhikang when compared with inositol nicotinate .
However, these promising findings need to be interpreted with caution due to methodological problems in CM trials [11, 12, 13, 14]. Outcome measures are one of the key factors . For instance, a systematic review of 35 trials studying the efficacy of a Chinese patented drug on hyperlipidemia reported as many as 30 different outcomes, but none focused on atherosclerotic cardiovascular disease events . The main issues concerning hyperlipidemia outcome measures are: (1) the outcome measures vary from one trial to the next and this heterogeneity jeopardizes data consistency, resulting in reporting bias and patient irrelevance ; (2) there is no agreed upon or standardized outcome; (3) surrogate endpoints, such as biochemical indicators, have been widely adopted [11, 12, 13, 14, 15, 16]; and (4) no standardized outcome sets to measure syndrome change for hyperlipidemia. Due to these problems, the practical value of clinical trials has been limited.
A core outcome set (COS) refers to a minimal set of important outcome measures identified by key stakeholders that should be collected and reported in all clinical trials of a specific health area, as a standard criterion [17, 18]. The Core Outcome Measures in Effectiveness Trials (COMET) Initiative brings together people interested in the development and application of COS. It collects and reports core outcomes that will promote the comparison and synthesis of separate trial results where appropriate . Several studies on the development of COS for CM have already begun [20, 21, 22], but none focus on a COS for hyperlipidemia. This study presents a protocol for the development and promotion of a COS for hyperlipidemia in clinical trials. It evaluates the efficacy of CM in the treatment of hyperlipidemia compared with conventional medications. Therefore, a COS for CM clinical trials for hyperlipidemia is urgently need.
This study aims to present a protocol of the development of a COS for CM clinical trials for hyperlipidemia (COS-CM-Hyperlipidemia).
Stage 1: Systematic review
A systematic review will be conducted to screen the range of potentially important outcomes according to the recommendations of the COMET Initiative . Previous COS studies on CM have also used systematic reviews to initiate research [20, 21, 22]. All clinical trials (regardless of study type) that report hyperlipidemia outcome(s) will be included in this study.
Types of studies, interventions, and participants
Both interventional and observational studies (e.g. case reports, case-control studies, cohort studies, randomized controlled trials, and systematic reviews) published in either English or Chinese, will be included in our systematic review. Patients with hyperlipidemia, including hypercholesterolemia, hypertriglyceridemia, mixed hyperlipidemias, and lowered high-density lipoprotein cholesterol, treated by CM alone or combined with conventional medicine, will be considered for addition into our systematic review. Only those in which the hyperlipidemia diagnosis and the effectiveness assessment standard were clear will be eligible for inclusion. All patients, regardless of sex, age, or race, will be included. Patients who suffer from coronary heart disease, hypertension, diabetes, hypothyroidism, or liver disease will be excluded. If a study is mainly on secondary hyperlipidemia, as opposed to other serious diseases, the full text will be read to decide whether to include it.
Data will be searched in CENTRAL, PubMed, Embase, Wangfang database, China National Knowledge Infrastructure, and Chinese BioMedical Database using a comprehensive, electronic search strategy. Two independent reviewers will assess the titles and abstracts to identify whether studies should be included. Any disagreements will be resolved through discussion after a thorough reading of the paper or by consulting a third researcher. Any duplicate studies will be excluded. Studies will also be excluded if they do not describe hyperlipidemia outcomes.
Data extraction and analysis
Two reviewers will independently extract the data by reading the full texts. For each included study, the following data will be extracted if available: author(s); year of publication; country; patient source; study type; treatment duration; length of follow-up; number of patients included; number of patients who withdrew; patients’ age; type of hyperlipidemia; CM syndrome types; interventions details; outcomes and their definitions; time-point; and method of outcome measurement. If any data are incomplete, the reviewers will contact the studies’ authors by email or telephone to get the missing data. Before the data are extracted, consistency evaluation will be conducted between the two reviewers to ensure all analytical details are reliable. Discussion and consulting with a third researcher will be conducted to resolve any disagreements. The data will then be entered into a form and the outcomes will be classified into different domains (patient-reported, clinician-reported, and proxy-reported outcomes) by one of our researchers. Classification will be confirmed by another researcher. The number of definitions of outcomes, the number of outcomes of each domain, and the number of measurement methods will be calculated and presented.
Study advisory group
A SAG composed of at least two hyperlipidemia patients and two clinicians, methodologists, researchers, ethicists, and statisticians will be created. The patients will be selected out of outpatients at the Guangdong Provincial Hospital of Chinese Medicine (GPHCM); clinicians will be selected from endocrinologists and cardiovascular experts of CM/integrated Chinese and Western medicine from both China and abroad; methodologists, researchers, and statisticians will be chosen from GPHCM and ethicists from GPHCM’s Ethics Committee.
Development of COS candidate items
The identified outcomes will be presented on a checklist which will be distributed to the SAG. The SAG will evaluate the checklist and domains of each outcome and will list additional outcomes if they think important ones have been left off the checklist. The final outcomes will be the candidate items for the COS in Questionnaire 1 and will be scored in Delphi process 1. The SAG will not participate in the Delphi process, but will take part in the consensus meeting.
Stage 2: Delphi survey
Stakeholder groups in the study
CM/integrated Chinese and Western medicine
(1) The identification of stakeholders begins with a preliminary list of experts and patients
(2) The preliminary list will be augmented with authors of studies included in the systematic review
(3) Stakeholders will recommend whomever else they think should be included as a relevant stakeholder
(4) An email will be sent to the identified panelists who registered with informed consents to explain this study and the importance of the Delphi survey
Editors of important relevant journals
Editors from important relevant journals on hyperlipidemia will be invited via email
Candidate items will be grouped by topic. In each group, candidate items will be sorted alphabetically. In this way, it will be convenient to both score items and avoid ranking bias in each group. For patient panelists, lay equivalents of each outcome will be presented instead of scientific terms . Candidate items will be measured using a 9-point Likert scale, where 7, 8, and 9 mean “critical importance;” 4, 5, and 6 mean “not critically important;” and 1, 2, and 3 mean “low importance” . This scale has been recommended by the COMET Initiative for measuring outcomes  and has been widely adopted by other COS development studies [17, 30, 31, 32].
Delphi round 1
In round 1 of the Delphi survey, all panelists will be asked to score all candidate items, along with their name, contact information, and other relevant personal information registered. This will confirm their stakeholder groups. After scoring all of the candidate outcomes, panelists will have the opportunity to add one or two additional outcomes they believe are important to hyperlipidemia but which had not been included in the questionnaire. A three-week period is planned for data collection and a reminder will be sent via email or SMS to panelists who have not completed their questionnaires by the end of the second week. A second reminder will be sent via email or SMS when only two days are left.
The number of panelists who complete Questionnaire 1 will be recorded. Each outcome will be analyzed using descriptive statistics, both by stakeholder group and as a whole. The SAG will assess the additional outcomes to decide whether they are representative of new outcomes. Those identified as new outcomes will be included in Delphi round 2. All scored outcomes will be included into Delphi round 2 (Questionnaire 2).
Delphi round 2
Panelists who complete the Delphi round 1 will be invited to take part in the Delphi round 2. They will re-score the outcomes listed on Questionnaire 2, where the scores they assigned in Delphi round 1 will be presented. The descriptive statistical analysis results of Delphi round 1, including the responses and distribution of each outcome from each different stakeholder group, will be attached. Panelists can make their scores in light of insight from others in his/her stakeholder group.
Descriptive statistics will be performed again. For each stakeholder group, we will calculate the number of participants and describe the distribution of each outcome score. A comparison will be conducted between each stakeholder group and the response group as a whole. Outcomes whose median is ≥ 4 (by any stakeholder group) will continue to Delphi round 3 (Questionnaire 3).
Delphi round 3
Panelists who complete Delphi rounds 1 and 2 will participate in round 3 to re-score Questionnaire 3. Questionnaire 3 will include those scores they gave in round 2. Descriptive statistical analysis of round 2, including the responses and distribution of each outcome, will be attached. Panelists will be asked to make their scores in light of the insight of all stakeholder groups.
Consensus that outcome should be excluded from the core outcome set
≥ 70% scoring 1–3 AND < 15% scoring 7–9 in each stakeholder group
Consensus that outcome should be included in the core outcome set
< 15% scoring 1–3 AND ≥ 70% scoring 7–9 in each stakeholder group
No consensus reached on whether the outcome should be included in or not
Stage 3: Consensus meeting
To ensure a range of opinions of all hyperlipidemia stakeholder groups, the purposeful sampling method will be adopted to select participants from those who have completed all three rounds of the Delphi survey. The consensus group will be composed of at least two representatives from patients, endocrinologists, cardiologists, hyperlipidemia specialist nursing, and hyperlipidemia researchers, as well as all members of the SAG.
A roundtable consensus meeting will be held at GPHCM to identify the final COS-CM-Hyperlipidemia. The results of each Delphi round and the classification of candidate outcomes from round 3 will be represented. Outcomes of “consensus in” will be voted either “yes” or “no” electronically and anonymously. Those voted for by at least 70% of the participants will be included in the final COS-CM-Hyperlipidemia. Outcomes scored “consensus out” will be excluded from the final COS-CM-Hyperlipidemia. Outcomes of “without consensus” in round 3 will be discussed. Participants will be asked to re-score all of the “without consensus” outcomes anonymously using the same 9-point Likert scale at the meeting. Outcomes for which at least 70% were scored 1–3 and at most 15% received a score of 7–9 will be removed from the final COS-CM-Hyperlipidemia. Those for which at least 70% were scored 7–9 and at most 15% were scored 1–3 will be included in the final COS-CM-Hyperlipidemia. The remaining outcomes will be re-scored until a final consensus is reached [28, 31].
Stage 4: Promotion and update
A priority of this study is to improve the quality and value of CM trials for hyperlipidemia by applying the COS-CM-Hyperlipidemia. We will collaborate with systematic review groups, clinical guideline makers, clinicians, journal editors, and ethics committees to promote the broad application of COS-CM-Hyperlipidemia. We hope this COS-CM-Hyperlipidemia will be published in the COMET Initiative and be recommended by relevant industry associations.
As medical research continues, understandings of disease, diagnosis, treatment, and evaluation will be updated. Therefore, the COS-CM-Hyperlipidemia needs constant evaluation and upgrading in accordance with the latest achievements of basic and clinical research. In the process of promotion, weaknesses in the current COS-CM-Hyperlipidemia should be noted. They should then be revised in line with the standard procedures based on the number and importance of these shortcomings. With constant evaluation and updates, the COS-CM-Hyperlipidemia can maintain its value and advantages in clinical and basic research of CM for hyperlipidemia. This is accomplished by adding new outcomes and new evaluation tools to ensure practicality and advancement.
To date, there has been no COS for CM clinical trials on hyperlipidemia. The development of COS-CM-Hyperlipidemia will improve the design and operation of CM trials, keeping them in compliance with international standards and guaranteeing the credibility of their results . Outcomes important to key stakeholders, particularly to physicians and patients with hyperlipidemia, will be identified through the development of the COS-CM-Hyperlipidemia. We hope that the use of the developed COS-CM-Hyperlipidemia will ensure the consistency of important outcomes, lower reporting bias, improve the comparability of future studies, and improve the methodological quality of CM clinical research and the utility of study results.
The development of the COS-CM-Hyperlipidemia is currently in the systematic review stage.
This work was supported by the Specific Research Fund for TCM Science and Technology of Guangdong Provincial Hospital of Chinese Medicine (No. YN2015QN11).
Availability of data and materials
XC, GL, and ZW devised the study. GL, LZ, WO, MX, LL, and XL refined the study, contributed ideas for conducting it, and participated in data collection and analysis. GL drafted the manuscript. XC, ZW, LZ, WO, MX, LL, and XL contributed intellectual content to the paper. XC obtained funding and supervised the work. All authors read and approved the final version of the manuscript prior to submission.
Ethics approval and consent to participate
Ethical approval to conduct this study has been granted by the Ethics Committee of Guangdong Provincial Hospital of Chinese Medicine. Consent will be requested from all study participants.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
- 4.Chinese Joint Committee of Revising the Guidelines for Prevention and Treatment of Dyslipidemia in Chinese Adults. Guidelines for Prevention and Treatment of Dyslipidemia in Chinese Adults (2016). Chin J Cardiol. 2016;44(10):833–53.Google Scholar
- 9.World Health Organization. WHO International Standard Terminologies on Traditional Medicine in the Western Pacific Region. Geneva: World Health Organization; 2007.Google Scholar
- 10.Lai SL. Key points about clinical effectiveness assessment of traditional Chinese medicine. J Guangzhou Univ Tradit Chin Med. 2002;19(4):245–50.Google Scholar
- 13.Liu ZL, Liu JP, Zhang AL, et al. Chinese herbal medicines for hypercholesterolemia. Cochrane Database Syst Rev. 2011;7:CD008305.Google Scholar
- 16.Tai HX. Systematic review of Zhibituo in treatment of hyperlipidemia. Tianjin : Tianjin Medical University; 2015.Google Scholar
- 19.COMET Initiative. Overview. http://www.comet-initiative.org/about/overview. Accessed 9 Aug 2017.
- 20.Hongcai Shang, Ruijin Qiu, Xiaoyu Zhang, et al. A core outcome set for clinical trials of Traditional Chinese Medicine in non-valvular atrial fibrillation. http://www.comet-initiative.org/studies/details/941. Accessed 9 Aug 2017.
- 21.Yanan Sun, Changhe Yu, Liyun He, et al. Core Outcome Sets of integrity of modern and Traditional Chinese Medicine on treatment of chronic Hepatitis B. http://www.comet-initiative.org/studies/details/669. Accessed 9 Aug 2017.
- 22.Junhua Zhang, Dongmei Xing, Hongcai Shang. Developing a Core Outcome Set for Traditional Chinese Medicine for Stable Angina Pectoris. http://www.comet-initiative.org/studies/details/391. Accessed 9 Aug 2017.
- 23.Zehuai Wen, Xiankun Chen, Geng Li, et al. A core outcome set for clinical trials of Chinese Medicine in hyperlipidemia. http://www.comet-initiative.org/studies/details/983. Accessed 9 Aug 2017.
- 25.Prinsen CA, Vohra S, Rose MR, et al. Core Outcome Measures in Effectiveness Trials (COMET) initiative: protocol for an international Delphi study to achieve consensus on how to select outcome measurement instruments for outcomes included in a ‘core outcome set’. Trials. 2014;15:247.PubMedPubMedCentralCrossRefGoogle Scholar
- 27.Zhang D, Zhang M, Zheng W, et al. The Construction of Core Outcome Set of Traditional Chinese Medicine Clinical Trials (COS-TCM) and Implementation Specification of the Delphi Method. J Tradit Chin Med. 2017;58(1):20–2.Google Scholar
- 36.Gillies K, Entwistle V, Treweek SP, et al. Evaluation of interventions for informed consent for randomised controlled trials (ELICIT): protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey. Trials. 2015;16:484.PubMedPubMedCentralCrossRefGoogle Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.