COVID-19 vaccines have been administered in many countries; however, a sufficient vaccine coverage rate is not guaranteed due to vaccine hesitancy. To improve the uptake rate of COVID-19 vaccine, it is essential to evaluate the rate of vaccine hesitancy and explore relevant factors in different populations. An urgent need is to measure COVID-19 vaccine hesitancy among different population groups, hence a validated scale for measuring COVID-19 vaccine hesitancy is necessary. The present study aims to validate the COVID-19 vaccine hesitancy scale among different populations in China and to provide a scale measuring COVID-19 vaccine hesitancy with satisfactory reliability and validity.
Self-reported survey data were collected from different populations in China from January to March 2021. Based on the Parent Attitudes about Childhood Vaccines scale, 15 items were adapted to evaluate the COVID-19 vaccine hesitancy. Exploratory and confirmatory factor analysis were utilized to identify internal constructs of the COVID-19 vaccine hesitancy scale among two randomly split subsets of the overall sample. Reliability was analyzed with the internal consistency, composite reliability, and the test–retest reliability, and validity was analyzed with the criterion validity, convergent validity, and discriminant validity.
A total of 4227 participants completed the survey, with 62.8% being medical workers, 17.8% being students, 10.3% being general population, and 9.1% being public health professionals. The exploratory factor analysis revealed a three-factor structure that explain 50.371% of the total variance. The confirmatory factor analysis showed that models consisting of three dimensions constructed in different populations had good or acceptable fit (CFI ranged from 0.902 to 0.929, RMSEA ranged from 0.061 to 0.069, and TLI ranged from 0.874 to 0.912). The Cronbach’s α for the total scale and the three subscales was 0.756, 0.813, 0.774 and 0.705, respectively. Moreover, the COVID-19 vaccine hesitancy scale had adequate test–retest reliability, criterion validity, convergent validity, and discriminant validity.
The COVID-19 vaccine hesitancy scale is a valid and reliable scale for identifying COVID-19 vaccine hesitancy among different population groups in China. Given the serious consequences of COVID-19 vaccine hesitancy, future studies should validate it across regions and time to better understand the application of the COVID-19 vaccine hesitancy scale.
Coronavirus disease 2019 (COVID-19) has been wreaking havoc on the world, taking a heavy toll on human lives and economic activities. Countries in different regions are facing the challenge of containing its spread. To curb the COVID-19 pandemic, COVID-19 vaccines have been successfully developed and are being administered in many countries. The COVAX Facility, led by WHO, is designed to guarantee rapid, fair and equitable access to COVID-19 vaccines for every country in the world to achieve WHO’s vaccine equity target . However, it’s not vaccines that will stop the pandemic, it’s vaccination. Vaccination is considered to be one of the greatest achievements of public health and has contributed to the decline in mortality and morbidity of various infectious diseases. COVID-19 vaccine hesitancy may be a barrier to sustained public acceptance of vaccination and could lead to an insufficient vaccine coverage rate to achieve herd immunity .
Vaccine hesitancy refers to a refusal or delay in acceptance of vaccination despite the availability of vaccination services. It is complex and context-specific, varying across time, place, and vaccines . A certain amount of the population has been hesitant or unsure about vaccination since the first vaccine was made available, and those who are hesitant about vaccines may have varied levels of hesitancy about specific vaccines or vaccination in general [4, 5]. Vaccine hesitancy has emerged as a major public health crisis in recent decades, and World Health Organization (WHO) has listed it as one of the ten leading threats to global health in 2019 . Indeed, vaccine hesitancy has a negative impact on maintaining herd immunity, preventing outbreaks of vaccine-preventable diseases, and ensuring the vaccination of novel vaccines .
Regarding the factors related to vaccine hesitancy, WHO identified complacency, inconvenience in access, and a lack of confidence as the driving factors . The complacency refers to the belief that perceived risks of vaccine-preventable diseases are low and that vaccination is not a necessary preventive action; the convenience refers to vaccine availability and accessibility; and the confidence refers to the trust in the effectiveness and safety of vaccines, the delivery system, and the motivations of vaccination policymakers [8, 9]. Previous studies demonstrated that complacency, convenience, and confidence were equally significant factors influencing COVID-19 vaccine hesitancy [10, 11]. In addition, COVID-19 vaccine hesitancy was also depended on a variety of factors, such as concerns about side effects, perceptions of the benefits, perceptions of vaccination risks or harms, trust in authorities, vaccination knowledge, and their own vaccination history [12,13,14,15,16,17].
To measure COVID-19 vaccine hesitancy, most studies utilized one-item question: “Would the respondents be willing to vaccinate against COVID-19 if COVID-19 vaccine is available?” [18,19,20,21]. Several studies used the Vaccine Hesitancy Scale (VHS) or modified VHS, a 10-item scale developed to measure parental vaccine hesitancy [22, 23], and to measure adult’s COVID-19 vaccine hesitancy [24,25,26]. The Oxford COVID-19 vaccine hesitancy scale was a newly developed scale which comprises 7 items and focused mostly on the link between hesitancy and vaccine conspiracy beliefs to measure COVID-19 vaccine hesitancy among adults [27,28,29]. The COVID-19 vaccine attitudes and beliefs scale was proposed to measure adults’ hesitancy, consisting of 20 items in four dimensions: general COVID-19 vaccination beliefs and attitudes, COVID-19 vaccination adverse effects, perceived knowledge sufficiency, and return to “normal” life [30, 31]. However, there are no data on adapted or validated scales capable of assessing vaccine hesitancy among different populations, and a valid scale to identify COVID-19 vaccine hesitancy that could perform reliably across diverse populations would be especially valuable.
At present, WHO points out that COVID-19 transmission remains very high, vaccination coverage remains very low in too many countries, and certain numbers of healthcare workers and others at high risk are still unvaccinated, with the risk of new variants emerging . Measuring different populations’ hesitancy levels to receive the COVID-19 vaccine is essential to better understand the reasons for it and further increase the vaccination rate. The aim of the current study was to adapt and validate a COVID-19 vaccine hesitancy scale based on the Parent Attitudes about Childhood Vaccines (PACV) scale to measure COVID-19 vaccine hesitancy and then validated it among 4 different population groups in China. The PACV was originally developed by adapting items from a previous Health Beliefs Survey in 2011, which included an 18-item survey encompassing the domains: immunization behaviors, beliefs about vaccine safety and effectiveness, attitudes, and trust . Then Larson et al. adapted and validated a series of questions for greater global relevance on this basis in 2015, the results of this process was a 15-item scale, containing 15 items in 3 dimensions: general attitudes, safety and efficacy, and behavior. And PACV has been utilized or validated in different countries, such as China, Italy, Malaysia, and Switzerland [34,35,36,37,38]. Potentially, a COVID-19 vaccine hesitancy scale with satisfactory reliability and validity could provide timely feedback to health authorities and policy makers by assessing COVID-19 vaccine hesitancy in order to establish appropriate strategies to address vaccine hesitancy during the COVID-19 pandemic.
Survey design and sampling
Exploratory factor analysis requires a sample size 10 times larger than the number of items . Given the number of items in the scale was 15, the sample size was planned 150 at least for exploratory factor analysis among each studied population. The sample size for confirmatory factor analysis was planned 192 at least among each population group, with the set of \(\alpha\)= 0.05, \(\beta\) = 0.20, df = 87, RMSEA = 0.05 in the null hypothesis, and RMSEA = 0.01 in the alternative hypothesis [40, 41].
Firstly, the 31 provinces/municipalities/autonomous regions of China are divided into seven geographical divisions, namely, East China, South China, Central China, North China, Northwest China, Southwest China, and Northeast China. Second, in each of the 7 geographic divisions, one representative city is selected, which are Dalian, Jinan, Shenzhen, Sichuan, Xinjiang, Henan, and Inner Mongolia in China. Finally, snowball sampling was used to recruit the potential study participants. Investigators from the seven cooperative institutions were initially invited, and they distributed the questionnaire to those who met the inclusion criteria.
The cross-sectional survey was conducted in seven cities by generating an electronic questionnaire on a survey platform named Wenjuanxing. Then, investigators shared the link of questionnaire on WeChat among the potential participants. The participants included four different population groups: students, public health professionals, medical workers, and the general population. The medical workers were recruited from hospital departments such as respiratory and critical care medicine, general surgery, and nephrology department, while hospital administrators were excluded from medical workers group in our study. The public health professionals were recruited from local CDCs in China, mostly from the communicable disease control and prevention department, immunization program department, and preventive health department. The student group was recruited from students of local universities. The retest sample representing four different populations in seven cities was repeatedly surveyed over a four-week period . People who were 18 years old or above and could read, understand and complete an online questionnaire were eligible to participate. Those who were younger than 18 years old, had difficulty using a cell phone or computer, or had cognitive impairments were excluded. To ensure the quality of the online survey, study team members received training on standard procedures for data collection and inclusion.
This study has been approved by the Ethical Review Committee of Chinese Center for Disease Control and Prevention on December 4th, 2020 (approval number: 202020). An electronic informed consent was provided before the start of the questionnaire survey. Upon completion of the informed consent, the study participants filled in the online questionnaire.
Sociodemographic variables included age, gender (male or female), ethnicity (Han ethnicity or other), residence place (urban or rural), marital status (single, married, or others), education level (below high school or college and above), and household income (during past 1 year).
The COVID-19 vaccine hesitancy scale (CVHS)–the modified scale based on PACV
We used 15 items to identify the COVID-19 vaccine hesitancy, these items were adapted from the Parent Attitudes About Childhood Vaccines (PACV) scale, which was originally designed to measure the parents’ attitude towards childhood vaccines by the WHO Strategic Advisory Group on Experts (SAGE). The revised 15-item scale was named the COVID-19 vaccine hesitancy scale (CVHS). Based on the PACV, the revised 15-item scale was composed of the three domains: “Safety and Efficacy” with 4 items, “General Attitudes” with 9 items, and “Behavior” with 2 items [22, 33, 38]. Responses of scale items was divided into 3 categories: hesitant responses, “not sure or don’t know”, and non-hesitant responses. Additional file 1: Appendix A contains the specific items and scoring rules of this scale. The raw total score was calculated by summing up each item’s score, ranging from 0 to 30. Simple linear transformation was used to convert this raw score to a 0–100 scale, accounting for missing values. COVID-19 vaccine hesitancy was indicated by a score higher than or equal to 50 .
The revised flu vaccine hesitancy scale for measurement of COVID-19 vaccine hesitancy
On the basis of the flu vaccine hesitancy scale, we replaced “flu vaccine” in the items with “COVID-19 vaccine” to form the revised flu vaccine hesitancy scale. The flu vaccine hesitancy scale is comprised of 6 items and 3 dimensions (complacency, confidence and convenience) . In this study, complacency was measured by perceived necessity and importance of the vaccine; confidence was measured by perceived vaccine safety and effectiveness; convenience was measured by perceived convenience and affordability of the vaccine. Participants rated each item on a 5-point Likert scale (1 = strongly disagree; 5 = strongly agree). The items were as follows: (1) necessity: “Thinking specifically about the COVID-19 vaccine, do you think the COVID-19 vaccine is necessary?”, (2) importance: “Thinking specifically about the COVID-19 vaccine, do you think the COVID-19 vaccine is important?”, (3) safety: “Thinking specifically about the COVID-19 vaccine, do you think the COVID-19 vaccine is safe?”, (4) effectiveness: “Thinking specifically about the COVID-19 vaccine, do you think the COVID-19 vaccine is effective?”, (5) convenience: “Thinking specifically about the COVID-19 vaccine, do you think the COVID-19 vaccine is convenient?”, (6) affordability: “Thinking specifically about the COVID-19 vaccine, do you think the COVID-19 vaccine is affordable?”. In the current study, the revised scale had satisfactory reliability (Cronbach’s α = 0.942), and the Cronbach’s α values for the three dimensions were 0.930, 0.925 and 0.820, respectively.
The vaccine confidence scale
The vaccine confidence scale consisted of 8 items on an 11-point response scale ranging from 0 (“strongly disagree”) to 10 (“strongly agree”), assessing three domains of the benefits of vaccination ( “Benefits”), the harms of vaccination (“Harms”), and trust in healthcare providers (“Trust”) . Using the Vaccination Confidence Scale, we reverse-coded the negative attitudes in the Harms factor and calculated mean scores for each participant by averaging responses for all 8 items. The resulting scores had a possible range of 0 to 10 with higher scores indicating more positive attitudes about vaccination . In this study, it has good reliability (Cronbach’s α = 0.774), and the Cronbach’s α values for the three dimensions were 0.592, 0.857 and 0.803, respectively.
Translation of the COVID-19 vaccine hesitancy scale and pilot testing
First, two translators independently translated the scale from English to Chinese. These translators were bilingual, with professional competence in English and native proficiency in Chinese. Then, the two Chinese versions were combined into one Chinese version, and experts from seven partner institutions were invited to review the Chinese version and make partial changes to the wording of the questions in it, so that the scale could be accurately translated into the Chinese version. And a pilot test of the preliminary version was conducted in 7 cities among different populations. These participants were invited to comment on any questions that they found difficult to understand, difficult to answer, disturbing, confusing, or offensive. No major problems were found in the pilot test and only minor modifications were made to finalize the Chinese version.
The sample was randomly divided into two parts for performing exploratory factor analysis and confirmatory factor analysis respectively, and to identify factors and further test the factor structure of the scale. A random number generator was employed to divide the sample into two groups with a 50/50 split to ensure its equal distribution. The final sample size of the exploratory sample (sample 1) was 2123 and the confirmatory sample (sample 2) was 2104. Descriptive statistics such as mean (standard deviation) and frequency were used to elaborate the sociodemographic characteristics of different population groups. The level of significance was set at P < 0.05. AMOS Version 24.0 and SPSS Version 24.0 were used to perform the analysis.
Exploratory factor analysis
The KMO (Kaiser–Meyer–Olkin) measurement of sampling adequacy was used to test whether our data were suitable for EFA . Exploratory factor analysis for the scale was conducted in sample1 using principal components analysis with varimax-based rotation to account for possible correlations among factors. Items which had a factor loading of more than 0.40 and did not load on multiple factors were considered part of a factor .
Confirmatory factor analysis
Confirmatory factor analysis was then conducted in sample 2 among four different population groups (students, public health professionals, medical workers, and the general public) in order to demonstrate that the factor structure validates across an independent sample of different populations. Model fit was evaluated by a few goodness of fit indices, including the root mean square error of approximation (RMSEA), the comparative fit index (CFI), and the Tucker Lewis index (TLI) [48,49,50]. RMSEA values close to 0.06, or below, were regarded as good fit, 0.07 to 0.08 as moderate fit, 0.08 to 0.10 as marginal fit, and > 0.10 as poor fit . For the CFI and TLI, values close to 0.95 or above were regarded as good fit, values close to 0.90 and 0.95 as acceptable fit, and values approaching 0 as poor fit [52, 53].
For the evaluation of the reliability of the scales, Cronbach’s alphas were calculated across the overall scale and each subscale to determine the internal consistency (Cronbach’s α ≥ 0.70 was considered satisfied) . And Composite Reliability (CR) for each factor was also calculated to evaluate the reliability (CR \(>\) 0.70 was considered satisfied).
Convergent and discriminant validity
Convergent validity was assessed by Average Variance Extracted (AVE) and CR, and convergent validity was considered high if AVE was greater than 0.50, CR was greater than 0.70, and CR was greater than AVE. Discriminant validity was considered satisfactory if the correlation between the factor scores was significant and the correlation coefficient was less than the square root of the corresponding AVE [55, 56].
Criterion validity was examined through bivariate Pearson correlation analysis between the CVHS and the revised flu vaccine hesitancy scale, and correlations between the CVHS and the vaccine confidence scale. Significant correlations between the CVHS and the revised flu vaccine hesitancy scale and the vaccine confidence scale indicated adequate construct validity.
Test–retest reliability was probed by the intraclass correlation coefficients (ICCs) of the self-reported scores of retest samples that were repeatedly surveyed over a 4-week period in 4 different populations from 7 cities. ICCs above 0.40 were acceptable; ICCs above 0.60 or greater indicated satisfactory stability; and ICCs greater than 0.80 were excellent .
The sociodemographic characteristics of samples were summarized in Table 1. A total of 4289 respondents (response rate 95.37%) completed the online questionnaire, and 62 questionnaires were excluded due to age limitations. Among them, 62.8% were medical workers, 17.8% were students, 10.3% were general population, and 9.1% were public health professionals. The mean age of all participants was 33.02 years old. A total of 2818 (66.7%) respondents were female and 33.3% of them were male. 89.1% of them were of Han ethnicity. The majority of participants (85.6%) lived in urban areas. 56.9% of them were married and 41.3% were single. The education level of most of them (93.8%) were college or above. For the household income during the past 1 year, 43.4% of them were in the range of 50,000–100,000 Yuan per year, while 33.8% of them were in the range of 110,000–350,000 Yuan.
Exploratory factor analysis
The value of KMO measure of sampling adequacy was 0.755 and Bartlett’s test of Sphericity was significant (P < 0.001). The total variance of the 15 items explained by the 3 factors was 50.371%. Table 2 showed the factor loadings of the 15 items to the three factors extracted. The factor loadings of the 15 items were ranging from 0.294 to 0.880, and excepted the 6th and 12th item, all other items loaded on the expected factors as the original dimensions (> 0.40). The 6th and the 12th item had relatively weak loadings on the “General Attitudes” factor, they were hence classified as the “Safety and efficacy” factor.
Confirmatory factor analysis
The confirmatory factor analyses based on the original structure of “Safety and efficacy” dimension (4 items), “General Attitudes” dimension (9 items), and “Behavior” dimension (2 items) were performed in sample 2 among four different population groups. Initially, the results showed that the CFA analysis conducted among four different population groups and the whole sample 2 did not meet the criteria of RMSEA values close to 0.06 and CFI and TLI values greater than or equal to 0.90. Based on the results of EFA, we constructed new models that “Safety and efficacy” dimension included 6 items (added the 6th and 12th item), “General Attitudes” dimension included 7 items (excluded the 6th and 12th item), “Behavior” dimension included 2 items. However, the results of the new models based on the results of EFA still did not meet the above criteria and were worse compared with the original structure. The results of modification indices of the CFA model based on the original structure indicated that freeing error terms to covary could substantially improve model fit. After modification was made to the CFA model based on the original structure, RMSEA indicated the modification models were good fit, CFI and TLI indicated the modification models were acceptable fit (Table 3).
Figure 1 depicted the results of the modification model built in overall sample 2, including the three dimensions and its normalization. Model 1 built in overall sample 2 fit the data well, with the value of RMSEA of 0.069, the CFI of 0.909, and the TLI of 0.886. Besides, the fit indices of the modification model based on medical workers (RMSEA = 0.064, CFI = 0.922, TLI = 0.903), students (RMSEA = 0.069, CFI = 0.902, TLI = 0.874), the general population (RMSEA = 0.067, CFI = 0.927, TLI = 0.909) and public health professionals (RMSEA = 0.061, CFI = 0.929, TLI = 0.912) samples indicated good model fit, as shown in Figs. 2, 3, 4 and 5, respectively.
The results of the internal consistency analysis for the overall scale demonstrated satisfactory internal consistency, with Cronbach’s α of 0.756. And the values of Cronbach’s α of the three sub-scales (“Behavior”, “Safety and Efficacy”, “General Attitudes”) were 0.813, 0.774, and 0.705, respectively. All five models had CRs greater than 0.70 for all three dimensions, with the exception of the “general attitude” dimension of the models conducted among health workers, students, public health professionals, and all confirmatory samples, which had CRs below 0.70.
Convergent and discriminant validity
Table 4 showed the results of convergent and discriminant validity. For the convergent validity, the AVE values for the “Behavior” and “Safety and Efficacy” dimensions of the 5 models conducted in the medical workers, students, general population, public health professionals, and all confirmatory samples were greater than 0.50, except for the AVE value of 0.494 for the “Safety and efficacy” dimension of the model built in the student sample, while regarding the “General Attitudes” dimension, its AVE was lower than 0.50 in all 5 models. The CR values for the three dimensions of the five models were greater than 0.70, except for the CR of the “General Attitudes” dimension of the models conducted in medical workers, students, public health professionals and all confirmatory samples, which were below 0.70. In addition, the CR values in all factors among the 5 models were all greater than AVE. For the discriminant validity, the correlation coefficients of any two dimensions in the five populations are less than the square root of the corresponding AVE.
Table 5 showed the results of bivariate Pearson analysis between the COVID-19 vaccine hesitancy scale and the revised flu vaccine hesitancy and the vaccine confidence scale. For the revised flu vaccine hesitancy scale, the results showed that the three subscales of the COVID-19 vaccine hesitancy scale (the “Behavior”, “Safety and Efficacy”, and “General Attitudes”), and the total score of the COVID-19 vaccine hesitancy scale were positively correlated with the “Complacency” subscale, and negatively correlated with the “Confidence” and “Convenience” subscales, as well as the total scores of the revised flu vaccine hesitancy scale. For the vaccine confidence scale, the COVID-19 vaccine hesitancy scale total scores and its three subscales were found to have a significantly negative correlation with the vaccine confidence scale total score and scores of its three subscales (“Harms”, “Benefits”, “Trust”).
The ICCs of the overall COVID-19 vaccine hesitancy scale for a 4-week interval among 45 participants was 0.773 (P < 0.001), indicating satisfactory stability. The three subscales also yielded test–retest reliabilities of 0.878 (P < 0.001), 0.530 (P < 0.05), 0.746 (P < 0.001), respectively.
In the current study, the COVID-19 vaccine hesitancy scale was modified based on the items of Parent Attitudes about Childhood Vaccines scale to evaluate people’s hesitancy about the COVID-19 vaccine among different populations, thus satisfying the need to validate an appropriately structured and specific instrument during the COVID-19 pandemic. Based on the results of EFA and CFA, the original structure reflected well on its conceptual framework. In addition, Cronbach’s alpha values for the total scale and the three subscales showed satisfactory internal consistency and also confirmed that the 15 items based on the original dimensions could consistently measure COVID-19 vaccine hesitancy. Therefore, the scale based on the initial dimensions could provide a consistent and reliable assessment of the COVID-19 vaccine hesitancy among various population in China. Convergent and discriminant validity was assessed by the AVE and CR, indicating that the scale has good stability in reflecting the measured outcomes. Criterion validity was demonstrated by the significant correlations between the CVHS overall scale and its subscales and the revised flu vaccine hesitancy scale and its three subscales (complacency, confidence and convenience), as well as its significant correlation with the vaccine confidence scale and its subscales (harms, benefits and trust). The results of the current study demonstrated the validity and reliability of the COVID-19 vaccine hesitancy scale as a reliable multifactor instrument to measure COVID-19 vaccine hesitancy.
Results of the criterion validity indicated that complacency about the vaccination were positively associated with COVID-19 vaccine hesitancy. Complacent individuals who feel that the risks are minimal are less likely to take preventive measures. Previous studies also indicated that complacency is negatively related to the adoption of preventive behavior [10, 58]. The public might become complacent after the vaccine becomes available, since they believe that COVID-19 is preventable and the consequences of infection are not severe . This phenomenon may due to their limited knowledge about the risk of getting COVID-19 and the efficacy of COVID-19 vaccine, thus giving them a misperception of the disease and of the vaccine . Alternatively, they may think that the surrounding people’s vaccination is sufficient to prevent transmission and protect themselves from COVID-19 infection, and so they believe they are at low risk of contracting COVID-19. Besides, people who believed in their own immunity were hesitant to get vaccinated, due to their low perception of the risk of getting the COVID-19 . In addition, the large number and diversity of effective preventive measures, such as wearing masks, provide an alternative to prevent COVID-19, potentially weakening the intent or perceived necessity for vaccination for some people . To reduce the complacency on vaccination, proactive public health campaigns and communications, together with appropriate social media engagement, would be efficacious solutions. The healthcare system and relevant authorities should provide more solid knowledge about COVID-19 and COVID-19 vaccines, as well as disseminate more transparent and accurate information to increase the public’s awareness of the infection risk, necessity, and the significance of vaccination as well.
In contrast, confidence was negatively associated with COVID-19 vaccine hesitancy, indicating individuals who lack confidence have hesitancy to receive COVID-19 vaccine. Confidence is defined as trust in the effectiveness and safety of vaccines, the system that delivers them, and the motivations of policy-makers who decide on the need of vaccines [3, 63]. During the COVID-19 pandemic, the rapid pace of vaccine development and ubiquitous misinformation could cause public’s concerns or distrust about the effectiveness and safety of COVID-19 vaccines, which undermined their confidence of the vaccine [64,65,66]. Distrusting the COVID-19 vaccine may stem from having little confidence in the governments or the health system since vaccination is usually a government-led public health intervention . Moreover, lack of transparency in information about the development, approval, and use of vaccines may increase public distrust of the authorities involved. Therefore, ensuring and facilitating public access to reliable information from vaccine providers (e.g., health workers) and health authorities can facilitate the reduction of misinformation and increase public trust in authorities. In addition, health authorities should have the ability to utilize social media to monitor trends in public opinions and respond timely, thereby increasing the credibility of the government and relevant authorities in vaccination activities, thus improving or restoring public confidence in vaccines and reducing vaccine hesitancy.
As a specific instrument to assess COVID-19 vaccine hesitancy, the COVID-19 Vaccine Hesitancy Scale has advantages compared to other scales. The results of the current study indicated that the COVID-19 vaccine hesitancy scale could properly assess the population in a variety of backgrounds, including medical workers, students, public health professionals, and the general population. Additionally, this scale focused on three variables (“Behavior”, “Safety and Efficacy”, and “General Attitudes”) that are distinct from those of other scales and can be used to investigate multiple vaccine hesitancy reasons. Additionally, we concentrated on customizing the scale to the precise COVID-19 vaccine, which is more reliable to measure, in comparison to the previously validated general vaccine hesitancy scales. In the context of the ongoing global COVID-19 pandemic, there are still people who have not been vaccinated for various reasons, which undoubtedly hinders the global control of the COVID-19 pandemic. And, due to the emergence of variants, people who have already been vaccinated will need to receive a booster shot, a process that can also lead to vaccine hesitancy. The COVID-19 Vaccine Hesitancy Scale can measure COVID-19 vaccine hesitancy in this situation, which not only facilitates the identification of sources of hesitancy but also provides a reference for interventions to improve vaccination rates, which is crucial for lowering COVID-19 vaccine hesitancy.
Although the present study revised the COVID-19 vaccine hesitancy scale specifically for the COVID-19 vaccine and also validated the scale in different populations, limitations were still existed. First, all data were collected cross-sectionally using self-report, leading to the presence of information bias. Second, despite the large sample size containing a diverse population when considering the demographic data of the participants, a convenience sample was employed for the survey, which was not fully representative of the Chinese population. Besides, its psychometric properties have not been tested in populations from other countries because only Chinese people were recruited as study subjects in the present study. Further studies considering all these limitations are warranted to apply this scale among other different populations.
The COVID-19 vaccine hesitancy scale was adapted and validated among different populations in China, and had good validity and reliability, which will assist in assessing vaccine hesitancy status and influencing factors, developing prevention or intervention programs, and improving vaccination rates. The validity of the COVID-19 vaccine hesitancy scale would need to be further evaluated across regions and time to increase its validity and generalizability.
Availability of data and materials
The datasets generated and/or analysed during the current study are not publicly available due to the confidentiality related to the protection of research participants, but are available from the corresponding author on reasonable request.
World Health Organization
Coronavirus disease 2019
Vaccine Hesitancy Scale
Parent Attitudes about Childhood Vaccines scale
Centers for Disease Control and Prevention
COVID-19 vaccine hesitancy scale
Exploratory factor analysis
Confirmatory factor analysis
Intraclass correlation coefficients
Root mean square error of approximation
Comparative fit index
Tucker Lewis index
Average variance extracted
Organization WH. More than 150 countries engaged in COVID-19 vaccine global access facility 2020 [cited 2022 17 Apr]. Available from: https://www.who.int/news/item/15-07-2020-more-than-150-countries-engaged-in-covid-19-vaccine-global-access-facility.
Dror AA, Eisenbach N, Taiber S, Morozov NG, Mizrachi M, Zigron A, et al. Vaccine hesitancy: the next challenge in the fight against COVID-19. Eur J Epidemiol. 2020;35(8):775–9.
MacDonald NE. Vaccine hesitancy: definition, scope and determinants. Vaccine. 2015;33(34):4161–4.
Larson HJ, Jarrett C, Eckersberger E, Smith DM, Paterson P. Understanding vaccine hesitancy around vaccines and vaccination from a global perspective: a systematic review of published literature, 2007–2012. Vaccine. 2014;32(19):2150–9.
Kestenbaum LA, Feemster KA. Identifying and addressing vaccine hesitancy. Pediatr Ann. 2015;44(4):e71–5.
Organization WH. Ten Threats to Global Health in 2019 2019 [cited 2022 14 Apr]. Available from: https://www.who.int/vietnam/news/feature-stories/detail/ten-threats-to-global-health-in-2019.
Callender D. Vaccine hesitancy: more than a movement. Hum Vaccin Immunother. 2016;12(9):2464–8.
Mathieu P, Gautier A, Raude J, Goronflot T, Launay T, Debin M, et al. Population perception of mandatory childhood vaccination programme before its implementation, France, 2017. Euro surveillance: bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin. 2019;24(25).
González-Block M, Gutiérrez-Calderón E, Pelcastre-Villafuerte BE, Arroyo-Laguna J, Comes Y, Crocco P, et al. Influenza vaccination hesitancy in five countries of South America. Confidence, complacency and convenience as determinants of immunization rates. PLoS ONE. 2020;15(12):e0243833.
Liu XJ, Mesch GS. The adoption of preventive behaviors during the COVID-19 pandemic in China and Israel. Int J Environ Res Public Health. 2020;17(19):7170.
Gerretsen P, Kim J, Caravaggio F, Quilty L, Sanches M, Wells S, et al. Individual determinants of COVID-19 vaccine hesitancy. PLoS ONE. 2021;16(11): e0258462.
Solís Arce JS, Warren SS, Meriggi NF, Scacco A, McMurry N, Voors M, et al. COVID-19 vaccine acceptance and hesitancy in low- and middle-income countries. Nat Med. 2021;27(8):1385–94.
Joshi A, Kaur M, Kaur R, Grover A, Nash D, El-Mohandes A. Predictors of COVID-19 vaccine acceptance, intention, and hesitancy: a scoping review. Front Public Health. 2021;9: 698111.
Saied SM, Saied EM, Kabbash IA, Abdo SAE. Vaccine hesitancy: beliefs and barriers associated with COVID-19 vaccination among Egyptian medical students. J Med Virol. 2021;93(7):4280–91.
Thunström L, Ashworth M, Finnoff D, Newbold SC. Hesitancy toward a COVID-19 vaccine. EcoHealth. 2021;18(1):44–60.
Patwary MM, Bardhan M, Disha AS, Hasan M, Haque MZ, Sultana R, et al. Determinants of COVID-19 vaccine acceptance among the adult population of Bangladesh using the health belief model and the theory of planned behavior model. Vaccines. 2021;9(12):1393.
Patwary MM, Alam MA, Bardhan M, Disha AS, Haque MZ, Billah SM, et al. COVID-19 vaccine acceptance among low- and lower-middle-income countries: a rapid systematic review and meta-analysis. Vaccines. 2022;10(3):427.
Alfageeh EI, Alshareef N, Angawi K, Alhazmi F, Chirwa GC. Acceptability of a COVID-19 vaccine among the Saudi population. Vaccines. 2021;9(3):226.
El-Elimat T, AbuAlSamen MM, Almomani BA, Al-Sawalha NA, Alali FQ. Acceptance and attitudes toward COVID-19 vaccines: a cross-sectional study from Jordan. PLoS ONE. 2021;16(4): e0250555.
Kasrine Al Halabi C, Obeid S, Sacre H, Akel M, Hallit R, Salameh P, et al. Attitudes of Lebanese adults regarding COVID-19 vaccination. BMC Public Health. 2021;21(1):998.
Qunaibi EA, Helmy M, Basheti I, Sultan I. A high rate of COVID-19 vaccine hesitancy in a large-scale survey on Arabs. eLife. 2021;10.
Larson HJ, Jarrett C, Schulz WS, Chaudhuri M, Zhou Y, Dube E, et al. Measuring vaccine hesitancy: the development of a survey tool. Vaccine. 2015;33(34):4165–75.
Shapiro GK, Tatar O, Dube E, Amsel R, Knauper B, Naz A, et al. The vaccine hesitancy scale: psychometric properties and validation. Vaccine. 2018;36(5):660–7.
Thaker J. The persistence of vaccine hesitancy: COVID-19 vaccination intention in New Zealand. J Health Commun. 2021;26(2):104–11.
Akel KB, Masters NB, Shih SF, Lu Y, Wagner AL. Modification of a vaccine hesitancy scale for use in adult vaccinations in the United States and China. Hum Vaccine Immunother. 2021;17(8):2639–46.
Shen X, Dong H, Feng J, Jiang H, Dowling R, Lu Z, et al. Assessing the COVID-19 vaccine hesitancy in the Chinese adults using a generalized vaccine hesitancy survey instrument. Hum Vaccine Immunother. 2021;17(11):4005–12.
Freeman D, Loe BS, Chadwick A, Vaccari C, Waite F, Rosebrock L, et al. COVID-19 vaccine hesitancy in the UK: the Oxford coronavirus explanations, attitudes, and narratives survey (Oceans) II. Psychol Med. 2020:1–15.
Freeman D, Loe BS, Yu LM, Freeman J, Chadwick A, Vaccari C, et al. Effects of different types of written vaccination information on COVID-19 vaccine hesitancy in the UK (OCEANS-III): a single-blind, parallel-group, randomised controlled trial. Lancet Public health. 2021;6(6):e416–27.
Duong TV, Lin CY, Chen SC, Huang YK, Okan O, Dadaczynski K, et al. Oxford COVID-19 vaccine hesitancy in school principals: impacts of gender, well-being, and coronavirus-related health literacy. Vaccines. 2021;9(9):985.
Sherman SM, Smith LE, Sim J, Amlôt R, Cutts M, Dasch H, et al. COVID-19 vaccination intention in the UK: results from the COVID-19 vaccination acceptability study (CoVAccS), a nationally representative cross-sectional survey. Hum Vaccin Immunother. 2021;17(6):1612–21.
Fernandes N, Costa D, Costa D, Keating J, Arantes J. Predicting COVID-19 vaccination intention: the determinants of vaccine hesitancy. Vaccines. 2021;9(10):1161.
Organization WH. WHO Director-General’s opening remarks on Member State Information Session on COVID-19 2022 [cited 2022 18 Apr]. Available from: https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-on-member-state-information-session-on-covid-192022.
Opel DJ, Taylor JA, Mangione-Smith R, Solomon C, Zhao C, Catz S, et al. Validity and reliability of a survey to identify vaccine-hesitant parents. Vaccine. 2011;29(38):6598–605.
Xu Y, Xu D, Luo L, Ma F, Wang P, Li H, et al. A cross-sectional survey on COVID-19 vaccine hesitancy among parents from Shandong vs. Zhejiang. Front Public Health. 2021;9: 779720.
Bianco A, Mascaro V, Zucco R, Pavia M. Parent perspectives on childhood vaccination: how to deal with vaccine hesitancy and refusal? Vaccine. 2019;37(7):984–90.
Abd Halim H, Abdul-Razak S, Md Yasin M, Isa MR. Validation study of the Parent Attitudes About Childhood Vaccines (PACV) questionnaire: the Malay version. Hum Vaccin Immunother. 2020;16(5):1040–9.
Olarewaju VO, Jafflin K, Deml MJ, Zimmermann C, Sonderegger J, Preda T, et al. Application of the Parent Attitudes about Childhood Vaccines (PACV) survey in three national languages in Switzerland: exploratory factor analysis and Mokken scale analysis. Hum Vaccin Immunother. 2021;17(8):2652–60.
Opel DJ, Mangione-Smith R, Taylor JA, Korfiatis C, Wiese C, Catz S, et al. Development of a survey to identify vaccine-hesitant parents: the parent attitudes about childhood vaccines survey. Hum Vaccin. 2011;7(4):419–25.
Spector PE. Summated rating scale construction: an introduction. Bms Bull Sociol Methodol. 1992;82(35):59.
Preacher KJ, Coffman DL. Computing power and minimum sample size for RMSEA 2006 [Available from: http://quantpsy.org/rmsea/rmsea.htm.
MacCallum RC, Browne MW, Sugawara HM. Power analysis and determination of sample size for covariance structure modeling. Psychol Methods. 1996;1(2):130–49.
Switzer GE, Wisniewski SR, Belle SH, Dew MA, Schultz R. Selecting, developing, and evaluating research instruments. Soc Psychiatry Psychiatr Epidemiol. 1999;34(8):399–409.
Quinn SC, Jamison AM, An J, Hancock GR, Freimuth VS. Measuring vaccine hesitancy, confidence, trust and flu vaccine uptake: results of a national survey of White and African American adults. Vaccine. 2019;37(9):1168–73.
Gilkey MB, Magnus BE, Reiter PL, McRee AL, Dempsey AF, Brewer NT. The Vaccination Confidence Scale: a brief measure of parents’ vaccination beliefs. Vaccine. 2014;32(47):6259–65.
Gilkey MB, Reiter PL, Magnus BE, McRee AL, Dempsey AF, Brewer NT. Validation of the vaccination confidence scale: a brief measure to identify parents at risk for refusing adolescent vaccines. Acad Pediatr. 2016;16(1):42–9.
Flerd A. Discovering statistics using SPSS. 3rd ed. SAGE Publications; 2009.
Kocoglu-Tanyer D, Dengiz KS, Sacikara Z. Development and psychometric properties of the public attitude towards vaccination scale—health belief model. J Adv Nurs. 2020;76(6):1458–68.
Steiger JH. Structural model evaluation and modification: an interval estimation approach. Multivar Behav Res. 1990;25(2):173–80.
Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107(2):238–46.
Tucker LR, Lewis C. A reliability coefficient for maximum likelihood factor analysis. Psychometrika. 1973;38(1):1–10.
Lt H, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6(1):1–55.
Gomez R, Stavropoulos V. Confirmatory factor analysis and exploratory structural equation modeling of the structure of attention-deficit/hyperactivity disorder symptoms in adults. Assessment. 2021;28(6):1570–82.
Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Long JS, editors. Testing structural equation models. Newbury Park: Sage; 1993. p. 136–62.
DeVon HA, Block ME, Moyle-Wright P, Ernst DM, Hayden SJ, Lazzara DJ, et al. A psychometric toolbox for testing validity and reliability. J Nurs Scholarship. 2007;39(2):155–64.
Fornell C, Larcker DF. Structural equation models with unobservable variables and measurement error: algebra and statistics. Los Angeles: Sage Publications Sage CA; 1981.
Bagozzi RP, Yi Y. Specification, evaluation, and interpretation of structural equation models. J Acad Mark Sci. 2012;40(1):8–34.
Landis JRK, Gary G. The measurement of observer agreement for categorical data. Biometrics. 1977:159–74.
Leung GM, Ho LM, Chan SK, Ho SY, Bacon-Shone J, Choy RY, et al. Longitudinal assessment of community psychobehavioral responses during and after the 2003 outbreak of severe acute respiratory syndrome in Hong Kong. Clin Infect Dis. 2005;40(12):1713–20.
The L. COVID-19 vaccines: no time for complacency. Lancet (London, England). 2020;396(10263):1607.
González-Block M, Pelcastre-Villafuerte BE, Riva Knauth D, Fachel-Leal A, Comes Y, Crocco P, et al. Influenza vaccination hesitancy in large urban centers in South America. Qualitative analysis of confidence, complacency and convenience across risk groups. PLoS ONE. 2021;16(8):e0256040.
Kose S, Mandiracioglu A, Sahin S, Kaynar T, Karbus O, Ozbel Y. Vaccine hesitancy of the COVID-19 by health care personnel. Int J Clin Pract. 2021;75(5):e13917.
Chan NN, Ong KW, Siau CS, Lee KW, Peh SC, Yacob S, et al. The lived experiences of a COVID-19 immunization programme: vaccine hesitancy and vaccine refusal. BMC Public Health. 2022;22(1):1–13.
Betsch C, Schmid P, Heinemeier D, Korn L, Holtmann C, Böhm R. Beyond confidence: development of a measure assessing the 5C psychological antecedents of vaccination. PLoS ONE. 2018;13(12): e0208601.
Lin C, Tu P, Beitsch LM. Confidence and receptivity for COVID-19 vaccines: a rapid systematic review. Vaccines. 2020;9(1).
Schwarzinger M, Watson V, Arwidson P, Alla F, Luchini S. COVID-19 vaccine hesitancy in a representative working-age population in France: a survey experiment based on vaccine characteristics. Lancet Public Health. 2021;6(4):e210–21.
Noronha V, Abraham G, Bondili SK, Rajpurohit A, Menon RP, Gattani S, et al. COVID-19 vaccine uptake and vaccine hesitancy in Indian patients with cancer: a questionnaire-based survey. Cancer Res Stat Treat. 2021;4(2):211.
Li M, Zheng Y, Luo Y, Ren J, Jiang L, Tang J, et al. Hesitancy toward COVID-19 vaccines among medical students in Southwest China: a cross-sectional study. Hum Vaccines Immunother. 2021;17:4021.
The authors acknowledge the contributions of all associated workers and students of the seven collaborating institutions for the support in data collection.
This study was funded by the Asian Regional Special Cooperation Fund of National Health Commission of the People’s Republic of China (BLXM01).
Ethics approval and consent to participate
This study has been approved by the Ethical Review Committee of Chinese Center for Disease Control and Prevention on December 4th, 2020 (approval number: 202020). An electronic informed consent was provided before the start of the questionnaire survey. Upon completion of the informed consent, the study participants filled in the on-line questionnaire. And all methods in this study were carried out in accordance with relevant guidelines and regulations.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Huang, Y., Wu, Y., Dai, Z. et al. Psychometric validation of a chinese version of COVID-19 vaccine hesitancy scale: a cross-sectional study. BMC Infect Dis 22, 765 (2022). https://doi.org/10.1186/s12879-022-07746-z
- Vaccine hesitancy