Skip to main content

Analysis of seven SARS-CoV-2 rapid antigen tests in detecting omicron (B.1.1.529) versus delta (B.1.617.2) using cell culture supernatants and clinical specimens

Abstract

Purpose

Omicron is rapidly spreading as a new SARS-CoV-2 variant of concern (VOC). The question whether this new variant has an impact on SARS-CoV-2 rapid antigen test (RAT) performance is of utmost importance. To obtain an initial estimate regarding differences of RATs in detecting omicron and delta, seven commonly used SARS-CoV-2 RATs from different manufacturers were analysed using cell culture supernatants and clinical specimens.

Methods

For this purpose, cell culture-expanded omicron and delta preparations were serially diluted in Dulbecco’s modified Eagle’s Medium (DMEM) and the Limit of Detection (LoD) for both VOCs was determined. Additionally, clinical specimens stored in viral transport media or saline (n = 51) were investigated to complement in vitro results with cell culture supernatants. Ct values and RNA concentrations were determined via quantitative reverse transcription polymerase chain reaction (RT-qPCR).

Results

The in vitro determination of the LoD showed no obvious differences in detection of omicron and delta for the RATs examined. The LoD in this study was at a dilution level of 1:1,000 (corresponding to 3.0—5.6 × 106 RNA copies/mL) for tests I–V and at a dilution level of 1:100 (corresponding to 3.7—4.9 × 107 RNA copies/mL) for tests VI and VII. Based on clinical specimens, no obvious differences were observed between RAT positivity rates when comparing omicron to delta in this study setting. Overall positivity rates varied between manufacturers with 30–81% for omicron and 42–71% for delta. Test VII was only conducted in vitro with cell culture supernatants for feasibility reasons. In the range of Ct < 23, positivity rates were 50–100% for omicron and 67–93% for delta.

Conclusion

In this study, RATs from various manufacturers were investigated, which displayed no obvious differences in terms of analytical LoD in vitro and RAT positivity rates based on clinical samples comparing the VOCs omicron and delta. However, differences between tests produced by various manufacturers were detected. In terms of clinical samples, a focus of this study was on specimens with high virus concentrations. Further systematic, clinical and laboratory studies utilizing large datasets are urgently needed to confirm reliable performance in terms of sensitivity and specificity for all individual RATs and SARS-CoV-2 variants.

Introduction

The new SARS-CoV-2 variant of concern (VOC) omicron, first described publicly on November 24th 2021 in South Africa, is spreading rapidly throughout the globe, overtaking delta as the dominant variant in many countries [1, 2]. Thus, the question whether this new variant has an impact on diagnostic equipment performance is of utmost importance. Besides multiple mutations in the spike (S)-protein, omicron BA.1 shows three single characteristic point mutations and one deletion in the nucleocapsid protein (N-protein) (P13L, Δ31-33, R203K, G204R) [3]. Since the N-protein is the target protein of most SARS-CoV-2 rapid antigen tests (RATs), it is crucial to investigate whether the corresponding mutations affect RAT performance [4].

To obtain a basic understanding of the performance of RATs in detecting omicron, RATs for professional use and self-application from seven widely used manufacturers (Table 1) listed by the German Federal Institute for Drugs and Medicinal Devices ‘BfArM’ (except for test VII) were tested with in vitro cell culture supernatants (test I to VII) and clinical specimens (test I to VI). All deployed tests target the SARS-CoV-2 N-protein and are lateral flow immunoassays [5]. Tests I–VI are colourimetric, whereas test VII is fluorescence based and requires the manufacturer’s specific analyser for interpretation of results.

Table 1 RATs investigated in this study

Laboratory performance of RATs with cell culture-derived omicron and delta

RAT performance was evaluated in vitro by determining analytical Limits of Detection (LoDs) in DMEM (Dulbecco’s modified Eagle’s Medium) with infectious SARS-CoV-2 omicron (B.1.1.529, BA.1) and delta (B.1.617.2, AY.122) samples (confirmed by Whole Genome Sequencing (WGS)) derived from cell culture in Vero E6 cells (ATCC CRL-1586, RRID: CVCL_0574) as described [6]. Viral load was determined by quantitative real-time PCR using the EDX SARS-CoV-2 Positive Run Control (Exact Diagnostics, Fort Worth, TX USA) as calibrator. The LoD was determined as the dilution level at which all three replicates were considered positive by two or, in case of ambiguity, three investigators, independently and blinded regarding dilution levels. For test VII, a sample analyser from the manufacturer was used for result interpretation.

Based on the analytical LoD of RATs evaluated, no distinct performance differences between omicron and delta were determined in this investigated setting (Table 2). Five tests were able to detect both virus variants at least up to a dilution of 1:1,000 (corresponding to 3.0—5.6 × 106 RNA copies/mL), while tests VI and VII displayed positive results only up to a dilution level of 1:100 (corresponding to 3.7—4.9 × 107 RNA copies/mL) (Table 2). Some tests showed positive results at higher dilution levels for either omicron or both VOCs under study but in less than three replicates. It is important to mention here that there were small differences in concentrations between the omicron and delta dilutions, with omicron containing slightly higher genome copy numbers (Table 2).

Table 2 Limits of detection with cell culture supernatants from omicron and delta based on virus dilution series in DMEM

Investigating SARS-CoV-2 RATs with clinical specimens containing omicron or delta

Additionally, RAT performance was analysed utilizing clinical specimens positive for omicron or delta to complete in vitro results with cell culture supernatants. Clinical specimens (n = 51; nasopharyngeal/oropharyngeal swabs in viral transport media (VTM) or saline (NaCl)) were collected between the 13th and 24th of December 2021 at the Bavarian Health and Food Safety Authority as remnants of routine diagnostics and stored at 2–8 °C (Supplementary Table). Lineages were confirmed by variant-specific RT-qPCR (ViRSNiP assay, TIB MOLBIOL, Berlin, Germany) and WGS (except no. 24: due to low virus concentration confirmation only by variant-specific RT-qPCR). All clinical omicron samples were B.1.1.529, BA.1, while delta samples varied in subtype. To minimize the influence of confounding factors (e.g. medium, sampling date) and to generate comparable groups of samples containing the variants under study, panels with an approximately equal distribution of I) Cycle threshold (Ct) values, II) transport media and III) sampling dates were assembled. Even so, minor differences between panels remained present (Supplementary Table). Only viral transport media without guanidinium were included because of its protein-denaturizing effects. To obtain comparable Ct values within this study, all clinical samples were quantified in one collective RT-qPCR run prior to RAT testing as described previously (AmpliCube Coronavirus SARS-CoV-2 RT-qPCR Kit (Mikrogen, Neuried, Germany) on a BioRad CFX96 Real-Time RT-qPCR Detection System (BioRad, Feldkirchen, Germany), extraction on a Microlab STAR (Hamilton, Reno, NV, USA), using the RNAdvance Viral Large GRP Kit (Beckmann-Coulter Life Sciences, Nyon, Switzerland)) [6, 7]. A majority of clinical samples utilized in this study contained high viral loads.

For reasons of standardization and comparability, the methodological procedure was largely based on the protocol developed by the Paul-Ehrlich-Institute but without prior pooling of samples [5]. In short, 50 µL of each sample was inoculated into the respective RAT buffers by pipetting. Sample media were either Virocult (Medical Wire and Equipment Virocult, Wiltshire, UK) or saline. RT-qPCR-negative patient samples in Virocult and saline were included as negative controls for all RATs evaluated. Further procedural steps, such as number of drops or incubation time, were carried out strictly according to the manufacturer’s instructions of the respective RATs in a BSL-3 laboratory. Samples were considered positive if two investigators could still clearly visually identify positive test bands in duplicates independently and blinded regarding concentration levels. In case of ambiguity, a third investigator was consulted.

Statistical analysis of results with clinical specimens

Delta and omicron samples were tested for differences of their Ct values in respect to their variance (F-test, two sided) and mean (t-test, two sided, equal variance). No differences were found at a statistical significance level (p-values ≥ 0.05). Based on these data, three hypotheses were tested in an exploratory approach (null hypotheses) at a statistical significance level of p < 0.05 (total samples without stratification for Ct values; using Fisher’s exact test and Chi-square test for homogeneity):

  1. 1.

    Test results of sensitivity towards omicron do not differ from test results for delta (Fisher’s exact test).

  2. 2.

    Tests do not perform significantly different in respect to sensitivity towards delta (Chi-square test for homogeneity).

  3. 3.

    Tests do not perform significantly different in respect to sensitivity towards omicron (Chi-square test for homogeneity).

Hypothesis testing was performed using statistical tools available online [8, 9].

SARS-CoV-2 RAT results with clinical specimens of omicron and delta

In this study setting, the investigation of RATs with clinical samples showed no distinct differences between positivity rates of omicron versus delta, which is in line with in vitro laboratory analysis using cell culture supernatants. For Ct > 23, the sample panel was too small and partially unbalanced in terms of viral loads between omicron and delta, so that no distinct conclusions based on clinical specimens could be drawn for Ct > 23.

A total of 27 omicron and 24 delta clinical samples were examined. Depending on the manufacturer, 30–81% of all omicron samples and 42–71% of all delta samples were detected as positive (Table 3). There were no clear performance differences between the VOCs omicron and delta for high viral loads (Ct < 23; mean Ct 20.3 (omicron) and 19.4 (delta)) in five out of six tests (positivity rate: 80–100%). Test VI showed lower positivity rates with 50% for omicron and 67% for delta at Ct < 23. For Ct values 23–25.5, positivity rates varied strongly (0–100%) between different RATs with higher rates for omicron. For Ct > 25.5, only one positive sample was detected in one of the RATs (test V, omicron). Test V showed very faint test bands in duplicates in a negative saline control (Supplementary Table).

Table 3 Positivity rates of RATs with clinical samples of omicron and delta

The null hypotheses 1 and 2 could not be rejected. Null hypothesis 3 was rejected at a p-level < 0.05, indicating inhomogeneity of test performance among the six tests under investigation in respect to omicron. The Chi-square test for homogeneity was repeated in respect to the sensitivity of this result towards the performance of test VI and regarding the possible influence of high Ct values. After exclusion of test VI, the null hypothesis could not be rejected anymore (p = 0.569). This finding remained stable when also excluding samples with Ct values > 23 (p = 0.465).

Discussion

Clinical and laboratory data evaluating the effects of the new VOC omicron on RAT performance are crucial but scarce. We investigated seven widely used SARS-CoV-2 RATs and found no distinct differences in analytical LoDs and positivity rates with clinical samples comparing the two VOCs omicron and delta, whereas test VII was only tested for analytical LoD. However, there were partial differences in results between various manufacturers. Differences in the general sensitivity of tests were also reported by the German Paul-Ehrlich-Institute and the Robert-Koch-Institute who tested 122 CE-certified RATs systematically with 26 RATs missing the sensitivity criteria [5]. In this study, results of the laboratory investigations using cell culture supernatants showed no clear differences in detecting omicron compared to delta. This result is in line with results of investigations utilizing clinical samples at high viral loads. Slightly lower positivity rates for omicron compared to delta with clinical samples for test VI were not observed in results with cell culture supernatants. In general, this study utilizes analytical laboratory data. Data based on clinical specimens were only considered in a descriptive way due to small clinical sample size and partly varying Ct value distributions in samples positive for omicron or delta. Nevertheless, among the six RATs tested with clinical specimens, exploratory hypothesis testing indicated a statistically significant heterogeneity in test performance regarding test sensitivity towards omicron (p < 0.05) between RATs of different manufacturers. This heterogeneity could be linked to the poorer performance of one of the six tests under investigation. Within these six tests under investigation, no significant difference in respect to test sensitivity towards omicron as compared to test sensitivity towards delta was detected.

Results of this study agree with those of Deerain et al. (2021), who described no distinct performance differences between omicron and delta for ten evaluated RATs using cultured virus dilutions, which included two RATs (test I and test IV) also present in our study [10]. Similarly, Regan et al. (2021) reported comparable performances between omicron and delta, particularly at high virus concentrations, for the Abbott BinaxNow rapid antigen test using clinical samples [11]. Other studies, however, described lower sensitivity of tests evaluated with omicron compared to delta [12, 13]. In comparison to RATs investigated in our study, Bekliz et al. (2021) showed lower sensitivities with omicron compared to delta for test IV with clinical specimens and cell culture-expanded samples [12]. Osterman et al. (2022) showed lower sensitivities for omicron in test III with clinical samples, while results with cell culture-expanded samples were reversed [13].

It is important to mention some limitations of the current study, which could influence interpretation of the results. The experimental set-up with tenfold dilution steps may mask differences in LoDs smaller than the tested tenfold increments. Additionally, slight variations in the concentrations of omicron and delta preparations should be considered, as this fact could influence the results of the current study. In previous studies, DMEM was considered a suitable medium for investigations of cell culture samples [6]. As only few RATs are approved by the manufacturer for testing from specific VTMs and for purposes of consistency, a sample volume of 50 µL was elected. This, however, may not exactly reflect the suggested volumes of certain manufacturers and was chosen for comparability among the tests and to other studies [5, 6, 11, 13]. Moreover, there could be dilution effects introduced due to the variable buffer volumes provided by the RAT manufacturers and, additionally, the differences in sample loaded onto the test cassette according to the respective manufacturer’s instructions. Test band intensities of tests I–VI were quantified by visual inspection instead of software-guided image analysis, which does reflect the conditions present in the clinical and practical application of these tests but may lead to investigator-dependent variations in the perceived signal strength. An exemption is test VII, which was interpreted by a test-specific detection device as required by the manufacturer. RAT investigations with clinical specimens were conducted on two different test days. As residual samples from routine diagnostics were used, some samples exceeded the storage time specified by the manufacturers. Respective samples were submitted from different external sources and were processed on different days as part of routine diagnostics. Uniform conditions of sample storage (i.e. cold chain) on the part of the submitting institution or during transport can therefore not be guaranteed. A great benefit of this study is that examinations were not only conducted with cell culture supernatants, but also with clinical samples. In addition, the methodological approach was largely based on those of large studies, such as the Paul-Ehrlich-Institute and on earlier publications of the working group concerning the analytical LoD of RATs with other VOCs [5, 6]. A standardized procedure was established for RAT investigation with replicable virus and clinical samples of routine diagnostics, and all tests were examined on one, or in the case of clinical samples, a maximum of two consecutive test days.

Overall, the RATs examined detected omicron comparably to delta in this study. Analytical LoD and positivity rates were determined using cell culture-expanded virus and clinical samples with a focus on high virus concentrations. However, differences in LoD and positivity rates were found between the different manufacturers. Whether the general performance of RATs from different manufacturers is varying systematically in a relevant magnitude is subject to further research, taking the limited study size and possible chance effects into account. Besides sensitivity, specificity of RATs plays a decisive role within RAT evaluation. Corresponding systematic clinical and laboratory studies with large sample sizes for all individual tests and variants are urgently needed to confirm reliable RAT performance.

References

  1. World Health Organization. Classification of Omicron (B.1.1.529): SARS-CoV-2 Variant of Concern. 2021. https://www.who.int/news/item/26-11-2021-classification-of-omicron-(b.1.1.529)-sars-cov-2-variant-of-concern Accessed 20 Jan 2022.

  2. Viana R, Moyo S, Amoako DG, Tegally H, Scheepers C, Althaus CL, et al. Rapid epidemic expansion of the SARS-CoV-2 Omicron variant in southern Africa. Nature. 2022. https://doi.org/10.1038/d41586-021-03832-5.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Latif AA, Mullen JL, Alkuzweny M, Tsueng G, Cano M, Haag E, et al. B.1.1.529 Lineage Report. outbreak.info. 2022. https://outbreak.info/situation-reports?pango=B.1.1.529 Accessed 20 Jan 2022.

  4. Ferré VM, Peiffer-Smadja N, Visseaux B, Descamps D, Ghosn J, Charpentier C. Omicron SARS-CoV-2 variant: What we know and what we don’t. Anaesthesia Crit Care Pain Med. 2021;41(1):100998–100998. https://doi.org/10.1016/j.accpm.2021.100998.

    Article  Google Scholar 

  5. Scheiblauer H, Filomena A, Nitsche A, Puyskens A, Corman VM, Drosten C, et al. Comparative sensitivity evaluation for 122 CE-marked rapid diagnostic tests for SARS-CoV-2 antigen, Germany, September 2020 to April 2021. Eurosurveillance. 2021; 26(44). https://doi.org/10.2807/1560-7917.es.2021.26.44.2100441

  6. Jungnick S, Hobmaier B, Mautner L, Hoyos M, Haase M, Baiker A, et al. In Vitro Rapid Antigen Test Performance with the SARS-CoV-2 Variants of Concern B.1.1.7 (Alpha), B.1.351 (Beta), P.1 (Gamma), and B.1.617.2 (Delta). Microorganisms. 2021; 9(9): 1967. https://doi.org/10.3390/microorganisms9091967

  7. Eberle U, Wimmer C, Huber I, Neubauer-Juric A, Valenza G, Ackermann N, et al. Comparison of nine different commercially available molecular assays for detection of SARS-CoV-2 RNA. Eur J Clin Microbiol Infect Dis. 2021. https://doi.org/10.1007/s10096-021-04159-9.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Preacher KJ and Briggs NE. Calculation for Fisher's Exact Test: An interactive calculation tool for Fisher's exact probability test for 2 x 2 tables [Computer software]. 2001. http://quantpsy.org. Accessed 19 Apr 2022.

  9. Preacher KJ. Calculation for the chi-square test: An interactive calculation tool for chi-square tests of goodness of fit and independence [Computer software]. 2001. http://quantpsy.org. Accessed 19 Apr 2022.

  10. Deerain J, Druce J, Tran T, Batty M, Yoga Y, Fennell M, et al. Assessment of the analytical sensitivity of ten lateral flow devices against the SARS-CoV-2 omicron variant. J Clin Microbiol. 2021;60(2):e02479-e2521. https://doi.org/10.1128/jcm.02479-21.

    Article  Google Scholar 

  11. Regan J, Flynn JP, Choudhary MC, Uddin R, Lemieux J, Boucau J, et al. Detection of the Omicron Variant Virus With the Abbott BinaxNow SARS-CoV-2 Rapid Antigen Assay. Open Forum Infectious Diseases. 2022; 9(3). https://doi.org/10.1093/ofid/ofac022

  12. Bekliz M, Perez-Rodriguez F, Pulhach O, Adea K, Melanica SM, Baggio S, et al. Sensitivity of seven SARS-CoV-2 antigen-detecting rapid tests for Omicron variant. medRxiv. 2021; https://doi.org/10.1101/2021.12.18.21268018. Accessed 16 Mar 2022.

  13. Osterman A, Badell I, Basara E, Stern M, Kriesel F, Eletreby M, et al. Impaired detection of omicron by SARS-CoV-2 rapid antigen tests. Med Microbiol Immunol. 2022. https://doi.org/10.1007/s00430-022-00730-z.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the financial support of the Bavarian State Ministry of Health and Care with regard to the provision of rapid antigen tests and the funding of corresponding investigations. The authors specially thank Robert Ethan Rollins for the linguistic revision of this manuscript. Bavarian SARS-CoV-2 Public Health Laboratory Team: Balakin, Vadim; Bartha-Dima, Bernadett; Bengs, Katja; Berger, Anja; Boll, Kerstin; Carl, Anja; Christian, Jürgen; Drdlicek, Juliana; Eisenberger, David; Flechsler, Jennifer; Gerdes, Lars; Githure, George; Govindaswamy, Janani; Hupfer, Christine; Lutmayr, Johannes; Margos, Gabriele; Müller, Roswitha; Nickel, Silke; Pavlovic, Melanie; Pecoraro, Sven; Reichwald, Daniel; Rollins, Robert Ethan; Sahm, Isabel; Schauer, Melanie; Schmidt, Sandra; Schulze, Gesine; Schülein, Anika; Schürmann, Eva-Maria; Scuda, Nelly; Seebach, Judith; Singer, Stefanie; Stellberger, Thorsten; Tuschak, Christian; Zimmermann, Pia.

Funding

Open Access funding enabled and organized by Projekt DEAL. Partial financial support was received from the Bavarian Ministry of Health and Care via the projects Innovative Testing Strategies (ITT) and Bay-VOC.

Author information

Authors and Affiliations

Authors

Consortia

Contributions

SJ, BH and NP conceptualized and conducted the experiments and laboratory work with focus on the implementation of the RATs and RT-qPCR and wrote manuscript draft. LM, MHo, MHa and AB conceptualized and conducted the experiments and laboratory work with focus on the cultivation of virus preparations and supported project coordination. RK, MB, BT, MO, CW, VF, UE, SH, AS, AD, ASp, BS, CB and LW participated in lab testing, whole genome sequencing, bioinformatic analyses and supported project coordination. SI coordinated the procurement of RATs evaluated. MW, BL, VF, NA, ASi and WJ provided general supervision, administration and project coordination. The Bavarian SARS-CoV-2 Public Health Laboratory Team participated in material acquisition and virus isolation, lab testing, VOC sequencing and bioinformatic analyses.

Corresponding author

Correspondence to Volker Fingerle.

Ethics declarations

Conflict of interest

On behalf of all the authors, the corresponding author states that there is no conflict of interest.

Ethical statement

Ethical approval was not required because all samples used were residual samples from routine diagnostics in the context of secondary use of biological material and without the possibility of inferring patient data. This study is part of the quality assurance in accordance with the German Infection Protection Act.

Consent to participate and consent to publish statements

No consent to participate and no consent to publish were necessary for this study, as no identifiable personal data were gathered.

Additional information

Members of the Bavarian SARS-CoV-2-Public Health Laboratory Team are listed in acknowledgement.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 296 kb)

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Jungnick, S., Hobmaier, B., Paravinja, N. et al. Analysis of seven SARS-CoV-2 rapid antigen tests in detecting omicron (B.1.1.529) versus delta (B.1.617.2) using cell culture supernatants and clinical specimens. Infection (2022). https://doi.org/10.1007/s15010-022-01844-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s15010-022-01844-5

Keywords

  • SARS-CoV-2 rapid antigen test
  • Comparison
  • Omicron (B.1.1.529)
  • Delta (B.1.617.2)
  • Variants of concern