A Method for the Minimization of Competition Bias in Signal Detection from Spontaneous Reporting Databases

Abstract

Introduction

The two methods for minimizing competition bias in signal of disproportionate reporting (SDR) detection—masking factor (MF) and masking ratio (MR)—have focused on the strength of disproportionality for identifying competitors and have been tested using competitors at the drug level.

Objectives

The aim of this study was to develop a method that relies on identifying competitors by considering the proportion of reports of adverse events (AEs) that mention the drug class at an adequate level of drug grouping to increase sensitivity (Se) for SDR unmasking, and its comparison with MF and MR.

Methods

Reports in the French spontaneous reporting database between 2000 and 2005 were selected. Five AEs were considered: myocardial infarction, pancreatitis, aplastic anemia, convulsions, and gastrointestinal bleeding; related reports were retrieved using standardized Medical Dictionary for Regulatory Activities (MedDRA®) queries. Potential competitors of AEs were identified using the developed method, i.e. Competition Index (ComIn), as well as MF and MR. All three methods were tested according to Anatomical Therapeutic Chemical (ATC) classification levels 2–5. For each AE, SDR detection was performed, first in the complete database, and second after removing reports mentioning competitors; SDRs only detected after the removal were unmasked. All unmasked SDRs were validated using the Summary of Product Characteristics, and constituted the reference dataset used for computing the performance for SDR unmasking (area under the curve [AUC], Se).

Results

Performance of the ComIn was highest when considering competitors at ATC level 3 (AUC: 62 %; Se: 52 %); similar results were obtained with MF and MR.

Conclusion

The ComIn could greatly minimize the competition bias in SDR detection. Further study using a larger dataset is needed.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

References

  1. 1.

    Moore N. The past, present and perhaps future of pharmacovigilance: homage to Folke Sjoqvist. Eur J Clin Pharmacol. 2013;69:33–41.

    Article  CAS  PubMed  Google Scholar 

  2. 2.

    Pariente A, Gregoire F, Fourrier-Reglat A, Haramburu F, Moore N. Impact of safety alerts on measures of disproportionality in spontaneous reporting databases: the notoriety bias. Drug Saf. 2007;30:891–8.

    Article  PubMed  Google Scholar 

  3. 3.

    Gould A. Practical pharmacovigilance analysis strategies. Pharmacoepidemiol Drug Saf. 2003;12:559–74.

    Article  PubMed  Google Scholar 

  4. 4.

    Hauben M, Madigan D, Gerrits CM, Walsh L, Van Puijenbroek EP. The role of data mining in pharmacovigilance. Expert Opin Drug Saf. 2005;4:929–48.

    Article  CAS  PubMed  Google Scholar 

  5. 5.

    Pariente A, Didailler M, Avillach P, Miremont-Salamé G, Fourrier-Reglat A, Haramburu F, et al. A potential competition bias in the detection of safety signals from spontaneous reporting databases. Pharmacoepidemiol Drug Saf. 2010;19:1166–71.

    Article  PubMed  Google Scholar 

  6. 6.

    Evans SJW, Waller PC, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf. 2001;10:483–6.

    Article  CAS  PubMed  Google Scholar 

  7. 7.

    Juhlin K, Ye X, Star K, Norén GN. Outlier removal to uncover patterns in adverse drug reaction surveillance—a simple unmasking strategy. Pharmacoepidemiol Drug Saf. 2013;22:1119–29.

    PubMed  Google Scholar 

  8. 8.

    Maignen F, Hauben M, Hung E, Holle LV, Dogne J-M. A conceptual approach to the masking effect of measures of disproportionality. Pharmacoepidemiol Drug Saf. 2014;23:208–17.

    Article  PubMed  Google Scholar 

  9. 9.

    Pariente A, Avillach P, Salvo F, Thiessard F, Miremont-Salamé G, Fourrier-Reglat A, et al. Effect of competition bias in safety signal generation. Drug Saf. 2012;35:855–64.

    Article  PubMed  Google Scholar 

  10. 10.

    Salvo F, Raschi E, Moretti U, Chiarolanza A, Fourrier-Réglat A, Moore N, et al. Pharmacological prioritisation of signals of disproportionate reporting: proposal of an algorithm and pilot evaluation. Eur J Clin Pharmacol. 2014;70:617–25.

    Article  CAS  PubMed  Google Scholar 

  11. 11.

    Maignen F, Hauben M, Hung E, Van Holle L, Dogne J-M. Assessing the extent and impact of the masking effect of disproportionality analyses on two spontaneous reporting systems databases. Pharmacoepidemiol Drug Saf. 2014;23:195–207.

    Article  PubMed  Google Scholar 

  12. 12.

    Salvo F, Raschi E, Moretti U, Chiarolanza A, Fourrier-Réglat A, Moore N, et al. Pharmacological prioritisation of signals of disproportionate reporting: proposal of an algorithm and pilot evaluation. Eur J Clin Pharmacol. 2014;70:617–25.

    Article  CAS  PubMed  Google Scholar 

  13. 13.

    Trifiro G, Pariente A, Coloma PM, Kors JA, Polimeni G, Miremont-Salamé G, et al. Data mining on electronic health record databases for signal detection in pharmacovigilance: which events to monitor? Pharmacoepidemiol Drug Saf. 2009;18:1176–84.

    Article  PubMed  Google Scholar 

  14. 14.

    European Medicines Agency. Summary of product characteristics (EU). 2014. http://www.ema.europa.eu/ema/. Accessed 8 Sep 2015.

  15. 15.

    Vidal 2014: le dictionnaire. 90e ed. Issy-les-Moulineaux: Vidal; 2014.

  16. 16.

    Cantor SB, Kattan MW. Determining the area under the ROC curve for a binary diagnostic test. Med Decis Making. 2000;20:468–70.

    Article  CAS  PubMed  Google Scholar 

  17. 17.

    Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36.

    Article  CAS  PubMed  Google Scholar 

  18. 18.

    van Puijenbroek EP, Bate A, Leufkens HGM, Lindquist M, Orre R, Egberts ACG. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiol Drug Saf. 2002;11:3–10.

    Article  PubMed  Google Scholar 

  19. 19.

    Géniaux H, Assaf D, Miremont-Salamé G, Raspaud B, Gouverneur A, Robinson P, et al. Performance of the standardised MedDRA® queries for case retrieval in the French spontaneous reporting database. Drug Saf. 2014;37:537–42.

    Article  PubMed  Google Scholar 

  20. 20.

    Moore N, Kreft-Jais C, Haramburu F, Noblet C, Andrejak M, Ollagnier M, et al. Reports of hypoglycaemia associated with the use of ACE inhibitors and other drugs: a case/non-case study in the French pharmacovigilance system database. Br J Clin Pharmacol. 2003;44:513–8.

    Article  Google Scholar 

  21. 21.

    Bate A, Lindquist M, Edwards IR, Olsson S, Orre R, Lansner A, et al. A Bayesian neural network method for adverse drug reaction signal generation. Eur J Clin Pharmacol. 1998;54:315–21.

    Article  CAS  PubMed  Google Scholar 

  22. 22.

    DuMouchel W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. Am Stat. 1999;53:177–89.

    Google Scholar 

  23. 23.

    Ahmed I, Thiessard F, Miremont-Salame G, Haramburu F, Kreft-Jais C, Bégaud B, et al. Early detection of pharmacovigilance signals with automated methods based on false discovery rates. Drug Saf. 2012;35:495–506.

    Article  PubMed  Google Scholar 

  24. 24.

    Ryan PB, Schuemie MJ, Welebob E, Duke J, Valentine S, Hartzema AG. Defining a reference set to support methodological research in drug safety. Drug Saf. 2013;36:33–47.

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank all members of the 31 French regional pharmacovigilance centers, as well as the French National Agency for Drug Safety [Agence Nationale de Sécurité des Médicaments et des produits de santé (ANSM)] for providing the data.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Mickael Arnaud.

Ethics declarations

Funding

This study is part of a research project that has received funding from the French National Agency for Drug Safety (ANSM) under Grant agreement number 2013–2050—the SPOON-KIM project. The funding source had no role in study design; collection, analysis, and interpretation of data; writing of the report; and the decision to submit the paper for publication.

Conflict of interest

Mickael Arnaud, Francesco Salvo, Ismaïl Ahmed, Philip Robinson, Nicholas Moore, Bernard Bégaud, Pascale Tubert-Bitter, and Antoine Pariente have no conflicts of interest that are directly relevant to the content of this study.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 1066 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Arnaud, M., Salvo, F., Ahmed, I. et al. A Method for the Minimization of Competition Bias in Signal Detection from Spontaneous Reporting Databases. Drug Saf 39, 251–260 (2016). https://doi.org/10.1007/s40264-015-0375-8

Download citation

Keywords

  • Acute Myocardial Infarction
  • Acute Pancreatitis
  • Aplastic Anemia
  • Anatomical Therapeutic Chemical
  • Reference Dataset