An often key component to coordinating surveillance activities across distributed networks is the design and implementation of a common data model (CDM). The purpose of this study was to evaluate two drug safety surveillance CDMs from an ecosystem perspective to better understand how differences in CDMs and analytic tools affect usability and interpretation of results.
Humana claims data from 2007 to 2012 were mapped to Observational Medical Outcomes Partnership (OMOP) and Mini-Sentinel CDMs. Data were described and compared at the patient level by source code and mapped concepts. Study cohort construction and effect estimates were also compared using two different analytical methods—one based on a new user design implementing a high-dimensional propensity score (HDPS) algorithm and the other based on univariate self-controlled case series (SCCS) design—across six established positive drug-outcome pairs to learn how differences in CDMs and analytics influence steps in the database analytic process and results.
Claims data for approximately 7.7 million Humana health plan members were transformed into the two CDMs. Three health outcome cohorts and two drug cohorts showed differences in cohort size and constituency between Mini-Sentinel and OMOP CDMs, which was a result of multiple factors. Overall, the implementation of the HDPS procedure on Mini-Sentinel CDM detected more known positive associations than that on OMOP CDM. The SCCS method results were comparable on both CDMs. Differences in the implementation of the HDPS procedure between the two CDMs were identified; analytic model and risk period specification had a significant impact on the performance of the HDPS procedure on OMOP CDM.
Differences were observed between OMOP and Mini-Sentinel CDMs. The analysis of both CDMs at the data model level indicated that such conceptual differences had only a slight but not significant impact on identifying known safety associations. Our results show that differences at the ecosystem level of analyses across the CDMs can lead to strikingly different risk estimations, but this can be primarily attributed to the choices of analytic approach and their implementation in the community-developed analytic tools. The opportunities of using CDMs are clear, but our study shows the need for judicious comparison of analyses across the CDMs. Our work emphasizes the need for ongoing efforts to ensure sustainable transparent platforms to maintain and develop CDMs and associated tools for effective safety surveillance.
This is a preview of subscription content,to check access.
Access this article
Similar content being viewed by others
MedDRA® terminology is the international medical terminology developed under the auspices of the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH). MedDRA® trademark is owned by the International Federation of Pharmaceutical Manufacturers and Associations (IFPMA) on behalf of ICH.
Behrman RE, Benner JS, Brown JS, McClellan M, Woodcock J, Platt R. Developing the Sentinel system—a national resource for evidence development. N Engl J Med. 2011;364(6):498–9. doi:10.1056/NEJMp1014427.
Suissa S, Garbe E. Primer: administrative health databases in observational studies of drug effects–advantages and disadvantages. Nat Clin Pract Rheumatol. 2007;3(12):725–32. doi:10.1038/ncprheum0652.
Brown JS, Holmes JH, Shah K, Hall K, Lazarus R, Platt R. Distributed health data networks: a practical and preferred approach to multi-institutional evaluations of comparative effectiveness, safety, and quality of care. Med Care. 2010;48(6 Suppl):S45–51. doi:10.1097/MLR.0b013e3181d9919f.
Stang PE, Ryan PB, Racoosin JA, Overhage JM, Hartzema AG, Reich C, et al. Advancing the science for active surveillance: rationale and design for the Observational Medical Outcomes Partnership. Ann Intern Med. 2010;153(9):600–6.
Curtis LH, Weiner MG, Boudreau DM, Cooper WO, Daniel GW, Nair VP, et al. Design considerations, architecture, and use of the Mini-Sentinel distributed data system. Pharmacoepidemiol Drug Saf. 2012;21(S1):23–31.
McGraw D, Rosati K, Evans B. A policy framework for public health uses of electronic health data. Pharmacoepidemiol Drug Saf. 2012;21:18–22. doi:10.1002/pds.2319.
Observational Medical Outcomes Partnership. OMOP project website. http://omop.fnih.org. Accessed 15 May 2015.
Ryan PB, Madigan D, Stang PE, Overhage JM, Racoosin JA, Hartzema AG. Empirical assessment of methods for risk identification in healthcare data: results from the experiments of the Observational Medical Outcomes Partnership. Stat Med. 2012;31(30):4401–15. doi:10.1002/sim.5620.
FDA Mini-Sentinel Program. Mini-Sentinel project website. http://www.mini-sentinel.org/. Accessed 15 May 2015.
Exploring and Understanding Adverse Drug Reactions (EU-ADR) project. EU-ADR project website. http://euadr-project.org/. Accessed 15 May 2015.
Toh S, Avorn J, D’Agostino RB, Gurwitz JH, Psaty BM, Rothman KJ, et al. Re-using Mini-Sentinel data following rapid assessments of potential safety signals via modular analytic programs. Pharmacoepidemiol Drug Saf. 2013;22(10):1036–45.
Overhage JM, Ryan PB, Reich CG, Hartzema AG, Stang PE. Validation of a common data model for active safety surveillance research. J Am Med Inform Assoc. 2012;19(1):54–60. doi:10.1136/amiajnl-2011-000376.
Zhou X, Murugesan S, Bhullar H, Liu Q, Cai B, Wentworth C, et al. An evaluation of the THIN database in the OMOP common data model for active drug safety surveillance. Drug Saf. 2013;36(2):119–34.
Coloma P, Avillach P, Salvo F, Schuemie M, Ferrajolo C, Pariente A, et al. A reference standard for evaluation of methods for drug safety signal detection using electronic healthcare record databases. Drug Saf. 2013;36(1):13–23. doi:10.1007/s40264-012-0002-x.
Ryan PB, Schuemie MJ, Welebob E, Duke J, Valentine S, Hartzema AG. Defining a reference set to support methodological research in drug safety. Drug Saf. 2013;36(1):33–47.
Schneider G, Kachroo S, Jones N, Crean S, Rotella P, Avetisyan R, et al. A systematic review of validated methods for identifying anaphylaxis, including anaphylactic shock and angioneurotic edema, using administrative and claims data. Pharmacoepidemiol Drug Saf. 2012;21(S1):240–7.
Lo Re V, Haynes K, Goldberg D, Forde KA, Carbonari DM, Leidl KB, et al. Validity of diagnostic codes to identify cases of severe acute liver injury in the US Food and Drug Administration’s Mini-Sentinel Distributed Database. Pharmacoepidemiol Drug Saf. 2013;22(8):861–72.
Noren GN, Hopstadius J, Bate A, Edwards IR. Safety surveillance of longitudinal databases: methodological considerations. Pharmacoepidemiol Drug Saf. 2011;20(7):714–7. doi:10.1002/pds.2151.
Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H, Brookhart MA. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology. 2009;20(4):512–22.
FDA Mini-Sentinel Program. Prospective Routine Observational Monitoring Program Tools (PROMPT) users guide. http://www.mini-sentinel.org/methods/methods_development/details.aspx?ID=1045. Accessed 10 Jan 2014.
Whitaker HJ, Hocine MN, Farrington CP. The methodology of self-controlled case series studies. Stat Methods Med Res. 2009;18(1):7–26.
Madigan D. Implementing study designs in the OMOP environment. OMOP Symposium Day 1, Workshop 5 Nov. 2013. http://omop.org/sites/default/files/OMOP%20Symposium%20Day%201%20Workshop%205Nov2013%20Madigan%20study%20design.pdf. Accessed 11 May 2015.
Gagne JJ, Rassen JA, Walker AM, Glynn RJ, Schneeweiss S. Active safety monitoring of new medical products using electronic healthcare data: selecting alerting rules. Epidemiology. 2012;23(2):238.
Reich CG, Ryan PB, Suchard MA. The impact of drug and outcome prevalence on the feasibility and performance of analytical methods for a risk identification and analysis system. Drug Saf. 2013;36(Suppl 1):S195–204.
Ryan PB, Schuemie MJ, Gruber S, Zorych I, Madigan D. Empirical performance of a new user cohort method: lessons for developing a risk identification and analysis system. Drug Saf. 2013;36(Suppl 1):S59–72. doi:10.1007/s40264-013-0099-6.
OMOP CDM Specifications version 4.0 and OMOP Standard Vocabulary Specification version 4.4. http://omop.fnih.org. Accessed 30 Oct 2013.
Observational Medical Outcomes Partnership. OMOP methods library. http://archive-omop.fnih.org/MethodsLibrary. Accessed 10 Dec 2014.
This research was conceived, funded, and carried out collaboratively by Humana Inc., Pfizer Inc., and Comprehensive Health Insights, Inc. The research concept was approved by the Joint Research Governance Committee, consisting of Humana Inc. and Pfizer Inc. employees, and plans to publish results which were made known prior to commencing the study. We wish to thank Dr. James Harnett, Mr. Daniel Wiederkehr, and Dr. Robert Reynolds at Pfizer Inc. for their support and advice to this study. Prior to study initiation, the research protocol was reviewed and approved by an independent institutional review board (IRB) (Schulman Associates, study #13-03391).
Conflict of interest
Yihua Xu, Brandon Suehs, Keran Moll, and Margaret Pasquale are employees of Comprehensive Health Insights, a wholly owned subsidiary of Humana. Brandon Suehs is a stockholder of Humana. Vinit Nair is an employee of Comprehensive Health Insights, and serves as the primary investigator from Humana for both the Observational Medical Outcomes Partnership and the Mini-Sentinel program. Xiaofeng Zhou, Qing Liu, and Andrew Bate are employees and stockholders of Pfizer Inc. Abraham Hartzema, Michael Kahn, Brian Sauer, and Yola Moride received consulting fees and travel expenses in connection with providing input on the design of the study and interpretation of results.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Xu, Y., Zhou, X., Suehs, B.T. et al. A Comparative Assessment of Observational Medical Outcomes Partnership and Mini-Sentinel Common Data Models and Analytics: Implications for Active Drug Safety Surveillance. Drug Saf 38, 749–765 (2015). https://doi.org/10.1007/s40264-015-0297-5