Background

Osteoporosis, a condition characterized by loss of bone mass and increased risk of fracture, affects approximately 10 million individuals in the United States (US) [1]. Approximately 30% of postmenopausal Caucasian women have osteoporosis; their lifetime fracture risk is estimated as 40% [2]. Treatment includes abaloparatide, bisphosphonates, calcitonin, denosumab, raloxifene, and teriparatide, some of which have been linked with increased risk of hypocalcemia and adverse dermatologic events [3,4,5,6,7]. Post-marketing safety studies of medicines often rely on administrative claims databases to identify adverse outcomes following drug exposure [8,9,10,11]. Valid ascertainment of outcomes is essential for accurate results. Yet, the accuracy of identifying hypocalcemia and adverse dermatologic events, especially the more serious events, using claims-identified codes is not well described.

We conducted a study to assess the validity of diagnostic codes in claims data for hypocalcemia and dermatologic events, as compared with medical record confirmation of events, among a population of women with postmenopausal osteoporosis (PMO).

Methods

Data source

This observational study was a retrospective analysis of medical and pharmacy claims data that are part of the Optum Research Database (ORD), a proprietary research database built from provider, facility, and pharmacy claims of a large US health insurer affiliated with Optum. The individuals covered by this health insurance are geographically diverse, and represent 3–4% of the US population.

There was no active enrollment or active follow-up of patients, and no data were directly collected from patients. The New England Institutional Review Board provided oversight during the conduct of this study and its Privacy Board granted a Waiver of Authorization for linkage of claims and medical records.

Study population

The population included women with PMO who had medical and pharmacy coverage between June 2005 and May 2010. Women who were postmenopausal (age 55 years or older) and had diagnosis or treatment codes indicative of osteoporosis or osteoporotic fracture were eligible for the PMO population [12]. A list of relevant codes is provided in an Additional file 1: Table S1. We required at minimum 6 months of continuous enrollment in the health plan preceding the first code indicating PMO (baseline period). We used data from the baseline period to determine cohort eligibility and to characterize baseline attributes for study members.

Identification of potential outcomes through claims data

Potential events were identified through diagnosis codes for hypocalcemia (ICD-9 275.41) or dermatologic adverse events (bullous dermatoses [ICD-9 694.xx], erythematous events [ICD-9 695.1×, 695.5×], or urticaria or rash [ICD-9 708.x, 782.1]) associated with an emergency department (ED) visit or inpatient hospitalization. To capture serious events, we excluded potential cases recorded outside of the ED or inpatient setting, and additionally included only events with codes recorded in the first (primary) position on the claim. Within our data system, the primary diagnostic code on hospitalization-associated claims represents the principal diagnosis, the condition established after study to be chiefly responsible for the admission [13].

Case confirmation through medical record review

We sought medical records for all hypocalcemia, bullous dermatoses, and erythematous events and for a random sample of urticarial or rash events identified from the claims. A chronological list of claims for each of the potential cases was reviewed to determine the site of medical care most likely to yield the necessary information for case confirmation. A physician blinded to the patients’ osteoporosis medication use reviewed the medical records and classified each potential case as: definite case; definite non-case; or insufficient information. Definite cases of hypocalcemia were identified based on the designation of hypocalcemia diagnosis by either the admitting or consulting physician, with confirmation through the lab result that triggered the diagnosis. For both hypocalcemia and dermatologic events, a definite case classification also required attribution of the event as the leading cause for the hospitalization or ED visit (non-incidental cases).

Statistical analysis

We calculated the positive predictive value (PPV) and associated 95% confidence interval (CI) among potential cases for which we obtained a medical record. The PPV was defined as the proportion of potential cases classified as definite cases—calculated overall, and for dermatologic events, by subgroups (bullous dermatoses, erythematous events, and urticaria/rash). CIs were calculated using the exact binomial Wilson method [14]. We stratified the results by site of care, and, in a secondary analysis, by provider specialty. To evaluate the importance of our requirement that events led to the hospitalization or ED visit, we included both non-incidental and incidental cases in the numerator of PPV estimates in a sensitivity analysis. In this case, incidental events were those confirmed events based on medical record review but listed within the medical record as a secondary reason for the hospitalization or ED visit.

Results

The population consisted of 165,729 women with PMO, the majority of whom were between 55 and 64 years of age, and white, reflecting the underlying population of the database (Table 1). We sought charts for 55 patients with qualifying claims for hypocalcemia, for which 40 (73%) charts were received. Sixteen potential cases were confirmed hypocalcemia leading to hospitalization or ED visit, yielding a PPV of 40.0% (95% CI 24.9–56.7%) (Table 2). One potential case had insufficient information for adjudication. Claims associated with ED setting (PPV 81.8%, 95% CI 48.2–97.7%) performed better than those from inpatient setting (PPV 24.1%, 95% CI 10.3–43.5%). The PPV for hypocalcemia was higher for claims associated with emergency medicine (PPV 54.2%, 95% CI 32.8–74.4%) as compared with other provider specialties. The inclusion of incidental hypocalcemia events yielded a higher PPV (70.0%, 95% CI 54.6–81.9%).

Table 1 Baseline characteristics of PMO study population, June 2005 – May 2010
Full size table
Table 2 PPV and 95% CI of serious hypocalcemia and serious dermatologic adverse event claims from emergency departments or inpatient facilities within the PMO study population
Full size table

Medical records were sought for a random 265 of 441 potential dermatologic adverse events identified from the database (6 bullous, 15 erythematous, 247 urticaria/rash and 3 with multiple codes); 184 (69%) charts were received (all had sufficient information for adjudication). The physician confirmed 128 as dermatologic events (PPV 69.6%, 95% CI 62.4–76.1%) (Table 2). PPVs varied across subtypes of dermatologic events (highest for urticaria/rash [PPV 70.5%, 95% CI 63.1–77.2%]) and healthcare setting (highest for ED [PPV 77.0%, 95% CI 69.4–83.5%]). Additionally, the PPV was higher for claims associated with emergency medicine (PPV 78.1%, 95% CI 70.2–84.7%) relative to other provider specialties. The inclusion of incidental dermatologic adverse events had little impact (PPV 74.5%, 95% CI 67.7–80.2%).

Discussion

In this nationwide, observational study of women with PMO, the performance of diagnosis codes in identifying hypocalcemia and dermatologic adverse events from health insurance claims data varied across settings, and by provider specialty. Our definition of hypocalcemia (as the primary reason for obtaining ED or inpatient care) yielded a PPV of 40%, and for dermatologic adverse events, a PPV of 70%. The inclusion of incidental cases increased the PPV of hypocalcemia appreciably, suggesting that secondary hypocalcemia may frequently be recorded in the primary position on claims. Incidental cases were infrequent for dermatologic adverse events, possibly because these events generally represent the true primary reason for the patients’ care. With both outcomes, the diagnosis codes from ED claims were more accurate than inpatient claims. Serious hypocalcemia and dermatologic adverse events may be treated and resolved within the ED without requiring hospital admission, and if hospitalization does occur, these outcomes—hypocalcemia in particular—may be considered a secondary concern.

There are few published data for comparison. Strom et al. reported that within Medicaid claims, 60.9% of the erythematous events captured through presence of ICD-9 695.1 (erythema multiforme, Stevens-Johnson syndrome, and toxic epidermal necrolysis) were later confirmed as true cases [15]. Within a health plan database, Chan et al. reported that the presence of a discharge diagnosis of erythema multiforme yielded a PPV of 60.7% [16]. These are similar to our PPV finding of 56 to 67% (including incidental cases) for serious erythematous events, which also included ICD-9 695.5 (exfoliation due to erythematous conditions).

Historically, 70 to 80% of medical records requested by our research group (and similar institutions) are obtained [17, 18]. In this study, as expected, our retrieval rate was at the lower end of this spectrum as we sought medical records only from the principal site of care. This choice arose from the study objective to validate outcomes associated with a specific medical claim, rather than to confirm the presence of an outcome. While our lower retrieval rate decreased the precision of the PPVs, leading to broader confidence intervals, it likely did not bias the PPVs estimates, unless the chart retrieval rate was somehow differential with respect to the true case status. For example, if hospitals were more likely to provide charts for true cases of hypocalcemia, our PPV estimates would be biased toward 100%. However, this scenario seems unlikely. With other study objectives, it is generally feasible to seek charts from multiple providers or institutions (e.g., a dermatologist and a hospital) to increase the fraction of events for which at least one medical record is available.

In this study, we had expected that limiting our algorithm to the first-position diagnoses on claims would increase the PPV for capturing serious occurrences of adverse events that were the primary reason for seeking care. However, we found that clinically incidental or secondary events are also captured through diagnosis codes recorded in the primary position. Further, it is important to note that outcomes leading to hospitalization or ED visits may have had ICD-9 codes recorded in a secondary position on claims. These cases were not counted in this study, and thus, incidence derived with these code sets will be underestimated.

This study was conducted in a US commercially-insured population which, on average, tend to be slightly younger than the US general population. While we expect the results of this study to be generalizable to other insured populations, caution must be taken if there are differences in coding standards for reimbursement for hypocalcemia or dermatologic adverse events across insurers. Further, as PPVs vary according to disease prevalence, our PPVs may underestimate those observed in populations with a higher prevalence of hypocalcemia and/or dermatologic events than our study population, and overestimate those observed in populations with lower prevalence of these conditions than our study population. This highlights the need to assess the performance of case-identification algorithms within specific populations of interest. Lastly, we recognize that additional work is needed to assess the performance of algorithms for identifying other outcomes of interest that are associated with the use of osteoporosis medication, including osteonecrosis of the jaw, and atypical femur fractures.

Conclusion

Our results suggest that the current algorithms to identify serious hypocalcemia and dermatologic adverse events are moderately accurate for events leading to an ED visit (PPV 81.8% for hypocalcemia and PPV 77.0% for dermatologic adverse events) and has lower accuracy for events leading to hospitalization. In certain scenarios, estimates derived from the current claims definitions may be insufficient, and algorithms that include other components of insurance claims data should be explored to further refine the algorithm. Alternatively, the outcome definitions could be widened to include all occurrences of the events that result in healthcare services.