To the Editor,

Reliable and complete data are needed for monitoring and evaluating the provision of healthcare services, physician and hospital remuneration, and service planning.1 In the Canadian province of Ontario, the Institute for Clinical Evaluative Sciences (ICES) houses healthcare administrative databases obtained from physician and hospital billing records from a single-payer public health insurance physician reimbursement system, the Ontario Health Insurance Plan (OHIP). These databases may be linked, using unique patient identifiers, to a number of other databases. Such databases, containing data points such as procedures, diagnostic codes, as well as costing and resource utilization, are often used for research purposes that may subsequently inform clinical care and policy decisions. Within the medical specialties, a vast array of data related to the field of anesthesiology and perioperative medicine are accessible.2,3 Nevertheless, researchers need to be wary of the impact that administrative changes in the way data are coded and retrieved may have on the interpretation of results. For example, following the conversion of the International Classification of Diseases (ICD) diagnostic codes ICD-9 to the ICD-10, the sensitivity of comorbidity detection was variable for a period of six months to three years, as coders adapted to the new coding system in Canada and other countries.4

Another example relates to the field of pediatric anesthesia, where OHIP provides incentives to physicians in the form of age premiums, recognizing the unique skills needed in managing the pediatric population. Here, specific codes are added to physician remuneration related to age categories. Up until June 2008, these codes needed to be specified by the anesthesia provider in addition to the pertinent procedure and time codes. In June 2008, OHIP instituted a change in the billing system, whereby pediatric premium codes (aside from premature newborns) were automated rather than billed separately.5 Unfortunately, the methodology of this automation prevented the codes from being identified in ICES administrative data.

Our group sought to identify the volume of pediatric anesthetic procedures carried out in Ontario over a 12-year period from 2003 to 2015, linked to specific institutions and anesthesia providers. Using pediatric age premium codes, patients were stratified into the following categories: premature infants (< 37 gestational weeks), newborns (37 weeks to 28 days), infants (29 days to < one year), and children (one to eight years). We found that the total pediatric anesthesia volume involving anesthesia providers was 362,326 cases between 2003 and 2015 at 145 hospitals in Ontario, with wide variations between years (Figure). Billing for premature infants accounted for 0.7% of the total pediatric anesthesia volume; 1.6% of the volume was attributed to newborns; 8.9% to infants 29 days to < one year of age; and 92.4% to children aged one to eight. There was a sharp decrease observed in the data in pediatric anesthesia cases, aside from premature infants, between 2007 and 2009, coinciding with the change in the billing system. In 2015, data returned in an inconsistent way to pre-2008 numbers.

Figure
figure 1

Number of pediatric anesthesia cases by year (logarithmic scale). The abrupt decrease in numbers, beginning in 2008, coincided with the automation of pediatric billing premiums. Data obtained from billing statistics from the Ontario Health Insurance Plan, Ontario, Canada

Full size image

Although there are many benefits to automation of data, including the potential elimination of human error in billing entry, this scenario of conversion to the automatic billing of codes resulted in unreliable data. Similar to the impact of changes in ICD code methodology,4 data integrity was unstable during a wide changeover period.

Researchers and policy makers employing administrative database analysis need to be wary of changes in the way data are coded. Likewise, agencies providing such data must be cautious regarding the impact that changes may have on database research integrity.