Sequence symmetry analysis graphic adjustment for prescribing trends
Sequence symmetry analysis (SSA) is a signal detection method that can be used to assist with adverse drug event detection. It provides both a risk estimate and a data visualisation. Published methods provide results in the form of an adjusted sequence ratio, which adjusts for underlying market trends of medicine use, however no method for adjusting the visualisation is available. We aimed to develop and evaluate another method of adjustment for prescribing trends and apply it to the SSA visualisation.
The SSA method relies on incident prescriptions for pairs of medicines of interest. Smoothing curves were fitted to the frequency distributions of incident medicine use. When divided and normalised, these curves yielded a set of proportions related to differences in prescribing trends over follow-up. These were then used to adjust the unit counts for incident prescriptions in the SAA visualisation and to calculate the sequence ratio. Curve fitting was also used to identify the proportional relationship between sequences over time for SSA and is presented as a supplementary visualisation to the SSA. We compared the sensitivity and specificity of our method with that from the SSA method of Tsiropolous et al.
Curve-fit adjusted SSA visualisations yielded adjusted sequence ratios very close to those of Tsiropolous, with a p-value of 0.999 for the two sample Kolmogorov-Smirnov test. Results for sensitivity and specificity derived from adjusted sequence ratios were also practically the same. The curve-fit method graphically indicates the proportionality of the sequence and provides a useful supplement of net differences between the two sides of the SSA visualisation. Additionally, we found that incident prescriptions for patients common to both distributions are best precluded from adjustment calculations, leaving only incident prescriptions for patients unique to one or other distribution. This improved the accuracy of SSA in some atypical instances with negligible affect on accuracy elsewhere.
Our curve-fit method is equivalent to current methods in the literature for adjusting prescribing trends in SAA, with the advantage of providing adjustment incorporated in the SAA visualisation.
KeywordsSequence symmetry analysis (SSA) Waiting time distribution (WTD) Prescribing trend Rate ratio (RR) Curve fitting
Rate Ratio. (Number of patients undergoing a sequence of two events, where the first is suspected to cause the second, divided by number of patients undergoing the reversed sequence of the two events.)
Sequence Symmetry Analysis. (Consecutive complementary sequences of counts for two discrete incident events; in our case within a specified time interval.)
Waiting Time Distribution. (Two frequency distributions, each of discrete incident events in the data over the study follow-up period.)
Sequence symmetry analysis is a signal detection method used to assist with adverse drug event detection. The method is suitable for use in administrative claims data or electronic health records and has a similar performance to signal detection methods used with spontaneous adverse drug reaction reports [1, 2, 3].
SSA uses a case-only design that assesses the sequence of incident use of two medicines of interest within a specified time interval. If the use of each medicine is independent then chance alone determines the sequence order, hence the distribution of sequence orders in a population who have both medicines is effectively symmetrical. Alternatively, if a medicine is associated with an adverse event then we expect the distribution of sequence orders to be asymmetrical .
Specifically, if a medicine A has no adverse effect, the likelihood of starting a medicine B (as an indicator of treatment for an adverse event) after medicine A is the same as that for starting medicine B before medicine A. Where there is an adverse effect, the likelihood of starting medicine B after starting medicine A is greater than that for starting medicine B before medicine A. The method calculates the ratio of number of persons who start medicine B after medicine A, to number of persons who start medicine B before medicine A. This is called the crude sequence ratio. Because the method is sensitive to prescribing trends over time, a null-effect sequence ratio is calculated to adjust for temporal trends in medicine use. The null-effect sequence ratio is the expected sequence ratio in the absence of a causal association, given incident medicine use and events in the background population.
SSA also produces a visualisation of sequence orders, enabling the temporality of sequences to be assessed. The visualisation displays the frequency of patients by time between incident dispensing of two medicines of interest and is usually produced as a summary by weeks between sequences. It can be a valuable addition in determining the likelihood that a statistically significant result represents a signal for an adverse drug event as these are temporally associated with initiation of therapy, with the majority occurring within the first 4 months . Currently, while the null-effect sequence ratio adjusts for changes in prescribing trends over time, no method is available that provides an adjusted visualisation. We aimed to develop and evaluate a method of adjustment for prescribing trends applied to the SAA visualisation.
Waiting time distributions
Sequence symmetry analysis visualisation
The unadjusted, or crude, sequence ratio for the SSA visualisation is calculated by dividing the number of patients with incident dispensing to the right of week zero by the number of patients with incident dispensing to the left of week zero. This estimates the incidence rate ratio, which we refer to as rate ratio (RR). In Fig. 2, the crude RR is 1.58. The adjusted RR seeks to compensate for different background prescribing trends, which are changes in medicine use that have nothing to do with any adverse effect caused by a medicine but may influence the occurrence of a sequence of prescription. The two methods of adjustment adopted in the literature are Hallas  and Tsiropoulos  (see Additional file 1: Appendix 1 for more information). The Tsiropolous adjusted RR is 1.62.
The Hallas method was developed for studies where there is no limitation in the time interval between two treatments of interest. All incident observations in the study period are used. The Tsiropoulos method is an extension of the Hallas method and was developed for studies where there is an upper cap on the interval between two treatments of interest. The Tsiropoulos method applies to this study, in which it is inappropriate to make comparisons with the Hallas method.
The proportion of patients supplied both medicines, shown in Fig. 2 as proportion of pairs in WTDs, is calculated by dividing the total for patients supplied both medicines across the two WTDs by the total for all medicines (one or other medicine exclusively as well as both medicines). Over the follow-up time of 9.5 years, 3.2% of patients were supplied both medicines. This proportion is considered in the methods section.
In this paper we discuss adjustment methods for the summary statistic and introduce a new method to produce an adjusted summary statistic that also allows adjustment of the SSA visualisation. We compare the results from our method with the Tsiropoulos-style symmetry analysis, but it is implicitly adapted to the Hallas-style by not limiting the time between two treatments of interest in the SSA visualisation. In the latter instance, all incident observations in the study period are used.
There are often differences in prescribing trends between two WTDs that need to be reconciled. If a medicine goes off patent, for example, and uptake rapidly increases, this would not be due to any causal mechanism ascribed to the other medicine and would need compensating for by adjustment to the crude RR. The adjusted RR attempts to compensate for non-causal differences between two medicines for patients common to both WTDs.
The Hallas method of adjustment was included because it provides the context from which other developments ensue. (While ensuring our software program has the flexibility to apply any of the methods where appropriate, it also allowed us to examine the methods under various test conditions.) Hallas provides a null-effect sequence ratio representing an overall adjustment, which does not adjust for counts of first supply times based on where they occur in the WTDs. Tsiropoulos’ method yields a more locally derived null-effect sequence ratio from multiple, ‘moving window’ type calculations over shorter intervals of the WTDs. This captures the effect of relatively local trend differences between two WTDs. However, neither method provides an adjustment to the shape of the SSA visualisation.
We have developed a way of calculating adjustment to the SSA visualisation by curve fitting. Curve fitting is the process of constructing a curve that fits a series of data points. It determines characteristics of the data such as rate of change anywhere on the curve, local minimum and maximum points and area under the curve (probability estimate) and can produce parameter values that most closely match the data. Most tried and proven methods of curve fitting are suitable for this purpose. (See Additional file 1: Appendix 2 for a description of the method we used.) Our default curve-fit parameter selects smoothing values half way between no smoothing and full smoothing; the former tracing the original graph and the latter tracing a straight line of some orientation.
Unadjusted and adjusted counts forfrusemide-after-amlodipine supply lags 1, 2 … 101 … 365. The closer two WTDs come to the having the same shape, the closer the proportions come to having a value of one
Rate ratio adjustment compensates for different prescribing trends due to external causes. These are changes in prescribing frequency over time and need compensating for if each WTD is different in shape. Moreover, a medicine of interest may cause a change of prescribing frequency of another medicine, and this is additional to the underlying trend. Any such change of prescribing frequency is considered constant over follow-up; it is considered not to alter the difference between trends in the two WTDs. In practice, however, it can change over time independently in each WTD, which can be problematic if the proportion of medicine pairs is high. Hence applying any adjustment method yet discussed in the literature works best where the proportion of medicine pairs involved is typically low.
WTDs comprise exclusive components, which are supplies to persons who did not receive the other medicine of interest (i.e. not paired anywhere across the WTDs), and inclusive components, which are supplies to persons who had both medicines of interest (i.e. paired anywhere in the WTDs.) Patients can have medicine A exclusively, medicine B exclusively, or medicine A and medicine B. Where a substantial proportion of medicine pairs exist in relation to number of medicine starts in a WTD and these vary in frequency over follow-up, then this may differentially alter the original background prescribing trends between WTDs. Some of the difference in shape between two WTDs may be attributed to an effect, which would corrupt an adjustment—otherwise intended to compensate for different background prescribing trends—and tend to adjust out the effect. Consequently, only the exclusive components should be used in adjustment calculations.
The net effect proportion is calculated as the sum of positive magnitudes divided by the sums of both magnitudes. If the confidence interval for the net effect proportion straddles 0.5, then the effect is not statistically significant. Minimum and maximum possible limits for confidence intervals are implied at 0 and 1 respectively. (Of course, the net effect proportion bears a direct correspondence to the RR of the SSA visualisation.)
To validate our adjustment method, we compared it with the established method of Tsiropoulous for sensitivity and specificity to detect known adverse drug effects. The adverse drug effects assessed, along with the method, have been reported elsewhere . We assessed 59 known adverse reactions (true positives) that had been identified within randomized controlled trials, and 65 negative controls (adverse effects not documented in the product information of the medicines assessed but identified as adverse effects of medicines in the other medicine classes). We used a 10% sample of persons with medicines supplied by the Australian Pharmaceutical Benefits Scheme; the national scheme that provides subsidised access to pharmaceuticals in Australia .
Comparison of results from 8-year WTDs show improved sensitivity, specificity and predictive values when using only exclusive components. The Curve-fit method compares closely with Tsiropoulos and yields the same result over a follow-up of 9.5 years
Having validated the Curve-fit method against Tsiropoulos, the main result is the adjusted SSA visualisation. The overall height of each bar for each week, representing adjusted number of patient’s first prescriptions, is derived from a composite of individual heights adjusted according to where they come from in the WTDs.
Adjusted RRs for Hallas (used advisedly here), Tsiropoulos and Curve-fit. The first row shows an incorrect result for meloxicam and proton pump inhibitors for all three methods because the confidence intervals (CI) straddle the value 1. The second row shows the correct result for all three methods
Needing just three variables, SSA is economical with respect to dataset size and computing resources, and is easy to deploy. It has been applied to international as well as national datasets . Because it uses the experience of different persons before and after exposure, SSA cannot be considered self-controlled. However, time invariant confounders, such as age, smoking or hypertension, are effectively controlled because they do not predict prescription order given that patients become users of both drugs. That is, they do not contribute to the generation of an asymmetrical distribution of sequence orders [4, 11]. Accordingly, the SSA visualisation can usefully assess temporal relationships of effects.
SSA has proved useful as a signal detection tool to provide early alerts of drug safety issues. Results from the study by Wahab, Pratt, Wiese et al—which tested the validity of SSA to detect adverse drug reactions from an administrative claims database—show a sensitivity of 61%, specificity of 93%, positive predictive value of 77% and negative predictive value of 87% . These exceed those of our study owing to the quality of the database they utilised, which afforded complete data capture in their case. Their study did not consider conditions discussed in our study that seek to further improve utility. Still, they demonstrated that SSA has the potential to identify early drug safety issues and could complement existing pharmacosurveillance methods.
We have demonstrated that curve fitting produces a similar statistic to that of the Tsiropoulos method and can be extended to provide a mechanism to adjust the visualisation of sequences. Additionally, we found that curve fitting is useful for some related procedures, such as for showing the net effect integration of each side of the SSA visualisation. Displaying net raw values also eliminates the confusion of corresponding after and before weekly comparisons sometimes necessary when viewing an SSA visualisation, by clearly showing which week is the greater or otherwise.
Not all paired observations result from a medicine effect. Remaining pairs are coincidental and become more prevalent as the RR gets closer to 1. These are no more or less important than exclusive observations. They are part of the background trend and considered to be distributed as such—hence excluding them, along with signal pairs, should not prove problematic. Curve fitting produces parameter values that closely match the data. If two curves (one for all components and one for exclusive components) displayed on the one WTD run substantially parallel, then the shape of a WTD with or without pairs remains effectively unchanged. Excluding pairs from the WTDs in the RR adjustment calculation made no effective difference in most instances in our study. However, such exclusion precludes the possibility of disproportionality that might markedly affect the shape of a WTD due to a high proportion of medicine pairs that can change over time. While an overall high number of medicine pairs with respect to all medicine starts for both WTDs can cause such a problem, an overall low number of medicine pairs can likewise do so if the frequency balance between a pair of WTDs is too one sided. For the higher frequency WTD the proportion of medicine pairs may well be low with respect its medicine starts, but for the lower frequency WTD it can be quite high. There were a few instances in which disproportionality was an issue and for these we found improved adjusted RR accuracy reflected in sensitivity, specificity and predictive values when using only exclusive components.
Finally, curve fitting, as supplied in statistical packages, is a relatively simple substitute for the mathematical complexity involved in the Tsiropoulos method. They both implement moving average procedures that capture the effect of local trend differences between two WTDs. Curve fitting also has the advantage of not having to specify a cut-off time related to the time a medicine effect may diminish. (See Additional file 1: Appendix 1, equation 3 where a cut-off time for the Tsiropoulos method must be specified.)
Our curve-fit method is equivalent to the Tsiropoulos method for adjusting prescribing trends in SAA, with the advantage of providing adjustment incorporated in the SAA visualisation. An adjusted visualisation can more accurately assist in determining the likelihood that a statistically significant result represents a signal for an adverse drug event. Excluding pairs eliminates any effect signal pairs might have upon the shape of either WTD, which would otherwise tend to mitigate medicine cause-effect relationships upon adjustment. Further work is planned to test the curve-fit method on short period WTDs of about 31/2 years to see how it compares with the Tsiropoulos method. This matters because adjustment for prescribing trends at each end of WTDs can be compromised due to lack of data on one or other side of an index date. The longer the WTDs, the less this is an issue. Since different statistical methods can treat boundary conditions in different ways, such a comparison is considered worthwhile.
The authors thank John Barratt and Paul Baker for delivering the research database and technical support.
ER raised the issue of adjusting SSA visualisations for differences in prescribing trends between medicines. AP suggested the effect a relatively high proportion of medicine pairs might have on the shape of a WTD. NP suggested superposing the curve fitted WTDs of all components with the curve fit of exclusive components to visualise any effect. AP developed the curve fitting methodology using Hallas and Tsiropoulosas as points of departure, and prepared the manuscript. ER and NP helped to draught the manuscript. All authors have read and approved the submitted manuscript.
The authors declare that this research received no grant from any funding agency in the public, commercial or not-for-profit sectors.
Ethics approval and consent to participate
This research was approved by the Australian Government Department of Human Services External Requests Evaluation Committee—Request approval number MI5157. Since the research was undertaken on existing medicines data with no personal identifiers, ethics approval was not required.
Consent for publication
The authors declare that they have no competing interests.
- 2.Wahab IA, Pratt NL, Kalisch LM, and Roughead EE. “Sequence symmetry analysis and disproportionality analyses: what percentage of adverse drug reaction do they signal?” Adv Pharmacoepidemiol Drug Saf 2013; 2:4. https://doi.org/10.4172/2167-1052.1000140. Accessed 15 Jun 2017.
- 3.Pratt NL, Ilomaki J, Raymond C, Roughead EE. The performance of sequence symmetry analysis as a tool for post-market surveillance of newly marketed medicines: a simulation study. BMC Med Res Methodol. 2014;14(66). http://www.biomedcentral.com/1471-2288/14/66.
- 8.Department of Health. “Sources of epidemiological data for use in generating utilisation estimates 2015. Canberra: Commonwealth of Australia.” Accessed 25 Nov 2015.Google Scholar
- 10.Pratt N, Andersen M, Bergman U, Choi NK, Gerhard T, Huang C, Kimura M, et al. Multi-country rapid adverse drug event assessment: the Asian Pharmacoepidemiology network (AsPEN) antipsychotic and acute hyperglycaemia study. Pharmacoepidemiol Drug Saf. 2013;22(9):915–24. https://doi.org/10.1002/pds.3440.CrossRefPubMedGoogle Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.