Informative graphing of continuous safety variables relative to normal reference limits
Interpreting graphs of continuous safety variables can be complicated because differences in age, gender, and testing site methodologies data may give rise to multiple reference limits. Furthermore, data below the lower limit of normal are compressed relative to those points above the upper limit of normal. The objective of this study is to develop a graphing technique that addresses these issues and is visually intuitive.
A mock dataset with multiple reference ranges is initially used to develop the graphing technique. Formulas are developed for conditions where data are above the upper limit of normal, normal, below the lower limit of normal, and below the lower limit of normal when the data value equals zero. After the formulae are developed, an anonymized dataset from an actual set of trials for an approved drug is evaluated comparing the technique developed in this study to standard graphical methods.
Formulas are derived for the novel graphing method based on multiples of the normal limits. The formula for values scaled between the upper and lower limits of normal is a novel application of a readily available scaling formula. The formula for the lower limit of normal is novel and addresses the issue of this value potentially being indeterminate when the result to be scaled as a multiple is zero.
The formulae and graphing method described in this study provides a visually intuitive method to graph continuous safety data including laboratory values, vital sign data.
KeywordsScaling Multiples Multiplicative inverse Reference range
Greatest non-zero number below LLN used in the derivation formula
LLN used in the derivation formula
Abnormally high value
International Conference for Harmonization
Abnormally low value
Lower limit of normal
Lower limit of normal for the RESULTmax
MI used in the JMP column formula to derive MI0
- Max M
MI of the RESULTmax
Mock labs data set
Milligrams per deciliter
Derived multiplicative inverse of zero
- MIA → 0
Distance from MImax to MI0
Multiplicative inverse of the non-zero abnormal value, A, with the greatest distance from its LLN
Values equal to or falling between the ULN and LLN
Non-zero number greatest magnitude below LLN, also designated as A
Upper limit of normal
Unique subject identification
Multiples of LLN
Multiples of ULN
Graphic presentations have long been recognized as a useful method to present clinical trial data [1, 2]. This is particularly true for safety data because clinical studies are not typically designed to inferentially test safety and tolerability data. The International Conference on Harmonization (ICH) guidance to industry on Statistical Principles for Clinical Trials recommends graphic inspection for safety and efficacy, data, including dose-response relationships .
Accurately interpreting the graphs of safety data may be challenging because of differences in laboratory reference ranges between testing sites, genders, and age groups. For example, the reference range for plasma alkaline phosphatase concentrations varies between age, gender, and testing center. The reference ranges of many pediatric laboratories and vital sign tests vary considerably, particularly before adolescence, so interpretation of the graphical display from a clinical trial of one or more year’s treatment duration in the very young can be particularly challenging. Graphing the actual values can also result in a misleading mixture of normal and abnormal results in the regions of the upper and lower limits of normal because of overlapping differences in these ranges. These differences are sometimes due to variances in assay techniques or demographics of the reference population between sites.
Several studies recognize these issues and suggest using multiples of the reference range limits instead of the actual values, although these descriptions are typically limited to demonstrating values relative to the upper limit of normal (ULN). When this approach is applied to data with abnormal values above and below normal range, the normal and abnormal low values are ‘compressed’ between 1 and 0 multiples of the ULN, while the abnormal high values have, at least theoretically, no upper limit . This is not usually concerning when working with data from biomarkers of cell injury, such as lactate dehydrogenase (LDH), which can have multiples in the thousands over the ULN and where a low value is not clinically concerning. However, other analytes, such as serum glucose or hematocrit, often have abnormal values higher than the ULN and below the LLN. One approach to the problem of displaying values above and below the normal range is to plot the standard deviations of the sample mean; however, one is often testing a sample where almost all of the subjects have abnormal values, and so the result would look almost the same as if a similarly homogeneous population of normal subjects was tested.
The objective of this study is to develop a method of graphing continuous safety data in a manner so as to address the issue of compression, particularly with datasets having large numbers of data below the LLN and for data with abnormal values both above the ULN and below the LLN. The problems encountered with graphing the actual values using standard techniques are first demonstrated and then formulas are derived to graph data in a manner that addresses scalability above and below the normal range and to produce graphs that are visually intuitive.
Datasets used for methods development and analyses
Reference Limits for Plasma Transferrin used in the Analysis of the Mock Laboratory Dataset
Site # / Name
Upper Limit of Normal (mg/dL)
Lower Limit of Normal (mg/dL)
1 / Quest Diagnostic Center, Baltimore site (QBA)
2 / University of Kentucky Health Care
3 / University of Utah Health Sciences Library
4 / University of California, San Francisco Clinical Laboratories
Graphical display of untransformed lab datasets
Graphs are first created for the untransformed data using the variable for time (e.g., VISIT in MDLD.JMP) as a continuous variable for the X-axis. The RESULT is a continuous dependent variable on the Y-axis. For the purposes of demonstration, lines for the normal reference limits are added to the Y-axis designating the ULN and LLN for each of the laboratory testing sites. Colored row states are created designating H as blue + signs, N as orange dots, and L as black inverted triangles.
Graphic inspection of untransformed data
Derivation of formulas to transform data for graphing
The formulae needed to graph the possible scenarios (L, N, H) are found in the following set of equations:
Formulae needed to graph transformed lab datasets in JMP
Multiple of the ULN for RESULTS that are abnormally high,
Normal RESULTS scaled between 1 and − 1,
Multiple or multiplicative inverse of the LLN for RESULTS that are abnormally low but greater than 0, and for the
Multiple or multiplicative inverse of the LLN for RESULTS that are abnormally low but equal to 0. Reference lines are added to the Y-axis at (0, 1) and (0, − 1) to designate 1× multiple of the ULN and LLN, respectively.
Create columns for the RESULT and corresponding LLN
Create a Column MI using the formula LLN ÷ RESULT
Create a Column Max M using the formula Col Maximum(M). This is done using Col Maximum from the Statistical function of the Functions (grouped) dialogue box and designating Column M as the variable of operation.
A column (Row of Max M) with the conditional formula, IF [M] = [MAX M] → [ROW()] specifies the row containing the maximum M for the next step. All of the cells in this column will appear to be missing values, except the rows corresponding to the maximum M value.
The formula for MI0 is written as one of the scenarios (last line, Fig. 2) for the conditional function that will be graphed on the Y Axis. The MIo formula makes use of Subscript from the Row function of the Functions (grouped) dialogue box to exclusively use the RESULT and LLN of the maximum multiple of the LLN.
Graphical display of transformed data
Advantages of graphic presentation of safety data
Graphic presentation of safety variables provides a means to enhance the accuracy of signal detection. Evidence for this has been reviewed in a recent publication by Michard . Businesses, such as aeronautics, have long known this, designing key displays with graphics rather than numeric displays. The advantage of graphical presentation has been systematically studied with respect to vital sign data in the disciplines of cardiology [8, 9], anesthesiology , and critical care medicine . Evaluation of clinical data in graphic form or a combination of numeric and graphic data has been shown to produce improved data interpretation speed and accuracy in several clinical scenarios. In one study of home blood pressure monitoring, this difference in monitoring resulted in differences in patient management, as measured by medications used. While definitive evidence was not found that graphic display could have a meaningful effect on mortality, several studies mentioned above have demonstrated significant improvements in the detection of myocardial ischemia or acute coronary syndrome, which can be fatal.
An unmet need in graphical presentation of data
Potential discrepancies and deficiencies in standard numeric reporting of clinical safety data values have been long recognized. In the 1970’s, Duboff reported creatine kinase values for members of a family with malignant hyperthermia that differed based on the testing site . In this same period of time, the College of American Pathologists’ Enzyme Survey demonstrated that results of liver enzyme tests varied by site, presumably because of differences in instruments, reagents, and assay conditions [13, 14]. The findings from these studies in the 1970’s coincided with a revolution in the use of graphical presentation of data, prompted largely by the work of Cleveland [1, 15, 16] and similar biometricians.
Recent attention to the use of graphics in the presentation of safety data has brought refinements [17, 18] including the use of individual patient graphic profiles ; however, limitations and misperceptions still exist.
Rodbard discussed potential issues in perception when viewing data graphed for analytes such as glucose, which may have abnormal data that are both above the ULN and below the LLN. Despite being potentially more critical in the acute setting, hypoglycemic values may be overlooked when presented graphically because, in contrast, hyperglycemic values can be so relatively large that the appearance of the low values does not appear meaningfully different than those that are normal when graphed using standard methods . While the concept of compression discussed by Bottger and Balzer  relates more to distortions caused by two unequal stretching operations, the phenomenon described by Rodbard et al., seems to describe compression of abnormally low values relative to the scaling of those above the ULN.
Previous proposals for graphing safety data
Several authors have proposed alternative methodologies for graphing. The most commonly used is based on multiples of the ULN. This method has the most utility for monitoring plasma levels for enzyme biomarkers of tissue damage such as AST, ALT, and CPK. One of the original methods proposed “centrinormalized units” that divide the ULN by 100 and scale all results by this factor so the normal range would always be 100 (upper limit) to LLN*100/ULN (lower limit) . More recently, a system based on the ULN has been proposed as the method to evaluate drug-induced liver toxicity based on ‘Hy’s Law’ . A closely related alternative methodology proposes to use both multiples of the ULN and baseline in a statistical outlier technique to define the critical boundaries of toxicity methods . While the ULN-based methods have value in certain analyses, this methodology would not be useful when plasma analytes or vital sign measurements have values below the lower limit of normal. Normal values would also be scaled based on the ULN, which would make these data difficult to interpret.
Methods of graphing based on the standard deviation (SD) of data have also been proposed, where for example, data falling outside of 2  or 3  standard deviations of the mean would be considered abnormal. This method has the advantage of being able to described data above the ULN and LLN in a scale that is equivalent; however, the utility of this method is limited because not all sample populations are equivalent between hospital or lab testing sites and the population that comprises the mean ± 2 SD is not necessarily normal.
Rodbard  proposed use of a semilogarithmic plot to triple the percentage of the vertical axis allocated to values below the LLN, as is encountered in cases of hypoglycemia, while at the same time compressing the region of the graph containing hyperglycemic values. This method has value in expanding the detail for values below the LLN, yet it is not optimal for simultaneously visualizing data containing low, normal and high values.
Scaling of values with the proposed method
The primary objective in developing this method was to provide a means to graph data where values below the LLN were scaled the same as those above the ULN. The first step was relatively simple; transforming abnormally high values above the ULN is typically done by dividing the value by the ULN reference limit. The scaling of results below the LLN is not typically found in literature and maintaining a scalar, visually intuitive display is not as simple as the ULN. The multiple of LLN is multiplied by − 1 in the last step before graphing so that it occupies a position equidistant to normal values as those above the ULN; this final value is termed the multiplicative inverse. While the issue of zero in the denominator would not exist if the formula was configured the same as that of the ULN (e.g., RESULT/LLN), this relationship cannot be used because zero values would result in a multiple equal to zero, which would place the point in the middle of the normal values, which are scaled between 1 and − 1.
The calculation of the multiples of the abnormal low value below the LLN will be problematic when the RESULT equals zero, since this would make the multiplicative inverse an undefined number using the equation LLN/RESULT. This situation is not typically found with certain data, such as vital signs or chemistry laboratories, where having a result of zero is not typically observed. However, there are laboratories where zero is encountered (e.g., the absolute neutrophil count in profound neutropenia) and so the issue must be addressed. A RESULT of zero would intuitively have a multiple greater than any non-zero value in the column. An additional quantity would need to be added to the column maximum of the greatest multiple of the LLN equal to a ratio that is based on the LLN and the value remaining after the smallest result is subtracted from the LLN, as is demonstrated in the derivation of Eq. 4.
Limitations and concerns
Transformation of data almost always has potential issues with the audiences’ ability to comprehend the relationship of numbers when graphically displayed. Log transformation is one of the procedures most often used and it results in a display that may be uninterpretable by those outside of the scientific field. The method presented in this study has the advantage that it is visually intuitive; abnormal data of equal magnitude are equidistant from the normal reference limits. Most who routinely work with data are so accustomed to compression of results less than the lower limit of normal displayed in the Cartesian system relative to that above the upper limit of normal that visualization of the data in a more naturally expanded state will require familiarization. The advantage of this technique, particularly, the ability to look at data with multiple reference ranges in the same graph, will make such effort worthwhile. The method to graph points with a value of zero is not likely to be used often. Other techniques such as inclusion of a ‘zero-corrector’ (such as the use of ½ when an incidence is zero in the calculation of relative risk) may be simpler though this introduces error in the visual display.
A method to graph continuous safety variables is presented. This method addresses the issues that arise when graphing values that have different reference limits, such as sometimes seen with differences in gender, age, and testing site. The graphing method presents novel approaches to the plotting of normal values in graphs of multiples of the reference limits based on a classic scaling equation. The method also includes a novel approach to scaling the multiples of the abnormal low value below the LLN when the result is zero.
I thank Dr. Hylton Joffe for his critical reading of and suggestions for this manuscript.
Availability of data and materials
The data supporting this manuscript is found in the Summary Basis of Approvals at Drugs@FDA (http://www.accessdata.fda.gov/scripts/cder/daf/).
CDB is responsible for the conception, research, analysis, and writing of this manuscript. The author read and approved the final manuscript.
Ethics approval and consent to participate
Dr. Breder is an employee of the United States Food and Drug Administration. The views expressed in this manuscript represent the opinions of the author and do not necessarily represent the views of the FDA. The author does not have any financial interest in the SAS Institute® or any of its products.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
- 1.Cleveland WS. The elements of graphing data. Monterey: Wadsworth Advanced Books and Software; 1985.Google Scholar
- 3.International Conference on Harmonization. E9 Statistical Principles For Clinical Trials; 1998. p. 39.Google Scholar
- 5.Strickland S. Dynamically Scale a Range of Numbers. FMForums Worldwide Filemake Community 2013 [cited 2015 December 30]; Available from: http://fmforums.com/topic/90562-dynamically-scale-a-range-of-numbers/
- 6.user25658, A. How to normalize data to 0–1 range? Cross Validated 2013 9/23/13 11/11/16]; Available from: http://stats.stackexchange.com/questions/70801/how-to-normalize-data-to-0-1-range
- 12.Duboff GS. Expressing results of enzyme assays. Clin Chem. 1974;20(3):2.Google Scholar
- 13.Lott JA, Gilmore BE, Massion CG. Lnterlaboratory survey of enzyme analyses: V. A simplified method of reporting Participants' data. Am J Clin Pathol. 1983;80:8.Google Scholar
- 14.Lott JA, Tholen DW, Massion CG. Determination of reference ranges for serum enzymes via a large Interlaboratory survey. A1ch Patho!LabMed-V. 1987;111:7.Google Scholar
- 17.Chuang-Stein C, Le V, Chen W. Recent advancements in the analysis and Presentaton of safety data. Drug Inform J. 2001;35:377–97.Google Scholar
- 19.Ma H, et al. Graphical analyses of clinical trial safety data. In: Joint statistical meeting - statistical computing and statistical graphics. Salt Lake City: American Statistical Association; 2007. p. 42.Google Scholar
- 20.Rodbard D. A Semilogarithmic scale for glucose provides a balanced view of hyperglycemia and hypoglycemia. J Diabetes Sci Technol. 2009;3(6):7.Google Scholar
- 22.Administration, F.a.D. Guidance for industry drug-induced liver injury: premarketing clinical evaluation, D.O.H.A.H. Services, editor; 2009. p. 25.Google Scholar
- 24.Miyamoto M, Shimizu H, Niwa M. Delta check system for detection and further Management of Laboratory Errors. Tokai J Exp Med. 1986;11(2):11.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.