A NADE nomogram to predict the probability of 6-month unfavorable outcome in Chinese patients with ischemic stroke
Early prediction of unfavorable outcome after ischemic stroke is of great significance to the clinical and therapeutic management. A nomogram is a better visual tool than earlier models and prognostic scores to predict clinical outcomes, which incorporates different factors to develop a graphic continuous scoring system and calculates accurately the risk probability of poor outcome entirely based on individual characteristics. However, to date, no nomogram models have been found to predict the probability of 6-month poor outcome after ischemic stroke. We aimed to develop and validate a nomogram for individualized prediction of the probability of 6-month unfavorable outcome in Chinese patients with ischemic stroke.
Based on the retrospective stroke registry, a single-center study which included 499 patients from May, 2013 to May, 2018 was conducted in Nanjing First Hospital (China) for ischemic stroke within 12 h of symptoms onset. The main outcome measure was 6-month unfavorable outcome (mRS > 2). To generate the nomogram, NIHSS score on admission, Age, previous Diabetes mellitus and crEatinine (NADE) were integrated into the model. We assessed the discriminative performance by using the area under the curve (AUC) of receiver-operating characteristic (ROC) and calibration of risk prediction model by using the Hosmer–Lemeshow test.
A visual NADE nomogram was constructed that NIHSS score on admission (OR: 1.190, 95%CI: 1.125–1.258), age (OR: 1.068, 95%CI: 1.045–1.090), previous diabetes mellitus (OR: 1.995, 95%CI: 1.236–3.221) and creatinine (OR: 1.010, 95%CI: 1.002–1.018) were found to be significant predictors of 6-month unfavorable outcome after acute ischemic stroke in Chinese patients. The AUC–ROC of nomogram was 0.791. Calibration was good (p = 0.4982 for the Hosmer–Lemeshow test).
The NADE is the first nomogram developed and validated in Chinese ischemic stroke patients to provide an individual, visual and precise prediction of the risk probability of 6-month unfavorable outcome.
KeywordsStroke Cerebral ischemia Unfavorable outcome Prediction Nomogram
The area under the curve
Body Mass Index
Fasting blood glucose
High density lipoprotein
International normalized ratios
Modified Rankin Scale
NIHSS score on admission, Age, previous Diabetes mellitus and crEatinine
National Institute of Health stroke scale
Randomized controlled trial
Variation Inflation Factors
Globally, stroke is now the second most common cause of death and a major cause of disability [1, 2], and despite a reduction in mortality and hospitalization owing to improved acute management and implementation of preventive measures, the rate of stroke is on the rise owing to aging population. Therefore, early prediction of unfavorable outcome after ischemic stroke is of great significance to provide a reasonable approach to the clinical and therapeutic management and also help the patients and their families in understanding the challenging consequences of cerebral ischemia.
Nomogram is a visual statistical instrument that incorporates different data to develop a continuous scoring system which reflects the individual and precise risk probability. The accurate continuous probability power of the score emphasizes the potential of nomogram score as an important part of modern medical decision-making and a useful risk stratification tool that has been applied in routine clinical practice including cancer and surgery [3, 4, 5, 6].
To our knowledge, there are only three researches on nomograms for individualized prediction of poor outcome of ischemic stroke in Caucasians (Italian) [7, 8, 9]. The START nomogram is developed and validated for individualized prediction of 3-month unfavorable outcome in intravenous thrombolysis-treated stroke patients . Another nomogram model was established in 344 patients who started oral anticoagulants 1–7 days after atrial fibrillation-related stroke onset to predict the probability of 3-month poor outcome . They found that NIHSS score which reflects the severity of stroke is the strongest predictor of 3-month unfavorable outcome [7, 8, 9]. However, to date, no nomogram model has been established to predict the probability of 6-month (long-term) unfavorable outcome after ischemic stroke in Chinese patients. Furthermore, the correlation of NHISS score with the outcomes varies with the time passed from the onset of cerebral ischemia . It is unclear how National Institute of Health stroke scale (NHISS) score and other risk factors affect the 6-month (long-term) unfavorable functional outcome.
The objectives of this study were to develop and validate a nomogram model based on the integration of different degrees of stroke severity and other risk factors for individualized prediction of the probability of 6-month (long-term) unfavorable functional outcome after ischemic stroke in Chinese patients, which can help direct individual treatment for ischemic stroke patients.
Study design, participants, and procedures
Based on the retrospective stroke registry, a single-center study was conducted in Nanjing First Hospital, Nanjing Medical University, China. The study protocol was approved by the hospital medical ethics committee in accordance with the Helsinki declaration. All of the consecutive patients were recorded in the stroke registry between May, 2013 and May, 2018 for ischemic stroke within 12 h of symptoms onset.
All patients managed with endovascular procedures were excluded from the study. Patients with signs of intracranial hemorrhage (ICH) on baseline brain computed tomography (CT) scan, age < 18 years, lack of 6-month modified Rankin Scale (mRS) score and NIHSS score on admission unknown were excluded from the analysis. On admission, all clinical, demographic and laboratory characteristics were collected, including: age, sex, baseline NIHSS score, creatinine, platelet count, total cholesterol, fasting blood glucose, interval from onset to hospital within 4.5 h, medical history such as diabetes mellitus, hypertension, valvular heart disease, cerebral hemorrhage and so on. The quality of laboratory data was validated throughout the study period by regular internal quality control procedures and participation to an External Quality Assessment scheme. Baseline NIHSS and 6-month mRS were assessed by trained physicians with telephone questionnaires or face-to-face interviews. Mortality, and any complications were recorded. The primary outcome was unfavorable functional outcome after ischemic stroke, as reflected by a mRS comprised between 3 and 6 (i.e., poor prognosis) at 6-month .
Baseline characteristics were summarized with descriptive statistics, continuous variables were reported as median value with interquartile range or means with standard deviation [SD]. Categorical variables were instead expressed as number of events and percentage, dividing the number of events by the total number excluding missing and unknown cases. Differences between the two groups were explored using the Mann-Whitney U-test or Student t test for continuous variables. Differences between proportions were assessed by Fisher’s exact test or the χ2 test, when appropriate.
A NADE nomogram model was generated to predict the probability of 6-month unfavorable outcome. To develop a nomogram, the pre-established predictors and all variables with a probability value < 0.20 in the univariate analysis were entered into a multivariate logistic regression model. A final model selection was carried out by a backward stepdown selection process with the Akaike information criterion. Regression coefficients and odds ratios (OR) with two-sided 95% confidence intervals (95% CIs) for each of the variable included in the model were finally calculated. Collinearity of combinations of variables that entered the multivariate logistic regression analysis was assessed by the Condition Index (< 30 being considered non-significant) and Variation Inflation Factors (VIF, < 2 being considered non-significant). The performance of the model was assessed by discrimination (the ability of a proposed model to separate patients with different outcomes) and calibration (the relative distance of predictions from actual outcome). The predictive accuracy of the nomogram model was assessed by calculation of the area under curve (AUC) of the receiver-operating characteristic (ROC). Calibration was carried out using a calibration plot, in which the predicted probabilities were plotted against the frequency of the observed unfavorable outcome. The prediction of a well-calibrated model should be mirrored by a 45° diagonal line. Given that all predictive equations tend to be over-fitting to the original sample, the model was internally validated using bootstrap resampling. The statistical analysis was carried out using SPSS version 22.0 (IBM Corporation, Armonk, NY, USA), Stata version 13.0 (Stata Corporation, College Station, TX, USA) statistical software and the statistical software package R, version 3.3.3 (R Development Core Team, Auckland, New Zealand). All tests were two sided and P < 0.05 was considered statistically significant.
Demographics and clinical characteristics according to 6-month outcome
Favorable outcome (mRS 0–2))
Unfavorable outcome (mRS 3–6)
Medical history, n (%)
Coronary artery disease
Age (years), median (IQR)
NIHSS score, median (IQR)
BMI, kg/m2, mean (SD)
Systolic BP, mmHg, median (IQR)
Creatinine, umol/L, median (IQR)
significant predictors of 6-month unfavorable outcome after an acute ischemic stroke
NIHSS on admission
Previous Diabetes mellitus
The total number of patients with a risk probability < 10% was 116/499 (23.2%), and only 9 of these had an unfavorable outcome (0.936 sensitivity, 0.298 specificity, 0.922 negative predictive value and 0.342 positive predictive value). The total number of patients with a risk probability < 40% was 386/499 (77.4%), 66 of whom had unfavorable outcome (0.529 sensitivity, 0.891 specificity, 0.829 negative predictive value and 0.655 positive predictive value). Finally, the total number of patients with a high-risk probability (i.e., > 80%) was 25/499 (5%), the vast majority of whom (21/25; 84%) had a poor prognosis (0.15 sensitivity, 0.989 specificity, 0.749 negative predictive value and 0.84 positive predictive value).
Ischemic stroke is one of the leading causes of mortality and disability, and post-stroke disability has now become a public health care problem, so that the early and reliable identification of unfavorable outcome after ischemic stroke should be a valuable perspective on precise clinical and therapeutic management.
Some earlier models and prognostic scores identified that NIHSS score [12, 13, 14, 15, 16], age [12, 13, 14, 16, 17], creatinine [18, 19] and diabetes mellitus [14, 16, 20], size infarct  were independent predictor of unfavorable outcome in stroke patients. However, these above models and scores were limited by the use of dichotomization / categorization of predictors, because the process of categorizing discrete/continuous variables (size infarct, age and NIHSS score) into 2 or 4 risk groups was often statistically inefficient and may decrease the predictive accuracy. Moreover, the disadvantage of dichotomization is that it does not make use of within-category information and lead to the loss of information.
Using continuous variables, we are the first to present a visual NADE nomogram which is a better tool for individualized prediction of the probability from 5 to 95% of 6-month poor outcome following ischemic stroke in Chinese patients, as shown in Fig. 1. The NADE nomogram, defined as graphical computation instrument composing four predictors that age, NIHSS score on admission, creatinine, and diabetes mellitus were observed to be independently associated with 6-month unfavorable outcome for individual ischemic stroke, and the effect remained significant with good discriminative performance even after adjusting other clinical, and demographic variables. That is, we found that NIHSS score on admission, age, previous diabetes mellitus and creatinine were significant and independent predictors of poor outcome. Moreover, patients age, NIHSS score on admission and creatinine are the non-categorical variables by order of decreasing multivariate predictive effect on the probability of 6-month poor outcome with good discriminative performance for individual stroke patients.
The NADE nomogram was a reliable model to evaluate the risk of 6-month mRS score 3–6 in stroke individuals, which showed a significant superiority for prediction of unfavorable outcome. Differently from prognostic scores, The NADE nomogram assigns a probability (from 5 to 95%) of unfavorable outcome. For example, the NADE nomogram assigned > 95% probability of adverse consequence in an 80-year-old patient (74 points) stroke patient, with a history of diabetes mellitus (14 points), NHISS score of 10 (35 points) and creatinine level of 300 (58 points), with a total score of 181 points. On the other hand, < 10% probability of unfavorable outcome was nominated to a 40 years old (20 points), with no history of diabetes mellitus (0 points), NHISS score of 5 (18 points) and creatinine level of 100 (17 point), with a total score of 55 points. By converting the total score into a continuum of individual probability, the NADE nomogram can reclassify the risk of poor outcome more precisely. So the NADE nomogram could be better than previous prognostic scores and models based on outdated risk-grouping categorization for identifying different risk predictors in patients. Thus the NADE nomogram may provide more detailed information to facilitate the early identification of patients with very high probability of poor outcome and to discuss prognosis with patients and their families.
The NADE nomogram integrated patient age, NIHSS score, creatinine and diabetes mellitus for individualized prediction of 6-month unfavorable outcome following ischemic stroke in Chinese patients. Firstly, we found that age and NHISS on admission are the two strongest and independent predictors. For example, the NADE nomogram assigns a 45% probability of unfavorable outcome to a patient over 70 years old with modest creatinine and diabetes mellitus whose NIHSS score is 6, respectively. Instead, 80-year-old (74 points) and NHISS score of 10 (35 points) substantially influence the outcome in the patient because the probability of unfavorable outcome may remain higher than 75%. Moreover, age contributed to a long-term mortality and poor outcome because old age is associated with some cardiovascular diseases such as coronary artery disease, high blood pressure and high cholesterol. In patients > 80 years of age, the risk of poor outcome could be increased by the increased prevalence of cerebral small-vessel disease and comorbidities. Indeed, the NIHSS score indicates a more severe stroke, which can ultimately lead to a poorer outcome, whereas age means a less intense recovery. Secondly, the NADES nomogram also found that diabetes mellitus in ischemic stroke could predict worse outcome independently, because diabetes mellitus can affect the blood vessels and the nerves, which increases the chance of developing poor outcome. Finally, we observed that increased serum creatinine level was associated with unfavorable clinical outcome. Serum creatinine concentration is widely used as an index of renal function, and impaired renal function is associated with poor outcome after stroke .
Our study has some limitations. Firstly, it was an exploratory study in a single center and not a randomized controlled trial (RCT) designed to investigate the critical factors affecting 6-month poor outcome, and the clinical significance of this study may be attenuated by its design. Secondly, data of known neurobiological predictors such as infarct size  were not available in the present study, which may affect the development of the NADE nomogram model to predict the probability of 6-month poor outcome. So the neuroimaging predictors should also be integrated into our model in future studies, which may increase its discriminative performance of the nomogram model. Finally, an external validation in a completely different cohort of patients is warranted although the discriminative performance of the NADE model was good. Despite these limitations, as far as we know, the present study is the first attempt to construct and validate a nomogram to predict the probability of 6-month poor outcome in Chinese stroke patients.
The NADE nomogram may is a reliable tool with accurate continuous probability power to provide an individual and precise prediction of the risk probability of 6-month unfavorable outcome following ischemic stroke in Chinese patients.
We would like to thank all patients, researchers, and institutions for their collaboration and contribution to this study.
CS and XL contributed equally to this work. JSZ and JJZ concept, designed, and supervised the study. YKL, XLC, ZZ, and CL acquired the data. CS, XL, BLS and JJZ analyzed and interpreted the data, provided statistical analysis, had full access to all of the data in the study, and are responsible for the integrity of the data and the accuracy of the data analysis. LN, MI, DT, and CS drafted the manuscript, JJZ, JY, XDP, and MY critically revised the manuscript for important intellectual content. All authors read and approved the final manuscript.
The design of this study; the collection, analysis, and interpretation of data; and language editing were financially supported by National Natural Science Foundation of China grant 81673511, Jiangsu Key Research and Development Plan grant BE2017613, Jiangsu Six Talent Peaks Project grant WSN-151, and Nanjing Medical Science and Technique Development Foundation grant QRX17020 and ZKX15027.
Ethics approval and consent to participate
The study protocols were approved by the Ethics Committees of Nanjing First Hospital in accord with the Helsinki declaration and internal protocol. All patients have given their written informed consent.
Consent for publication
The authors declare that they have no competing interests.
- 15.Weimar C, Konig IR, Kraywinkel K, Ziegler A, Diener HC. Age and National Institutes of Health stroke scale score within 6 hours after onset are accurate predictors of outcome after cerebral ischemia: development and external validation of prognostic models. Stroke. 2004;35(1):158–62.CrossRefGoogle Scholar
- 16.Vora NA, Shook SJ, Schumacher HC, et al. A 5-item scale to predict stroke outcome after cortical middle cerebral artery territory infarction: validation from results of the diffusion and perfusion imaging evaluation for understanding stroke evolution (DEFUSE) study. Stroke. 2011;42(3):645–9.CrossRefGoogle Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.