Advertisement

BMC Immunology

, 20:11 | Cite as

A novel approach reveals that HLA class 1 single antigen bead-signatures provide a means of high-accuracy pre-transplant risk assessment of acute cellular rejection in renal transplantation

  • Nicole Wittenbrink
  • Sabrina Herrmann
  • Arturo Blazquez-Navarro
  • Chris Bauer
  • Eric Lindberg
  • Kerstin Wolk
  • Robert Sabat
  • Petra Reinke
  • Birgit Sawitzki
  • Oliver Thomusch
  • Christian Hugo
  • Nina Babel
  • Harald Seitz
  • Michal Or-GuilEmail author
Open Access
Research article
Part of the following topical collections:
  1. transplants, autoimmunity and vaccines

Abstract

Background

Acute cellular rejection (ACR) is associated with complications after kidney transplantation, such as graft dysfunction and graft loss. Early risk assessment is therefore critical for the improvement of transplantation outcomes. In this work, we retrospectively analyzed a pre-transplant HLA antigen bead assay data set that was acquired by the e:KID consortium as part of a systems medicine approach.

Results

The data set included single antigen bead (SAB) reactivity profiles of 52 low-risk graft recipients (negative complement dependent cytotoxicity crossmatch, PRA < 30%) who showed detectable pre-transplant anti-HLA 1 antibodies. To assess whether the reactivity profiles provide a means for ACR risk assessment, we established a novel approach which differs from standard approaches in two aspects: the use of quantitative continuous data and the use of a multiparameter classification method. Remarkably, it achieved significant prediction of the 38 graft recipients who experienced ACR with a balanced accuracy of 82.7% (sensitivity = 76.5%, specificity = 88.9%).

Conclusions

The resultant classifier achieved one of the highest prediction accuracies in the literature for pre-transplant risk assessment of ACR. Importantly, it can facilitate risk assessment in non-sensitized patients who lack donor-specific antibodies. As the classifier is based on continuous data and includes weak signals, our results emphasize that not only strong but also weak binding interactions of antibodies and HLA 1 antigens contain predictive information.

Trial registration

ClinicalTrials.gov NCT00724022. Retrospectively registered July 2008.

Keywords

Renal transplantation Acute cellular rejection Pre-transplantation risk assessment Anti-HLA-1 antibodies Single HLA antigen bead assay Immune signatures Machine learning 

Abbreviations

ABMR

Acute antibody-mediated rejection

ACR

Acute cellular rejection

AR

Acute rejection

AUC

Area under the curve

BACC

Balanced accuracy

DSA

Donor-specific antibodies

IQR

Interquartile range

KTR

Kidney transplantation

MAB

Mixed HLA antigen bead

MFI

Median fluorescence intensity

post-Tx

Posttransplantation

PRA

Panel Reactive Antibodies

pre-Tx

Pretransplantation

P-SVM

Potential Support Vector Machine

ROC

Receiver operating characteristic

SAB

Single HLA antigen bead

Sens.

Sensitivity

Spec.

Specificity

Background

The efficacy of immunosuppressive therapy in kidney transplantation has steadily increased over the last decade. As a consequence, the incidence of acute rejection (AR) episodes has decreased and short-term graft survival rates have improved [1, 2]. However, long-term transplant outcomes are still poor and episodes of AR are known to significantly exacerbate long-term outcomes [2, 3]. AR is associated with long-term complications, such as graft dysfunction and reduced graft survival and AR prevention continues to be a main focus in the design of new therapeutic strategies for renal transplantation [4, 5, 6, 7]. The most common form of AR is acute cellular rejection (ACR) [8]. ACR is a T cell cytotoxic immune response against the graft, leading to inflammatory cell infiltration with tubulitis and, eventually, damage of the donor tissue [9, 10]. The positive outcome of ACR if treated early, as well as its potentially irreversible damage, render it particularly relevant for prevention research [10, 11]. Regarding non-invasive diagnostics, a number of studies have obtained good results using tissue, blood or urine markers [11, 12, 13, 14, 15, 16, 17, 18]. For early risk assessment, the large majority of models are donor-dependent, as they either employ measurements from the early post-transplantation period or utilize donor-derived data (e.g. from crossmatch tests) [19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30]. The most common approach for pre-transplant risk assessment relies on the characterization of HLA antibodies in recipient serum samples by solid phase single HLA antigen bead (SAB) assay [24, 25, 26, 27, 28, 29, 31]. The assay facilitates detection and identification of anti-HLA antibody specificities and provides a method for monitoring the development of donor-specific antibodies (DSA). The detection of DSA through SAB assays is a well-established method for antibody-mediated rejection (ABMR) pre-transplantation risk assessment, but not for ACR [24, 25, 26, 27, 28, 29, 30, 32].

Approaches for risk assessment of ACR do not employ DSA for the prediction – as both patients with or without DSA experience episodes of ACR – but other risk markers, such as soluble CD30 levels or panel of reactive T cells [23, 33, 34, 35]. However, the inspection of SAB serum antibody reactivity profiles (irrespective of DSA status) may provide a means to an ACR risk assessment tool for two reasons: (1) serum antibody binding profiles against antigen/protein libraries are generally powerful in discriminating between different health or disease conditions [36, 37, 38, 39], and (2) antibody-mediated mechanisms have been shown to be involved in the T cell-mediated initiation, perpetuation, and progression of graft injury [40, 41].

In this work, as part of an exploratory study, we present a classifier achieving high-accuracy pre-transplant risk assessment of ACR. Remarkably, this classifier is based on continuous non-thresholded HLA 1 SAB data and does not rely on donor-specific HLA typing.

Results

Characteristics of the graft recipients included in the study

Pre-transplant HLA assay data were retrospectively analyzed as part of a systems medicine approach towards early risk assessment of ACR [42, 43]. The investigated study group comprised all kidney transplant recipients enrolled in the Harmony trial (N = 615) who experienced at least one ACR or borderline ACR event in the first year (N = 77) and all transplant recipients who experienced no serious adverse events (N = 80). Median time to the first ACR event was 20.5 days (range = 4–373 days) (Additional file 1: Figure S1). Demographics and clinical characteristics of the study groups are summarized in Additional file 4: Table S1.

Pre-transplantation HLA-1 and HLA-2 MAB data were available for N = 63 recipients of the ACR group and N = 54 recipients of the control group (for demographic and clinical characteristics, see Additional file 5: Table S2). Additionally, HLA-1 SAB data was measured for all those patients who tested positive for HLA-1 MAB screening (21 ACR + 13 control) and a random subset of patients who tested negative (13 ACR + 5 control). In total, pre-transplantation HLA-1 SAB data were available for N = 34 recipients of the ACR group and N = 18 of the control group. Due to the higher sensitivity of SAB assay compared to MAB, the former assay was considered a better candidate for ACR risk assessment.

Demographic and clinical characteristics of the SAB ACR (N = 34) and the SAB control group (N = 18) were compared and are summarized in Table 1. The majority of patients was male, received their first kidney transplant and had a deceased donor. There were no significant differences between the study groups for the above mentioned characteristics as well as immunosuppressive therapy. However, the mean age of patients in the ACR group (54.9 ± 11.0) was significantly higher than for the control group (51.6 ± 11.6; p = 0.04; Mann-Whitney U test) as was the body mass index (27.7 ± 5.4 vs. 24.2 ± 4.2; p = 0.01; Mann-Whitney U test). With respect to HLA mismatches, a significant difference was found for HLA-DR (p = 0.03; Pearson’s chi-squared test), with an elevated frequency of patients with two mismatches in the ACR group (32.4% vs. 11.1%). No significant differences were found for HLA-A or HLA-B. There were no significant differences regarding PRA between the groups. There was a near-significant difference in cold ischemia time with longer times being observed for the ACR group (739 ± 295 vs. 637 ± 302; P = 0.06; Mann-Whitney U test).
Table 1

Characteristics and medication details for the subset of patients included in the HLA class 1 SAB data seta

 

ACR

Control

p-value

Number of kidney transplant recipients

34

18

Age at time of transplantation (years)

57.9 ± 11.0

51.6 ± 11.6

0.04

Body mass index at time of transplantation (kg/m2)

27.7 ± 5.4

24.2 ± 4.2

0.01

Gender

 Female

17 (50.0%)

7 (38.9%)

nsb

 Male

17 (50.0%)

11 (61.1%)

Type of donor

 Living

7 (20.6%)

4 (22.2%)

nsc

 Deceased

27 (79.4%)

14 (77.8%)

Re-transplantation

3 (8.8%)

0 (0.0%)

nsc

HLA-A Mismatchesd

 0

11 (32.4%)

9 (50.0%)

nsc

 1

18 (52.9%)

8 (44.4%)

 2

5 (14.7%)

1 (5.6%)

HLA-B Mismatchesd

 0

3 (8.8%)

4 (22.2%)

nsc

 1

18 (52.9%)

11 (61.1%)

 2

13 (38.2%)

3 (16.7%)

HLA-DR Mismatches

 0

3 (8.8%)

7 (38.9%)

0.03c

 1

20 (58.8%)

9 (50.0%)

 2

11 (32.4%)

2 (11.1%)

PRA = 0%

31 (91.2%)

16 (88.9%)

nsc

Therapeutic Arm

 A

12 (35.3%)

6 (33.3%)

nsc

 B

10 (29.4%)

3 (16.7%)

 C

12 (35.3%)

9 (50.0%)

Cold ischemia time: only deceased donors (min)

739 ± 295

637 ± 302

ns

aData are given as mean ± standard deviation for quantitative variables and as number (frequency) for categorical variables. P values for quantitative variables were calculated by Mann-Whitney U test, for categorical variables either chi-squared (b) or Fisher’s exact test (c) were employed. (d) According to Fisher’s exact test, there is also no statistically significant differences between the ACR and Control groups when HLA-A and HLA-B mismatches are combined into one group

Conventional HLA SAB data analysis does not permit pre-transplant risk assessment of ACR

To assess whether HLA-1 SAB reactivity profiles provide a means for ACR risk assessment, we initially applied the conventional data analysis approach used in HLA-diagnostics to our data set. Central to this approach is the conversion of the quantitative SAB assay read-out data into qualitative binary data (1 = presence of antibody-antigen reactivity, 0 = absence of reactivity) based on a mean fluorescence intensity (MFI) threshold. We performed all analyses for a fixed threshold of 1000 MFI and an individually adjusted threshold in the range 253–1068 MFI (Additional file 2: Figure S2). In both cases, there were no statistically significant differences between the SAB ACR and the SAB control group in any of the individual reactivities (Additional file 6: Table S3 and Additional file 7: Table S4). That is, there are no individual HLA-1 antibody reactivities that allow for risk assessment of ACR.

To assess whether there is a combination of reactivities that allows for risk assessment of ACR, we extended the conventional approach by applying a support vector machine-based multiparameter classification method to the binarized data (for details, see Material and Methods). The resulting multiparameter classifiers did not achieve significant classification performance (p > 0.1, Table 2). Taken together, our results indicate that the conventional HLA SAB data analysis approach does not permit pre-transplant risk assessment of ACR.
Table 2

Multiparameter pre-transplant prediction of ACR

Data set

Data analysis approach

MFI-treshold

BACC [%]

Sens. [%]

Spec. [%]

p-value

HLA-1 SAB

Conventional, binary data input [0, 1]

1000

62.1

35.3

88.9

ns (p > 0.1)

Conventional binary data input [0, 1]

Individually adjusted [253–1068]

70.9

52.9

88.9

ns (p > 0.1)

Novel, continuous data input

82.7

76.5

88.9

0.002

MAB

Novel, continuous data input

63.9

55.6

72.2

0.040

A novel approach built on multiparameter classification and quantitative data input allows for high accuracy pre-transplant prediction of ACR

In spite of the widespread use of HLA SAB assays, the interpretation of results obtained following the conventional data analysis approach remains controversial [44]. A strict MFI threshold consistently identifying clinically relevant antibody-antigen reactivities is challenging to define [45]. Since it is likely that the choice of MFI threshold compromises classification efforts, we applied a novel approach to the HLA-1 SAB data set that does not rely on MFI thresholding. Key to this novel approach is the rank-normalization of the continuous SAB assay read-out data. Remarkably, a support-vector machine-based multiparameter classificator built on these data achieved highly significant prediction performance with a balanced accuracy of 82.7% (Sens. = 76.5%, Spec. = 88.9%, p = 0.002, Fig. 1a and Table 2). Receiver operating characteristic (ROC) analysis further emphasizes that the prediction performance was better than a random guess (area under the curve [AUC] = 0.86) and illustrates the trade-off between the probability of correctly predicting ACR (true positive rate, sensitivity) and the probability of incorrectly predicting ACR (false positive rate, specificity) (Fig. 1b). Importantly, we found that prediction performance was independent of a patient’s MAB screening test result, as patients who tested positive or negative for HLA-1 antibodies are predicted equally well (Fig. 1a). Moreover, the performance of the continuous data classifier was not due to age, BMI or HLA-DR mismatch frequency as confounding factors; significant classification was not achieved when HLA class 1 SAB continuous data were grouped according to either of those factors (≤50 y vs. > 50 y, ≤25 BMI vs. > 25 BMI, or no-mismatch vs. 1–2 mismatches). In addition, median-centered bead MFIs did not show any association with age or BMI (mean Spearman correlation coefficient r = 0.019 ± 0.129 and r = 0.009 ± 0.133, respectively). Taken together, our results show that continuous, rank-normalized HLA-1 SAB reactivity profiles provide a means of high-accuracy risk assessment of pre-transplant ACR.
Fig. 1

Predictive performance of the multiparameter ACR risk assessment classifier based on rank-normalized continuous pre-transplant HLA-1 antibody reactivity profiles. a Output of the classifiers decision function for each patient. The decision threshold is indicated by a dashed horizontal line. Patients with a decision value > 0 are classified as ACR, patients with a decision value < 0 are classified as control. Colors indicate whether patients tested positive (black) or negative (grey) for the presence of serum HLA-1 antibodies during MAB screening. b ROC curve of the multiparameter classifier

Diagnostics based on HLA antibody detection assays may generally benefit from the novel approach

The fact that continuous HLA-1 SAB reactivity data outperformed MFI-thresholded binary data in terms of pre-transplant prediction of ACR (Table 2) led us to the conjecture that the conventional approach entails a loss of information that may compromise HLA-diagnostics classification efforts in general. To substantiate this claim, we performed additional analyses on the MAB screening data (63 ACR + 54 controls). Conventional MFI-threshold based data analysis revealed no statistical differences between the two study group as to the prevalence of HLA class 1 and/or HLA class 2 antibodies (Fig. 2). A multiparameter classifier based on the continuous rank-normalized data, however, achieved statistically significant prediction of the patients who experience ACR (p = 0.04, Table 2 and Additional file 3: Figure S3). Even though the accuracy of the classifier was low and not sufficient for routine risk assessment (BACC = 63.9%, Sens. = 55.6%, Spec. = 72.2%), the fact that it was significant emphasizes that the use of continuous non-thresholded antigen bead assay data favorably affects classification performance.
Fig. 2

Conventional MFI-thresholded binary MAB screening data do not allow for pre-transplant risk assessment of ACR. Illustrated are the results of the MAB screening data of the cohort (117 graft recipients, 63 ACR + 54 controls; for demographics and clinical characteristics, see Additional file 5: Table S2).). Analyses were carried out on MFI-thresholded binary HLA MAB screening data (conventional approach); according to Fisher’s exact test, differences with respect to the prevalence of HLA antibodies are not significant (p > 0.05)

Discussion

The current study shows that pre-transplant HLA class 1 SAB signatures predict the risk of acute cellular rejection (ACR) with high accuracy. Importantly, it demonstrates that HLA antibody signatures contain information on cell-mediated events to come.

In contrast to the vast majority of existing pre-transplant risk assessment models [24, 25, 26, 27, 28, 29, 31], our model does not rely on DSA reactivity data. The key advantages of this approach are that i) it facilitates risk assessment for non-sensitized patients lacking DSA and ii) it can be carried out independently of donor assignment.

HLA SAB data usually feed into prediction models in the form of binary data derived from MFI thresholding – the focus lying on strong binding interactions. Strikingly, our study emphasizes that such an approach entails a loss of information and ultimately results in loss of or suboptimal prediction performance. The fact that continuous HLA reactivity data outperform thresholded binary data (Table 2) indicates that weak binding interactions hold high-value information for risk assessment of ACR. This is further emphasized by our finding that our risk assessment tool performs equally well for patients who tested positive or negative for the presence of HLA-1 antibodies during MAB screening. There is indeed sufficient evidence in the literature to show that weak binding events are of great importance to biological systems, e.g. protein-peptide interactions [46], virus-cell interactions [47], cell adhesion, and cell-cell interactions [48, 49, 50, 51]. Our data suggest that HLA SAB based diagnostics will profit from inclusion of weak interactions by feeding prediction models with non-thresholded continuous data. A further advantage of prediction models based on non-thresholded MFI data is that they are not affected by the prevailing uncertainties regarding the right choice of the threshold MFI level and by the yet missing internationally agreed standards [44, 52].

But why is the pre-transplant signature of serum antibodies against HLA-1 SAB predictive for the risk of T cell mediated rejection? There is evidence in the literature for an association between anti-HLA serum antibodies and ACR. Crosslinking of HLA-1 antigens expressed on the surface of donor cells by HLA class 1 antibodies has been shown to trigger the classical complement pathway through binding of C1q. The subsequent release of the complement peptides C3a and C5a then leads to enhanced allo-T cell responses and leucocyte recruitment [53]. That is, pre-transplant HLA class 1 antibodies may be involved in the initiation and perpetuation of ACR by boosting adaptive T cell activities after graft transfer. HLA class 1 antibodies have also been shown to be directly involved in mechanisms that cause severe graft injury such as endothelial cell activation or NK cell related FcγR-dependent processes [53]. However, these mechanisms usually result in histological manifestations strongly associated with ABMR [53]. Since our study cohort did not show such manifestations, these processes are unlikely to be relevant to the predictive value of our pre-transplant HLA 1 antibody signatures.

The HLA MAB and SAB data used in this study are part of a large multi-parameter database set up by the e:KID consortium that seeks to establish a systems medicine approach to personalized immunosuppressive treatment at an early stage after kidney transplantation (http://www.sys-med.de/en/consortia/ekid/). e:KID recorded a total of 478 parameters including, among others, gene expression, cytokine profile, epigenetics, metabolomics and viral load data as well as common clinical variables such as renal function or acute phase proteins. Evaluation of clinical parameters failed to identify any markers or combinations thereof which are predictive of ACR [4]. Additionally, no other single parameter or multiparameter set, other than HLA class 1 SAB signatures, achieved high accuracy pre-transplant prediction performance. This emphasizes the vast potential of serum antibodies in diagnostics in general, and, in particular, for diseases where the antigens are unknown.

The comparison of predictive performances between our classifier and classifiers in the literature underlines its relevance to pre-kidney transplant risk assessment (Additional file 8: Table S5). Its accuracy of 82.7% is i) one of the highest among all donor-independent risk assessment models [19, 22, 27, 34, 54], ii) comparable to any AR models [19, 20, 21, 22, 23, 24, 26, 27, 28, 29, 30, 34, 35, 54, 55] and iii) comparable to any SAB data based models for ABMR [24, 26, 27, 28, 29, 30]. Furthermore, our classifier is based on SAB, an established diagnostics laboratory tool, thereby facilitating its further use for ACR risk assessment.

Conclusions

Our study establishes a novel tool for pre-transplant risk assessment of acute cellular rejection. Once externally validated, patients classified as high risk by our model will benefit from its implementation through modified immunosuppression as well as closer monitoring leading to earlier detection of rejection onset and initiation of treatment. Consequently, the prognosis and survival rate of the graft will improve.

Methods

Study aim

The aim of this study is to determine whether SAB serum antibody reactivity profiles of renal transplantation recipients can be used for the prediction of ACR during the first post-transplantation year. For this goal, pre-transplant HLA assay data were retrospectively analyzed as part of a systems medicine approach.

Patient population and monitoring

Six hundred fifteen adult kidney transplant recipients were enrolled in the randomized, multicenter diagnostic trial Harmony (EudraCT-Nr. 2007–006516-31) [4]. Patients were treated with a quadruple (arm A) or triple (arms B and C) immunosuppressive therapy as described before [4]. The immunosuppressive therapy included induction with either monoclonal IL-2R antibody basiliximab (arms A and B) (Simulect®, Novartis) or rabbit ATG (arm C) (Thymoglobulin®, Sanofi). Maintenance immunosuppression consisted of tacrolimus (Advagraf®, Astellas) and mycophenolate mofetil (MMF) with (arm A) or without steroids (arms B and C) [4]. All transplantations were of low immunological risk, with recipient PRA scores ≤30% and no detectable DSA prior to transplantation (complement-dependent cytotoxicity crossmatch) [4]. Further inclusion and exclusion criteria can be found in Thomusch et al. [4]. Suspected episodes of acute rejection were confirmed through biopsy according to the Banff criteria of 2005 [56]. For the e:KID project, which aims at early risk assessment of ACR by following a systems medicine approach [42, 43], 157 recipients were retrospectively monitored for the presence of HLA antibodies in blood serum on day 0 (pre-transplantation). All patients who experienced ACR (borderline or Banff class 1 or higher) in the first year were assigned to the ACR group (N = 77). The control group included all patients who neither experienced a rejection episode nor other serious adverse events (N = 80).

HLA antibody detection by Luminex multiplex bead assay

Screening for HLA class 1 and class 2 antibodies was performed using a MAB assay (LABScreen® Mixed Kit, One Lambda, Canoga Park, CA, USA). All sera that tested positive and a random subset of negative sera were subject to SAB assays to identify antibody specificities (LABScreen Single Antigen HLA Class I kit and/or LABScreen Single Antigen HLA Class II kit, One Lambda). Both MAB and SAB were performed according to the manufacturer’s instructions. Briefly, following heat-inactivation at 56 °C for 30 min and clearance from debris (0.22 μm filter), 20 μl of undiluted serum was added to 3 μl of the LABScreen bead mix and incubated for 30 min in the dark at room temperature. After a washing step in 1x LABScreen wash buffer, the bead mix was incubated with 100 μl of a 1:100 dilution of the PE-conjugated goat anti-human IgG detection antibody for 30 min in the dark at room temperature. After a final washing step in 1x LABScreen wash buffer, data acquisition was performed using a FLEXMAP3D Analyser in combination with xPONENT software version 4.1 (Luminex Corporation, Texas, USA).

Conventional HLA data processing and analysis

Key to the conventional method for HLA data processing is the binarization of the continuous xPONENT median fluorescence intensity (MFI) raw data by means of a MFI threshold (1 = presence of reactivity, 0 = absence of reactivity). For generation of binary data, raw MFI data were normalized to an in-house negative control serum (MAB) or the One Lambda negative serum OLI.NS (SAB). In case of MAB data, a bead was considered positive if its normalized background ratio exceeded 3. For binarization of SAB data, we used both a fixed threshold of 1000 MFI and an individually adjusted MFI threshold. In case of the latter, a bead was considered positive when its baseline normalized MFI exceeded 30% of the MFI of the bead showing the highest strength in reaction. Single parameter prediction performance of binarized HLA class I SAB data was assessed using Fisher’s exact test. In the case of binarized HLA class reactivity screening data (MAB), study groups were compared using Fisher’s exact test.

Experimental design and statistical rationale: novel strategy for HLA data processing and analysis

In this work we applied a novel approach to the HLA data sets that does not rely on MFI- thresholding. Key to it is the use of non-thresholded unprocessed continuous data, that is unprocessed raw MFI data given out by the FLEXMAP3D Analyser in combination with the xPONENT software. In contrast to the conventional approach, reactivity may take any value and is not limited to a binary set of values (reactivity/no reactivity). For downstream analyses these raw data are rank-transformed; they are not processed in any other way or form. To assess the predictive potential of the rank-normalized data, we performed multiparameter classification using an non-public R implementation of the Potential Support Vector Machine (P-SVM) algorithm [57]. An equivalent public release that can be run via command line or MatLab interface is provided under: https://ml.jku.at/software/psvm/. To assess the predictive performance of the classifier, we followed a leave-one-out cross-validation approach. The latter provides a well-established solution to assess a classifier’s predictive performance for high-dimensional, low sample size data sets as ours [58]. To rate predictive performance, we used the statistical measures sensitivity (Sens.), specificity (Spec.) and balanced accuracy (BACC). To assess the statistical significance of the predictive performance, we used random class label-permutation testing. The significance level was set at p < 0.05. Training of the P-SVM classifier was performed using the function psvm(), providing the matrix of training data, the class labels of the training data, the cost parameter C, the regularization parameter ε and the parameter epsitol as arguments. In case of classification of SAB data, we performed grid search for the hyperparameter space ε = {0.25, 0.5, 0.75, 1} and C = {1, 6}; the hyperparameter space for MAB data-based classification was ε = {8, 9, 10, 11} and C = {1, 6}. The parameter epsitol was set to 0.05 for all analyses. To predict the class labels of test data, we called the function predict() with the arguments object and x set to the trained classifier and the test data that is to be predicted, respectively. True classification and p-value estimation were always carried out for the same grid of hyperparameters. To further specify the performance of classificators, receiver operating characteristic (ROC) curve analysis was performed. The numerical scores (decision values) that form the basis of P-SVM class identity label assignment were extracted by setting the argument decision.values of the predict() function to TRUE. After sorting the decision values in increasing order they were used as decision boundaries. For each boundary, both sensitivity and specificity were estimated. AUC was calculated based on Mann-Whitney U statistics [59].

Statistical analyses

To assess whether the two study groups (control/ACR) differed in any of the baseline population characteristics, Mann-Whitney U test (metric variables), Pearson’s chi-squared or Fisher’s exact test (categorical variables) were applied. A p-value < 0.05 was considered statistically significant. Variables are described with mean ± standard deviation or median [interquartile range (IQR)].

Notes

Acknowledgements

The authors thank Dr. Nils Lachmann, Center for Tumor Medicine, H&I Laboratory, Charité University Medicine Berlin, for helpful advice and discussions. The authors also thank Ulrich Bodenhofer, now with QUOMATIC.AI, Linz, Austria, and Sepp Hochreiter, Institute of Bioinformatics, Johannes Kepler University Linz, Austria, for insightful discussions.

Part of this work was presented on the Kongress für Nephrologie 2018 – 10. Jahrestagung der Deutschen Gesellschaft für Nephrologie (DGfN), 27-30 September 2018 in Berlin (Germany).

Funding

This work was supported by the German Federal Ministry of Education and Research (BMBF) within the framework of the e:Med research and funding concept (01ZX1312). Funders had no role in study and collection of data, analysis, interpretation of data and writing of the manuscript.

Availability of data and materials

The datasets analysed during the current study are available from the corresponding author on reasonable request.

Authors’ contributions

NW, EL, OT, KW, RS, CB, BS, PR, CH, HS, NB and MO contributed to sample collection and sample management. NW and MO contributed to study design. Data were generated by SH, data interpretation by NW, SH, CB, EL, ABN and MO and drafting of the manuscript by NW, ABN and MO. MO supervised the research and writing processes. All authors have contributed to the manuscript and approved the final version of the manuscript for submission.

Authors’ information

The authors are part of the e:KID consortium, an interdisciplinary group which has as its main aim the development and establishment a systems medicine approach to personalized immunosuppressive treatment at an early stage after kidney transplantation. The consortium’s backgrounds areas include expertise in bioinformatics and machine learning (NW, ABN, CB, EL and MO), HLA antibody experimental analysis and immunology (NW, SH, KW, RS, BS, HS and MO), and the clinic of renal transplantation (PR, OT, CH and NB).

Ethics approval and consent to participate

The study was carried out in compliance with the Declaration of Helsinki and Good Clinical Practice. All participants provided written informed consent prior to inclusion into the study. The trial was approved by the Ethics Committee of the Universitätsklinikum Carl Gustav Carus Dresden. The trial is registered with ClinicalTrials.gov, number NCT00724022 (https://clinicaltrials.gov/ct2/show/study/NCT00724022). Date of registration was July 2008 (retrospectively registered), date of enrollment was June 2008.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary material

12865_2019_291_MOESM1_ESM.pdf (127 kb)
Additional file 1: Figure S1. Cumulative frequency of ACR through the first post-transplantation year. Frequency of patients in the ACR group who experienced at least one ACR event at different time points post-transplantation. At week 2 and month 3, 34% and 79% of patients in the ACR group had experienced an ACR event. (PDF 126 kb)
12865_2019_291_MOESM2_ESM.pdf (54 kb)
Additional file 2: Figure S2. Distribution of MFI cutoffs for generation of binary HLA class 1 SAB data. Illustrated are the MFI cutoff values of the 52 pre-transplant serum samples (control group = 18; ACR group, Rej = 34) of the binary HLA class 1 SAB data (1: presence of reactivity = above the MFI cutoff). (PDF 53 kb)
12865_2019_291_MOESM3_ESM.pdf (211 kb)
Additional file 3: Figure S3. Predictive performance of multiparameter ACR classification based on rank-normalized continuous pre-transplant MAB screening data. (A) Output of the classifiers decision function for each patient. The decision threshold is indicated by a dashed horizontal line. Patients with a decision value > 0 are classified as ACR, patients with a decision value < 0 are classified as control. Colors indicate whether patients tested positive (black) or negative (grey) for the presence of serum HLA-1 antibodies during MAB screening. (B) ROC curve of the multiparameter classifier. (PDF 211 kb)
12865_2019_291_MOESM4_ESM.docx (19 kb)
Additional file 4: Table S1. Baseline characteristics and medication details for the Harmony patient ACR-control sub-cohort. Baseline characteristics and medication details for the Harmony patient ACR-control sub-cohort.a (DOCX 18 kb)
12865_2019_291_MOESM5_ESM.docx (21 kb)
Additional file 5: Table S2. Baseline characteristics and medication details for the patients with HLA MAB data.a (DOCX 20 kb)
12865_2019_291_MOESM6_ESM.docx (18 kb)
Additional file 6: Table S3. Single parameter pre-transplant prediction of ACR based on binarized HLA class 1 SAB data (fixed MFI threshold of 1000 MFI). (DOCX 18 kb)
12865_2019_291_MOESM7_ESM.docx (18 kb)
Additional file 7: Table S4. Single parameter pre-transplant prediction of ACR based on binarized HLA class 1 SAB data (individually adjusted MFI threshold). (DOCX 18 kb)
12865_2019_291_MOESM8_ESM.docx (19 kb)
Additional file 8: Table S5. Summary of existing AR prediction models. (DOCX 19 kb)

References

  1. 1.
    Djamali A, Kaufman DB, Ellis TM, Zhong W, Matas A, Samaniego M. Diagnosis and management of antibody-mediated rejection: current status and novel approaches. Am J Transplant. 2014;14(2):255–71.PubMedPubMedCentralCrossRefGoogle Scholar
  2. 2.
    Meier-Kriesche HU, Schold JD, Srinivas TR, Kaplan B. Lack of improvement in renal allograft survival despite a marked decrease in acute rejection rates over the Most recent era. Am J Transplant. 2004;4(3):378–83.PubMedCrossRefGoogle Scholar
  3. 3.
    Meier-Kriesche HU, Schold JD, Kaplan B. Long-term renal allograft survival: have we made significant progress or is it time to rethink our analytic and therapeutic strategies? Am J Transplant. 2004;4(8):1289–95.PubMedCrossRefGoogle Scholar
  4. 4.
    Thomusch O, Wiesener M, Opgenoorth M, Pascher A, Woitas RP, Witzke O, et al. Rabbit-ATG or basiliximab induction for rapid steroid withdrawal after renal transplantation (harmony): an open-label, multicentre, randomised controlled trial. Lancet. 2016;388:3006–16.PubMedCrossRefGoogle Scholar
  5. 5.
    Ciancio G, Burke GW, Gaynor JJ, Carreno MR, Cirocco RE, Mathew JM, et al. A randomized trial of three renal transplant induction antibodies: early comparison of tacrolimus, mycophenolate Mofetil, and steroid dosing, and newer immune-Monitoring1. Transplantation. 2005;80(4):457–65.PubMedCrossRefGoogle Scholar
  6. 6.
    Ekberg H, Tedesco-Silva H, Demirbas A, Vítko Š, Nashan B, Gürkan A, et al. Reduced exposure to Calcineurin inhibitors in renal transplantation. N Engl J Med. 2007;357(25):2562–75.PubMedCrossRefGoogle Scholar
  7. 7.
    El-Zoghby ZM, Stegall MD, Lager DJ, Kremers WK, Amer H, Gloor JM, et al. Identifying specific causes of kidney allograft loss. Am J Transplant. 2009;9(3):527–35.PubMedCrossRefGoogle Scholar
  8. 8.
    Roberts DM, Jiang SH, Chadban SJ. The treatment of acute antibody-mediated rejection in kidney transplant recipients—a systematic review. Transp J. 2012;94(8):775–83.CrossRefGoogle Scholar
  9. 9.
    Magil AB. Monocytes/macrophages in renal allograft rejection. Transplant Rev. 2009;23(4):199–208.CrossRefGoogle Scholar
  10. 10.
    Becker LE, Morath C, Suesal C. Immune mechanisms of acute and chronic rejection. Clin Biochem. 2016;49(4–5):320–3.PubMedCrossRefGoogle Scholar
  11. 11.
    Suthanthiran M, Schwartz JE, Ding R, Abecassis M, Dadhania D, Samstein B, Knechtle SJ, et al. Urinary-cell mRNA profile and acute cellular rejection in kidney allografts. N Engl J Med. 2013;369:20–31.PubMedPubMedCentralCrossRefGoogle Scholar
  12. 12.
    Afaneh C, Muthukumar T, Lubetzky M, Ding R, Snopkowski C, Sharma VK, et al. Urinary cell levels of mRNA for OX40, OX40L, PD-1, PD-L1, or PD-L2 and acute rejection of human renal allografts. Transplantation. 2010;90(12):1381–7.PubMedPubMedCentralCrossRefGoogle Scholar
  13. 13.
    Hricik DE, Nickerson P, Formica RN, Poggio ED, Rush D, Newell KA, et al. Multicenter validation of urinary CXCL9 as a risk-stratifying biomarker for kidney transplant injury. Am J Transplant. 2013;13(10):2634–44.PubMedPubMedCentralCrossRefGoogle Scholar
  14. 14.
    Roedder S, Sigdel T, Salomonis N, Hsieh S, Dai H, Bestard O, et al. The kSORT assay to detect renal transplant patients at high risk for acute rejection: results of the multicenter AART study. PLoS Med. 2014;11(11):e1001759.PubMedPubMedCentralCrossRefGoogle Scholar
  15. 15.
    Wang JN, Zhou Y, Zhu TY, Wang X, Guo YL. Prediction of acute cellular renal allograft rejection by urinary metabolomics using MALDI-FTMS. J Proteome Res. 2008;7(8):3597–601.PubMedCrossRefGoogle Scholar
  16. 16.
    Reeve J, Einecke G, Mengel M, Sis B, Kayser N, Kaplan B, et al. Diagnosing rejection in renal transplants: a comparison of molecular- and histopathology-based approaches. Am J Transplant. 2009;9(8):1802–10.PubMedCrossRefGoogle Scholar
  17. 17.
    Desvaux D, Schwarzinger M, Pastural M, Baron C, Abtahi M, Berrehar F, et al. Molecular diagnosis of renal-allograft rejection: correlation with histopathologic evaluation and antirejection-therapy resistance. Transplantation. 2004;78(5):647–53.PubMedCrossRefGoogle Scholar
  18. 18.
    Ting YT, Coates PT, Marti H-P, Dunn AC, Parker RM, Pickering JW, et al. Urinary soluble HLA-DR is a potential biomarker for acute renal transplant rejection. Transplantation. 2010;89(9):1071–8.PubMedCrossRefGoogle Scholar
  19. 19.
    Poggio ED, Augustine JJ, Clemente M, Danzig JM, Volokh N, Zand MS, et al. Pretransplant cellular Alloimmunity as assessed by a panel of reactive T cells assay correlates with acute renal graft rejection. Transplantation. 2007;83(7):847–52.PubMedCrossRefGoogle Scholar
  20. 20.
    Simon T, Opelz G, Wiesel M, Pelzl S, Ott RC, Süsal C. Serial peripheral blood Interleukin-18 and perforin gene expression measurements for prediction of acute kidney graft rejection. Am J Transplant. 2003;3(1121–1127):1589–95.Google Scholar
  21. 21.
    Hauser IA. Prediction of acute renal allograft rejection by urinary Monokine induced by IFN- (MIG). J Am Soc Nephrol. 2005;16(6):1849–58.PubMedCrossRefGoogle Scholar
  22. 22.
    Mancebo E, Castro MJ, Allende LM, Talayero P, Brunet M, Millán O, et al. High proportion of CD95+ and CD38+ in cultured CD8+ T cells predicts acute rejection and infection, respectively, in kidney recipients. Transpl Immunol. 2016;34:33–41.PubMedCrossRefGoogle Scholar
  23. 23.
    Dong W, Shunliang Y, Weizhen W, Qinghua W, Zhangxin Z, Jianming T, et al. Prediction of acute renal allograft rejection in early post-transplantation period by soluble CD30. Transpl Immunol. 2006;16(1):41–5.PubMedCrossRefGoogle Scholar
  24. 24.
    Malheiro J, Tafulo S, Dias L, Martins LS, Fonseca I, Beirão I, et al. Analysis of preformed donor-specific anti-HLA antibodies characteristics for prediction of antibody-mediated rejection in kidney transplantation. Transpl Immunol. 2015;32(2):66–71.PubMedCrossRefGoogle Scholar
  25. 25.
    Vlad G, Ho EK, Vasilescu ER, Colovai AI, Stokes MB, Markowitz GS, et al. Relevance of different antibody detection methods for the prediction of antibody-mediated rejection and deceased-donor kidney allograft survival. Hum Immunol. 2009;70(8):589–94.PubMedCrossRefGoogle Scholar
  26. 26.
    Riethmüller S, Ferrari-Lacraz S, Müller MK, Raptis DA, Hadaya K, Rüsi B, et al. Donor-specific antibody levels and three generations of crossmatches to predict antibody-mediated rejection in kidney transplantation. Transp J. 2010;90(2):160–7.CrossRefGoogle Scholar
  27. 27.
    Lefaucheur C, Loupy A, Hill GS, Andrade J, Nochy D, Antoine C, et al. Preexisting donor-specific HLA antibodies predict outcome in kidney transplantation. J Am Soc Nephrol. 2010;21(8):1398–406.PubMedPubMedCentralCrossRefGoogle Scholar
  28. 28.
    Song EY, Lee YJ, Hyun J, Kim YS, Ahn C, Ha J, et al. Clinical relevance of pretransplant HLA class II donor-specific antibodies in renal transplantation patients with negative T-cell cytotoxicity crossmatches. Ann Lab Med. 2012;32(2):139–44.PubMedPubMedCentralCrossRefGoogle Scholar
  29. 29.
    Salvadé I, Aubert V, Venetz JP, Golshayan D, Saouli AC, Matter M, et al. Clinically-relevant threshold of preformed donor-specific anti-HLA antibodies in kidney transplantation. Hum Immunol. 2016;77(6):483–9.PubMedCrossRefGoogle Scholar
  30. 30.
    Shaikhina T, Lowe D, Daga S, Briggs D, Higgins R, Khovanova N. Decision tree and random forest models for outcome prediction in antibody incompatible kidney transplantation. Biomedical Signal Processing Control. 2017; In press. DOI:  https://doi.org/10.1016/j.bspc.2017.01.012.
  31. 31.
    Ho EK, Vasilescu ER, Colovai AI, Stokes MB, Hallar M, Markowitz GS, et al. Sensitivity, specificity and clinical relevance of different cross-matching assays in deceased-donor renal transplantation. Transpl Immunol. 2008;20(1–2):61–7.PubMedCrossRefGoogle Scholar
  32. 32.
    Kannabhiran D, Lee J, Schwartz JE, Friedlander R, Aull M, Muthukumar T, et al. Characteristics of circulating donor human leukocyte antigen-specific immunoglobulin G antibodies predictive of acute antibody-mediated rejection and kidney allograft failure. Transplantation. 2015;99(6):1156–64.PubMedPubMedCentralCrossRefGoogle Scholar
  33. 33.
    Poggio ED, Clemente M, Hricik DE, Heeger PS. Panel of reactive T cells as a measurement of primed cellular Alloimmunity in kidney transplant candidates. J Am Soc Nephrol. 2006;17:564–72.PubMedCrossRefGoogle Scholar
  34. 34.
    Vondran FWR, Timrott K, Kollrich S, Steinhoff AK, Kaltenborn A, Schrem H, et al. Pre-transplant immune state defined by serum markers and alloreactivity predicts acute rejection after living donor kidney transplantation. Clin Transpl. 2014;28(9):968–79.CrossRefGoogle Scholar
  35. 35.
    Nafar M, Farrokhi F, Vaezi M, Entezari AE, Pour-Reza-Gholi F, Firoozan A, et al. Pre-transplant and post-transplant soluble CD30 for prediction and diagnosis of acute kidney allograft rejection. Int Urol Nephrol. 2009;41(3):687–93.PubMedCrossRefGoogle Scholar
  36. 36.
    Hughes AK, Cichacz Z, Scheck A, Coons SW, Johnston SA, Stafford P. Immunosignaturing can detect products from molecular markers in brain cancer. PLoS One. 2012;7(7):e40201.PubMedPubMedCentralCrossRefGoogle Scholar
  37. 37.
    Restrepo L, Stafford P, Johnston SA. Feasibility of an early Alzheimer’s disease immunosignature diagnostic test. J Neuroimmunol. 2013 Jan;254(1–2):154–60.PubMedCrossRefGoogle Scholar
  38. 38.
    Stafford P, Cichacz Z, Woodbury NW, Johnston SA. Immunosignature system for diagnosis of cancer. Proc Natl Acad Sci U S A. 2014 Jul;111(30):E3072–80.PubMedPubMedCentralCrossRefGoogle Scholar
  39. 39.
    Legutki JB, Johnston SA. Immunosignatures can predict vaccine efficacy. Proc Natl Acad Sci U S A. 2013;110(46):18614–9.PubMedPubMedCentralCrossRefGoogle Scholar
  40. 40.
    Cascalho MI, Chen BJ, Kain M, Platt JL. The paradoxical functions of B cells and antibodies in transplantation. J Immunol. 2013;190(3):875–9.PubMedPubMedCentralCrossRefGoogle Scholar
  41. 41.
    Randhawa P. T-cell-mediated rejection of the kidney in the era of donor-specific antibodies: diagnostic challenges and clinical significance. Curr Opin Organ Transplant. 2015;20(3):325–32.PubMedCrossRefGoogle Scholar
  42. 42.
    Blazquez-Navarro A, Schachtner T, Stervbo U, Sefrin A, Stein M, Westhoff TH, et al. Differential T cell response against BK virus regulatory and structural antigens: a viral dynamics modelling approach. PLoS Comput Biol. 2018;14(5):1–20.CrossRefGoogle Scholar
  43. 43.
    Blazquez-Navarro A, Dang-Heine C, Wittenbrink N, Bauer C, Wolk K, Sabat R, et al. BKV, CMV, and EBV interactions and their effect on graft function one year post-renal transplantation: results from a large multi-Centre study. EBioMedicine. 2018;34:113–21.PubMedPubMedCentralCrossRefGoogle Scholar
  44. 44.
    Sullivan HC, Liwski RS, Bray RA, Gebel HM. The road to HLA antibody evaluation: do not rely on MFI. Am J Transplant. 2017;XX:1–7.Google Scholar
  45. 45.
    Konvalinka A, Tinckam K. Utility of HLA antibody testing in kidney transplantation. J Am Soc Nephrol. 2015;26(7):1489–502.PubMedPubMedCentralCrossRefGoogle Scholar
  46. 46.
    Fairchild PJ, Wraith DC. Lowering the tone: mechanisms of immunodominance among epitopes with low affinity for MHC. Immunol Today. 1996;17(2):80–5.PubMedCrossRefGoogle Scholar
  47. 47.
    Haywood AM. Virus receptors: binding, adhesion strengthening, and changes in viral structure. J Virol. 1994;68(1):1–5.PubMedPubMedCentralGoogle Scholar
  48. 48.
    Hakomori S. Structure and function of Sphingoglycolipids in transmembrane Signalling and cell-cell interactions. Biochem Soc Trans. 1993;21(3):583–95.PubMedCrossRefGoogle Scholar
  49. 49.
    van der Merwe PA, Brown MH, Davis SJ, Barclay AN. Affinity and kinetic analysis of the interaction of the cell adhesion molecules rat CD2 and CD48. EMBO J. 1993;12(13):4945–54.PubMedPubMedCentralCrossRefGoogle Scholar
  50. 50.
    Reilly PL, Woska JR, Jeanfavre DD, McNally E, Rothlein R, Bormann BJ. The native structure of intercellular adhesion molecule-1 (ICAM-1) is a dimer. Correlation with binding to LFA-1. J Immunol. 1995;155(2):529–32.PubMedGoogle Scholar
  51. 51.
    Hage DS. Weak affinity chromatography. In: Affinity chromatography methods and protocols; 2000. p. 7–23.Google Scholar
  52. 52.
    Szatmary P, Jones J, Hammad A, Middleton D. Impact of sensitivity of human leucocyte antigen antibody detection by Luminex technology on graft loss at 1 year. Clin Kidney J. 2013;6(3):283–6.PubMedPubMedCentralCrossRefGoogle Scholar
  53. 53.
    Thomas KA, Valenzuela NM, Reed EF. The perfect storm: HLA antibodies, complement, FcγRs and endothelium in transplant rejection. Trends Mol Med. 2015;21(5):319–29.PubMedPubMedCentralCrossRefGoogle Scholar
  54. 54.
    Pike R, Thomas N, Workman S, Ambrose L, Guzman D, Sivakumaran S, et al. PD1-expressing T cell subsets modify the rejection risk in renal transplant patients. Front Immunol. 2016;7:126.PubMedPubMedCentralCrossRefGoogle Scholar
  55. 55.
    Zhang Q, Liu YF, Su ZX, Shi LP, Chen YH. Serum fractalkine and interferon-gamma inducible protein-10 concentrations are early detection markers for acute renal allograft rejection. Transplant Proc. 2014;46(5):1420–5.PubMedCrossRefGoogle Scholar
  56. 56.
    Solez K, Colvin RB, Racusen LC, Sis B, Halloran PF, Birk PE, et al. Banff ‘05 meeting report: differential diagnosis of chronic allograft injury and elimination of chronic allograft nephropathy ('CAN’). Am J Transplant. 2007;7(3):518–26.PubMedCrossRefGoogle Scholar
  57. 57.
    Hochreiter S, Obermayer K. Support vector machines for dyadic data. Neural Comput. 2006 Jun;18(6):1472–510.PubMedCrossRefGoogle Scholar
  58. 58.
    Molinaro AM, Simon R, Pfeiffer RM. Prediction error estimation: a comparison of resampling methods. Bioinformatics. 2005;21(15):3301–7.PubMedCrossRefGoogle Scholar
  59. 59.
    Mason SJ, Graham NE. Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: statistical significance and interpretation. Q J R Meteorol Soc. 2002;128(584):2145–66.CrossRefGoogle Scholar

Copyright information

© The Author(s). 2019

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors and Affiliations

  • Nicole Wittenbrink
    • 1
  • Sabrina Herrmann
    • 2
  • Arturo Blazquez-Navarro
    • 1
    • 3
  • Chris Bauer
    • 4
  • Eric Lindberg
    • 4
  • Kerstin Wolk
    • 3
    • 5
  • Robert Sabat
    • 5
    • 6
  • Petra Reinke
    • 3
    • 7
    • 8
  • Birgit Sawitzki
    • 3
    • 9
  • Oliver Thomusch
    • 10
  • Christian Hugo
    • 11
  • Nina Babel
    • 3
    • 12
  • Harald Seitz
    • 2
  • Michal Or-Guil
    • 1
    Email author
  1. 1.Systems Immunology Lab, Department of BiologyHumboldt University BerlinBerlinGermany
  2. 2.Fraunhofer Institute for Cell Therapy and Immunology, Bioanalytics und BioprocessesPotsdamGermany
  3. 3.Berlin-Brandenburg Center for Regenerative Therapies (BCRT)BerlinGermany
  4. 4.MicroDiscovery GmbHBerlinGermany
  5. 5.Psoriasis Research and Treatment Center, Institute of Medical Immunology, Department of Dermatology and AllergyCharité University Medicine BerlinBerlinGermany
  6. 6.Interdisciplinary Group of Molecular Immunopathology, Institute of Medical Immunology, Department of Dermatology and AllergyCharité University Medicine BerlinBerlinGermany
  7. 7.Department of Nephrology and Internal Intensive CareCharité University Medicine Berlin, Campus Virchow ClinicBerlinGermany
  8. 8.Berlin Center for Advanced Therapies (BeCAT)Charité University Medicine Berlin, Campus Virchow ClinicBerlinGermany
  9. 9.Molecular Immune Modulation, Institute for Medical ImmunologyCharité University Medicine Berlin, Campus Virchow ClinicBerlinGermany
  10. 10.Klinik für Allgemein- und ViszeralchirurgieUniversitätsklinikum FreiburgFreiburgGermany
  11. 11.University Hospital Carl Gustav Carus, Dresden University of TechnologyDresdenGermany
  12. 12.Medical Clinic I, Marien Hospital HerneRuhr University BochumHerneGermany

Personalised recommendations