Retrospective cohort study to develop a diagnostic predictive model with patients evaluated between 2004 and 2014.

Clinical records of all patients diagnosed with SLE of any age and any gender who were treated from January 2004 to October 2014 in two third level hospitals from Medellin Colombia were revised. To be eligible for inclusion, patients were required to have had an available result of a kidney biopsy performed within the first 6 months since onset of symptoms or laboratory abnormalities attributable to nephritis. The 1982 American College of Rheumatology classification criteria, which were updated in 1997, were used to diagnose SLE.20,21 All the data were collected in standard case report forms from clinical records by the investigators, independently of any result on kidney biopsy report.

The outcome variable was the indication to use intensive immunosuppressive treatment with cytostatics (either cyclophosphamide or mycophenolate mofetil) depending on the kidney biopsy report (Class III, IV, V, III+V, or IV+V LN, versus Class I, II or VI LN). The median time between the first tests and the performing of a biopsy was 1 week, and the outcome was independent from the predictor variables considered. Performing a biopsy was not conditioned on the results of the candidate predictor variables. In every SLE patient with possible nephritis, a kidney biopsy was performed in accordance with guidelines and common practices unless there was a clinical contraindication. In both hospitals a cutoff of 500mg of proteinuria is used to order a kidney biopsy.

When kidney biopsy was indicated during outpatient or inpatient care, the initial values from medical records for the following candidate predictor variables were obtained: age in years, gender, black race or another, systolic and diastolic blood pressure (mm Hg), creatinine (mg/dl), blood urea nitrogen (BUN, mg/dl), potassium (mEq/l), hematuria, leukocyturia, red blood cell (RBC) casts and white blood cell (WBC) casts (maximum number of erythrocytes, leukocytes, RBC casts or WBC casts per high power field, respectively), proteinuria (mg in 24h), C3 complement (mg/dl), C4 complement (mg/dl), anti-DNA antibodies (positive ≥1/80), hemoglobin (g/dl), platelets (α/4l∧1), serum albumin (g/dl) and the disease activity measured by SLEDAI score (0•105). These variables were selected based on an extensive literature review1,13•17,22•27 and our own clinical experience. A variable called “modified SLEDAI” was created excluding the kidney-related parameters (proteinuria, pyuria, hematuria and cellular casts) platelets, anti-DNA and complement to avoid the collinearity with these same variables measured individually. The first values recorded in medical records were consistently obtained. Almost all of these variables corresponded to laboratory results; therefore, their results and definitions were independent among them.

We expected, in accordance with the literature review, that 15%•20% of the patients had a form of LN which would not need an addition of intensive therapy with cytostatics. Since the most relevant clinical outcome is needing to use cytostatics, sample size calculation was focused on that outcome. According to the rule of at least 10 outcomes for each independent variable, a sample size of approximately 300 patients was calculated, considering that the outcome would be presented for about 240 patients and this allowed us to consider all of those predictors previously explained.28,29

Missing values were not imputed. The variables with more than 10% missing data were not considered for the model. All analyses were based on the cases with complete data. It was assumed that the missing data of predictor variables occurred completely at random.

Quantitative variables were described as means or medians with their respective measure of dispersion depending on data distribution; and qualitative variables as absolute number and percentages. The relation between independent variables and the outcome was explored using Student's t test and Mann•Whitney U test depending on the distribution of the quantitative variables, and for categorical variables Chi square or Fisher's exact test were used according to the expected values in the cells.

We conducted univariate logistic regression with each one of the independent variables previously defined. Monotony of quantitative variables was graphically evaluated using a non-parametric locally weighted regression model (LOWESS),30 and accordingly, its inclusion as continuous or transformed dummy variables was defined.

For each univariate logistic regression analysis the Odds Ratio (OR), the Wald test, the R2 and the corresponding ROC curve were analyzed. Thus, the discrimination and calibration ability of each one of the candidate predictor variables was explored and for the final model those with a P value<.25 on the Wald test and/or an AUC>0.75 were considered.

Two interaction terms were built based on clinical considerations: between proteinuria and creatinine and also between hematuria and leukocyturia. Then, we compared the log likelihood of each complete model with the interaction term being studied versus the respective nested model which presumes the absence of interaction, using the likelihood ratio test with a P value<.10 considered as statistically significant.31

Afterwards, all the candidate variables proposed plus the significant interaction terms were included in a multivariate logistic regression model, and different alternative models with fewer number of variables were explored seeking the one that would offer the best performance with the most reasonable number of predictors. For this selection, manual comparisons of the discrimination and calibration properties of each model were made using ROC curves and Hosmer•Lemeshow goodness-of-fit tests, respectively. We were always guided by clinical criteria and the principle of parsimony to select the final model. In addition, the presence of collinearity was discarded using the correlation matrix of the independent variables expecting values lower than 0.532 and using the variance inflation factor method (VIF) expecting values lower than 10.33 We decided in each case how much the coefficients and the prediction ability were affected by the variables that showed high collinearity. Internal validity was assessed with a bootstrapping procedure in 200 samples drawn without replacement from the original population. After obtaining the model with the best performance and its respective predictor variables, a spreadsheet was designed to obtain the probability of the outcome in accordance with the individual values of the variables included in the logistic regression equation.

A total of 301 clinical records were studied and 242 patients older than 16 years were included. Those younger than 16 years (n=59 children) were excluded because of considerations regarding their representativeness, the predictive capacity of the model and the external validity of the results. Furthermore, just 6 of these minors did not need immunosuppression (Fig. 1).

Fig. 1.

Participants
tm) flow chart.

(0.31MB).

Table 1 shows the clinical and demographic characteristics of patients included depending on the needing for intensive immunosuppression. The median time of SLE evolution was 3 (IQR=1•32) months, the median time of evolution of symptoms attributable to nephritis was 1 (1•3) month and the median time elapsed between nephritis and the performance of the biopsy was 1 (1•4) week. The median SLEDAI score was 16 (12•20) and there were missing data on race (n=83), albumin (n=40) and potassium (n=37). There were 198 patients with proliferative forms of nephritis that required the addition of cytostatics. The different classes of nephritis found are shown in Table 2.

Table 3 shows the univariate and multivariate logistic regression between each candidate predictor and the outcome variable. For an initial analysis, race, albumin and potassium were excluded because of the high proportion of missing data. The RBC and WBC cellular casts variables had medians of 0 (0•0) in both groups; therefore, they were also omitted on the multivariate logistic regression. Possible collinearity between BUN and creatinine (r=0.651, VIF=1.91), between systolic BP and diastolic BP (r=0.788, VIF=2.92) and between complement C3 and C4 (r=0.713, VIF=2.46) was detected. Therefore, BUN, systolic BP and C4 were excluded and kept the other because they presented a higher univariate OR. Then, for the final model, age, diastolic BP, creatinine, proteinuria, complement C3, hematuria, leukocyturia and anti-DNA were considered; with complete data for 223 cases. The variables that showed coefficients with statistical significance were diastolic BP, creatinine, proteinuria and C3. The model with these four variables had an AUC=0.909 (95% CI=0.869•0.949) and a P value on Hosmer•Lemeshow test of .999.

A significant interaction was detected between hematuria and leukocyturia (P=.007). Thus, in the final model the sediment variable (hematuria*leukocyturia) was included with significant improvement in discrimination (AUC=0.929; 95% CI=0.894•0.963) without calibration changes (P=.956). We also found a significant interaction between creatinine and proteinuria (P=.028). Nevertheless, its inclusion did not improve the model's performance nor did it show changes in the creatinine's effect at different proteinuria levels. Therefore, these two variables were left independent. The assumption of linearity and monotony was confirmed for all the variables included in the model. Table 4 shows the coefficients and their respective OR for the variables included in the final model.

Bootstrapping on 200 repetitions showed inconsistency in sediment (P=.052) and complement C3 (P=.062) coefficients. Despite the apparent instability of these variables, their importance and clinical significance as measures of general activity of SLE and renal inflammation, forced to consider them in external validation and in future studies of clinical use.

A spreadsheet calculator (supplementary appendix) was developed to estimate the probability of needing aggressive immunosuppression with cytostatics using a logistic regression equation with the variables included in the model:

As an exploratory approach, different cut-off points were sought for the needing of aggressive immunosuppression with high specificity or with high sensitivity to confirm or discard, respectively. Based on this equation, and considering as a positive cut-off point the probability of 0.8, an 81.3% sensitivity and 92.7% specificity with a positive likelihood ratio (LR)=11 was obtained. On the other hand, under a probability of 0.4 a 95.6% sensitivity and 46.4% specificity with a negative LR=0.09 was obtained. On the whole cohort, 178 patients (80%) were classified as above 0.8 or under 0.4. In the gray zone between these two cut-off points, 45 of 223 patients (20%) might have to undergo kidney biopsy to decide their treatment.

The authors declare that no experiments were performed on humans or animals for this study.

The authors declare that no patient data appear in this article.

The authors declare that no patient data appear in this article.