This retrospective study aimed to perform the first external validation of the ACR/EULAR classification criteria for inflammatory myopathy (IIM) in a Mexican dynamic cohort where the patients were evaluated with clinical and laboratory values. As secondary objectives, we presented the clinical characteristics of the patients and included antibodies other than anti Jo1 to evaluate their impact on our population.
MethodologyThis study included 70 patients with IIM and 70 patients with differential diagnoses of IIM, according to the absolute score of the classification criteria. We obtained sensitivity and specificity in the modality without biopsy, and as an exploratory analysis, we added other antibodies from the myositis extended panel. We analyzed the area under the curve (AUC) of three models: score without antibodies, with anti Jo1 and with any antibody.
ResultsThe ACR/EULAR criteria showed increased specificity and at least similar sensitivity to that of the original cohort (85% sensitivity and 92% specificity), with a cohort point of >55%. When we classified patients into definite, probable, possible, and no IIM categories, by adding the extended myopathy panel, 6 of the 10 patients initially classified as “no IIM” changed their classification to “Probable IIM” and 4 to “Definite IIM”; of the 16 patients classified as “probable IIM,” 15 changed their classification to “Definite IIM.”
ConclusionConsidering the limitations of this study, we concluded that the 2017 EULAR/ACR criteria for IIM classification are sensitive and specific for classifying patients with IIM in the Mexican population. Additionally, the addition of antibodies other than anti-Jo1 may improve performance in certain populations.
Este estudio retrospectivo tuvo como objetivo realizar la primera validación externa de los criterios de clasificación ACR/EULAR para miopatía inflamatoria (MII) en una cohorte dinámica de pacientes mexicanos que fueron evaluados en consulta y con muestras de laboratorio. Como objetivos secundarios presentamos las características clínicas de los pacientes e incluimos anticuerpos distintos al anti-Jo1 para evaluar su impacto en nuestra población.
MetodologíaEste estudio incluyó a 70 pacientes con MII y 70 pacientes con diagnóstico diferencial de MII, según la puntuación absoluta de los criterios de clasificación. Obtuvimos la sensibilidad y la especificidad en la modalidad sin biopsia, y como análisis exploratorio añadimos otros anticuerpos del panel extendido de miositis. Analizamos el área bajo la curva (AUC) de tres modelos: puntuación sin anticuerpos, con anti-Jo1 y con cualquier otro anticuerpo.
ResultadosLos criterios ACR/EULAR mostraron una mayor especificidad y una sensibilidad, al menos similar a la de la cohorte original (85% de sensibilidad y 92% de especificidad), con un punto de cohorte de >55%. Cuando clasificamos a los pacientes en las categorías de definitiva, probable, posible y sin MII, al agregar el panel ampliado de miopatía, 6 de los 10 pacientes clasificados inicialmente como «Sin MII» cambiaron su clasificación a «Probable MII» y 4 a «MII Definitiva»; de los 16 pacientes clasificados como «Probable MII», 15 cambiaron su clasificación a «MII Definitiva».
ConclusiónConsiderando las limitaciones de este estudio, concluimos que los criterios de 2017 de la EULAR/ACR para la clasificación de la MII son sensibles y específicos para clasificar a los pacientes con MII en la población mexicana. Además, la adición de anticuerpos que no sean anti-Jo1 puede mejorar la estadificación en ciertas poblaciones.
Immune-mediated inflammatory myopathies (IIM) are an uncommon and heterogeneous group of diseases that mainly affect skeletal muscles and have a wide variety of manifestations in various organs.1 Various classification criteria have been used to include these patients in previous studies. Although they do not exclude muscular dystrophies with inflammation, those of Bohan and Peter are among the most widely used.2,3 The discovery of specific antibodies (MSA) and myopathy-associated antibodies (MAA) in recent decades has been related to the specific phenotypes of IIM and their clinical course. Although it seems that they may help in the classification, diagnosis, and prognosis of different groups with/without IIM,4 the inclusion of these to improve accuracy in classification has not been adequately demonstrated.
In 2017, the European League Against Rheumatism (EULAR) and the American College of Rheumatology (ACR) published the classification criteria for adults and children with IIM. Considering the two models of patients, one with biopsies and one without biopsies, this classification included 16 variables of clinical features, histopathological findings, laboratory findings, and anti Jo1 antibody positivity as the only serological variable. Assigning different scores to each variable provided an aggregate total score, and a probability was obtained using a formula. These criteria allow the modification of the cut points for trials that require stricter classifications, classifying patients into definite, probable, or possible IIM. The new criteria were found to have high sensitivity and specificity,5 however, they still need to be validated in populations that may be under-represented in the original cohort.
External assessments have already been performed; in a Japanese cohort, Jinnin M, et al., found higher sensitivity and specificity of the criteria in their population for the model without biopsies, and similar sensitivity but lower specificity in patients with biopsies.6 In an Australian validation, Luu et al. found lower sensitivity but higher specificity for the new criteria in their population and proposed that the possible addition of antibodies other than anti-Jo1 and MRI findings may improve the performance of these criteria.
In the original cohort, the Hispanic population comprised approximately 5% of patients. In previous studies, a higher prevalence of dermatomyositis (DM) was found in the Mesoamerican population. These patients have clinical and genetic differences compared with other patients worldwide.7 Furthermore, it is known that the prevalence of MSA and MAA differs in different populations, even within the same country. In Mexico, a low prevalence of anti-Jo1 antibodies and a higher prevalence of anti-Mi2 antibodies have been reported,8 perhaps associated with genetic and environmental traits such as UV exposure.9 The purpose of this study was to perform the first external validation of the ACR/EULAR classification criteria for inflammatory myopathies by calculating their sensitivity and specificity in the Mexican population. As a secondary objective, we present the clinical characteristics of the patients and include antibodies other than anti Jo1 to assess their impact on sensitivity and specificity in our population.
Patients and methodsStudy design, subjects, and data collectionThis population-based, observational, descriptive, cross-sectional study of diagnostic tests included subjects with a diagnosis of IIM, using the gold standard, the clinical diagnosis of certified rheumatologists, and patients with differential diagnoses of IIM. We used the database from the rheumatology consultation center and from the specialized laboratory of the Arthritis and Rheumatism Specialist Center of the University Hospital “José Eleuterio Gónzalez” in Monterrey N.L, Mexico, from January 2017 to July 31, 2021.
For the group of patients with IIM (cases), we applied the following inclusion criteria: patients older than 18 years who had complete clinical data for IIM at the time of diagnosis and a complete determination of the 17-element myositis panel, with or without biopsies. For the group of patients without IIM (controls), we identified those over 18 years of age whose final diagnosis was not IIM but who had complete clinical data of muscle weakness and/or dermatitis, with or without biopsy results, and a complete myositis panel requested. Exclusion criteria included patients lacking complete clinical data or a myositis panel assessment. Additionally, patients were not paired by age or sex.
Clinical data were collected retrospectively in the order of new patient visits from the medical records using questionnaires. The study was approved by the local ethics committee with registration key RE21-00002, and informed consent was not required.
VariablesWe collected the following clinical and demographic variables: sex, age, time of disease onset, weight, height, and body mass index. Muscle assessment included the deltoid, biceps, wrist extensors, quadriceps, ankle dorsiflexors, neck flexors, gluteus medius, gluteus medius, and gluteus maximus using the MMT8 manual muscle test.10 Skin involvement was assessed using Gottron's papules, heliotrope rash, Gottron's sign, V sign, shawl sign, Holster sign, periungual erythema, calcinosis, and mechanic's hands. We also described any pulmonary, joint, cardiac, or gastrointestinal involvement, including dysphagia.
The paraclinical studies evaluated were Aspartate aminotransferase (AST), alanine aminotransferase (ALT), creatine phosphokinase (CPK) lactate dehydrogenase (LDH). Electromyography and muscle biopsy results were obtained.
The myositis antibody panel included myositis-specific autoantibodies and associated antibodies: recombinant Mi2 alpha protein (Mi2a), recombinant Mi2 beta protein (Mi2b), transcription intermediary factor 1γ/α (TIF1γ/α, p155/140), anti-melanoma differentiation-associated gene 5 (MDA5), anti-MJ/nuclear matrix protein 2 (NXP-2), activating enzyme (SAE1), anti-Ku antibody (Ku), anti-PM/Scl (PM100 and PM75), anti-histidyl-tRNA synthetase (Jo-1), anti-signal recognition particle (SRP), anti-threonyl-tRNA synthetase (PL-7), anti-alanyl-tRNA synthetase (anti-PL12), anti-glycyl-tRNA synthetase (EJ), anti-isoleucyl-tRNA synthetase (OJ), and anti-Ro 52 (Ro52), anti-cytosolic 5′-nucleotidase 1A (cN1A) antibodies tested by Immunoblot.
Methodology for assigning classificationThe new criteria include clinical and laboratory findings. Each item was assigned a weighted score and the aggregate scores were calculated by adding the score points. Aggregate scores can be converted to the probability of IIM using the following formula: IIM probability=1/[1+exponential (6.49−score)] when muscle biopsy data are present or IIM probability=1/[1+exponential (5.33−score)] when muscle biopsy data are not present. Patients with a probability >55%, corresponding to a score of ≥5.5, or ≥6.7 if biopsies were included, were considered classifiable as IIM. We classified as “definite IIM” if the probability was >90% (corresponding to total score of ≥7.5 without biopsy and ≥8.7 with biopsy), “probable IIM” at the level of probability between 55% and 90%, “possible IIM” between 50% and 55% probability and “no IIM” if the probability was <50% (score <5.3 without biopsies; <6.5, with biopsies). We calculated the total aggregate score and classified patients into the above categories. We did not use the web calculator for the correlation between the aggregate total scores and the variable probability of having IIM, as shown in the original article.5
Statistical analysisDescriptive statistics were performed for demographic, clinical, and serological variables using percentages for categorical variables and means (SD) or medians (IQR) for continuous variables according to their distribution. The sensitivity, specificity, and positive and negative odds ratios of the EULAR/ACR scoring criteria against the gold standard clinical diagnosis were calculated and used to assess the diagnostic accuracy of the criteria.
Collins et al. reported that small external validation studies are unreliable and may be inaccurate and biased. However, the average standardized bias for all models and performance measures fell below 10% when the number of events increases to 75–100.11 Therefore, to assess the significance of the data obtained, we consider that although the number is not ideal due to our cohort having 70 patients with IIM, the results obtained from their analysis can be considered valid.
In the exploratory analyses, we included the antibodies from the extended myopathy panel, considering as positive any antibody, and weighting a score similar to Jo1. We then calculated the area under the curve (AUC) in three different models: one model where only scores without antibodies were considered; a second one with the original criteria and anti Jo1 antibodies; and a third where any antibody was included to determine the accuracy of including the extended panel of antibodies as covariates of the EULAR/ACR scoring criteria. This was performed to determine the probability of a definitive diagnosis of IIM. In addition, we divided the patients into the following categories: definite, probable, possible, and no IIM, and showed the change in patient classification with the addition of other antibodies.
Statistical analyses were performed using SPSS v 25 software.
ResultsClinical and demographic characteristics of patients with and without IIMWe found 84 patients with a diagnosis of IIM from January 2017 to July 2021; we eliminated 14 patients due to lack of data or myopathy panel. We collected data from 70 patients in the case group with IIM and either of the following diagnoses: dermatomyositis (DM) (n=48), Amyopathic Dermatomyositis (AMD) (n=8), polymyositis (PM) (n=39), or immune-mediated necrotizing myopathy (IMNM) (n=1); and 70 patients in the control group with 14 different diseases (Table 1 – Supplemental Material 1). In both groups, most patients were female (74.3% in the group with IIM and 90% in the group without IIM). Among the clinical characteristics, proximal weakness was more frequent in the IIM group (n=61, 87.1% in the IIM group and n=19, 27.1% in the non-IIM group), where the most affected muscle in both groups was the deltoid (n=51, 72.9% in the IIM group and n=10, 14.3% in the non-IIM group). We found Gottron's papules in 47.1% of patients with IIM, while none were found in the group without IIM. While heliotrope rash was only found in 5.7%, it was the most common finding in the control group along with joint involvement (n=55, 78.6% in the group without IIM and n=38, 54.3% in the group with IIM). Only 11 patients in the IIM group (15.7%) and 5 patients (7.1%) in the group without IIM had muscle biopsies. Electromyography was performed in six patients (8.6%) in the group of patients with IIM and two patients (2.9%) in the group without IIM (Table 1 – Supplemental Material 1).
Of the IIM-specific antibodies, Mi2 (alpha and beta) was the most common antibody in both groups (37.2% in the group with IIM and 15.7% in the group without IIM), while Ro52 was the most common antibody in both groups (34.3% in the IIM group and 18.6% in the non-IIM group) (Table 1 – Supplemental Material 1).
Performance of EULAR/ACR criteria vs. clinical diagnosisDue to the small number of patients with muscle biopsies and the heterogeneous data collected from them, we only performed the classification model without biopsies. Using the clinical diagnosis of IIM as the gold standard, the EULAR/ACR criteria showed higher specificity and at least similar sensitivity to the original cohort (85% sensitivity and 92% specificity). The LK+ values found (10.63) indicate the correct association of the criteria score to classify patients with the disease, and with an LK-(0.16) below 0.1, the criteria distinguish patients who do not have the disease well (Table 2 – Supplemental Material 2).
Addition of the extended myositis panel to the original criteriaWe found no significant differences in the AUC for the definitive classification of IIM among the three models (Table 3 – Supplemental Material 3).
When we classified patients into definite, probable, possible, and no IIM categories by adding the extended myopathy panel, 6 of the 10 patients initially classified as “no IIM” changed their classification to “Probable IIM” and 4 to “Definite IIM”; of the 16 patients classified as “probable IIM,” 15 changed their classification to “Definite IIM” (Table 4 – Supplemental Material 4).
DiscussionThe present study was developed to assess the applicability of the 2017 EULAR/ACR criteria to the Mexican population for the classification of patients with IIM. This is relevant because the original cohort of Latinos only represented 5.2% of the patients.5 In our cohort, we found a sensitivity similar to that of the original cohort, but with higher specificity (85 and 92%, respectively). With a sensitivity of 62% and specificity of 97% to classify patients with definitive IIM, our cohort confirms that the new criteria should be applied in the classification of Mexican patients.
Since these patients have a higher prevalence of comorbidities such as diabetes mellitus, hypothyroidism, and chronic steroid consumption,12 which may contribute to clinical myopathies, our cohort had more rheumatologic diseases than the original cohort (74.28% vs. 36%).5 This raises the effectiveness of the criteria for discerning inflammatory myopathy and inflammatory myopathy mimickers among complex patients with comorbidities and underlying autoimmune diseases.
Due to the low prevalence of anti-Jo-1 antibodies in Mexican patients,8 we evaluated the impact of a single antibody as a criterion. Therefore, we developed three models to assess AUC: one in which the Jo-1 variable was not included, another with Jo-1 corresponding to the original study, and one with any antibody from the extended myositis panel. We found practically no difference in the AUC between the model without antibodies and the model with antibodies. Although not statistically significant, a larger area was observed with the addition of any antibody. These differences were more evident when the categories of possible, probable, and definite were changed. While the criteria perform well for patient classification in the clinical setting, the addition of other antibodies may help improve the classification of patients with definitive disease when dealing with clinical trials with stricter selection criteria. The addition of these antibodies increased the sensitivity of the criteria by 28% without significantly decreasing the specificity of identifying patients with definitive IIM (90% sensitivity and 87% specificity using all antibodies vs. 62% sensitivity with 97% specificity using Jo-1 alone). However, in our study, we did not weigh the various types of associated or specific antibodies, as they are found in different proportions in patients with IIM and are responsible for certain clinical features 13 that may not be considered within the original criteria.
In other studies where myopathy-specific and myopathy-associated antibodies were performed in patients classified according to the 2017 criteria, those with positive myositis-specific antibodies were more likely to have a definitive classification than those without. Patients with myopathy-associated antibodies did not appear to be associated with a definitive diagnosis.14
One of the most important limitations of our study was that we only developed a model without biopsy because of the absence of muscle biopsies and the heterogeneity of the data in the reports of those performed. In our patients, biopsies were performed only in cases with a broad differential diagnosis, a limitation that was also found in a Japanese cohort.6 This practice seems to be becoming more common in the real world, which could give more importance to the consideration of the rest of the IIM antibodies in classification. As mentioned by Riddell et al., diagnosis by clinicians can be made by clinical variables and antibodies, leaving the use of biopsy only for cases with negative antibodies or positive antibodies in which the characteristics of the biopsies can change the treatment.15
Another weakness of the study is that, as controls, we included only patients from the rheumatology office whose rheumatologist had considered performing a myopathy panel as a diagnostic approach. Consequently, the number of patients with differential neurological diagnoses was low. A certain number of patients have started their diagnostic approach with suspicion of inflammatory myopathy due to pulmonary involvement. Some studies have evaluated patients in whom a myositis panel was requested for purely pulmonary involvement and found high antibody positivity rates. In some cases, the diagnosis changed from idiopathic pulmonary disease to IIM because of antibody positivity.16–18 Nevertheless, the diagnosis and classification of patients with pulmonary involvement and other signs not included within the criteria, such as arthritis, mechanic's hands, Raynaud's syndrome, or typical muscular or dermatologic involvement, cannot be classified.19
Among other limitations, the retrospective nature of the study meant that we did not have control over variables, or the number of biopsies performed. In addition, poor documentation of variables results in missing and unmeasurable bias.
Our study had several strengths, such as being the first Mexican cohort to report the validity of the classification criteria in this population, including patients with complete myositis panels, and complete clinical variables for both cases and controls.
ConclusionsAs conclusion, the 2017 EULAR/ACR criteria for IIM classification are sensitive and specific for classifying patients with IIM in the Mexican population. Due to the variability in the prevalence of IIM antibodies in different geographic distributions and the low performance of biopsies in the real world, the addition of antibodies different from Jo1 in the classification criteria can be useful in clinical trials and other research studies in this area.
FundingThis research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Conflict of interestThe authors declare that there are no financial or non-financial interests directly or indirectly related to this work; they have full control of all primary data and agree to allow the journal to review their data upon request.
The following are the supplementary data to this article:
