Rheumatoid factor (RF) testing is used in primary care in the diagnosis of rheumatoid arthritis (RA); however a positive RF may occur without RA. Incorrect use of RF testing may lead to increased costs and delayed diagnoses. The aim was to assess the performance of RF as a test for RA and to estimate the costs associated with its use in a primary care setting.
Material and methodsA retrospective cohort study using the Information System for the Development of Research in Primary Care database (contains primary care records and laboratory results of >80% of the Catalonian population, Spain). Participants were patients ≥18 years with ≥1 RF test performed between 01/01/2006 and 31/12/2011, without a pre-existing diagnosis of RA. Outcome measures were an incident diagnosis of RA within 1 year of testing, and the cost of testing per case of RA.
Results495,434/4,796,498 (10.3%) patients were tested at least once. 107,362 (21.7%) of those tested were sero-positive of which 2768 (2.6%) were diagnosed with RA within 1 year as were 1141/388,072 (0.3%) sero-negative participants. The sensitivity of RF was 70.8% (95% CI 69.4–72.2), specificity 78.7% (78.6–78.8), and positive and negative predictive values 2.6% (2.5–2.7) and 99.7% (99.6–99.7) respectively. Approximately €3,963,472 was spent, with a cost of €1432 per true positive case.
ConclusionsAlthough 10% of patients were tested for RF, most did not have RA. Limiting testing to patients with a higher pre-test probability would significantly reduce the cost of testing.
El factor reumatoide (FR) se usa en atención primaria para el diagnóstico de la artritis reumatoide (AR); sin embargo, un FR positivo puede observarse en sujetos sin AR, y su uso inapropiado puede conllevar costes y retraso diagnóstico. En este contexto, estudiamos la utilidad y costes del FR como test diagnóstico de la AR en atención primaria.
MétodosEstudio de cohortes retrospectivas basadas en datos de historia clínica informatizada de >80% de la población de Cataluña (SIDIAP). Se incluyeron sujetos de edad ≥18 años y con ≥1 medida de FR entre el 1/1/2006 y el 31/12/2011, sin diagnóstico previo de AR. El diagnóstico incidente de AR durante el año posterior a la medida de FR, y el coste por caso de AR fueron las medidas de interés.
Resultados495.434/4.796.498 (10,3%) pacientes tuvieron al menos una medida de FR 107.362 (21,7%) de estos fueron sero-positivos, de los cuales solo 2.768 (2,6%) fueron diagnosticados de AR en el año siguiente, comparado a 1.141/388.072 (0,3%) diagnósticos en sero-negativos. La sensibilidad del FR fue del 70,8% (IC 95%: 69,4 a 72,2%), especificidad 78,7% (78,6 a 78,8%), y valor predictivo positivo y negativo 2,6% (2,5 a 2,7%) y 99,7% (99,6 a 99,7%), respectivamente. El coste total estimado fue de 3.963,472€, alrededor de 1.432€ por caso de AR diagnosticado.
ConclusionesAunque el 10% de participantes (casi medio millón de personas) fueron sujetos de medición/es de FR, la mayoría no desarrollaron AR. El uso de FR en pacientes con mayor probabilidad pre-test reduciría de forma significativa su coste.
Early treatment of rheumatoid arthritis (RA) prevents disability1,2 and requires prompt diagnosis and rapid referral. Rheumatoid factor (RF) testing is used to support diagnosis despite the poor positive predictive value (PPV).3 In one study, only 11–20% of those with musculoskeletal symptoms and a positive RF had RA; conversely, only 60–70% of those with RA were RF positive.4
The pre-test probability of RA is likely to be higher5 in rheumatology clinics than in primary care, where the predictive value of RF is lower.3 NICE (National Institute for Health and Clinical Excellence) guidelines advise prompt secondary care referral of patients with suspected inflammatory joint disease based on clinical symptoms only.6 We have shown that 67% of laboratory RF requests made to a UK laboratory came from primary care of which only 5.8% were positive. A positive RF did not lead to a diagnosis of RA in any patient in whom clinical suspicion of RA had not been previously documented.7 If RF testing is used by GPs to screen low-risk patients, the value is diminished and costs increase. However, there is little evidence regarding the use of RF testing in primary care from a health economic perspective.
The aims of this study were to identify the frequency of RF testing in a primary care setting in Spain, to estimate the sensitivity and specificity of RF as a test for RA, to determine the proportion of patients tested who had an eventual diagnosis of RA and to explore the costs of testing RF in this setting.
Material and methodsSource of data and study participantsData was obtained from the Information System for the Development of Research in Primary Care (SIDIAP8). This contains the primary care clinical records of over five million people (more than 80% of the population aged ≥14 years) in Catalonia, Spain. GPs use ICD-10 codes to record clinical diagnoses. Each patient is assigned a unique identifier for confidentiality and data protection. The data is highly representative of the population of Catalonia,9 and has been used to study the epidemiology of musculoskeletal conditions such as osteoporotic fractures10 and osteoarthritis.11 In addition, SIDIAP contains complete information on blood test results performed in primary care. Pharmacy invoicing data is available for all subsidized medications dispensed in community pharmacies. Medicines data is classified using the World Health Organization Anatomical Therapeutic Chemical classification/Defined Daily Dose (ATC/DDD) index.
The study protocol was approved by the SIDIAP ethics review. We included all participants registered in the SIDIAP Database aged ≥18 years that were tested for RF between 01/01/2006 and 31/12/2011 and followed them up until the end of 2012. We considered that testing contributed to the diagnosis of RA only if the time between testing and subsequent diagnosis was less than 1 year.
We excluded participants with a diagnosis of RA at the start of the study period, and those diagnosed during the study period that had not been tested for RF during the previous year. We identified the results for all RF tests performed in primary care, and used the result nearest to a subsequent diagnosis of RA where patients were tested more than once. We used the laboratory upper limit of normal to define a positive RF result (≥10IU/mL). RF testing was performed using a latex-enhanced immunoturbidimetric assay.
Identification of incident RA casesWe identified incident RA cases between 01/01/2006 and 31/12/2012, using ICD-10 codes M05 and M06 as registered in primary care records. We used a modified algorithm previously developed in the UK Clinical Practice Research Datalink (CPRD) database to confirm a diagnosis of RA in patients with Read codes for RA.12 This excluded patients with no prescription records for Disease Modifying Anti-Rheumatic Drugs (DMARDs) including azathioprine, ciclosporin, gold, hydroxychloroquine, leflunomide, methotrexate, mycophenolate, penicillamine, and sulfasalazine; or who had a subsequent alternative diagnosis (ankylosing spondylitis, dermato-polymyositis, fibromyalgia, gout, osteoarthritis, psoriatic arthritis, reactive arthritis, scleroderma, septic arthritis, systemic lupus erythematosus and other spondylo-arthropathies). This algorithm was subsequently validated for use in SIDIAP.13
Statistical analysesWe used a receiver operating characteristic (ROC) curve for RF against an incident diagnosis of RA made in the following year and estimated the area under the curve (95% confidence intervals) to explore the sensitivity and specificity. We identified the best “theoretical threshold” as the cut-off with the highest value of the total of sensitivity plus specificity.
We calculated the sensitivity, specificity, likelihood ratios, and positive/negative predictive values for both the pre-specified threshold and best theoretical threshold for RF. We estimated age and gender-adjusted odds ratios (OR) for a diagnosis of RA in the year following testing using the Mantel-Haeszel test.
Finally, we calculated the total cost of RF testing during the study period and the cost per true positive and negative case.
All statistical analyses were performed using Stataversion 12.0 SE for Mac.
ResultsOut of a population of 4,796,498, 10.3% (495,434) patients were tested for RF between 2006 and 2011. Of these, 4912 (1.0%) had an incident diagnosis of RA between 01/01/2006 and 31/12/2012. Table 1 shows the baseline characteristics for tested patients with and without a diagnosis of RA in the year following RF testing.
Baseline characteristics for patients undergoing RF testing.
| RA casesn=3909 | Not RA casesn=491,525 | |
|---|---|---|
| Age in years | 56.5 (16.1SD) | 53.7 (17.2SD) |
| Females | 2747 (70.3%) | 333,570 (67.9%) |
| Body Mass Index kg/m2 (SD) | 27.9 (5.0SD)840 (21.5%) missing data | 28.1 (5.3SD)113,981 (23.2%) missing data |
| Current smoker | 692 (25.0%)1140 (29.2%) missing data | 85,723 (25.5%)154,983 (31.5%) missing data |
| Sero-positive (RF≥10IU/mL) | 2768 (70.8%) | 104,594 (21.3%) |
| Prescription for DMARDsa | 2842 (72.7%) | 13,616 (2.8%) |
| Prescription for systemic corticosteroidsa | 3015 (77.1%) | 124,024 (25.2%) |
Of the 495,434 patients tested, 107,362 (21.7%) were sero-positive but only 2768 (2.6%) of these were diagnosed with RA in the following year. Out of 388,072 sero-negative participants, 1141 (0.3%) were diagnosed with RA in the following year (Fig. 1).
The area under the ROC curve was 79.9% [95% CI 79.0–80.8%]. The best theoretical cut-off value for RF was 13.4IU/mL. Using this value increased the specificity from 78.7% [95% CI 78.6–78.8%] to 88.7% [95% CI 88.6–88.8%] but lowered sensitivity from 70.8% [95% CI 69.4–72.2%] to 65.4% [95% CI 63.9–66.9%]. RF had a low PPV of 2.6% [95% CI 2.5–2.7%] using the original cut-off, or 4.4% [95% CI 4.2–4.6%] using the theoretical cut-off.
Age and gender-adjusted OR for a diagnosis of RA in the year following RF testing were 8.57 [95% CI 7.92–9.27] for RF ≥10IU/mL, and 13.83 [95% CI 12.75–15.00] for the theoretical threshold of 13.4IU/mL.
An estimated €3,963,472 was spent on RF testing during the 6-year study period; an average expenditure of over €660,000 per annum. €22,144 (0.6%) was spent testing true positive cases. 179 tests were performed for each true positive case giving a cost of €1432 per case.
DiscussionWe investigated the use of RF in primary care in Spain using a large database with almost complete recording of laboratory requests. Approximately 10% of the source population was tested between 2006 and 201114 despite the incidence of RA being only 8.3/100,000 in Spain.15 This high frequency of testing suggests that RF is requested in patients with a low pre-test probability of RA, in keeping with previous work in primary care.7 Only 2.6% of sero-positive patients were diagnosed with RA in the following year. The recommended cut off for RF was relatively low and we found that the best theoretical cut-off was 13.4IU/mL.
The high frequency of testing and the low PPV for RF resulted in a very high cost of testing which we estimated to be €1432 per true positive RA diagnosis. This value is based on RF testing alone and does not include additional costs such as referral.
Strengths and limitationsThe main strength is the comprehensive dataset, which includes all blood tests and results in primary care, and all primary care prescribing data. The study relies on the correct use of ICD-10 codes by GPs, which may be inaccurate. There was no access to clinical data to confirm ACR/EULAR classification criteria. However, we used a previously validated algorithm to select the most probable RA cases.12,13 We used a relatively low cut-off for defining a positive RF however the PPV remained low despite using the calculated best theoretical cut-off of 13.4IU/mL.
Comparison with existing literatureThe sensitivity and specificity obtained for RF were in keeping with previous studies.16,17 We considered that a positive RF only contributed to the diagnosis of RA if it occurred within the year prior to diagnosis; this approach may exclude earlier RF tests that were relevant to diagnosis. However, calculating sensitivity and specificity based on a longer period between test and diagnosis (5 years) did not substantially change the results (data available on request). Only 955 patients were tested for anti-cyclic citrullinated peptide and so sensitivity and specificity was not calculated for this test due to the inherent selection bias of this group.
Implications for practicePrimary care services in Spain have state-funded GPs acting as gate-keepers to secondary care.18 For the best use of limited resources, it is important to use tests appropriately. Less than 1% of patients tested had a subsequent diagnosis of RA suggesting that GPs may be using RF as a screening tool. This will increase the overall cost of testing.7 In this 6 year study period almost €4 million were spent and the cost of testing per true positive case was €1432. In comparison, the estimated cost of a GP consultation in the Catalan healthcare system is €40, a hand radiograph €9, and a first rheumatology appointment €101 to €143.19 Limiting RF testing to patients with a higher pre-test probability would significantly reduce the cost of testing.
Inappropriate testing also increases the laboratory workload and may lead to extra consultations to discuss results. Patient care is also impacted; if GPs use the results of RF testing to support referral decisions, there may be a delay for patients with clinical evidence of RA but who have a negative RF result, even though early diagnosis and treatment of RA leads to better outcomes20,21 and keeps patients in work.22
Potential solutions include criteria restricted primary care access to RF testing, issuing guidelines or reminder aids for when RF testing should be requested with local audit and feedback, and primary care educational interventions such as outreach by rheumatologists and peer facilitated workshops. The literature so far is unclear on which of these is the best approach; a review of research on the effectiveness of interventions to improve laboratory test ordering in primary care found low levels of evidence with poor quality of studies.23
In conclusion, RF testing is inefficient and costly when used in primary care patients with a low risk of inflammatory arthritis.
FundingDPA is funded by NIHR through an NIHR Clinician Scientist award. Partial funding was provided by SIDIAP Database (Idiap Jordi Gol) for data extraction and data management and partial support was received from the Oxford NIHR Biomedical Research Centre (BRC). The research was independently conducted.
ContributorsAM, RL and DPA designed the study. DPA and RPV analyzed the data. AM, KM, DPA, RL and RPV interpreted the data, and drafted the article. All the co-authors revised the manuscript for important intellectual content, and approved the final version. RL is guarantor.
Ethical approvalThe study protocol was approved by the SIDIAP ethics review.
Conflict of interestAll authors have completed the Unified Competing Interest form atwww.icmje.org/coi_disclosure.pdf (available on request from the corresponding author). AM, KM, RPV, CE, KJ, MM, SC, FF have no disclosures. RL reports non-financial support and other from GSK, personal fees from Roche, personal fees from Janssen, other from Nordic, other from Chemocentryx, personal fees from UCB, outside the submitted work; CC reports personal fees from consultancy, lecture fees and honoraria from AMGEN, GSK, Alliance for Better Bone Health, MSD, Eli Lilly, Pfizer, Novartis, Servier, Merck, Medtronic and Roche, outside the submitted work; DPA reports grants from BIOIBERICA, grants from AMGEN SPAIN, outside the submitted work; NA reports personal fees from FLEXION (PharmaNet), personal fees from Lily, personal fees from Merck, personal fees from Q-Med, personal fees from Roche, personal fees from Smith & Nephew, grants from NOVARTIS, grants from PFIZER, grants from Schering-Plough, grants from Servier, personal fees from AMGEN, personal fees from GSK, personal fees from NiCox, personal fees from Smith & Nephew, outside the submitted work.
