Telemedicine in rheumatology: high specificity and sensitivity of follow-up virtual video consultations during COVID-19 pandemic

Abstract

Objective

To evaluate the reliability of virtual video-assisted visits, added to the tight-control strategy for inflammatory rheumatic diseases (IRDs), in identifying patients who need treatment adjustment.

Methods

Tightly followed-up adult patients with RA, PsA, AS or SLE took part in a video consultation during COVID19 lockdown and repeated the same rheumatology evaluations through a face-to-face visit within 2 weeks. The sensitivity and specificity of the virtual visits for treatment decisions (categorized as: unchanged, adjusted/escalated, tapered/discontinued, need for further examinations), and the intraclass correlation coefficient (ICC) for virtually measured disease activity and patient-reported outcomes (PROs) were calculated with 95% CIs using face-to-face visits as the reference method.

Results

In 89 out of 106 patients (84.0%), face-to-face visits confirmed the remotely delivered treatment decision. Video-visiting showed excellent sensitivity (94.1% with 95% CI: 71.3%, 99.9%) and specificity (96.7%; 95% CI: 90.8%, 99.3%) in identifying the need for treatment adjustment due to inadequate disease control. The major driver for the low sensitivity of virtual video consultation (55.6%; 95% CI: 21.2%, 86.3%) in identifying the need for treatment tapering was SLE diagnosis [odds ratio (OR) 10.0; 95% CI: 3.1, 32.3; P <0.001], mostly because of discordance with face-to-face consultation in glucocorticoid tapering. Remotely evaluated PROs showed high reliability (ICC range 0.80–0.95), while disease activity measures had less consistent data (ICC range 0.50–0.95), especially for those diseases requiring more extensive physical examination, such as in SLE and PsA.

Conclusion

Video-visiting proved to have high reliability in identifying the need for treatment adjustment and might support the IRDs standard tight-control strategy.

Introduction

Tightly monitoring and adjusting treatment according to disease activity, safety concerns, and comorbidities, in a shared decision-making process, is pivotal for improving outcomes of inflammatory rheumatic diseases (IRDs) [1–4]. Face-to-face consultation represents the standard approach in rheumatology, but telemedicine applications have gained a growing role in recent years and boomed during COVID-19 pandemics, allowing rheumatologists to continue caring for patients remotely [5, 6]. However, virtual video-assisted visits are still far from widespread, their application to rheumatology has been only addressed by only a small number of studies, and research into their reliability compared with in-person visits is lacking [6]. Two recent systemic literature reviews independently came to the same conclusion that insufficient information on outcomes was available about the effectiveness of virtual visits for IRDs [7, 8].

The present study aimed to evaluate the reliability of virtual video-assisted follow-up visits in detecting the need for adjusting treatment (due to inadequate disease control) in patients affected by IRDs routinely followed up in a tight-control clinical setting.

Patients and methods

Study population

During the first SARS-CoV-2 Italian outbreak, from 9 March 2020 to 9 June 2020, our tertiary referral outpatient rheumatology clinic was converted into a telemedicine service due to the lockdown imposed in an attempt to halt the viral spreading. Adult patients, routinely followed up in our tight-control clinics for RA, PsA, AS and SLE, attended a virtual video-assisted consultation during the last 2 weeks of lockdown and later completed the rheumatology evaluation through a face-to-face visit within 2 weeks. Four consultant rheumatologists (A.F., M.C., E.C. and I.C.) delivered both the virtual consultations with the patients and the face-to-face visits to the same patients already entrusted to their care. Both rheumatologists and patients had no previous experience with virtual consultations. The Ethics Committee of the Azienda Ospedaliero Universitaria of Cagliari approved the present study (protocol no. 8557), and all patients provided written informed consent to participation in the study. The remotely delivered consultations were performed using a freely available web-based video-conferencing platform supported by smartphone and desktop.

Covariates and criteria for defining active disease

Data on demographic details, disease activity measures, patient-reported outcomes (PROs), laboratory results, and ongoing medications were collected during the video-assisted and face-to-face consultations (Table 1). During video-visiting, patients were coached by physicians through self-assessment of swollen and tender joint counts (SJCs and TJCs), applying a prespecified standardized procedure to increase reliability [9]. Patients were instructed on how to recognize inflamed swollen and tender joints and how to assess them for softness, elasticity and pain; afterward, they were asked to perform a physician-driven examination of their joints, showing their joints on video and reporting whether they were painful, swollen or tender. In the case of cutaneous rash and other inspectable signs, patients were asked to show them on video for physician assessment. Laboratory results were collected before (e.g. via email) or during virtual visits (shared by the patient through the video) and taken into account for both the telemedicine and the face-to-face evaluations. The criteria for defining active disease with high impact were set as a 28-joint DAS (DAS28) of >3.2 and RA impact of disease (RAID) of >4 for RA, Disease Activity index for PsA (DAPSA) of >14 and PsA Impact of Disease 12-item (PSAID 12) of >4 for PsA, BASDAI ≥ 4 and ASDAS of ≥2.1 for AS, clinical SLEDAI developed by the SELENA trials (SELENA-SLEDAI) of >0 (excluding serologic abnormalities) and lupus impact tracker (LIT) of >10 for SLE. However, the individually based treatment decision was not exclusively grounded on criteria for disease activity and PROs but also took into account patient history (e.g. duration of disease, amount of established organ damage, previous treatment), as well as safety concerns, comorbidities, and the patient’s expectations.

Outcome of interest

The treatment decision at the end of the each visit was categorized as being: (i) unchanged (i.e. having enough information collected to make a decision; continue with the same treatment); (ii) adjusted treatment for inadequate disease control (i.e. having enough information; new prescription or increasing dosage of any DMARDs, glucocorticoids or NSAIDs—the latter only for PsA and AS); (iii) treatment tapering/cessation for persistently adequate disease control (i.e. having enough information; increase time-intervals for administration, reduce the dosage, or discontinue any DMARDs or glucocorticoids); or (iv) needing further examinations (i.e. not having enough information; requiring a physical examination, new blood tests or imaging to better assess disease activity; or safety issues—excluding screening for biologics). Adverse events were also recorded. The sensitivity and specificity of virtual consultations in identifying the need for adjusting treatment due to inadequate disease activity control was the primary outcome of the study; other treatment decisions were secondary outcomes. Patients were not involved in the audit planning and design but were asked to assess the impact of and their satisfaction with the remotely delivered consultations by completing an anonymous, self-administered, web-based, 5-point Likert scale questionnaire.

Statistics

The sensitivity and specificity of virtual consultations were calculated using face-to-face visits as the reference method. A forward logistic regression model was built to identify factors independently associated with a modification of the remotely delivered treatment decision during the face-to-face consultation. Age, gender, education level, working status, diagnosis, disease duration, ongoing treatment, treatment duration, prednisone use, and examining physician were included in the model. The odds ratio (OR) with 95% CI was calculated. A P-value of <0.05 was considered significant. The intraclass correlation coefficient (ICC) single measure was used in subgroup analysis for each IRD to assess the reliability (ICC < 0.5 poor, 0.5–0.75 moderate, 0.75–0.90 good, >0.90 excellent) of disease activity measures and PROs performed during video-visiting and face-to-face consultations.

Results

Overall, 106 (25 RA, 30 PsA, 22 AS, 29 SLE) out of 120 patients who performed a virtual video-assisted consultation between 25 May and 9 June 2020 agreed to participate. The demographics and treatment information are reported in Table 2. The high prevalence of patients treated with biologic and targeted synthetic DMARDs, and with prednisone in the case of SLE, is related to their referral to our tertiary-level tight-control clinics, which serves those patients that need frequent monitoring for active disease, evaluation of their response to treatment changes, or supervision of steroid-tapering.

Face-to-face visits confirmed the remotely delivered treatment decisions in 89 out of the 106 patients (84.0%) (Fig. 1). Virtual consultations showed 94.1% (95% CI: 71.3%, 99.9%) sensitivity and 96.7% specificity (95% CI: 90.8%, 99.3%) in detecting the need for adjusting treatment due to inadequate disease activity control, using face-to-face visits as the gold standard (Supplementary Table S1, available at Rheumatology online). Excellent sensitivity (91.9%; 95% CI: 78.1%, 98.3%) and specificity (92.8%; 95% CI: 83.9%, 97.6%) were found also for the detection of patients who did not need treatment modification. Sensitivity dropped to 55.6% (95% CI: 21.2%, 86.3%) for treatment tapering/cessation and to 36.4% (95% CI: 10.9%, 66.2%) for the need for further examinations, while specificity maintained excellent values (93.8% with 95% CI: 87.0%, 97.7% and 95.8% with 95% CI: 89.6%, 98.8%, respectively) (Table 3). Drug adverse events were recorded during 3 virtual visits (i.e. site injection pain, headache, vertigo) and confirmed during standard consultations; no adverse events occurred between visits.

Having SLE (OR 10.0; 95% CI: 3.1, 32.3; P < 0.001) was independently associated with receiving adiscordant treatment decision during the face-to-face consultation, mostly because of different decision between virtual visit and standard visit in terms of treatment tapering (Supplementary Table S2, available at Rheumatology online). Difference in treatment decision between visits was mainly related to the low accuracy of the virtual approach in identifying clinical signs that may be revealed only through physical examination and in discriminating FM from active SLE.

Disease activity measures showed heterogeneous results in terms of the agreement between virtual and face-to-face consultations, ranging from moderate to excellent reliability (Table 4). PROs measured at the time of virtual visits and face-to-face consultations showed good to excellent agreement (Table 4).

Overall, 101 patients answered the questionnaire, reporting a high level of satisfaction with the telemedicine service, the virtual consultation, and the established patient–doctor relationship (Fig. 2).

Discussion

This study demonstrated that virtual video-assisted rheumatology consultations have very high sensitivity and specificity in identifying the need for adjusting treatment in patients affected with IRDs routinely followed up in a tight-control outpatient clinical setting. Such results provide the evidence that has been missing to date on the effectiveness of video-visiting, when applied in support of the standard approach, to increase the number of follow-up rheumatology consultations and support tight monitoring for patients with IRDs [10], while limiting the number of hospital visits, thus protecting spread of infection [11]. Further advantages of telerheumatology include reducing travel time, stress related to the visit, and costs. Moreover, we confirmed the generally reported high levels of acceptance and satisfaction with telemedicine [12, 13].

Along with these observations, we provided novel evidence on the high reliability of PROs in remotely delivered consultations, but less consistent data on disease activity measures, especially those requiring more extensive physical examination such as in SLE and PsA. Rheumatologists strongly rely on physical examination to get a better evaluation of disease activity measures and the patient’s general state of health [13]. The vast majority of patients in our cohort (78%) agreed that is important to get a physical examination at the rheumatology clinic. Considering the low accuracy of some disease activity measures when virtually assessed, it could be misleading to rely exclusively on them to remotely make treatment decisions. Nevertheless, integrating patients’ opinions with physician-driven joint counts self-assessment was effective in overcoming telemedicine barriers to physical examination, resulting in proper shared decisions for patients with IRDs. In this cohort, the only noteworthy exception to the excellent reliability of video-visiting was represented by SLE, for which virtual visits have shown moderate sensitivity in identifying those patients eligible for tapering of a daily prednisone dose. Considering the impact of prednisone on damage accrual [14], the extent to which video-visiting can be implemented in patients with SLE under tight monitoring will have to be further investigated in larger studies.

This study has some limitations. The major limitation is that both virtual and face-to-face visits were performed by the same observer, which could have led to a potential confirmation bias, so we suggest caution in interpreting the study results. However, changing the observer between visits would have introduced an additional confounding factor (i.e. interobserver reliability) and hampered the direct comparison between video-assisted and face-to-face visits Therefore, more than one physician was enrolled as an independent observer in an effort to minimize the potential effect of the confirmation bias by accordingly adjusting the regression model. The small IRD subgroups prevented an in-depth disease-specific analysis, which is another study limitation.

In conclusion, video-visiting should not replace the standard approach for consultations but might be effectively used in support of the tight-control strategy to increase the number of consultations while reducing outpatient visits, thus recognizing any need for adjusting treatment in routinely followed up patients affected with IRDs. Although our findings cannot be generalized to different contexts, and especially to first visits, this evidence could be extremely useful in setting up telerheumatology clinics during COVID19 outbreaks, when local measures of social distancing are in effect, or even beyond pandemics in cases where a patient cannot attend a face-to-face visit or whenever the capacity of rheumatology services is severely reduced.

Funding: No specific funding was received from any bodies in the public, commercial or not-for-profit sectors to carry out the work described in this article.

Disclosure statement: The authors have declared no conflicts of interest.

Data availability statement

The data underlying this article will be shared at reasonable request to the corresponding author.

Supplementary data

Supplementary data are available at Rheumatology online.

References