Accuracy of an adherence score and time in therapeutic range among patients on chronic use of vitamin K antagonists

To validate and analyze the accuracy of the Simonetti adherence score among patients on chronic use of vitamin K antagonists and their time in therapeutic range (TTR) of the international normalized ratio as a measure of quality of anticoagulation. A prospective cohort study with a nonrandomized intervention in patients from an anticoagulation center of a public hospital. Baseline data were collected from May to September 2017, and follow-up data were obtained eight months after a nurse-led educational intervention, which was given to all patients after consent form and after applying the adherence score (N=205). The intervention was undertaken through 30-40 min conversation about relevant factors that had been previously identified in the score derivation study, which comprised drug-drug interactions, inadequate OAC use, comorbidities, effects of food on vitamin K absorption, and invasive procedures. A receiver operating characteristic (ROC) curve was applied to validate the adherence score in terms of prediction of INR out of recommended therapeutic range. At baseline, mean adherence score was 44.69 and standard deviation (SD) was 18.37, and mean TTR was 41.07 (SD 15.40). Patients were reassessed after 8 months. At follow-up, there were significant improvements in mean adherence score 54.28 (SD 13.13), and in mean TTR 50.99 (SD 26.10). The Simonetti adherence score yielded high performance and accuracy in clinical practice among patients on chronic use of vitamin K antagonists. Our data indicate that nurse led educational intervention yielded favorable impact in terms of adherence score and TTR improvements.


Introduction
The guidelines of the European (2016), American (2014), and Brazilian (2016) Cardiology Societies recommend the use of oral anticoagulants (OACs) for patients with atrial fibrillation (AF) and with a CHA₂DS₂-VASc score ≥1 in men and ≥2 in women (level 1A evidence) regarding new OACs (January CT et al., 2014;Kirchhof P et al., 2016;Magalhães LP et al., 2016). Vitamin K antagonists (VKA) are used for stroke prevention in patients with AF and have been graded as level 1B evidence in maintaining an adjustable dosing for an international normalized ratio (INR) target between 2.0 and 3.0, and a time in therapeutic range (TTR) ˃70%. Warfarin currently remains the most frequently prescribed OAC (January CT et al., 2014;Kirchhof P et al., 2016;Magalhães LP et al., 2016). Furthermore, the TTR has been used as a measure of quality of warfarin therapy and usually is calculated as the number of days within target range divided by the total number of days in the observation period. However, if the INR value is on the therapeutic target and time between checks is long, the TTR will be falsely low (Kirchhof et al., 2016;January et al., 2014;Magalhães et al., 2016).
Recent studies have reported that, in clinical trials comparing the use of new OAC with warfarin, the TTR levels varied from 55% to 64%, which indicated that more than one out of three patients who used warfarin were either poorly or excessively anticoagulated. However, the authors have also reported results from a study conducted in Sweden that showed a mean TTR of 73.4% for patients managed in clinical practice (Mandrola, 2016).
The Simonetti score is a new clinical tool that can be used to improve patient adherence to and quality of OAC therapy. The applicability and accuracy of a clinically validated adherence score for OAC users is likely to enable the maintenance of the INR target value within a recommended therapeutic range . Therefore, we hypothesized that a nurse-led specific educational intervention delivered to VKA users, identified by the Simonetti score as having low level of adherence/poorer INR and TTR values would improve these measures of OAC quality. This educational intervention was Research, Society andDevelopment, v. 10, n. 8, e59510816523, 2021 (CC BY 4.0) | ISSN 2525-3409 | DOI: http://dx.doi.org/10.33448/rsd-v10i8.16523 3 given to all patients after consent form and after the use of the adherence score. This was undertaken through 30-40 min conversation about relevant factors that were identified in the score derivation study, which comprised drug-drug interactions, inadequate use, health problems, food interaction on vitamin K absortion, and invasive procedures (Simonetti, 2019).
Current cardiology guidelines recommend the use of oral anticoagulants (OAC) for patients presenting with at least one risk factor for stroke or systemic embolism. Many differing factors interfere with anticoagulant therapy, and it has been reported that understanding drug treatment and patient adherence are both essential to promote an effective anticoagulant therapy (Simonetti, 2014). These findings highlight the importance of implementing clinical and educational strategies to optimize patients' understanding and behavior in relation to pharmacotherapy.
Concerning nonadherence to treatment, intervening factors may adversely affect INR control, e.g., drug and diet interactions, inadequate drug use, complications and comorbidities, physical activities, alcohol consumption, stress and significant weight variations, as well as socioeconomic, physiological, psychological, emotional, and cognitive conditions (Simonetti, 2014). Low adherence to a proposed treatment has a negative effect on all aspects of healthcare, including serious complications and an increase in negative outcomes of chronic diseases, in addition to excessive costs and underutilization of available treatment resources (Simonetti, 2014). The incidence of thromboembolic events has been found to progressively increase along with an increase in elderly population numbers, accompanied with comorbidities and prior medical conditions (Edelmuth et al., 2018).
With an increase of the elderly population and in length of hospital stays, interventions to promote health and address the burden on the public healthcare system and reduce length of hospital stays, costs, and mortality due to stroke have been less effective. Therefore, there is a need for better nursing and program improvement (Edelmuth et al., 2018).
To develop an adherence score for OAC users, statistical methods recommended for logistic regression models were applied with adequate adjustment. The rationale and a detailed description of the methods used to build the Simonetti adherence score have been previously published . Validation of a new instrument that allows for the INR to be maintained within the recommended range and improves therapeutic adherence to OAC is likely to generate positive patient perspectives concerning clinical management, with higher safety and optimized quality control of the anticoagulation level (Neto Almeida et al., 2016).
In the clinical practice, nurses face challenges when aiming to improve adherence of patients on chronic OAC therapy, such as psychosocial factors, economic restraints, family environment, educational level, among others including unmeasured factors inherent to clinical practice. Adherence depends on a patient's awareness and response to the relevant guidelines as well as on follow-up undertaken by nurse, among other factors, particularly regarding issues arising when INR values fall outside of the therapeutic range.
To validate and analyze the accuracy of an adherence score among patients on chronic use of VKA and their TTR of the INR as a measure of quality of anticoagulation.

Design
This was a prospective cohort study with a nonrandomized intervention plan, including patients on chronic OAC therapy. It was conducted at an outpatient anticoagulation center of a public hospital of the Health Department of the State of São Paulo, Brazil (Pereira, 2018). Research, Society and Development, v. 10, n. 8, e59510816523, 2021 (CC BY 4.

Participants
For clinical validation of the adherence score, the study sample consisted of OAC users who attended daily appointments at the outpatient anticoagulation center. Currently, approximately 5,000 patients are active in the follow-up system, with a multidisciplinary team caring for approximately 300 patients daily. In this setting, nursing care also includes health education and pedagogical strategies to minimize the risk of thromboembolic events and major bleeding that may occur during chronic use of VKA Marevan® (Warfarin) or Marcoumar® (Femprocumone).
Patient enrollment occurred in accordance with the eligibility criteria, namely, being literate OAC users aged ≥ 18 years, who had taken VKA for at least 3 months, and all clinical indications for anticoagulation. Exclusion criteria were only cognitive impairment and those not willing to participate. All participants signed an informed consent form. Screening was performed during 01 week of daily work at the anticoagulation clinic. Potentially eligible patients received information regarding enrolment into the study. Overall, 359 were screened, and 205 provided informed consent form (Figure 1). The calculation for the sample size was based on a moderate AUC ROC of 0.7 (Shiffman & Low, 2020), a two-tailed alpha = 0.05, and a power level= 0.95. Therefore, the minimum estimated sample size was 198 patients.

Data collection
For the clinical validation, 205 patients were recruited and the first data collection was performed from May to September 2017. The minimum time interval from first evaluation to conducting the second data collection was 8 months.
After patient screening and explanations to inform them of the study purpose, those willing to participate signed an informed consent form. Those patients who consented to participate in the study completed a semi-structured form that comprised information concerning sociodemographic variables. The adherence score was derived from intervening factors likely to lead to an INR, and we referred to a study that presented a blinded assessment of a patient's condition to identify whether or not the INR value was inadequate.
Independent predictors of INR values outside the recommended range are as follows: inadequate OAC use, drug and food interaction, invasive procedure, clinical conditions, and other factors such as cigarette smoking, stress, weight gain or loss, and start or interruption of physical activity (Simonetti, Faro & Bianchi 2019).
The agreement level between nurses assessing patient adherence was determined by the adherence score estimates in 25 patients, as defined by Pearson correlation test.
An initial TTR (baseline) was calculated according to the Roosendaal linear interpolation method with the use of a time interval of the last 12 months. All patients received educational intervention, which was implemented according to the identification of factors associated with adherence to treatment and maintenance of INR in the recommended therapeutic range.
Thus, the guidelines and actions aimed at providing health education to those patients were based on pedagogical strategies (Pelegrino et al., 2014), such as the following: simulating the proper use of the medication, stable dietary pattern without significant variations in the amount of vitamin K, assessment of drug-drug interactions between OAC and other therapies, health problems that may interfere, increase or decrease body weight, exacerbated stress, chronic alcohol or tobacco use, and any interference with liver physiology.
These guidelines and educational actions were carried out individually with each patient for approximately 30 minutes. Eight months after the intervention, the adherence score and the TTR were recalculated to assess the association between adherence score variation and post-educational interventions results.

Ethical considerations
This study was approved by the Local Ethics Committee of the Dante Pazzanese Institute of Cardiology under the protocol 4729 and CAAE number: 63877517.5.0000.5462. Informed consent forms were obtained from all participants or their families as per current national ethics requirements in clinical research.

Data analysis
Quantitative variables are presented as mean and standard deviation, while qualitative variables are presented as absolute frequency and percentage. The Spearman correlation was used to assess the relationship between quantitative variables. Moreover, to assess if there was a difference before and after the educational intervention, either a paired Student's t-test or a paired Wilcoxon test was used, depending on normality. Significance was set at p<0.05 for all statistical tests We applied statistical software IBM SPSS Statistics (Version 19) and R Core (Version 3.6.1) for all the analyses.

Validity, reliability and rigour
To validate the score, an area under the receiver operation curve (AUC ROC) was calculated to identify patients with an altered INR. To assess concordance between two professionals, Lin's concordance correlation coefficient (CCC) and bias correction factor (C.b) were calculated to measure precision and accuracy (Figure 3). A Bland-Altman graph was used to evaluate errors among the health professionals concerning the magnitude of the measures (Figure 4).  Research, Society andDevelopment, v. 10, n. 8, e59510816523, 2021 (CC BY 4.0) | ISSN 2525-3409 | DOI: http://dx.doi.org/10.33448/rsd-v10i8.16523 7 was noteworthy: patients with mechanic valve prostheses (51.2%), other cardiovascular conditions (48.8%, such as atrial fibrillation/atrial flutter, pulmonary and venous thromboembolism, acute arterial occlusion, intraventricular thrombus, and hypertension). The area under the ROC curve between the Simonetti score and the reference (recommended INR) was 0.84 (95% CI 0.78-0.89, p<0.001), as shown in (Figure 2). Thus, there are scores with good discriminative power for the adequate INR. To measure the reproducibility of the score between validators, a Lin's CCC of 0.99 was used. This provided good concordance as to both precision and accuracy (Figure 3). The C.b was approximately 1, indicating very high accuracy. The line represents perfect agreement. The results outlined in Figure 4 show there was no relationship between the measures in terms of the differences between the evaluators and the magnitude of the score, that is, any error among the evaluators were not affected due to the magnitude of the score. There was good concordance concerning precision and accuracy between the two evaluators, regardless of patient adherence.

The relationship between the TTR and Simonetti's score
The correlation between Simonetti's score and the initial TTR was 0.183 (p=0.009), indicating that patients with greater adherence to the score had a better TTR. The correlation after 8 months was 0.250 (p≤0.001), corroborating the initial TTR and strengthening the score's effect on TTR improvement.

The effect of education on the improvement of the TTR and on Simonetti's score
In patients with a baseline TTR <65%, the mean initial score was 44.69 and the standard deviation (SD) 18.37, while the mean initial TTR was 41.07 (SD 15.40). After 8 months, patients were reassessed. Mean score at follow-up was 54.28 (SD 13.13), (p<0.001), and mean TTR at follow-up was 50.99 (SD 26.10), (p<0.001). These results indicate that educational intervention had an effect on improving the adherence score and the TTR.
This study evaluated the discriminative power of an adherence score for OAC users. We found that the area under the ROC (AUROC) was 0.84 compared to the AUROC in Simonetti's adherence score (Simonetti, Faro & Bianchi, 2018). Our study showed similar adequate results (Simonetti, Faro & Bianchi, 2018), indicating an adherence tool performed sufficiently well to determine the likelihood of the outcome of interest. A cut-off point for the score was required; however, this study did not aim to replace the INR by the score. Nonetheless, information derived from the adherence score and the TTR enabled the nurses to intervene to achieve target INR values.
Adequate overall performance was observed in the nurses' ability to discriminate in the score development sample.
Factors associated with intervening events, in terms of OAC user adherence, have been reported to determine changes to the INR, enabling patients to remain within a recommended therapeutic range (Simonetti, Faro & Bianchi, 2019). Patient education has been suggested in several studies as a means to improve clinical outcomes, such as greater adherence, better INR control within the therapeutic range, better understanding of signs and symptoms, and a significant reduction of rehospitalizations and costs (Pelegrino et al., 2014;Simonetti, Faro & Bianchi, 2019). These factors influenced adherence to treatment identified according to the score. Nurses involved in decision-making focused on OAC user education have observed an improvement with regards to adequate use of medication at long-term follow-up (Agnelo, Alexandra & Matias, 2014;Colet, et al., 2016;Simonetti, Faro & Bianchi, 2019).
In terms of agreement on the applicability of the tool among nurses who measured the reproducibility of the score, good concordance on precision and accuracy was found. The adherence score was applicable and may likely affect the population evaluated, enabling behavioral changes and improving adherence to OAC treatment. In this scenario, nurses and other health professionals managing patients who are taking OACs are considered highly important to the educational process.
Studies have shown that health education has a significant effect on knowledge and greatly influences changes in patients' behavior. This effect has been shown to facilitate maintenance of the therapeutic range (recommended INR) within this population on OAC (Colet, Amador & Heineck, 2017;Cabellos-García, et al., 2018).
Adherence to medication generates self-care behavior among patients concerning treatment for chronic diseases.
Moreover, precise and accurate adherence measurement tools can help health professionals to provide better patient care.
These tools have been shown to facilitate proactive conditions for patients to acquire knowledge and improve their behavior in terms of proper use of medication and following guideline-based recommendations (Figueirêdo et al., 2018).
Our study found evidence of improvement in adherence when comparing the initial TTR value with the subsequent value post-intervention. This may have been due to the patient education given by nurses who were better informed of factors that could influence the INR out of the recommended therapeutic range and, consequently, maintain the appropriate target INR (Costa et al., 2016). Active methods of patient education have been found to be a better alternative to healthcare assistance for this population, and may be linked to better learning, and positively influence adherence and patient care (Simonetti, Massa & França, 2018;Coelho et al., 2021).
Moreover, the need to improve knowledge and provide continuous educational intervention measures have been reported to have enabled OAC users to recognize the need to modify their behavior sufficient to remain within a specified therapeutic range (Groia et al., 2015). In this study, we observed that an educational intervention that focused on the identification of influencing factors, based on an adherence score, significantly led to improvements in the adherence score and the TTR.
Simonetti's adherence score tool was validated in terms of its precision and accuracy within a population sample recruited from an oral anticoagulation center. The TTR is a sequence of INR values used to calculate the percentage of time that a patient remains within an appropriate therapeutic range. Due to this factor, certain limitations apply to the analyses. For example, the adherence score should not be linked to a single TTR per se. Furthermore, using data from a single center (rather than a multicenter study) may not necessarily reflect other settings, populations, socioeconomics and healthcare systems aspects, thus requires further exploration on broader levels, i.e., future research is needed to assess national and international generalizability. Therefore, future multicenter studies are needed to further evaluate the adherence score in relation to nurse-led patient education intervention concerning the use of OACs.

Conclusion
Our results demonstrate that Simonetti's adherence score is a valid tool with adequate performance and accuracy in measuring patient adherence to VKA and good correlation with INR and TTR, especially when used by nurses in the setting of anticoagulation clinics.
It is expected that the Simonetti score will contribute as a research tool among patients using oral anticoagulants, providing maintenance within the therapeutic range and adequate adherence to medication. Patient education provided by nurses is essential to promote adherence and prevent adverse events related to vitamin K antagonist use. The score applied as a nurse led intervention was able to promote patient behavioral change and improved adherence to anticoagulant treatment, especially in maintaining the TTR of the recommended international normalized ratio.