Compliance of RCTs in endodontic instrumentation with the CONSORT Statement: A systematic review

This systematic review aimed to evaluate the adherence to the CONSORT Statement in clinical studies comparing different instrumentation techniques in Endodontics. Methodology: A systematic search was carried out through Medline databases. There was no restriction on publication year or idiom. Solely randomized controlled trials (RCTs) were included. The articles were evaluated in compliance with CONSORT. Descriptive analyses of the number of studies by journal, country, and quality assessments were performed with RoB 2.0 Cochrane Collaboration’s tool for assessing risk of bias in RCTs. A total of 6779 studies were identified, and only 33 meet the eligibility criteria. In relation to the overall risk of bias, eight out of 33 were classified as at ‘low’ risk of bias. The overall CONSORT score for the included studies in this review was 22 ± 6.2 points. Significant differences among countries were observed (p = 0.03). The range of years had a significant influence on the average CONSORT score (p = 0.01). The adherence of RCTs of endodontic instrumentation to the CONSORT Statement requires improvements. Adherence to the CONSORT statement will also reduce the high or some concerns risk of bias of studies in the field. CRD42020160552


Introduction
Root canal treatment aims to restore a tooth and its underlying tissues to normal condition and function (Mohammadi et al. 2017). It is performed in several interdependent and continuous stages, in which the step of biomechanical preparation of the root canal should clean and remodel the endodontic canal system before obturation procedures (Chugal et al. 2017).
Until the 1960s, endodontic instrumentation was made by serial or conventional technique, using stainless steel instruments in an increasing order of diameter, maintaining the same working length. This protocol presents a higher risk of accidents, especially in curved and overlapped channels (Tortini et al. 2007). Over time, instruments with greater flexibility were developed. They were recommended to be used throughout the working length, while more calibrated instruments, would be used at lower working lengths. This was the rationale being the step-back technique, which significantly reduced accidents when compared to the conventional technique (Kfir et al. 2003).
In 1988, Marshall and Pappin described the Oregon technique, based on the concept of crown-down instrumentation, that is, the preparation is initiated by the cervical third of the canal, followed by the middle third and finally the third apical, where the files would work without producing pressure and minimizing the debris' extrusion to the apical foramen (Marshall & Pappin 1988).
In order to decrease the working time required for root canal shaping, the employment of nickel-titanium rotary instruments was proposed. These instruments were intended to replace the Gates Glidden drills and hand-files previously used since they present greater flexibility and, consequently, more safety when in use, due to the lower risk of fracture of the instrument. They are instruments used in the enlargement movement in the cervical-apical direction (Del Fabbro et al. 2018).
Reciprocating instrumentation emerged after rotary instrumentation, to use a single device produced for single use only. These new systems made the preparation faster, reduced cyclic fatigue of the instruments, and reduced cross-contamination due to their single-use (Jardine et al. 2016).
Various clinical trials have compared different endodontic shaping techniques (Arslan et al. 2016, Comparin et al. 2017, Gathani et al. 2016, Jain et al. 2016, Nekoofar et al. 2015Neves et al. 2016, Pasqualini et al. 2016, and they led to most of the conclusions clinicians rely on to select the instrumentation system for endodontic treatment. Although randomized Research, Society andDevelopment, v. 10, n. 8, e41910817350, 2021 (CC BY 4.0) | ISSN 2525-3409 | DOI: http://dx.doi.org/10.33448/rsd-v10i8.17350 3 controlled trials (RCTs) are considered the study design capable of minimizing most of the bias that threatens the quality of the evidence, they only produce reliable evidence if well delineated with good reporting. Incomplete or inadequate reporting of information on the planning and conduction of the study impairs the identification of possible methodological flaws and makes it difficult for the interested parties to use their conclusions since they cannot critically evaluate its clinical applicability (Altman et al. 2001, Schriger & Altman 2006. Frequent mistakes in the report of RCTs encouraged a group of editors and researchers to formulate the Consolidated Standards of Reporting Trials Statement (CONSORT) (Altman et al. 2001). This statement assists authors in the reporting process of RCTs in a way to allow readers for a critical interpretation of the study results. Additionally, it prevents the omission of possible systematic errors that would compromise the validity and reliability of the results and, consequently, their applicability within the context of evidence-based dentistry.
Given the importance of RCTs in endodontic instrumentation to make decisions regarding protocols, and commercial brand of instruments, the aim of this study was to systematically review the literature in peer-reviewed journals to evaluate a) the compliance of RCTs with the CONSORT Statement and b) the risk of bias in these RCT studies through the Cochrane Collaboration risk of bias tool.

Methodology
This is a systematic review of methodology of randomized clinical trials (RCTs) studies (Pereira et al. 2018).
Methodology described here follow the PRISMA Statement and was based on previous systematic reviews of methodology (Loguercio et al. 2017, Reis et al. 2018).

Protocol and registration
The study was registered at the protocol at the PROSPERO database (CRD42020160552).

Eligibility criteria
This systematic review included only RCTs published in peer-review journals that compared different endodontic instrumentation techniques in adults.

Information sources
A search was made in the scientific literature in July 2018, through the Pubmed database (primary search, Table 1), which was transferred and adapted to the other databases (Scopus, Web of Science, LILACS, BBO and Cochrane), without restriction of date or language of publication.

Study selection
The search strategy was based on the concepts of participant and intervention in which the participant comprised patients in need of endodontic treatment, and the intervention involved terms that included rotary instrumentation, at least in one of the groups of comparisons. Initially, the search strategy was elaborated for PubMed. For this purpose, Mesh terms and keywords were used, combined with the Boolean operator OR within each concept, and the concepts were combined with the Boolean operator AND. Then, this search strategy was adapted to the other databases described earlier.
After running the search strategy, a reference management program was used (EndNote X6, Clarivate Analytics, Philadelphia, PA, USA) to store the files of all databases. Then, the duplicate articles were removed using a software tool, and this was also followed by manual removal after the organization of titles in alphabetical order. The relevance of the articles was screened by titles, abstracts, and finally, with the complete full-text of the remaining articles. The full-texts were obtained and read to check if they met the inclusion and exclusion criteria. All these steps were performed independently by two authors (J.L.G. and L.M.W.), who reached a consensus whenever disagreements occurred.

Data collection process
Two authors collected data related to the study design, study setting, number of participants per group, age of participants, percentage of male, endodontic instruments used, instrumentation protocol, anesthetic technique, anesthetic salt and amount used, tooth endodontic treatment, number of sessions, intracanal medication used, irrigation solution, obturation technique and endodontic cement, sample losses and evaluated outcomes were collected by two authors (J.L.G. and L.M.W.).
If the study evaluated postoperative pain, the criteria for pain evaluation, and the assessment times were collected.

Risk of bias in individual studies
The Cochrane Collaboration's recommended tool for assessing risk of bias (RoB) is neither a scale nor a checklist. It is a domain-based evaluation, in which critical assessments are made separately for different domains. The RoB of the Research, Society andDevelopment, v. 10, n. 8, e41910817350, 2021 (CC BY 4.0) | ISSN 2525-3409 | DOI: http://dx.doi.org/10.33448/rsd-v10i8.17350 individual studies was evaluated by two independent evaluators (J.L.G. and L.M.W.) using the RoB version 2.0, a revised tool to assess the risk of bias in RCTs. This tool evaluated the following items: bias arising from the randomization process, bias due to deviations from intended interventions, bias due to missing outcome data, bias in measurement of the outcome, bias in selection of the reported result, and overall bias.
Risk of bias judgments followed from answers to signaling questions: 'low risk of bias', 'some concerns' or 'high risk of bias'. Overall risk of bias judgment: the study was considered to be at 'low risk of bias' if it was at 'low risk' for all domains, the study was considered to be at 'some concerns' if it was judged to be at 'some concerns' in at least one domain and the study was considered to be at 'high risk of bias' if it was at 'high risk of bias' in at least one domain.
During data selection and quality assessment, any disagreements between the reviewers were solved through discussion, and if needed, by consulting a third reviewer (A.R.).

Adherence to CONSORT statement
An evaluation tool based on the items related to the methods and results from the 2010 CONSORT Statement (Schulz et al. 2010) was used to evaluate the reporting completeness of RCTs (Table 2). This evaluation tool was adapted from previous publications to match the research question of this investigation (Loguercio et al. 2017, Reis et al. 2018. The items related to the title and abstract, introduction, and discussion were not included since their evaluation is very subjective, and the adherence to these items would not weaken the quality of the study report or the risk of bias of the studies.
A total of 12 items of the CONSORT Statement were included in this CONSORT evaluation tool. As some of these items were subdivided, a total of 16 items was evaluated. The given score per item ranged from 0 to 2. In general words, 0 = no description, 1 = poor description and 2 = adequate description. More details regarding the scoring process for each score of each item are displayed in Table 2. Each item was given an equal weight (Loguercio et al. 2017, Reis et al. 2018. Two reviewers (J.L.G. and L.M.W.) performed the round of scoring using the CONSORT evaluation tool as a guide (Table 2). In case of disagreement, a discussion followed, and the consensus was used to determine the final score. Evaluators were not blinded to the authors of the studies. Unfortunately, this blinding would not be feasible, as reviewers were familiar with the studies and could easily guess the researchers' affiliation by reading the paper. The interventions for each group are described with sufficient details to allow replication, including how they were actually administered.
There is no description.

Poor [1]
There is missing information that prevents the replication of the interventions/comparators.

Positive [2]
At least the primary outcomes were defined in details, including how and when they were assessed.
Consider it as clear when the details are clear, but the authors did not use the term "primary outcome" or related synonyms.

Negative [0]
There is no definition of the primary outcome and/or secondary outcomes.

Poor [1]
1. The authors only report they have used specific criteria without detailing the most important outcomes of such criteria. 2. The description of the primary outcome and/or secondary outcomes is very superficial and does not allow replication of the method.

Positive [2]
Method of sample size calculation is described in a way to allow replication. It should identify the primary outcome for each sample size calculated. Elements of the sample size calculation are (1) the estimated outcomes in each group (which implies the clinically important target difference between the intervention groups); (2) the α (type I) error level; (3) the statistical power (or the β (type II) error level); and (4), for continuous outcomes, the standard deviation of the measurements should be reported. For equivalence trials, the equivalence limit, instead of the effect size should be reported.

Negative [0]
There is no description in the article.

Poor [1]
The sample size is described but some parameters are missing so that it prevents replication.

Randomization
Sequence generation Positive [2] 1. Clear description of the random sequence generation. 2. or clear description of a non-random sequence method.

Negative [0]
There is no information in the text.

Poor [1]
The authors only provide a very superficial description (such as the "groups were randomly allocated") or do not provide sufficient information to allow replication of the randomization process.

Positive [2]
Clear description of the allocation concealment. See next columns for evaluation of the Risk of Bias.

Negative [0]
There is no information in the text.
1) The authors describe who is blinded in the study. 2. In single-blind studies (when this is clearly reported by the authors), just the description of participant or evaluator (the one blinded) is enough; however, when the study is double blind or triple blind all blinded people should be described.
2) The study describes just the participant or examiner blinded but one of these people cannot be blinded due to the intrinsic features of the study design.

Negative [0]
There is no description of the blinding.
(2) The authors describe the study was blind or double-blind but do not specify who was blinded. 1) There is not enough information to evaluate the statistical method used by the authors and/or the type of statistical tests employed by the authors are inadequate for the trial design and/or nature of the data (for instance, tests that do not take into account the paired nature of the data when this is the case).
2) The authors describe several statistical tests but do not specify each outcome they were applied to.
Estimated effect size Positive [2] Authors report at least for the primary outcome the effect size and its precision (such as 95% confidence interval). Odds ratio, risk ratio, risk difference, mean difference, etc.

Negative [0]
There is no description of the effect size and 95% confidence interval For each group, the number of participants who were randomly assigned, received intended treatment and were analyzed for the primary outcome is described in the flow chart CONSORT diagram.
The flow-chart is not presented in the article. For each group and for each outcome, the number or participants (denominator) included in the analysis is clear.
Authors do not report the numbers analyzed.

Poor [1]
There is no clear description of the number of participants (denominator) included in the analysis of at least one of the outcomes. 2. Instead of reporting the raw number of participants, the authors report their data in percentages. 3. The authors fail to report the baseline number of patients included in each analysis.

Data can be obtained indirectly in the study.
Registration and protocol The study was registered in a trial registry and the protocol number is provided.
This information is not available in the manuscript. Registration in an Ethics Committee is valid as trial registry

Poor [1]
The authors describe that the study was registered but does not provide the registration number and/or the number provided does not link to the study.

Scoring system and statistical analysis
The number of studies by journal, follow-up period, and country were analyzed descriptively. Compliance with individual items of the CONSORT Statement was analyzed to identify areas in which authors could improve the description. A chart with the percentage of studies per score in each item was provided.
To achieve an overall compliance score, the scores for the 16 items were summed in each article. A trial with Research, Society and Development, v. 10, n. 8, e41910817350, 2021 (CC BY 4.0) | ISSN 2525-3409 | DOI: http://dx.doi.org/10.33448/rsd-v10i8.17350 8 adequate descriptions (score 2) for all CONSORT items would receive a maximum score of 32 units. A mean average score was calculated by period, journal, and country. Comparison within each factor was performed with the one-way ANOVA and Tukey's test at 95%. Linear correlation analysis between 2015 ISI journal impact factor, and the average CONSORT score was also performed.

Study selection
A total of 6779 studies were identified in the databases. After removing duplicates, 4898 articles remained. With the reading of the titles, this number lowered to 872, and after abstract screening the number reduced to 39. Some articles were still excluded for some reasons: five of them were not RCTs (Bartols et al. 2016, Darda et al. 2009, Ritt et al. 2012, Sanfelice et al. 2010, Wan et al. 2017, and one did not compare different instrumentation techniques (Kfir et al. 2006). In summary, only 33 studies remained for assessment ( Figure 1).

Figure 1. Flow diagram of study identification.
Source: Authors.
Different outcomes were evaluated in the primary studies: risk, intensity and duration of pain, analgesic consumption, bacterial reduction, and success of endodontic treatment.

Study compliance with each of the CONSORT instrument tool items
The overall CONSORT score for the studies included in this review was 22.2 ± 6.2 points, which represents 69% of the maximum CONSORT score of 32 points. Figure 3 shows the percentage of studies per CONSORT score for each CONSORT item evaluated. The items classified as score 0 or 1 in most studies were: protocol, flow chart, effect size, sample size, and trial design. Random sequence had a high percentage of scores 0 and 1, while allocation concealment received mostly score 0.
The items scored as 2 in most of the studies were the following: numbers analyzed, baseline data, losses/exclusions, hypothesis testing, blinding, outcomes, interventions, settings, and eligibility. Correlation between journal impact factor and overall CONSORT score (r = 0.16; p = 0.359; Figure 4) was week and not significant, representing only 3% of the dispersion of data (R2 = 0.03, Figure 4). No influence of the journal on the average CONSORT score was observed (p = 0.36; Table 5). On the other hand, significant differences among countries were observed (p = 0.03; Table 5).
The vast majority of the RCT comparing endodontic treatments was published from 2011 to 2018 (25 out of 33). The year range had a significant influence on the average CONSORT score, with higher average CONSORT score in the more recent field of years (p = 0.01; Table 5).
The RoB of the studies had a significant influence on the average CONSORT score, with higher average CONSORT score in studies classified at 'low risk of bias' (p = 0.001; Table 5).

Discussion
In order to evaluate the adherence of the RCTs with the CONSORT Statement, a scale from 0 to 2 was used, with zero meaning absence of reporting, 1 bad report, and 2 adequate report (Reis et al. 2018). To the best of authors' knowledge, this is the first study that attempted to evaluate the adherence of endodontic instrumentation RCTs to the CONSORT Statement, which was one of the objectives of this study.
A total of 12 items of the CONSORT Statement were included in this CONSORT evaluation tool: trial design, participants, interventions, outcomes, sample size, randomization, blinding, statistical methods, participant flow, baseline data, numbers analyzed and registration and protocol of these items. Those who contributed the most to lowering the CONSORT score were random sequence and allocation concealment.
Adequate randomization balances, both known and unknown prognostic factors in the allocation of treatments. In addition to randomization, concealment of allocation is also important as it protects the randomization process so that the treatment to be allocated is not discovered before the patient has participated in the study. Proper management of these two domains minimizes selection bias (Higging et al. 2011). There is evidence that improper conduction of these steps in RCTs increases the likelihood of systematic errors (Pocock 1983); studies with poor methodological quality tend to overestimate the results, favoring the intervention in the test group (Schulz et al. 1995, Khan et al. 1996. Masking or blinding is the process of retaining information about the interventions assigned to each group out of examiners and patients, and it is a critical element in the design of RCTs. When done successfully, it prevents the introduction of information bias (Feys et al. 2014). Blinding can be applied to participants, researchers, and evaluators of study results. In clinical trials of endodontic instrumentation, masking poses a more significant challenge than for drug therapies. It is impossible to prevent the care provider, i.e., the dentist, from knowing which technique and instruments will be used for endodontic treatment. Therefore, the masking of the operator is not feasible. Evaluators can, and should, preferably, be masked on group assignment.
Compared with other areas of Dentistry or with the number of laboratory studies in Endodontics, there are still few RCTs on endodontic instrumentation. Only 33 studies were identified for inclusion in this systematic review, and only 24% (8/33) of the studies were classified at low risk of bias in the field, which shows a new need for new well-designed randomized controlled clinical studies.
Ideally, an average CONSORT score of 32 points should be obtained, meaning that the studies adhered entirely to the CONSORT statement. However, these 33 eligible studies on endodontic instrumentation had an overall average CONSORT score of 22.2 ± 6.2, which was slightly superior to that observed in bleaching studies (16.7 ± 5.4) (Loguercio et al. 2017) and studies of bonding conducted in non-carious cervical lesions (15.0 ± 4.8) (Reis et al. 2018). This superior average score in the endodontic instrumentation area is probably attributed to the fact that most studies were conducted in the most recent years when authors had a better understanding of the steps required for the conduction of and rigorous and adequate RCTs.
In addition to the overall CONSORT score, comparisons were made related to the journal, country of publication, and period of publication. These additional analyses aimed to provide information on whether improvements in the average CONSORT scores occurred over time and whether these improvements were related to the journal and its impact factor, as well as the country of origin of the first author. No influence of the journal on the average CONSORT score was observed.
However, significant differences among the publication period were observed with a higher average CONSORT score in the most recent, in agreement with earlier systematic reviews (Loguercio et al. 2017, Reis et al. 2018. One should point out the limitations of the studies included in the present study. A systematic review is a collection of the eligible studies and cannot report on findings not addressed by the primary studies. The first limitation of the eligible studies is that they reported surrogate outcomes (risk or intensity of pain, analgesic consumption, and bacterial reduction), instead of true outcomes (complete remission of the disease, absence of periapical lesion, maintenance of the tooth in the mouth or patient's preference; mean survival rate of the teeth). Surrogate endpoints might be problematic on some occasions since they are not always excellent representatives of the real clinical outcome. Unfortunately, however, the use of surrogate outcomes that try to predict the effect of specific treatment on an unobserved exact outcome have substantial economic and ethical advantages, reducing the duration and size of clinical trials (Ensor et al. 2016) they have been used very often in RCTs.
Some essential items such as study flow chart, sample size calculation, method of sample randomization, and allocation concealment should be added to the RCTs' methodology in order to improve the quality of the studies. It is important to be aware of the quality of RCTs in the field of endodontic instrumentation, as RCTs are considered the "gold standard" for treatment evaluation and serve as a reference for dentists' decision making. However, its presentation is not always sufficiently clear to allow adequate analysis of the findings and conclusions; therefore, a systematic review evaluating adherence to CONSORT and the risk of bias of RCTs evaluating endodontic instrumentation systems is relevant.
A limitation of this systematic review is that not all articles published in journals are available. To evaluate the CONSORT items, it is necessary to obtain the full text of the articles; this is the reason the gray literature was not explored.
Future studies of endodontic instrumentation should be better delineated, with complete information on the study methodology so that their risk of bias can be evaluated.

Conclusion
The adherence of RCTs that evaluate endodontic instrumentation to the CONSORT statement requires improvements.
Adherence to the CONSORT statement will also reduce the high or some concerns risk of bias of studies in the field.
More well-designed clinical studies evaluating endodontic instrumentation should be carried out in order to have more reliable results.