Abstract
Background
Effectiveness of computer-assisted joint replacement (CA-TJR) compared to conventional TJR has been evaluated by a large body of literature. Systematic reviews provide a powerful, widely accepted, evidence-based approach to synthesize the evidence and derive conclusions, yet the strength of these conclusions is dependent on the quality of the review. Multiple systematic reviews compared CA-TJR and conventional TJR with conflicting results. We aimed to assess the quality of these reviews.
Methods
We searched MEDLINE, EMBASE, the Cochrane, and Epistemonikos to identify SRs published through May 2017. Full-text articles that met inclusion criteria were retrieved and assessed independently by two reviewers. Evidence was qualitatively synthesized and summarized. Outcome measures were categorized into functional, radiological, and patient safety related. The corrected covered area (CCA) was calculated to assess the degree of overlap between SRs in analyzing the same primary studies. The AMSTAR 2, a valid and reliable tool, was applied to rate the confidence in the results of the SRs (Shea et al., 2017). AMSTAR-2 has 16 domains, of which 7 are critical (e.g., justification for excluding individual studies) and 9 are non-critical (e.g., not reporting conflict of interest for individual studies). Reviews are rated as high (no critical or non-critical flaws), moderate (only non-critical flaws), low (1 critical flaw) and critically low (more than one critical flaw). Disagreement between the 2 reviewers was resolved by discussion with the senior author to achieve consensus. We reported the quality ratings of these studies and the frequency of critical and non-critical flaws.
Results
Of 384 citations originally identified, 37 systematic reviews were included. Meta-analyses that addressed TKA showed discrepancy on functional (e.g. KSS), radiological (mechanical axis malalignment), and patient safety (e.g. adverse events) outcomes. Meta-analyses that addressed THA showed more consistent results. Moderate overlap was observed among TKA SRs (CCA=7%) and very high overlap among THA SRs (CCA=26%). Based on the AMSTAR 2 tool, 35 studies were rated critically low and two studies were rated low. Low rating was due to failure in: developing a review protocol (94.6%); using a comprehensive search strategy (56.8%), providing a list of excluded studies (89%); accounting for risk of bias in the primary RCTs (44%), accounting for the risk of bias of the primary studies when discussing the results (70%), performing appropriate statistical methods (53% for RCTs and 88% for non-RCTs), and adequately investigating publication bias (53%).
Conclusions
Given the very low confidence in the results of the SRs comparing CA to conventional total joint arthroplasty, clinicians should interpret the results of these SRs with caution. High methodological quality SRs are needed to inform evidence-based clinical practice.
