Registries, the ever increasing evidence gap, and what to do with that open talus

There are some things that registries can’t tell us. How to treat a talus fracture is one of them. This statement is heresy in many circles in these days of the National Inpatient Sample, National Joint Registry, HES data and accountability, but a statement I believe to be true. It’s not just the little questions, like the talus, that registries can’t answer but some quite big ones as well.

The issue with registries is that they are by their very nature observational studies, big ones, but observational studies nonetheless. Inherently biased (with patients allocated to treatments based on expert opinion), they do however serve an essential function for monitoring outcomes and can provide excellent prognostic and outcome data. We seem, however, to be being goaded, cajoled and incentivised into a more and more registry obsessed scientific community. Macro studies of tens or hundreds of thousands of patients are alluring in their data size, but the data resolution is, by its very nature, poor. This doesn’t mean inferences cannot be drawn from the data, but without knowing the ‘signal:noise’ it is difficult. Sabah et al.¹ have, to a certain extent, attempted to quantify the problem by turning this on its head. In what may very well prove to be one of the most important papers this year for patients with known outcome (where the implants ended up in the London Implant Retrieval Centre), they set out to establish the quality of data supplied by surgeons to the NJR. The results are staggering. Only 61% of procedures could be linked to a known NJR outcome, and of these, the NJR dataset had an incorrect outcome 16.6% of the time. In other areas of science with high signal:noise ratios, complex adjustments are made before inferences are drawn, allowing for ever more accurate conclusions to be drawn from the same data.² This does not happen in orthopaedic studies for the most part. The problems with registry data are two fold – it is observational, and the accuracy of the dataset is unclear. Essential for safety, and helpful for outcome studies, it nevertheless cannot be completely relied upon.

So what does this mean for orthopaedics? With a worrying spotlight shone on the ever-increasing evidence gap, Lohmander et al.³ describe the evidence for much of orthopaedics as “scandalously poor in parts”. Understandable given the range and rarity of some of the conditions within the specialty. There are no randomised controlled trials to support my decision making in the treatment of the talus fracture, indeed there never will be. Talus fractures are rare and devastating injuries too diverse to study in a controlled trial. Lohmander and colleagues reveal their own lack of understanding of the difficulties in informing practice in unusual conditions (making up large parts of much orthopaedic and trauma surgery) in their editorial, but they do highlight a significant problem. Where evidence does exist, it is often ignored. They cite examples of ACL reconstruction, knee arthroscopy and vertebroplasty, all still widely practiced but with large question marks over their efficacy following randomised sham controlled efficacy trials.

Where does this leave the modern orthopaedic surgeon? Stuck between big (but potentially unreliable) data, a multiplicity of conflicting, randomised controlled (but often ignored) trials and conditions all but impossible to study. In this month’s edition, two feature articles give a mature look at hip arthroplasty and attempt to answer some of the outstanding questions: How much innovation is required? What are the challenges for the future? Both provide an outstanding synthesis of registry, trial and study data. This type of comprehensive data synthesis, including but not limited to, registry data is becoming more and more essential in evidence-based practice.

This edition is somewhat of a landmark, with 360 now reaching surgeons throughout the UK and New Zealand as membership benefits of their national societies. I hope you will all continue to enjoy what is an innovative concept and interesting read.