Interpretation of randomised controlled trials: perhaps caution required?

In the short space of just a few years the evidence base for orthopaedic surgery has been transformed. Seminal papers such as Schatzker’s tibial plateau paper¹ and Gustillo & Anderson’s seminal work on open fractures,² simply would not get published in todays enlightened world of evidence based orthopaedics. Yet we still rely on these papers in our day to day practice. They were exploratory papers, exploring new themes, treatments and ideas.

Randomised controlled trials are the biggest studies now performed, often evaluating thousands of patients randomised to one treatment or another. The methodology has been borrowed from medical doctors where life is simple. The patient is randomised to one drug or another (often unkindly referred to as ‘red pill/blue pill’ studies). These trials are easy to conceptualise, with everything controlled for (intervention length, diagnosis, treatment setting and intervention), making the comparison easy. Understand the inclusion and exclusion criteria and you understand the study. Randomised controlled trials in medicine have yielded some very important information.

The setting is completely different in surgery or orthopaedics. The interventions are complex. If testing a comparison of two different implants, it is not just the implant being tested, there is the way in which the surgery is performed, the anaesthetic type, the rehabilitation strategy, even if clips or sutures are used is known to affect outcomes. These are complex interventions and as such more difficult to study. Starting with the Canadians and now also the Brits, there have been a rash of trauma studies significantly improving our knowledge by taking a pragmatic approach. Instead of trying to control for all the variables, taking a pragmatic approach assumes these factors will even out if the study is large enough, thus allowing effective research into complex interventions. In recent months we have seen the publication of the DRAFFT study³ and also the UK Heel Fracture study (UKHeFT).⁴ Two large randomised controlled trials. Two studies with negative answers.

The abstracts for both of these studies follow the format of the journal, in this case the BMJ. Both are pragmatic studies comparing two accepted interventions. Neither adds an intervention or treatment to our armamentarium. Both were designed carefully as ‘health technology appraisals’ funded by the UK government. Both say there is no difference.

This message has been variably interpreted, and in common orthopaedic conversation in the theatre coffee room (a realistic measure of current opinion), this seems to have settled on the idea that there is no place for fixation of calcaneal fractures, and that K-wires are likely to be as good as plates for all distal radial fractures. This of course is not what either study suggests. Both studies were carefully designed to look at patients where the optimal treatment is not clear. In the case of DRAFFT, the results only apply to patients where the fracture is within 3 cm of the joint line, a closed reduction could be achieved and the surgeon felt open reduction was not required. In these circumstances the DRAFFT investigators found the variety of fixation did not matter and that with no differences in the reported health economics, the investigators were in favour of percutaneous Kirschner wire fixation. A very different message to that given in coffee room chatter.

The UKHeFT reported that there were no differences in functional outcome measures between operatively and non-operatively managed calcaneal fractures, but a higher risk of complications in the operative group. This study was set up to evaluate depressed intra-articular fractures of the calcaneus. On the face of it, not an attractive intervention. The devil as they say is in the detail. The study team designed a trial in which there is equipoise. They excluded patients with fibular impingement, lateral wall blow out or severe hindfoot deformities. Their findings therefore apply only to patients who do not have any of these – the most common indications (in my hands at least), for intervention.

The coffee room reader could be forgiven in the case of the UKHeFT study for the misinterpretation as the exclusion criteria are hidden in the main article, not in the abstract. This does, however, underline the importance of reading the whole article for significant papers and ensuring that the message that is put across is the correct one. To me the message of the DRAFFT study is that in fractures reducible by closed means, k-wires are as good as plating. The UKHeFT study, however, has not changed my practice, very few surgeons with experience would operate for subtalar depression alone.

Randomised controlled trials are not the ‘be-all and end-all’ of orthopaedic research. They are a valuable tool, and it is a great step forwards to be able to perform these studies in a pragmatic way. However, we must all be careful to understand the inclusion and exclusion criteria as well as the inherent strengths and weaknesses of study design. In the process of striving for evidence based medicine, we must ensure that we do not stop innovating like our predecessors, and that we understand the evidence we are trying to apply to our patients.