As Douglas Adams famously said, the answer to life, the universe and everything is “42”. (Question of course as yet unknown). I was taught as a trainee and a young academic that the perennial problem in conducting research is asking the right question – a point Adams makes very eloquently. If you don’t ask the correct question you clearly will not get a reasonable answer – “garbage in – garbage out”. Over the last few years the quality of orthopaedic research in terms of methodology has skyrocketed. Randomised controlled trials abound and there doesn’t seem to be an edition of 360 that goes by without a number of excellent randomised controlled trials to comment upon.
Controversially I have become however increasingly concerned about the RCT. Whilst methodologically better at addressing biases there is a risk in the orthopaedic community that we are missing the point and putting methodology above substance. We have moved so far from the days of ‘notes to friends’ describing in elegant English the treatment of a few cases (anyone with any time should read Sir Astley Coopers 1822 masterpiece ‘A treatise on dislocations and fractures’; essentially a series of case descriptions) with complex statistics and involvement of methodologists, health economists and the all-important ‘qualitative’ arm of the research, I can’t help wondering if the pendulum has swung too far in the wrong direction.
Research questions should of course include a measure of uncertainty, and for a randomised trial to work, the questions should be asked where the treating teams are genuinely in equipoise. However with much funding for health sciences research based around health economics, questions being posed are based more around the funder’s framework than that of the clinicians. This all too often results in research questions that are not based on patient focus, but on costs of healthcare delivery.
Much of what we do in Orthopaedics is life-enhancing surgery, but not necessarily at a large enough impact to provide a quality of life benefit in a pragmatic study. Does this make it any less valuable? No. If orthopaedics and trauma is to survive, smaller studies need to improve their methodology answering questions that are more patient-focussed, using outcome measures that reflect patients’ quality of life. There is no perceptible impact of Dupytren’s surgery on hand function using the QuickDASH or quality of life scores in some studies. Does this mean it is valueless? Patients and surgeons would say not. Perhaps then we are asking the wrong questions; or the right questions in the wrong manner. By random chance 50% of completed randomised controlled trials should provide a positive answer, however the chances of doing so appear dramatically affected by the outcome measure chosen.
There is an urgent need for validated and appropriate outcome scores that are either disease- and limb-specific, or have been validated with a known MCID. There is a danger that with so many intervention studies with ‘negative’ health economic outcomes for established treatments, and the majority of other studies establishing that many new treatments do not offer improved disease-specific outcomes, that the evidence base for orthopaedics is looking rather shaky. There are very few RCTs in orthopaedics demonstrating a significant and clinically relevant treatment effect. With the lack of validated outcome scores in many domains the use of general quality of life scores such as the EQ5D and SF36 offers the tantalising option of a ‘one size fits all’ outcome measure which not only provides an easy route to undertake quality of life analysis, but also health economic outcomes. The question we need to ask ourselves as a surgical and scientific community is, are we happy to accept outcomes that are up to and including affecting overall quality of life as equivalent? If we aren’t we need to generate, validate and implement scores, pilot observations and definitive studies to evaluate clinically relevant outcome measures – after all whilst a patient with Dupytren’s will tell you his quality of life is impacted upon by loss of hand function, sadly the same patient’s global quality of life scores do not.
The issue here is not the methodology, but high-quality methodology alone does not substitute from asking the right question in the right way.
