102 – THE DYNAMICS OF OUTCOME ASSESSMENT: AN ANALYSIS OF THE SPRINT TIBIAL FRACTURE TRIAL

Abstract

Purpose: The purpose of this study was to evaluate how outcome assessment committees of various sizes, and the biases and personalities of its members, potentially impact a trial’s results.

Method: We conducted a retrospective analysis of the available individual and consensus data from an adjudication committee in a multinational trial (the SPRINT trial) of fracture fixation alternatives. The trial committee members included six members (5 surgeons, 1 methodologist) who independently determined the outcome of reoperation, and any discordant cases were discussed in the committee until a consensus was achieved. We described the pattern of agreement among adjudicators, modeled the adjudication process, and predicted the results if a smaller committee had been used. We also tested for adjudicator biases based upon their preferences for reamed or unreamed intramedullary nails, the presence of a potentially dominant adjudicator, and evaluated the resource implications of reducing the size of an adjudication committee.

Results: Overall, committee member agreement was moderate (Kappa Free=0.6). We found that reducing the number of adjudicators from six to three would have changed the consensus outcome in less than 15% of cases. Regardless of committee size, per-patient analyses also demonstrated very little change in the final study results across all fracture types or in the open fracture subgroup. Results from the original SPRINT adjudication indicated a significant decrease in the rate of reoperations associated with reamed intramedullary nailing among patients with closed fractures (relative risk 0.65; 95% confidence interval 0.46 to 0.93; p=0.02). Under the model, in committee sizes of three or less persons, these estimates of treatment effect were no longer significant. There was a significant difference between adjudicators with respect to the number of times their independent decision was in the minority but nevertheless became the final consensus decision (p=0.046), suggesting a dominant adjudicator was present in the committee. There were large predicted savings in cost and time with a reduced committee size.

Conclusion: In this study, smaller committees (i.e., four or five rather than six adjudicators) would likely have produced similar results, substantially reducing costs of research.