Abstract
Aims
A core outcome set for adult, open lower limb fracture has been established consisting of ‘Walking, gait and mobility’, ‘Being able to return to life roles’, ‘Pain or discomfort’, and ‘Quality of life’. This study aims to identify which outcome measurement instruments (OMIs) should be recommended to measure each core outcome.
Methods
A systematic review and quality assessment were conducted to identify existing instruments with evidence of good measurement properties in the open lower limb fracture population for each core outcome. Additionally, shortlisting criteria were developed to identify suitable instruments not validated in the target population. Candidate instruments were presented, discussed, and voted on at a consensus meeting of key stakeholders.
Results
The Wales Lower Limb Trauma Recovery scale was identified, demonstrating validation evidence in the target population. In addition, ten candidate OMIs met the shortlisting criteria. Six patients, eight healthcare professionals, and 11 research methodologists attended the consensus meeting. Consensus was achieved for the EuroQol five-dimension five-level questionnaire (EQ-5D-5L) and the Lower Extremity Functional Scale (LEFS) to measure ‘Quality of life’ and ‘Walking, gait and mobility’ in future research trials, audit, and clinical assessment, respectively. No instrument met consensus criteria to measure ‘Being able to return to life roles’ and ‘Pain or discomfort’. However, the EQ-5D-5L was found to demonstrate good face validity and could also be used pragmatically to measure these two outcomes, accepting limitations in sensitivity.
Conclusion
This study recommends the LEFS and EQ-5D-5L to measure the core outcome set for adult open lower limb fracture.
Cite this article: Bone Joint Res 2023;12(6):352–361.
Article focus
-
Which outcome measurement instruments (OMIs) should be used to measure core outcomes for adult patients recovering from open lower limb fracture?
-
What constitutes the UK multistakeholder consensus for a core OMI set for adults recovering from open lower limb fracture?
Key messages
-
The Core Outcomes for Open Lower Limb Fracture study recommends the EuroQol five-dimension five-level questionnaire (EQ-5D-5L) and the Lower Extremity Functional Scale (LEFS) to measure previously identified core outcomes: ‘Quality of life’ and ‘Walking, gait and mobility’, respectively.
-
The EQ-5D-5L and the LEFS are recommended for use in all future clinical research, local and national audit, and as part of routine clinical practice.
Strengths and limitations
-
This study involved the opinions and views of patients and used robust consensus methods to ensure that all stakeholder voices were heard. It was conducted in line with joint guidance from the Core Outcome Measures in Effectiveness Trials (COMET) and the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiatives.
-
This study overcame a lack of well-validated OMIs in the open lower limb fracture population by developing a novel and pragmatic approach to instrument selection. Using a panel of patients, clinicians, and experts in medical measurement, a set of well-respected OMIs with a proven track record of use in past research on patients following open lower limb fracture were identified.
-
Consensus was not reached for the recommendation of an OMI to measure the core outcomes ‘Being able to return to life roles’ and ‘Pain or discomfort’. However, the EQ-5D-5L was demonstrated to show good face validity at the consensus meeting for the no consensus core outcomes. Therefore, it could be used pragmatically to measure these two outcomes in addition to ‘Quality of life’ while accepting limitations in its sensitivity.
Introduction
Open fracture is a life-altering injury affecting 30.7 per 100,000 adults each year.1 Open lower limb fracture causes high levels of disability and reduced quality of life in the short-to-medium term,2 as well as substantial resource demands on trauma infrastructures.3-5 There is currently insufficient high-quality evidence for treatment options used to manage open lower limb fractures.6 Where high-quality studies have been conducted, results cannot be compared and pooled effectively in systematic reviews due to heterogeneity in outcomes used and how they have been measured; this hampers evidence-based clinical decision-making.6
A core outcome set (COS) is an agreed, standardized set of outcomes to be measured and reported, as a minimum, in all trials for a specific population and health condition.7 COSs do not preclude using additional outcome measurement instruments (OMIs) or specify which should be used as the primary outcome measure.8 COSs have been endorsed as a method to reduce outcome heterogeneity and outcome-reporting bias, and to promote better quality research that is patient-centred through the involvement of key stakeholders.7-11
The Core Outcomes for Open Lower Limb Fracture (CO-OLLF) study aims to create a COS for outcome domains and a core outcome measurement instrument set (COMIS) for how these domains should be measured for patients recovering from open lower limb fracture. The COS should be feasible for use in all trials of surgical, medical, and rehabilitation interventions in the setting of clinical research and routine clinical practice, e.g. local and national audits.
Following a systematic review,6 qualitative research, and a consensus process, a set of core outcomes for open lower limb fractures has been established and published separately.12 The COS defines which domains should be measured. This study aims to establish consensus among patients, healthcare professionals, and experts in medical measurement over how core outcomes should be measured.
Methods
Ethics and registration
The South-Central Research Ethics Committee (REC) granted ethical approval for this study on 1 March 2018 (REC reference: 18/SC0051, IRAS project ID: 235150). The CO-OLLF study was prospectively registered on the COMET database (688)13 and was adopted by the National Institute for Health and Care Research (NIHR) Clinical Research Network (CRN) (Protocol number: 13257).
Design
This study follows joint guidance from the Core Outcome Measures in Effectiveness Trials (COMET) and the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiatives.14 The COSMIN preferred approach for selecting OMIs consists of four steps: 1) conceptual considerations (defining the core outcomes and target population); 2) identifying existing OMIs; 3) undertaking a quality assessment of OMIs; and 4) selecting an OMI for each core outcome using a consensus procedure.
Due to a lack of OMIs demonstrating validation evidence for the assessment of measurement properties in the open lower limb fracture population, an additional pragmatic approach was used to identify suitable candidate outcome measurement instruments. See Figure 1 for a study design schematic detailing the adapted COSMIN approach.
Fig. 1
Schematic flow diagram summarizing the approach used for the selection of candidate outcome measurement instruments (OMIs) for consideration at a consensus meeting for measuring core outcomes. Adapted from Prinsen et al, 2016.14 COSMIN, COnsensus-based Standards for the selection of health Measurement INstruments; OLLF, open lower limb fracture.
Conceptual considerations
The population and health condition were defined as all skeletally mature patients (age greater than 18 years) following open lower limb fracture of any grade distal to the acetabulum.15 Health economic outcomes were considered out of scope.16 The COS defined for this population was: ‘Walking, gait and mobility’, ‘Being able to return to life roles’, ‘Pain or discomfort’, and ‘Quality of life’.12
Identifying existing outcome measurement instruments
The COSMIN initiative recommends searching for all existing OMIs with evidence of measurement properties in the target population by: 1) searching for systematic reviews of OMIs; and 2) conducting a literature search.14
A systematic review inclusive of all studies on open lower limb fracture published between January 2009 and July 2019, and a search of the COSMIN database of systematic reviews of OMIs13 on 25 May 2022 using the search term ‘fracture’, identified no studies or systematic reviews of measurement properties of OMIs validated in the open lower limb fracture population.6 Therefore, an additional systematic review was conducted in line with COSMIN guidance17 and reported with the aid of the PRISMA statement. The systematic review aimed to critically appraise, compare, and summarize the quality of measurement properties of all OMIs for each of the four core outcomes validated in the open lower limb fracture population. Four search strategies were devised with the aid of validated ‘search blocks’ for core outcomes and search filters for identifying studies on measurement properties18 and patient-reported outcome measures (PROMs).19 Search strategies are summarized in Figure 2 and detailed in Supplementary Material 1. MEDLINE and Embase databases were searched using the Ovid advanced search function (Ovid Technologies, Wolters Kluwer, USA) on 17 July 2019. All titles and abstracts were screened, and data were extracted independently by ALA and HC, using Rayyan (Qatar Computing Research Institute, Qatar) against eligibility criteria (Table I). Data were extracted into study overview templates recommended by COSMIN. Disagreements were resolved by discussion.
Fig. 2
Search strategies to identify validated outcome measurement instruments (OMIs) for each core outcome in the open lower limb fracture population. BMI,Biomedische Informatie Group; COSMIN, COnsensus-based Standards for the selection of health Measurement INstruments; PROM, patient-reported outcome measure.
Table I.
Eligibility criteria for studies evaluating the measurement properties of a patient-reported outcome measure to be included in the systematic review.
| Key area | Criteria for inclusion |
|---|---|
| Constructs | The studies should evaluate OMIs measuring the following core outcomes that constitute the constructs of interest: 1) walking, gait and mobility; 2) being able to return to life roles; 3) pain or discomfort; and 4) quality of life. |
| Population | At least 80% of the study sample should represent the open lower limb fracture target population. |
| Type of OMI | Except for studies on 'walking, gait and mobility', all studies should concern the assessment of a PROM. 'Walking, gait and mobility' can be measured using a physical performance measure subject to feasibility assessment. |
| Evaluation of measurement properties | The aim of included studies should be the evaluation of one or more measurement property as defined by COSMIN guidance14 in the development of the OMI. |
| Exclusion criteria | |
| Study type | Studies where an OMI of interest was used to measure a research outcome, e.g. a research study or systematic review not assessing the measurement properties of the OMI of interest. |
| Study language | Any study where the full text was not published in English |
-
COSMIN, COnsensus-based Standards for the selection of health Measurement INstruments; OMI, outcome measurement instrument; PROM, patient-reported outcome measure.
Feasibility screening of included outcome measurement instruments
OMIs were screened for feasibility after a full-text review by ALA and HC independently. Feasibility criteria are detailed in Table II and developed by the study management group to align with the CO-OLLF project scope.
Table II.
Feasibility criteria for outcome measurement instrument selection.
| OMI feasibility criteria |
|---|
| OMIs selected should be feasible to use in clinical effectiveness trials. As such, they should preferably be patient-reported such that they could be collected over telephone or by online questionnaire. |
| OMIs should be feasible to collect as part of routine clinical practice. For example, they should be able to be collected as part of national audit programmes, e.g. the Trauma, Audit and Research Network (TARN) routine follow-up data. As such, physical performance measures would not be feasible to collect. |
| OMIs should not require access to technology that is not widely available in routine clinical practice. For example, the use of gait analysis software or worn accelerometers will not be feasible. |
-
OMI, outcome measurement instrument.
Quality assessment of outcome measurement instruments
Shortlisted OMIs underwent a three-step quality assessment process recommended by COSMIN,14,17,20,21 detailed in Figure 1. Each included validation study on a shortlisted OMI was evaluated independently by ALA and HC for methodological quality using the COSMIN methodology for evaluating content validity20 and Risk of Bias checklist.21 The result of each study was rated against the COSMIN criteria for good measurement properties.14 All assessed validation studies for each shortlisted OMI were qualitatively summarized, and the quality of evidence was graded for trustworthiness. A modified Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach was used to grade the quality of evidence for each OMI as ‘high’, ‘moderate’, ‘low’, or ‘very low’.14,22
Pragmatic approach
A novel approach was developed post hoc to identify and shortlist additional candidate OMIs for consideration of inclusion in the COS that do not have evidence of validation in the open lower limb fracture population. The study management group agreed upon four shortlisting criteria for each additional OMI (Figure 1). All OMIs used to measure outcomes on patients following open lower limb fracture were identified in a previously published systematic review.6 ALA and HC independently screened all identified OMIs against shortlisting criteria, and disagreements were resolved by discussion.
All OMIs shortlisted using the COSMIN preferred approach or the novel approach were considered at a consensus meeting.
Consensus meeting
A one-day face-to-face consensus meeting was held on 6 February 2020 to review and discuss the measurement properties and feasibility of each shortlisted OMI before undertaking a consensus process to select a set of OMIs. Participants were sampled purposively using a sampling matrix to maximize participant diversity within three stakeholder groups: patients, healthcare professionals, and researchers with experience in the field of measurement in medicine. Academic healthcare professionals and researchers were sampled from four academic institutions representing several research groups. Participants were sent a pre-meeting reading pack which detailed characteristics of OMIs shortlisted and rejected for the meeting. As a final check for the inclusion of relevant OMIs, participants were invited to suggest any additional instruments in advance of or on the day for consideration.
The meeting structure, good practice principles, and consensus methods were guided by the COMET Handbook8 and followed the methodology used in the COS consensus meeting published previously.12 The meeting was chaired by MLC, who has experience in running consensus meetings.
A summary including validity properties of each shortlisted OMI was presented with an accompanying printout of how the OMI would look to a patient completing it. Participants were asked to review the degree to which the OMI content/items adequately measure each of the four core outcomes (face validity check). After a period of group discussion, participants voted for which core outcomes they believed the OMI demonstrated face validity. Voting was conducted using TurningPoint software and handsets (Turning Technologies, USA). Participants could vote for any combination of core outcomes for each OMI before discussing and voting on the next candidate OMI.
After face validity checks, collated results were presented before two rounds of facilitated discussion and voting was conducted on which single OMI should be recommended for measuring each core outcome. Participants could only vote for one OMI for each core outcome per round of voting. Before each poll opened, the facilitator chaired a discussion inviting participants to share their views over which OMIs may be best and explain their rationale to the group.
Following voting rounds over which OMI should be recommended to measure each core outcome, two candidate OMI combinations were presented. After a discussion period, the final vote was conducted to define how the COS should be measured. Voting reached ‘consensus in’ where 70% or more of the vote was reached.
Results
Identifying existing outcome measurement instruments and quality assessment
Searches were undertaken for each core outcome; ‘walking, gait and mobility’, ‘return to life roles’, ‘pain or discomfort’, and ‘quality of life’ returned 372, 90, 111, and 163 articles, respectively. After rejecting duplicates, 580 articles were identified for the title and abstract review. Before the consensus meeting, authors became aware of the publication of one additional article23 and, after citation searching, a further article;24 both were included for full-text screening and eligibility assessment (Figure 3). Of the 582 articles, three were identified for feasibility assessment.23-25 Following assessment, one paper evaluating the Hamlyn Mobility Score’s measurement properties was rejected due to the inclusion of physical performance measures and the use of technology not widely available in routine clinical practice.25 Two articles by Trickett et al23,24 on the development and assessment of the measurement properties of the Wales Lower Limb Recovery (WaLLTR) scale, a novel PROM designed to measure recovery from open tibial fractures, were included for data extraction and OMI quality assessment.
Fig. 3
PRISMA flow diagram. CO-OLLF, Core Outcomes for Open Lower Limb Fracture; OMI, outcome measurement instrument.
A quantitative summary of studies evaluating the measurement properties of the WaLLTR scale against the COSMIN criteria for good measurement properties was not possible due to a lack of published studies. Thus, evaluation of the measurement properties for the WaLLTR scale from Trickett et al23,24 was used to grade the quality of evidence detailed in Table III and provided in full in Supplementary Material 2. The WaLLTR scale was rated as having ‘sufficient’ measurement properties for internal consistency, reliability, construct validity, and responsiveness. The WaLLTR scale received a content validity rating of ‘inconsistent’ due to inadequacies in conducting an appropriate cognitive interview study; in particular, an insufficiency in assessing the comprehensibility of questionnaire items to patients and professionals. It was demonstrated to be relevant to the open lower limb fracture population.
Table III.
Evaluation of the Wales Lower Limb Trauma Recovery scale against the updated COnsensus-based Standards for the selection of health Measurement Instruments criteria for good measurement properties and rating the quality of evidence using a modified Grading of Recommendations Assessment, Development, and Evaluation approach.
| Criteria | Overall rating* | Quality of evidence† |
|---|---|---|
| Content validity | ± | Moderate |
| Relevance | + | Moderate |
| Comprehensiveness | ? | Unable to rate |
| Comprehensibility | – | Unable to rate |
| Structural validity | Not applicable (formative model) | Unable to rate |
| Internal consistency | + | High |
| Cross-cultural validity | ? | Very low |
| Measurement invariance | ? | Very low |
| Reliability | + | Moderate |
| Measurement error | – | Very low |
| Criterion validity | Not undertaken | Unable to rate |
| Construct validity | + | Moderate |
| Responsiveness | + | High |
-
*
Score: + = sufficient; – = insufficient; ? = indeterminate; ± = inconsistent.
-
†
High, moderate, low, very low.
A novel pragmatic approach identified 77 OMIs from a previously published systematic review,6 which were assessed against inclusion criteria detailed in Figure 2; where OMIs were rejected, reasons were given (Supplementary Material 3). Assessor conflict was limited to a 2% disagreement rate on face validity checks resolved by discussion. Ten OMIs were shortlisted for inclusion at the consensus meeting along with the WaLLTR scale (Table IV for candidate OMIs, and Supplementary Material 3 for descriptions and source references).
Table IV.
Electronic voting scores on outcome measurement instruments (OMIs) for each core outcome, and final vote on OMI combination (number of voters = 25).
| Core outcome voting rounds | Number of votes (%) for OMIs | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| EQ-5D-3L | EQ-5D-5L | SF-12 | SF-36 | SIP | LEFS | SMFA | DRI | WaLTTR scale | BPI | VAS | |
| Walking, gait, and mobility | |||||||||||
| Voting round 1 | - | 2 (8) | - | - | 2 (8) | 18 (72)* | 2 (8) | 1 (4) | - | ||
| Voting round 2 | - | 2 (8) | - | - | - | 20 (83)* | 1 (4) | 1 (4) | - | ||
| Being able to return to life roles | |||||||||||
| Voting round 1 | - | - | 1 (4) | 8 (32) | - | 1 (4) | 1 (4) | - | 14 (56) | ||
| Voting round 2 | 1 (4) | - | 2 (8) | 4 (16) | - | - | 1 (4) | - | 17 (68) | ||
| Pain or discomfort | |||||||||||
| Voting round 1 | - | 15 (60) | - | - | - | - | - | - | 7 (28) | 3 (12) | |
| Voting round 2 | - | 17 (68) | - | - | - | - | - | - | 2 (8) | 6 (24) | |
| Quality of life | |||||||||||
| Voting round 1 | 1 (4) | 16 (64) | 1 (4) | 2 (8) | - | - | - | - | 5 (20) | ||
| Voting round 2 | 1 (4) | 18 (72)* | - | 1 (4) | - | - | - | - | 5 (20) | ||
| Proposed OMI combination for COMIS | Number of votes for OMI combination (%) | ||||||||||
| Voting round 1 | Voting round 2 | ||||||||||
| OMI combination 1 | 9 (36) | 21 (84)* | |||||||||
| EQ-5D-5L + LEFS | |||||||||||
| OMI combination 2 | 16 (64) | 4 (16) | |||||||||
| EQ-5D-5L + LEFS + WaLTTR scale | |||||||||||
-
*
Consensus threshold met at 70%.
-
BPI, Brief Pain Inventory; COMIS, core outcome measurement instrument set; DRI, Disability Rating Index; EQ-5D-3L, EuroQol five-dimension three-level questionnaire; EQ-5D-5L, EuroQol five-dimension five-level questionnaire; LEFS, Lower Extremity Functional Scale; LEFS, Lower Extremity Functional Scale; OMI, outcome measurement instrument; SF-12, 12-Item Short-Form Health Survey questionnaire; SF-36, 36-Item Short-Form Health Survey questionnaire; SIP, Sickness Impact Profile; SMFA, Short Musculoskeletal Function Assessment questionnaire; VAS, visual analogue scale for pain; WaLTTR, Wales Lower Limb Trauma Recovery; WaLTTR, Wales Lower Limb Trauma Recovery.
Each additionally shortlisted OMI has well-established validation evidence in the measurement of other areas of health, which was checked using scoping searches, including the COSMIN database of systematic reviews of measurement instruments.
Consensus meeting
The consensus meeting was attended by 25 participants, consisting of: five patients; a Patient and Public Involvement/patient advocate lead; eight healthcare professionals, of whom six also held academic positions; and 11 researchers with academic backgrounds relevant to measurement in medicine (Table V). All patients had sustained an open tibial fracture requiring plastic surgical intervention between six and 24 months before the meeting. The meeting was well represented by academics with relevant experience in medical measurement.
Table V.
Consensus meeting participant demographic data.
| Demographic | Data, n (%) (n = 25) |
|---|---|
| Patients | 5 (20) |
| Anatomical area of open fracture | |
| Open fracture of the leg including ankle (tibia/fibula) | 5 (20) |
| Indication of severity of open fracture | |
| Open fracture required a skin graft or muscle flap | 5 (20) |
| Sex, n (%) | |
| Male | 3 (12) |
| Female | 2 (8) |
| Age (yrs), n (%) | |
| 20 to 29 | 1 (4) |
| 50 to 59 | 2 (8) |
| Over 70 | 2 (8) |
| Patient and Public Involvement lead | 1 (4) |
| Healthcare professionals (academic position if held) | 8 (32) |
| Nurse Consultant for Trauma Care | 1 (4) |
| Physiotherapist (Professor of Trauma and Orthopaedic Rehabilitation) | 1 (4) |
| Plastic and Reconstructive Surgeon (Professor of Hand, Plastic and Reconstructive Surgery) | 1 (4) |
| Plastic and Reconstructive Surgeon | 1 (4) |
| Trauma and Orthopaedic Surgeon (Professor of Orthopaedic Trauma) | 2 (8) |
| Trauma and Orthopaedic Surgeon (Academic Clinical Fellow) | 1 (4) |
| Trauma and Orthopaedic Surgeon (PhD candidate) | 1 (4) |
| Researchers (academic background relevant to project) | 11 (44) |
| Senior Research Fellow (Qualitative Researcher, patient experience of recovery from trauma) | 1 (4) |
| Research Fellow (PROMs Researcher) | 3 (12) |
| Professor of Public Health and Primary Care (measurement in medicine) | 1 (4) |
| Associate Professor (evaluation of health and care services) | 1 (4) |
| Medical Statistician (use of PROMs in trials) | 4 (16) |
| Executive Director of the Trauma, Audit and Research Network (PROMs in trauma) | 1 (4) |
-
PROM, patient-reported outcome measure.
Participants voted to confirm that all shortlisted OMIs shortlisted face validity for measuring one or more core outcomes (see Supplementary Material 4 for full face validity check voting results). Two rounds of discussion and voting were undertaken for each of the four core outcomes. Consensus was reached for the Lower Extremity Functional Scale (LEFS)26 and the EuroQol five-dimension five-level questionnaire (EQ-5D-5L)27 to measure ‘Walking, gait and mobility’ and ‘Quality of Life’, respectively. Consensus was not achieved for selecting an OMI for ‘Being able to return to life roles’ and ‘Pain or discomfort’ (Table IV).
A pragmatic decision was made to conduct an additional two rounds of discussion and voting over whether the OMIs recommended for the COMIS should consist of the EQ-5D-5L and LEFS, or include the WaLLTR scale in addition. Initially, opinion appeared to favour including the WaLLTR scale, with 64% (n = 16) of the participant votes in round 1. However, following the discussion, 84% (n = 21) of participants voted to exclude the WaLLTR scale. Thus, a consensus decision was made for the recommendation of the EQ-5D-5L and the LEFS only (Table IV).
The Study Management Group decided to give participants an additional opportunity to consider the WaLLTR scale’s inclusion. This was considered appropriate, as the WaLLTR scale was the only OMI presented at the meeting that had been specifically developed and validated for use on the open lower limb population. However, throughout the consensus meeting, several concerns were raised over the use of language in the WaLLTR scale. Participants felt that the scale could benefit from a linguistic assessment and further work to establish good content validity around comprehensibility of the question items identified as problematic at quality assessment using the COSMIN checklist. Researcher participants felt that there were problems with the scale by including the middle response option as ‘neither agree or disagree’, which creates issues regarding structural validity, e.g. such a response would be identified as problematic when undertaking a Rasch analysis.
Finally, a feasibility concern was raised that the inclusion of the WaLLTR scale, the EQ-5D-5L, and the LEFS would result in 35 question items and may risk the recommended COMIS becoming overly burdensome for implementation in all future trials.
Discussion
This study followed guidelines laid out by the COSMIN initiative demonstrating methodological rigour. By considering which currently available OMIs are best for measuring core outcomes, this study goes beyond work undertaken to develop the COS by recommending a set of two ‘ready to use’ OMIs. Recommendation on OMI selection when measuring the open lower limb fracture COS will result in further reductions in outcome-reporting heterogeneity in future open lower limb fracture research.
There are a number of strengths and limitations in this study. A novel approach was taken to overcome a lack of OMIs validated in the open lower limb fracture population to recommend a set of robust and well-respected OMIs with a proven track record of use in past research on patients following open lower limb fracture. This was achieved by drawing on a panel of expert researchers in the field of measurement in medicine. The practicality and acceptability of the OMIs were also considered by including patients and healthcare professionals with expertise in caring for and conducting research on patients following open lower limb fracture.
OMIs recommended in this study do not currently have any evidence for validation in the open lower limb fracture population. According to COSMIN guidance, a new instrument should be developed before a set of OMIs can be recommended. Developing a well-validated OMI and demonstrating its practical utility is a demanding process that is costly, lengthy, and requires specialist expertise. This study recognizes a need for an OMI designed for and validated in the open lower limb fracture population for recommendation. However, the study authors and a panel of professional experts and patients assembled at the consensus meeting supported the view that in the short-to-medium term, it is more important to gain consensus on and recommend an appropriate set of OMIs that have already demonstrated a good utility for measurement in the open lower limb fracture population. This will promote immediate continuity of outcome measurement in research studies and national audits on the open lower limb fracture population going forward.
Recommendation of the EQ-5D-5L and LEFS, which have been widely used for outcome measurement in previous open lower limb fracture research, suggests that they have beneficial measurement properties and practical utilities that previous study designers found desirable. As both the EQ-5D-5L and LEFS have been previously used to measure outcomes in this research field, study designers are more likely to be familiar with the instruments, which may increase implementation, uptake, and the inclusion of historical studies in future meta-analyses.
The development of the WaLLTR scale shows promise, and it is the recommendation of this study that additional work is conducted on WaLLTR to address issues raised at the consensus meeting. The COS for open lower limb fracture will remain flexible to change, and where evidence emerges for additional suitable OMIs, the consensus process should be revisited. The Study Management Group intend to act as guardians over the open lower limb fracture COS and action updates to ensure that the best available OMIs are recommended to measure core outcomes. Future updates will be supported under the oversight of interested national parties, including the Orthopaedic Trauma Society and the Trauma, Audit and Research Network.
In conclusion, the CO-OLLF project recommends the EQ-5D-5L and LEFS to be used in future trials and as part of routine clinical assessment of adult patients recovering from open lower limb fracture to measure the core outcomes ‘Quality of life’ and ‘Walking, gait and mobility’, respectively. While consensus was not achieved for an OMI to measure ‘Being able to return to life roles’ or ‘Pain or discomfort’, the EQ-5D-5L does demonstrate face validity in measuring both of these outcomes.
References
1. Court-Brown CM , Bugler KE , Clement ND , Duckworth AD , McQueen MM . The epidemiology of open fractures in adults. A 15-year review . Injury . 2012 ; 43 ( 6 ): 891 – 897 . Crossref , PubMed Google Scholar
2. Costa ML , Achten J , Parsons NR , WOLLF collaborators . Five-year outcomes for patients sustaining severe fractures of the lower limb: mid-term results from the Wound management for Open Lower Limb Fracture (WOLLF) trial . Bone Joint J . 2022 ; 104-B ( 5 ): 633 – 639 . Crossref , PubMed Google Scholar
3. Costa ML , Achten J , Bruce J , et al. Effect of Negative Pressure Wound Therapy vs Standard Wound Management on 12-Month Disability Among Adults With Severe Open Fracture of the Lower Limb: The WOLLF Randomized Clinical Trial . JAMA . 2018 ; 319 ( 22 ): 2280 – 2288 . Crossref , PubMed Google Scholar
4. Rees S , Tutton E , Achten J , Bruce J , Costa ML . Patient experience of long-term recovery after open fracture of the lower limb: a qualitative study using interviews in a community setting . BMJ Open . 2019 ; 9 ( 10 ): e031261 . Crossref , PubMed Google Scholar
5. Png ME , Petrou S , Knight R , Masters J , Achten J , Costa ML . Economic outcomes associated with deep surgical site infection from lower limb fractures following major trauma . Bone Jt Open . 2022 ; 3 ( 5 ): 398 – 403 . Crossref , PubMed Google Scholar
6. Aquilina AL , Claireaux H , Aquilina CO , et al. What outcomes have been reported on patients following open lower limb fracture, and how have they been measured? Bone Joint Res . 2023 ; 12 ( 2 ): 138 – 146 . Crossref , PubMed Google Scholar
7. Williamson PR , Altman DG , Blazeby JM , et al. Developing core outcome sets for clinical trials: issues to consider . Trials . 2012 ; 13 ( 1 ): 132 . Crossref , PubMed Google Scholar
8. Williamson PR , Altman DG , Bagley H , et al. The COMET Handbook: version 1.0 . Trials . 2017 ; 18 ( Suppl 3 ): 280 . Crossref , PubMed Google Scholar
9. Kirkham JJ , Davis K , Altman DG , et al. Core Outcome Set-STAndards for Development: The COS-STAD recommendations . PLoS Med . 2017 ; 14 ( 11 ): e1002447 . Crossref , PubMed Google Scholar
10. Ollivere BJ , Marson BA , Haddad FS . Getting the right answer: core outcome sets in orthopaedics . Bone Joint J . 2019 ; 101-B ( 3 ): 233 – 235 . Crossref , PubMed Google Scholar
11. Kirkham JJ , Gorst S , Altman DG , et al. Core Outcome Set-STAndards for Reporting: The COS-STAR Statement . PLoS Med . 2016 ; 13 ( 10 ): e1002148 . Crossref , PubMed Google Scholar
12. Aquilina AL , Claireaux H , Aquilina CO , et al. Development of a core outcome set for open lower limb fracture . Bone Joint Res . 2023 ; 12 ( 4 ): 294 – 305 . Crossref , PubMed Google Scholar
13. No authors listed . COSMIN database of systematic reviews of outcome measurement instruments . COSMIN . 2023 . https://database.cosmin.nl/ ( date last accessed 12 April 2023 ). Google Scholar
14. Prinsen CAC , Vohra S , Rose MR , et al. How to select outcome measurement instruments for outcomes included in a “Core Outcome Set” - a practical guideline . Trials . 2016 ; 17 ( 1 ): 449 . Crossref , PubMed Google Scholar
15. Trompeter AJ , Knight R , Parsons N , Costa ML . The Orthopaedic Trauma Society classification of open fractures . Bone Joint J . 2020 ; 102-B ( 11 ): 1469 – 1474 . Crossref , PubMed Google Scholar
16. Png ME , Petrou S , Bourget-Murray J , Knight R , Trompeter AJ , Costa ML . Association between the orthopaedic trauma society classification of open fractures and economic costs . Bone Joint J . 2022 ; 104-B ( 3 ): 408 – 412 . Crossref , PubMed Google Scholar
17. Prinsen CAC , Mokkink LB , Bouter LM , et al. COSMIN guideline for systematic reviews of patient-reported outcome measures . Qual Life Res . 2018 ; 27 ( 5 ): 1147 – 1157 . Crossref , PubMed Google Scholar
18. Terwee CB , Jansma EP , Riphagen II , de Vet HCW . Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments . Qual Life Res . 2009 ; 18 ( 8 ): 1115 – 1123 . Crossref , PubMed Google Scholar
19. No authors listed . PROM Group Construct & Instrument Type Filters: February 2010 . Patient-Reported Outcome Measurement Group, Department of Public Health, University of Oxford . 2010 . https://cosmin.nl/wp-content/uploads/prom-search-filter-oxford-2010.pdf ( date last accessed 9 May 2023 ). Google Scholar
20. Terwee CB , Prinsen CAC , Chiarotto A , et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study . Qual Life Res . 2018 ; 27 ( 5 ): 1159 – 1170 . Crossref , PubMed Google Scholar
21. Mokkink LB , de Vet HCW , Prinsen CAC , et al. COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures . Qual Life Res . 2018 ; 27 ( 5 ): 1171 – 1179 . Crossref , PubMed Google Scholar
22. Schünemann H , Brożek J , Guyatt G , Oxman A . GRADE Handbook: Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach . Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Working Group . 2013 . https://gdt.gradepro.org/app/handbook/handbook.html ( date last accessed 12 April 2023 ). Google Scholar
23. Trickett RW , Mudge E , Price P , Pallister I . The development of a novel patient-derived recovery scale for open tibial fractures . Bone Joint J . 2020 ; 102-B ( 1 ): 17 – 25 . Crossref , PubMed Google Scholar
24. Trickett RW , Mudge E , Price P , Pallister I . A qualitative approach to recovery after open tibial fracture: the road to A novel, patient-derived recovery scale . Injury . 2012 ; 43 ( 7 ): 1071 – 1078 . Crossref , PubMed Google Scholar
25. Kwasnicki RM , Hettiaratchy S , Jarchi D , et al. Assessing functional mobility after lower limb reconstruction: a psychometric evaluation of a sensor-based mobility score . Ann Surg . 2015 ; 261 ( 4 ): 800 – 806 . Crossref , PubMed Google Scholar
26. Binkley JM , Stratford PW , Lott SA , Riddle DL . The Lower Extremity Functional Scale (LEFS): scale development, measurement properties, and clinical application. North American Orthopaedic Rehabilitation Research Network . Phys Ther . 1999 ; 79 ( 4 ): 371 – 383 . PubMed Google Scholar
27. Herdman M , Gudex C , Lloyd A , et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L) . Qual Life Res . 2011 ; 20 ( 10 ): 1727 – 1736 . Crossref , PubMed Google Scholar
Author contributions
A. L. Aquilina: Conceptualization, Formal analysis, Project administration, Writing – original draft.
H. Claireaux: Data curation, Formal analysis, Writing – original draft.
C. O. Aquilina: Data curation, Formal analysis, Writing – original draft.
E. Tutton: Conceptualization, Data curation, Formal analysis, Methodology, Writing – review & editing.
R. Fitzpatrick: Conceptualization, Data curation, Formal analysis, Methodology, Writing – review & editing.
M. L. Costa: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing.
X. L. Griffin: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing.
Funding statement
The authors disclose receipt of the following financial or material support for the research, authorship, and/or publication of this article: funding from an AO UK Major Research grant (WE/VMC 24 October 2017), and financial support from the National Institute for Health and Care Research and Oxford Biomedical Research Centre.
ICMJE COI statement
X. L. Griffin reports institutional grants (paid to Queen Mary University of London) from the National Institute for Health and Care Research, not related to this study.
Data sharing
Referenced in press articles: 1) What outcomes have been reported on patients following open lower limb fracture, and how have they been measured? Submitted to Bone & Joint Research on 18 April 2022, ID: BJR-2022-0116.R1 (accepted); 2) The Core Outcomes for Open Lower Limb Fracture study. Development of a core outcome set for open lower limb fracture: what core outcomes should be measured? Submitted to Bone & Joint Research on 10 May 2022, ID: BJR-2022-0164.R2 (accepted pending a minor revision).
Acknowledgements
We are grateful for the support of the England and Wales Major Trauma Network, The Trauma, Audit and Research Network, the Orthopaedic Trauma Society, and the British Association of Plastic Reconstructive and Aesthetic Surgeons. CO-OLLF Delphi survey and consensus meeting collaborators (patients, healthcare professionals, and researchers). A panel of experts in trauma, orthopaedic, and plastic surgery as well as measurement in medicine, including Michele Peters, Jagdeep Nanchahal, David Morley, Ines Rombach, Mairead Murphy, Bethan Copsey, Jill Dawson, Julie Wright, Ruth Knight, Michael Schlussel, Antoinette Edwards, Rebecca Kearney, David Wallace, and Chetan Khatri. A special thank you to Elizabeth Baird for her role as patient and public representative lead and participation in all patient-facing activities ensuring the patient's voice was heard, and Lauren Exell for all her administrative help throughout the study.
Ethical review statement
The South-Central Research Ethics Committee (REC) granted ethical approval for this study on 1 March 2018 (REC reference: 18/SC/0051, IRAS project ID: 235150). The CO-OLLF study was prospectively registered on the COMET database (http://www.comet-initiative.org/Studies/Details/688) and was adopted by the National Institute for Health and Care Research (NIHR) Clinical Research Network (CRN) (Protocol number: 13257).
Open access funding
The authors report that they received open access funding for their manuscript from the National Institute for Health and Care Research (paid to Queen Mary University of London).
Follow A. Leonard. Aquilina @alexaquilina87
Follow H. Claireaux @harryclax
Follow E. Tutton @Oxford_Trauma
Follow M. L. Costa @Oxford_Trauma
Follow X. L. Griffin @xlgriffin
Supplementary material
Search blocks, filters, and search results identifying validated outcome measurement instruments for each core outcome in the open lower limb fracture population; COnsensus-based Standards for the selection of health Measurement Instruments quality assessment tables for the Wales Lower Limb Recovery scale; outcome measurement instruments not validated in target population assessed against pragmatic inclusion criteria for inclusion at the consensus meeting; and results of face validity checks for each outcome measurement instrument against core outcomes.
© 2023 Author(s) et al. This is an open-access article distributed under the terms of the Creative Commons Attribution Non-Commercial No Derivatives (CC BY-NC-ND 4.0) licence, which permits the copying and redistribution of the work only, and provided the original author and source are credited. See https://creativecommons.org/licenses/by-nc-nd/4.0/
