Abstract
Aims
This study aims to define a set of core outcomes (COS) to allow consistent reporting in order to compare results and assist in treatment decisions for idiopathic clubfoot.
Methods
A list of outcomes will be obtained in a three-stage process from the literature and from key stakeholders (patients, parents, surgeons, and healthcare professionals). Important outcomes for patients and parents will be collected from a group of children with idiopathic clubfoot and their parents through questionnaires and interviews. The outcomes identified during this process will be combined with the list of outcomes previously obtained from a systematic review, with each outcome assigned to one of the five core areas defined by the Outcome Measures Recommended for use in Randomized Clinical Trials (OMERACT). This stage will be followed by a two round Delphi survey aimed at key stakeholders in the management of idiopathic clubfoot. The final outcomes list obtained will then be discussed in a consensus meeting of representative key stakeholders.
Conclusion
The inconsistency in outcomes reporting in studies investigating idiopathic clubfoot has made it difficult to define the success rate of treatments and to compare findings between studies. The development of a COS seeks to define a minimum standard set of outcomes to collect in all future clinical trials for this condition, to facilitate comparisons between studies and to aid decisions in treatment.
Cite this article: Bone Jt Open 2021;2(4):255–260.
Introduction
Idiopathic congenital talipes equinovarus (CTEV) is a congenital 3D deformity of the foot1,2 with an incidence of 1 to 2 per 1,000 live births per year.3-5 Left untreated, the foot remains deformed, resulting in pain and functional limitations.1,5
The gold standard treatment for the idiopathic clubfoot is serial casting,1 following the Ponseti method, with a high success rate in achieving a primary correction.6-8 The treatment is low-cost and can be delivered by various members of the healthcare team. However, the long-term outcomes of the Ponseti method have yet to be established.7 The lack of definition of a relapse and the inconsistency in outcomes reporting among studies have made results difficult to compare. Indeed, a systematic review of 124 clinical trials confirmed that outcomes reporting was not standardized, identifying 20 different isolated outcomes and 16 different outcome tools.9
This lack of standardization of outcomes in studies investigating treatment effectiveness in clubfoot is the primary barrier in defining the success rate of the intervention.9 Core outcomes sets (COSs), developed following the COMET-Initiative10 guidelines, are minimum standardized sets of outcomes that should be reported in all studies investigating a specific clinical condition.11,12 Their implementation in clinical trials aims to reduce the heterogeneity of presented outcomes, allowing meaningful comparisons between studies.11,12 The use of COSs are well-established in clinical research and for some paediatric conditions,13-17 although less common in orthopaedic practice.14,18,19 To date no COS is available to define the success of intervention in idiopathic clubfoot.
The aim of this study is to develop a COS for clinical studies assessing idiopathic clubfoot management. The specific objectives of this study are to: identify outcomes of importance to patients and their parents through self-reported questionnaires; identify outcomes important to key stakeholders, such as surgeons, physiotherapists, and other clubfoot practitioners using a Delphi survey; and to hold a consensus meeting where the outcomes list will be discussed with all stakeholders to form the final core outcomes list.
Methods
A systematic review identifying all outcomes reported in the recent literature has been completed and the results have been published.9
Identification of key outcomes to patients and parents
Patient and parent involvement is of primary importance when defining effective COS that embrace a comprehensive view of outcomes relevant to key stakeholders.10 As stated by the COMET Handbook,12 several stages are needed to form the list that will comprise the final COS, and a qualitative approach is suggested as a way to collect patients’ and caregivers’ experience of the disease for which the COS is being developed. The aim of this process is to identify outcomes of relevance to the key stakeholders, that cannot be done merely through presentation of a list of outcomes to score, which could strongly influence patients’ answers.12 Instead, this process allows patients and caregivers to freely describe their experience of the illness and of the treatment in their own words,12 using qualitative methods such as group discussion; semi-strucured interviews; and/or open-ended questionnaires. In line with the suggestions of the COMET Handbook,12 to identify patient-reported outcomes (PROs) to be integrated into the COS for idiopathic clubfoot, patients and parents will be invited to complete a questionnaire, which will be used as a prompt for further discussion, to assess the life impact of idiopathic clubfoot.20 PROs identified through this process will be added to the outcome list obtained from a previously published systematic review.9 The full list will then be submitted for evaluation to key stakeholders through a Delphi Survey.
Patient-reported outcomes identification
PROs for patients and parents will be determined through the completion of questionnaires, including (Figure 1a): i) a dedicated questionnaire for parents; ii) a dedicated questionnaire for children (divided into age groups: 5 to 7 years; 8 to 11 years; 12 to 16 years).
Fig. 1
a) Core outcomes set (COS) development process. b) Final consensus meeting steps.
Participants
Patients and their families with direct experience of idiopathic clubfoot treated according to the Ponseti method and with the ability to converse in English will be invited to participate in the key outcomes’ identification process. Participants will be recruited from the Ponseti clinics of two local children’s hospitals. Patients and parents for whom English is not their first language will be provided with a translation service if needed. A minimum of 20 patients treated for idiopathic clubfoot, both males and females, and with an age of between five to 16 years, and their related parents will be recruited at this stage.
The information the patient and parents are required to provide in the questionnaires is no different from routine clinical questions that may be asked during their regular clinical appointments. Consultation with the institutional R&D offices has deemed this a service evaluation with no requirement for ethical approval.
Questionnaire format
Parent questionnaire
The view of the parent(s) of each child will be sought using a questionnaire that will comprise a series of open-ended questions on their experiences and on the impact that idiopathic clubfoot has on their child’s everyday life (Supplementary Material). The initial part of the questionnaire will gather data on sex and age of the parent participating in the survey. Subsequent questions will investigate areas such as influence of the condition on patients and family, how the condition affects their daily living activities, and the results of the clinical management. The questionnaire will also ask to identify possible outcomes in the management of idiopathic clubfoot. The questionnaire can be completed as a self-reported questionnaire by the parent; or used as a semi-structured interview schedule to be completed with the researcher, who will read the questions and take note of the parents’ answers using them as a prompt for further discussion. Completion of this questionnaire will take no longer than 25 minutes and will be anonymous. No identifiable personal data will be collected via the questionnaire. Following the questionnaire, parents will be also presented with a list of outcomes already collected from the systematic review to discuss further with the researcher, giving a personal opinion on their relevance. Any addition or suggestion of an existing outcome will be added to the outcome list and processed in the next stage.
Child questionnaire
Each child will complete the patient questionnaire (with their parents’ help where needed). The questionnaire aims to identify outcomes relevant to children, and it contains questions related to the influence of the condition on their daily living activities (Supplementary Material). It is designed to meet the developmental needs of a broad array of children, and each question is graded using emoji to ensure ease of completion (Table I). Each emoji corresponds to the equivalent relevance category for each outcome (critically relevant; important but not relevant; and not relevant) and will help the children to communicate the importance they attribute to specific outcomes and as a means of introducing the topic to this young audience. The questions are formulated in an appropriate manner to indirectly assess the relevance of the outcome discussed,21 as direct questions need clear explanation of what an outcome is, making things confusing for patients (especially for children) and possibly influencing their answers.12 The use of emoji as a grading system in questionnaires seeking PROs in children has already been adopted with good results.22 At the end of the questionnaire, there is a free narrative section where children can report the difficulties that they experience during daily activities to get their view on the condition and obtain possible insight on outcomes not considered previously in literature. Completion of this questionnaire will take no longer than 15 minutes and will be anonymous. No identifiable personal data will be collected by the questionnaire.
Table I.
Children questionnaire: emoji grading system.
| Emoji | Score (relevance)* |
|---|---|
|
|
Critical importance7-9 |
|
|
Important but not critical4-6 |
|
|
Not important1-3 |
-
*
Simplified version of the nine points scale used to score the outcomes during the Delphi Survey.
Patients and their parents will have the option to complete the questionnaires either during their outpatient clinics or at home, based on their preferences. If completed at home, the questionnaires will be sent to participants by either email or post/mail. Informed consent will be assumed if participants complete the questionnaire. Parents’ contact details will be collected on a separate form with their consent, and used to invite them to participate in the subsequent Delphi survey. The questionnaires have been co-designed with patients, who have fed back and “sense checked” in an iterative process throughout their development.
Questionnaire analysis
No identifiable participant data will be collected/retained. The parents’ questionnaire follows an open-ended structure and thus will be analyzed using a qualitative approach.23 Data from this questionnaire will be uploaded to a dedicated qualitative analysis software (NVivo Software, UK), and themes will be generated using the six-step thematic analysis method by Braun and Clarke.24 The themes identified will then be reviewed and discussed by all members of the research team in dedicated meetings.
Conversely, the results of the children’s questionnaire (except for the narrative section, which will be analyzed through thematic analysis) will be reported using descriptive statistics (i.e. median/range for continuous variables and frequencies/percentages for categorical data) and in tables and figures (as appropriate). No formal statistical significance testing will be undertaken and emphasis will be on description and trends.
The process of analysis of the questionnaires will summarize and define the key outcomes based on the stakeholders’ opinion. The PROs identified during this process will be assigned to one of the five core domains of the OMERACT (Outcome Measures in Rheumatology) filter,11 and added to the list of outcomes obtained from a systematic review published previously.9
Identification of key outcomes to clinicians
Overview
The list of outcomes collected from the systematic review9 and from the patients’/parents’ involvement described in this protocol will act as the basis for an international Delphi survey, aiming to collect feedback and distil the key outcomes important to patients, parents, orthopaedic surgeons, physiotherapists, and other healthcare professionals involved in the management of clubfoot. The Delphi method is a forecasting process framework comprising sequential surveys answered anonymously by a panel of participants with relevant knowledge and expertise in the given area, giving equal influence to all who participate in order to reach consensus.25 This is a standard process in the development of COS as per COMET-Initiative guidelines.10
Participants
No agreed standard sample size has been defined for a Delphi Survey26,27 as it is usually determined by practicality, research area, aim of the study, and time available for analysis.27,28 We will recruit the largest possible sample with at least ten participants for each stakeholder group (parents’ group; patients’ group; orthopaedic surgeons’ group; and clubfoot trained healthcare professionals’ group).27 Participants in the survey will be key stakeholders in the management of idiopathic clubfoot. The survey will be open to an international audience, looking for opinion from both UK and overseas stakeholders. Participants will be recruited from international partner hospitals/university institutions and from international patients’/parents’ organisations. Selection criteria are: experience of the disease for parents (e.g. parents who have at least one child with clubfoot) and expertise in clubfoot treatment for professionals (e.g. clinical interest of the surgeon). Participants will be contacted and invited to participate in the survey by email, and to complete the survey through a bespoke core outcome set Delphi management tool.29 Informed consent will be assumed if participants complete the survey.
Delphi Survey
The survey will be divided into two rounds (Figure 1a). Participants involved in the study will have three weeks to complete each round. A reminder email will be sent at the end of week 2 of each round to encourage completion of the survey and reduce dropout rate.
Delphi Round 1
Each participant in the survey will complete the electronic data collection form, which collects details of participants’ demographic data (participant name; role; institution; and contacts). Participants will then be asked to review the list of selected outcomes to be graded on a score of 1 to 9 (with “1 to 3 = not relevant”; “4 to 6 = important but not critical”; and “7 to 9 = extremely relevant”). Each participant will have the opportunity to add additional outcomes they consider important.
Analysis of Delphi Round 1
The data analysis will summarize the distribution of scores, and will be conducted separately for each stakeholder group. Additional outcomes added by the participants will be reviewed by two assessors in order to ensure that they do not refer to outcomes already listed. The response rate will also be recorded.
Delphi Round 2
Participants who respond to Round 1 of the survey will be invited to participate in Round 2; they will be able to see the summary of the data obtained in round one, and will be asked to review again the list of outcomes, with the opportunity to confirm or change their previous ratings. Participants who do not respond to round one will be excluded from Round 2.
Analysis of Delphi Round 2
The total number of participants that participate in Round 2 will be recorded. The analysis of the data will have separate results for each stakeholder group, with each outcome classified as “consensus in”, “consensus out”, and “no consensus” using the consensus criteria defined below. As this is a preliminary analysis of the results, all classified outcomes (consensus in; consensus out; no consensus) will be brought forward to the consensus meeting for final considerations and possible inclusion/exclusion.
Consensus meeting
The final list of the outcomes obtained from the Delphi study will be discussed either in a face-to-face or in an online consensus meeting between a selected group of international clinicians/healthcare professionals and an international representative group of patients/parents (Figure 1b). The aim is to recruit approximately 24 participants adhering to the OMERACT guidelines for the consensus meeting structure.11 Informed consent will be assumed if participants decide to take part in the final consensus meeting.
The meeting will be chaired by an independent researcher who is familiar with the Delphi process. He/she will not participate in the voting procedure. Before the meeting, participants will be able to review the score of the outcomes from the Delphi survey, and these data will then be discussed during the consensus meeting following these steps (Figure 1b): i) presentation of the results from the Delphi survey; ii) group discussion; iii) anonymous scoring of each outcome by the participants (using an online platform);30 and iv) formal endorsement (sign off) of the final core outcomes set by all participants in the consensus meeting. Any item categorized as ‘consensus in’ will be proposed to be included in the final document, while any item categorized as ‘consensus out’ will be excluded. Items that are categorized as ‘no-consensus’ will be discussed individually. The final consensus document will be agreed upon in the consensus meeting.
A report of the consensus meeting will be written up and published.
Definition of consensus
To define consensus, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidelines will be employed.31 Outcomes inclusion (consensus in) will be indicated as the agreement by the vast majority (> 70% of the group) that the discussed outcome is “extremely relevant” (7 to 9 points range), with only a minority (< 15% of the group) of participants considering it as “not relevant” (1 to 3 points range). Outcomes exclusion (consensus out) will be indicated as the agreement by the vast majority (> 70% of the group) that the discussed outcome is “not relevant” (1 to 3 points range), with only a minority (< 15% of the group) of participants considering it as “extremely relevant” (7 to 9 points range).
Discussion
The absence of a minimum standardized outcomes set for clinical trials in idiopathic clubfoot has led to difficulties in comparing research and in defining the rate of success of investigated treatments. Thus, the aim of this study is to develop a COS for clinical trials assessing idiopathic clubfoot management. The present protocol states the methodology of the study, which employs the well-established and widely used guidelines developed by the COMET Group.12
Patients (children) and families affected by idiopathic clubfoot will be engaged in the COS development to identify outcomes relevant to them. Indeed, patients’ involvement in COS development has become standard practice to ensure that the final COS embrace the view of all relevant stakeholders of the care process.12 The outcomes identified from the patients’ and parents’ involvement will provide, together with the list of outcomes previously identified from a systematic review, a comprehensive list of all relevant outcomes to be included in a Delphi survey for further scrutiny.
A final consensus meeting among international representatives of patients, parents, surgeons, and other relevant healthcare professionals will then score the results of the Delphi survey to reach consensus on the final list of outcomes to be include in the COS.
References
1. Colaço HB , Patel S , Lee MH , Shaw OM . Congenital clubfoot: a review . Br J Hosp Med . 2010 ; 71 ( 4 ): 200 – 205 . Crossref PubMed Google Scholar
2. Nordin S , Aidura M , Razak S , Faisham W . Controversies in congenital clubfoot: literature review . Malays J Med Sci . 2002 ; 9 ( 1 ): 34 – 40 . Google Scholar
3. Dobbs MB , Gurnett CA . Update on clubfoot: etiology and treatment . Clin Orthop Relat Res . 2009 ; 467 ( 5 ): 1146 – 1153 . Crossref PubMed Google Scholar
4. Smythe T , Kuper H , Macleod D , Foster A , Lavy C . Birth prevalence of congenital talipes equinovarus in low- and middle-income countries: a systematic review and meta-analysis . Trop Med Int Health . 2017 ; 22 ( 3 ): 269 – 285 . Crossref PubMed Google Scholar
5. Chen C , Kaushal N , Scher DM , Doyle SM , Blanco JS , Dodwell ER . Clubfoot etiology: a meta-analysis and systematic review of observational and randomized trials . J Pediatr Orthop . 2018 ; 38 ( 8 ): e462 – e9 . Crossref PubMed Google Scholar
6. Ganesan B , Luximon A , Al-Jumaily A , Balasankar SK , Naik GR . Ponseti method in the management of clubfoot under 2 years of age: a systematic review . PLoS One . 2017 ; 12 ( 6 ): e0178299 . Crossref PubMed Google Scholar
7. Gelfer Y , Wientroub S , Hughes K , Fontalis A , Eastwood DM . Congenital talipes equinovarus: a systematic review of relapse as a primary outcome of the Ponseti method . Bone Joint J . 2019 ; 101-B ( 6 ): 639 – 645 . Crossref PubMed Google Scholar
8. Radler C . The Ponseti method for the treatment of congenital club foot: review of the current literature and treatment recommendations . Int Orthop . 2013 ; 37 ( 9 ): 1747 – 1753 . Crossref PubMed Google Scholar
9. Gelfer Y , Hughes KP , Fontalis A , Wientroub S , Eastwood DM . A systematic review of reported outcomes following Ponseti correction of idiopathic club foot . Bone Jt Open . 2020 ; 1 ( 8 ): 457 – 464 . Crossref PubMed Google Scholar
10. COMET-Initiative . Core Outcome Measures in Effectiveness Trials . 2020 . http://www.comet-initiative.org/ (date last accessed 14 April 2021 ). Crossref PubMed Google Scholar
11. Boers M , Kirwan JR , Wells G , et al. Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0 . J Clin Epidemiol . 2014 ; 67 ( 7 ): 745 – 753 . Crossref PubMed Google Scholar
12. Williamson PR , Altman DG , Bagley H , et al. The COMET Handbook: version 1.0 . Trials . 2017 ; 18 ( Suppl 3 ): 280 . Crossref PubMed Google Scholar
13. Crudgington H , Rogers M , Bray L , et al. Core health outcomes in childhood epilepsy (choice): development of a core outcome set using systematic review methods and a Delphi survey consensus . Epilepsia . 2019 ; 60 ( 5 ): 857 – 871 . Crossref PubMed Google Scholar
14. Leo DG , Jones H , Murphy R , Leong JW , Gambling T , Long AF . The outcomes of Perthes’ disease: Development of a core outcomes set for clinical trials in Perthes’ disease . Bone Joint J . 2020 ; 102 ( 5 ): 611 – 617 . Google Scholar
15. Marson BA , Manning JC , James M , et al. CORE-Kids: a protocol for the development of a core outcome set for childhood fractures . BMJ Open . 2020 ; 10 ( 2 ): e036224 . Crossref PubMed Google Scholar
16. Sinha IP , Gallagher R , Williamson PR , Smyth RL . Development of a core outcome set for clinical trials in childhood asthma: a survey of clinicians, parents, and young people . Trials . 2012 ; 13 ( 1 ): 103 . Crossref PubMed Google Scholar
17. Tong A , Samuel S , Zappitelli M , et al. Standardised Outcomes in Nephrology—Children and Adolescents (SONG-Kids): a protocol for establishing a core outcome set for children with chronic kidney disease . Trials . 2016 ; 17 ( 1 ): 1 – 11 . Google Scholar
18. Ollivere BJ , Marson BA , Haddad FS . Getting the right answer: core outcome sets in orthopaedics . Bone Joint J . 2019 ; 101-B ( 3 ): 233 – 235 . Crossref PubMed Google Scholar
19. Dorman SL , Shelton JA , Stevenson RA , Linkman K , Kirkham J , Perry DC . Management of medial humeral epicondyle fractures in children: a structured review protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey . Trials . 2018 ; 19 ( 1 ): 1 – 7 . Crossref PubMed Google Scholar
20. Macefield RC , Jacobs M , Korfage IJ , et al. Developing core outcomes sets: methods for identifying and including patient-reported outcomes (pros) . Trials . 2014 ; 15 ( 1 ): 1 – 12 . Crossref PubMed Google Scholar
21. Rattray J , Jones MC . Essential elements of questionnaire design and development . J Clin Nurs . 2007 ; 16 ( 2 ): 234 – 243 . Crossref PubMed Google Scholar
22. Leo DG , Murphy R , Gambling T , Long A , Jones H , Perry DC . Perspectives on the Social, Physical, and Emotional Impact of Living With Perthes’ Disease in Children and Their Family: A Mixed Methods Study . Glob Pediatr Health . 2019 ; 6 : 2333794X1983523 . Google Scholar
23. Popping R . Analyzing open-ended questions by means of text analysis procedures . Bulletin of Sociological Methodology/Bulletin de Méthodologie Sociologique . 2015 ; 128 ( 1 ): 23 – 39 . Google Scholar
24. Braun V , Clarke V . Using thematic analysis in psychology . Qual Res Psychol . 2006 ; 3 ( 2 ): 77 – 101 . Google Scholar
25. Smith J , Firth J . Qualitative data analysis: the framework approach . Nurse Res . 2011 ; 18 ( 2 ): 52 – 62 . Crossref PubMed Google Scholar
26. Keeney S , Hasson F , McKenna HP . A critical review of the Delphi technique as a research methodology for nursing . Int J Nurs Stud . 2001 ; 38 ( 2 ): 195 – 200 . Crossref PubMed Google Scholar
27. Hsu C-C , Sandford BA . The Delphi technique: making sense of consensus . Pract Assess Res Evaluation . 2007 ; 12 ( 1 ): 10 . Google Scholar
28. Green B , Jones M , Hughes D , Williams A . Applying the Delphi technique in a study of GPs' information requirements . Health Soc Care Community . 1999 ; 7 ( 3 ): 198 – 205 . Crossref PubMed Google Scholar
29. COMET-Initiative . The COMET Initiative Delphi manager . http://www.comet-initiative.org/delphimanager/ Google Scholar
30. VoxVote . VoxVote free and easy mobile voting tool for any SPEAKER or teacher . 2014 . https://www.voxvote.com/ Google Scholar
31. Schünemann HJ , Oxman AD , Brozek J , et al. Grade: assessing the quality of evidence for diagnostic recommendations . Evid Based Med . 2008 ; 13 ( 6 ): 162 – 163 . PubMed Google Scholar
Author contributions
D. G. Leo: Conceptualized the study and participated in its design, Developed the study protocol, Designed the parents' and children's questionnaires, Involved in the data analysis from the patients'/parents’ questionnaires, Involved in developing the Delphi survey, Wrote the first draft of the manuscript.
A. Russell: Will be involved in recruiting participants to complete the questionnaires.
A. Bridgens: Will be involved in recruiting participants to complete the questionnaires.
D. C. Perry: Developed the study protocol, Contributed to editing the manuscript.
D. M. Eastwood: Conceptualized the study and participated in its design, Developed the study protocol, Will be involved in recruiting participants to complete the questionnaires, Contributed to writing and editing the manuscript.
Y. Gelfer: Conceptualized the study and participated in its design, Developed the study protocol, Will be involved in recruiting participants to complete the questionnaires, Involved in data analysis from the patients'/parents’ questionnaires, Involved in developing the Delphi survey, Contributed to writing and editing the manuscript.
Funding statement
No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article.
Data sharing
Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.
Trial registration
Core Outcome Measures in Effectiveness Trials (COMET) Initiative, 1738. Registered on 24 November 2020.
Ethical review statement
Consultation with the institutional R&D offices has deemed this project as a “patient service evaluation study” with no requirement for ethical approval. Registration and approval for audit and service evaluation was granted in both hospitals (St George’s University Hospitals NHS Foundation Trust and Great Ormond Street Hospital for Children NHS Foundation Trust).
Supplementary material
The supplementary material includes the patient and parent questionnaires.
© 2021 Author(s) et al. This is an open-access article distributed under the terms of the Creative Commons Attribution Non-Commercial No Derivatives (CC BY-NC-ND 4.0) licence, which permits the copying and redistribution of the work only, and provided the original author and source are credited. See https://creativecommons.org/licenses/by-nc-nd/4.0/.
