Abstract
Aims
The aim of this study is to define a core outcome set (COS) to allow consistency in outcome reporting amongst studies investigating the management of orthopaedic treatment in children with spinal dysraphism (SD).
Methods
Relevant outcomes will be identified in a four-stage process from both the literature and key stakeholders (patients, their families, and clinical professionals). Previous outcomes used in clinical studies will be identified through a systematic review of the literature, and each outcome will be assigned to one of the five core areas, defined by the Outcome Measures in Rheumatoid Arthritis Clinical Trials (OMERACT). Additional possible outcomes will be identified through consultation with patients affected by SD and their families.
Results
Outcomes identified in these stages will be included in a two-round Delphi process that will involve key stakeholders in the management of SD. A final list including the identified outcomes will then be summarized in a consensus meeting attended by representatives of the key stakeholders groups.
Conclusion
The best approach to provision of orthopaedic care in patients with SD is yet to be decided. The reporting of different outcomes to define success among studies, often based on personal preferences and local culture, has made it difficult to compare the effect of treatments for this condition. The development of a COS for orthopaedic management in SD will enable meaningful reporting and facilitate comparisons in future clinical trials, thereby assisting complex decision-making in the clinical management of these children.
Cite this article: Bone Jt Open 2022;3(1):54–60.
Take home message
The development of a core outcome set for orthopaedic management in spinal dysraphism will enable meaningful reporting and facilitate comparisons in future clinical trials, thereby assisting complex decision-making in the clinical management of these children.
Introduction
Spinal dysraphism (SD) encompasses a spectrum of congenital abnormalities resulting from abnormal cell migration and differentiation of the neural tube during the first trimester of pregnancy. Generally, the lesion can be classified depending on the external appearance: aperta (open, visible lesions) or occulta (closed with no external manifestation). The most common presentations of SD seen in clinical practice are meningocele, myelomenigocele, and lipomyelomenigocele, with myelomeningocele (MMC) accounting for 80% of cases of SD. The incidence of all forms of SD, both open and closed, is 0.5 to eight cases per 1,000 live births, with an increased incidence in developing countries.1,2
The aetiology of SD is multifactorial. Nutritional deficiency is the most common risk factor reported, associated with up to 50% of cases; however, antenatal folic acid supplementation reduces the incidence of MMC significantly. Genetics and ethnicity are also potential aetiological factors.
The severity of the neurological defect is determined by the level and extent of the lesion. The spectrum of neurological abnormalities include hydrocephalus, Chiari malformations, syrinx, and tethered cord, as well as bladder and bowel dysfunction, musculoskeletal weakness, and abnormal sensation.1,3
Weakness leads to muscle imbalance with secondary complications, such as scoliosis, hip dislocation, talipes equinovarus, and torsional abnormalities of the growing bones,4,5 with an associated and potentially severe impairment in terms of walking ability. These children require collaborative multidisciplinary care due to the spectrum of coexisting comorbidities. For this reason, their care is often challenging with no clear consensus among clinical practitioners on best practice for management of the different musculoskeletal problems. This in turn leads to variability in care and guidelines,6,7 which may be exacerbated further by the lack of standardization in outcome reporting in studies reporting different orthopaedic management strategies for SD. This is often the main limitation in defining the rate of success for each clinical intervention.8
To reduce the heterogeneity of reported outcomes, and to allow meaningful comparisons between studies, the Core Outcome Measures in Effectiveness Trials (COMET) Initiative has established guidelines to develop core outcomes sets (COSs),9 a minimum standardized sets of outcomes to be reported in all studies investigating a specific clinical condition. The use of COSs is now well-established in clinical research and practice; however, its implementation has been slower within the orthopaedic community.10-14 A study protocol registered in 2020 aims to develop a COS looking specifically at the outcomes of open fetal surgery for open MMC.15 To date, no COS is available to define the success of interventions in the later management of orthopaedic conditions associated with SD. The standardization of outcomes reporting for studies investigating treatment options will allow meaningful meta-analyses to be made, and will lead to clear guidelines for the orthopaedic management of this condition.
The aim of this study is to define a COS allowing consistency in outcomes reporting among clinical studies investigating the orthopaedic management in SD.
The specific objectives of this study (Figure 1) are to:
-
identify all relevant outcomes reported in previous studies through a systematic review of the literature;
-
identify outcomes relevant to patients and families affected by SD through an interview/questionnaire process;
-
identify outcomes relevant to key healthcare professionals (surgeons, physiotherapists, etc), as well as to patients and their families through a Delphi process; and
-
to hold a final consensus meeting in order to review the outcomes obtained from the previous steps and develop the final list of outcomes to be included in the COS.
Fig. 1
Core outcomes set development process.
Methods
The systematic review aims to identify the primary and secondary outcomes that have been reported in previous studies investigating both surgical and non-surgical interventions to address orthopaedic outcomes in SD. Studies of all methodologies will be included, with exclusion of case series/case reports, papers reporting a surgical technique, intrauterine operations, neurosurgical or urological surgeries. Inclusion criteria, based on the PICO (Population, Intervention, Comparator, Outcome), are:
-
population (patients aged 0 to 18 years diagnosed with SD with related orthopaedic involvement (i.e. deformity, malrotation, foot deformity, hip deformity, knee deformity, scoliosis, spine deformity));
-
intervention (any orthopaedic treatment);
-
comparator; and
-
outcome(s) (any).
Studies with human participants conducted in any setting or country will be included where a published version in English is available. The systematic review aims to generate a list of outcomes measures used in the current literature.
Selection of studies
The databases searched to identify eligible studies will include NICE Healthcare Databases Advanced Search, MEDLINE, Scopus, and the Cochrane Central Register of Controlled Trials (CENTRAL). The search strategy will include MeSH terms and synonyms for the different key terms, which will be combined with Boolean operators, proximity operators, truncations, and wildcards. The databases will be searched from the date of inception to the search date. Grey literature will also be screened.
Eligibility of studies
Two reviewers (DL, GG) will independently assess all of the titles and abstracts against the inclusion and exclusion criteria. Full text of all the manuscripts that match the inclusion criteria or manuscripts in which the abstract does not give enough information to make a clear decision about their inclusion will be obtained. Full-text screening will be completed independently by the two reviewers. Any disagreements will be resolved through consensus or by consulting with senior authors (AB, DME, YG). The whole process will be documented using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram.16
Data extraction
Data extraction sheet (Supplementary Material) will include the following data:
-
year of publication;
-
demographics (age, sex, comorbidities);
-
classification system used;
-
level of involvement;
-
orthopaedic involvement (foot, knee, hip or spine deformity, malrotation);
-
orthopaedic treatment non-surgical (brace, splint);
-
orthopaedic treatment surgical (type of surgery, age of surgery);
-
follow-up time;
-
outcomes reported (patient-reported outcomes measures (PROMs), clinical, radiological); and
-
outcomes measurement tools used.
Data analysis and presentation
The primary and secondary outcomes of eligible studies will be extracted and tabulated with their definition and measurement method(s), and then assigned to one of the five core domains (adverse event, life impact, resource use, pathophysiological manifestation, and death) of the OMERACT framework.17 The treatments reported in the eligible studies will be categorized based on the orthopaedic-related impairment treated, and on the percentage in which the treatment is adopted.
Risk of bias assessment
Two authors (GG, DGL) will independently assess each included study for risk of bias through dedicated assessment tools pertinent to the study design of each paper (e.g. randomized, non-randomized, etc),18,19 and any disagreements will be resolved through consensus or consulting with the senior author (AB, DME, YG). Results of the risk of bias assessment for each study will be reported in dedicated tables.
Parents and patients interview/questionnaire process
The involvement of patients with SD and their family is of primary importance when defining an effective COS that aims to embrace outcomes of relevance to key stakeholders.9 With this in mind, patients and parents will be asked to share their experience of the disease. This process aims to identify possible outcomes that have been overlooked or missed in previous literature, as per the COMET handbook guidelines.20
Participants and questionnaire format
Patients and their families with direct experience of SD will be invited to participate in the key outcomes identification process. This will act as a consultation process, as the small sample size and the recruitment from a single centre will not allow for generalizability of the results. Despite this, the insight of patients and parents will help us to better understand their needs, and to identify possible overlooked outcomes of relevance to them. Patients and parents for whom English is not the first language will be provided with a translation service if needed. Participants will be recruited from the outpatient clinics of two hospitals. A purposive sample of 20 patients treated for SD,10 both males and females aged five to 16 years, and their respective parents will be recruited in this process. The questionnaire for parents will comprise of a series of open-ended questions on their experiences and the impact that SD has on their child’s everyday life, while simultaneously asking to identify possible outcomes in the management of SD.
The questionnaire can be completed as a self-reported questionnaire by the parent, or used as a semi-structured interview schedule to be completed with the researcher, who will use it as a prompt for further discussion. Completion of this questionnaire will take no longer than 15 minutes and will be fully anonymous. No identifiable personal data will be collected via the questionnaire. The researcher will also discuss with the parents the list of outcomes identified during the systematic review, asking for the relevance that they attribute to each listed outcome. Any new outcomes suggested will be added to the Delphi process.
The questionnaire for children will contain questions related to the influence of the condition on their daily living activities, and will be scored using an emoji system already reported in details in previous papers related to COS identification.10,12 A free narrative section will be included at the end of the questionnaire, where children can report the difficulties that they experience during daily activities, aiming to obtain possible insight on outcomes not previously considered in the literature. The child’s questionnaire will take no longer than 15 minutes to complete and will be fully anonymous. No identifiable personal data will be collected by the questionnaire.
Patients and parents will have the option to complete the questionnaires either during their outpatient clinic appointment or at home (and return by email to the study team). Informed consent will be assumed if participants decide to complete the questionnaire. Parents’ contact details will be collected on a separate form with their consent, and used to invite them to participate in the subsequent Delphi survey.
Both the parent and the child’s questionnaire will ask for information that does not differ from routine clinical questions commonly asked during their regular clinical appointments. Consultation with the institutional R&D offices has deemed this a “service evaluation” with no requirement for ethical approval.
Questionnaires analysis
The open-ended structure of the parents’ questionnaire/interview, and of the narrative part of the child’s questionnaire, follows the qualitative approach,21 and will be analyzed using the reflective approach thematic analysis.22 The analysis will be performed by members of the research team with adequate training and experience in qualitative research. The main part of the child’s questionnaire (emoji scores) will be reported using descriptive statistics (i.e. median/range for continuous variables and frequencies/percentages for categorical data), and in tables and figures (as appropriate).
Delphi process
The Delphi process will summarize the list of outcomes obtained from the systematic review and from the interview/questionnaire process, and will seek the feedback of key stakeholders (patients and their families, surgeons, physiotherapists, and other healthcare professionals involved in the management of MMC) in identifying the most relevant outcomes to be reported in future clinical studies. The Delphi method is a process to reach group decision by anonymously surveying a panel of selected participants with relevant knowledge and expertise in the given area, who have equal influence in the scoring procedure.23 The implementation of a Delphi process in COS development is strongly advised by the COMET Initiative guidelines.20
Participants in the Delphi process
Participants in the Delphi process will be key stakeholders with direct experience of SD, or of its management. The list will include international representation. Participants will be recruited from international charities/patients’ organisations and other partner hospitals/universities. Inclusion criteria for the survey will be experience of the condition for patients and parents, and expertise in the treatment of the condition for clinical practitioners. An invitation letter will be sent by email to identified potential participants, which will include all the relevant details of the Delphi process and a link to access the survey. Informed consent will be assumed if participants complete the survey. There is no agreement in the literature on the minimum number of participants required for a Delphi process,23-25 and the sample size is often decided upon case-specific considerations, such as practicality, research area, aim of the study, and time available for analysis.26 The aim for this study is to recruit at least ten participants (representatives) for each of the two stakeholder groups (patients and parents’ group, and clinical practitioners’ group). The size of this sample is similar to other studies that have used the Delphi process in COSs development.10,12
Structure of the Delphi survey
The survey will be divided into two rounds (Figure 1). Participants involved in the study will have three weeks to complete each round. A reminder email will be sent at the end of week two of each round to encourage completion of the survey, and to reduce the dropout rate.
Delphi round one
The electronic data collection form for the first round of the survey will include collecting demographic data (participant name, role, institution, and contact information). Each participant will then be able to review the list of the outcomes included in the survey, and to grade them based on their relevance with a score from 1 to 9 (with 1 to 3 being not relevant, 4 to 6 being important but not critical, and 7 to 9 being extremely relevant). At the end of round one, participants will have the possibility to suggest additional outcomes that they feel may be of relevance.
Analysis of the first round
The analysis of the first round of the survey will summarize the distribution of scores, based on their percentage, and the response rate will be recorded. Outcomes that have reached consensus in or consensus out, will be excluded from round two (see paragraph titled “Definition of consensus” ). Two assessors will review additional outcomes suggested by the participants (if any) to ensure that there is no repetition of outcomes already listed.
Delphi round two
An invitation to the second and last round of the survey will be sent only to participants who have successfully completed round one. Participants that do not respond to round one will be excluded from the second round. In round two, participants will be able to score again the outcomes that did not reach consensus during round one, and will be able to confirm or change their previous score.
Analysis of the second round
Each outcome will be classified as “consensus in”, “consensus out”, or “no consensus” using the consensus criteria defined below (see paragraph titled “Definition of consensus” ). Response rate and drop-out rate from round one will also be recorded. Outcomes that do not reach consensus during rounds one and two will be forwarded to the consensus meeting for final consideration of inclusion/exclusion. Outcomes that will have reached consensus (in or out) during the survey will also be forwarded to the consensus meeting for confirmation.
Consensus meeting
A final consensus meeting will include a selected number of international representatives for each of the two groups (patients/parents, and clinical professionals) either in person or online (through a video conference software) (Figure 1). The consensus meeting will aim to recruit at least 24 participants, as suggested by the COMET handbook, for the consensus meeting structure.20
An independent researcher who is familiar with the consensus procedure, and who will not participate in the voting procedure, will be invited to chair the session. The data from the Delphi process will be presented to the participants in the meeting before the start of the voting procedure. The consensus meeting will briefly consist of four steps:
-
summary of the results of the Delphi process;
-
group discussion;
-
anonymous scoring of each outcome which did not reach a consensus by the participants; and
-
formal endorsement (sign off) of the final core outcomes set by all participants in the consensus meeting.
A report of the consensus meeting will be written up and published. Informed consent will be assumed if participants decide to take part in the final consensus meeting.
Definition of consensus
The Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidelines will be impletemented in the definition of consensus,26 both for the Delphi process and for the final consensus meeting. Following these guidelines, the definition of “consensus in” (which denotes inclusion of the outcome in the COS) is indicated as the agreement by the vast majority ( > 70% of the group) on the “extreme relevance” (7 to 9 points) of the outcome, with only a minority ( < 15% of the group) of participants scoring it as “not relevant” (1 to 3 points). The definition of “consensus out” (which denotes exclusion of the outcomes in the COS) is indicated as the agreement by the vast majority ( > 70% of the group) of the “non-relevance” (1 to 3 points) of the outcome, with only a minority ( < 15% of the group) of participants scoring it as “extremely relevant” (7 to 9 points).
Discussion
Orthopaedic management of patients with SD requires clarity and a standardization of guidelines to obtain the best results. This ambition is currently hindered by the inconsistency in outcome reporting among clinical studies investigating the orthopaedic interventions of SD. The aim of this study is to develop a COS for clinical studies assessing orthopaedic management of SD. The present protocol states the methodology of the study, which follows the recommendations defined by the COMET group.9
A systematic review of the literature will be conducted to identify outcomes that have been reported in previous studies, which will then be summarized in a Delphi survey. Further outcomes will be identified from patients and parents through a structured questionnaire/interview process. Any additional outcomes will be added to the list for the Delphi survey. Key stakeholders (patients and their families, and clinical practitioners) will be engaged in the identification of the core outcomes through completion of the survey, which requires scoring for the importance of each outcome in the list. A subsequent consensus meeting among international representatives of key stakeholders will then discuss and vote on the items from the Delphi survey, and will reach consensus on the outcomes to be include in the final COS.
References
1. Iftikhar W , De Jesus O . Spinal Dysraphism And Myelomeningocele . Treasure Island, Florida, USA : StatPearls Publishing , 2021 . Google Scholar
2. Copp AJ , Stanier P , Greene NDE . Neural tube defects: recent advances, unsolved questions, and controversies . Lancet Neurol . 2013 ; 12 ( 8 ): 799 – 810 . Crossref PubMed Google Scholar
3. Bakaniene I , Prasauskiene A , Vaiciene-Magistris N . Health-related quality of life in children with myelomeningocele: a systematic review of the literature . Child Care Health Dev . 2016 ; 42 ( 5 ): 625 – 643 . Crossref PubMed Google Scholar
4. Swarup I , Talwar D , Howell LJ , Adzick NS , Horn BD . Orthopaedic outcomes of prenatal versus postnatal repair of myelomeningocele . J Pediatr Orthop B . 2020 . Crossref PubMed Google Scholar
5. Thomson JD , Segal LS . Orthopedic management of spina bifida . Dev Disabil Res Rev . 2010 ; 16 ( 1 ): 96 – 103 . Crossref PubMed Google Scholar
6. Conklin MJ , Kishan S , Nanayakkara CB , Rosenfeld SR . Orthopedic guidelines for the care of people with spina bifida . J Pediatr Rehabil Med . 2020 ; 13 ( 4 ): 629 – 635 . Crossref PubMed Google Scholar
7. Dicianno BE , Beierwaltes P , Dosa N , et al. Scientific methodology of the development of the Guidelines for the Care of People with Spina Bifida: An initiative of the Spina Bifida Association . Disabil Health J . 2020 ; 13 ( 2 ): 100816 . Crossref PubMed Google Scholar
8. Macefield RC , Jacobs M , Korfage IJ , et al. Developing core outcomes sets: methods for identifying and including patient-reported outcomes (PROs ). Trials . 2014 ; 15 ( 1 ): 1 – 12 . Crossref PubMed Google Scholar
9. COMET Initiative . Core outcome measures in effectiveness trials . http://www.comet-initiative.org/ ( date last accessed 10 January 2022 ). Crossref PubMed Google Scholar
10. Leo DG , Russell A , Bridgens A , Perry DC , Eastwood DM , Gelfer Y . Development of a core outcome set for idiopathic clubfoot management . Bone Jt Open . 2021 ; 2 ( 4 ): 255 – 260 . Crossref PubMed Google Scholar
11. Crudgington H , Rogers M , Bray L , et al. Core Health Outcomes in Childhood Epilepsy (CHOICE): development of a core outcome set using systematic review methods and a Delphi survey consensus . Epilepsia . 2019 ; 60 ( 5 ): 857 – 871 . Crossref PubMed Google Scholar
12. Leo DG , Jones H , Murphy R , Leong JW , Gambling T , Long AF . The outcomes of Perthes’ disease: Development of a core outcomes set for clinical trials in Perthes’ disease . Bone Joint J . 2020 ; 102 ( 5 ): 611 – 617 . Google Scholar
13. Marson BA , Manning JC , James M , et al. CORE-Kids: a protocol for the development of a core outcome set for childhood fractures . BMJ Open . 2020 ; 10 ( 2 ): e036224 . Crossref PubMed Google Scholar
14. Ollivere BJ , Marson BA , Haddad FS . Getting the right answer: core outcome sets in orthopaedics . Bone Joint J . 2019 ; 101-B ( 3 ): 233 – 235 . Crossref PubMed Google Scholar
15. Altoukhi S , Whitehead CL , Ryan G , et al. Development of a core outcome set for fetal Myelomeningocele (COSMiC): study protocol . Trials . 2020 ; 21 ( 1 ): 732 . Crossref PubMed Google Scholar
16. Page MJ , McKenzie JE , Bossuyt PM , et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews . BMJ . 2021 ; 372 :n 71 . Crossref PubMed Google Scholar
17. Boers M , Kirwan JR , Wells G , et al. Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0 . J Clin Epidemiol . 2014 ; 67 ( 7 ): 745 – 753 . Crossref PubMed Google Scholar
18. Sterne JA , Hernán MA , Reeves BC , et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions . BMJ . 2016 ; 355 : i4919 . Crossref PubMed Google Scholar
19. Higgins JP , Savović J , Page MJ , Elbers RG , Sterne JA . Assessing risk of bias in a randomized trial . In : Higgins JP , Thomas J , Chandler J . Cochrane Handbook for Systematic Reviews of Interventions . 2019 . Google Scholar
20. Williamson PR , Altman DG , Bagley H , et al. The COMET Handbook: version 1.0 . Trials . 2017 ; 18 ( S3 ). Crossref PubMed Google Scholar
21. Popping R . Analyzing open-ended questions by means of text analysis . Bull Methodol Sociol . 2015 ; 128 ( 1 ): 23 – 39 . Google Scholar
22. Braun V , Clarke V . Using thematic analysis in psychology . Qualitative Research in Psychology . 2006 ; 3 ( 2 ): 77 – 101 . Google Scholar
23. Hsu CC , Sandford BA . The delphi technique: making sense of consensus . Pract Assess Res Eval . 2007 ; 12 ( 12 ): 2 – 9 . Google Scholar
24. Green B , Jones M , Hughes D , Williams A . Applying the Delphi technique in a study of GPs’ information requirements . Health Soc Care Community . 1999 ; 7 ( 3 ): 198 – 205 . Google Scholar
25. Keeney S , Hasson F , McKenna HP . A critical review of the Delphi technique as a research methodology for nursing . Int J Nurs Stud . 2001 ; 38 ( 2 ): 195 – 200 . Crossref PubMed Google Scholar
26. Schunemann HJ , Oxman AD , Brozek J , et al. GRADE: assessing the quality of evidence for diagnostic recommendations . BMJ Evid Based Med . 2008 ; 13 ( 6 ): 162 – 163 . PubMed Google Scholar
Author contributions
D. G. Leo: Project administration, Writing – original draft, Writing – review & editing.
G. Green: Conceptualization, Writing – original draft, Writing – review & editing.
D. M. Eastwood: Conceptualization, Writing – review & editing.
A. Bridgens: Conceptualization, Writing – original draft, Writing – review & editing.
Y. Gelfer: Conceptualization, Writing – original draft, Writing – review & editing.
Funding statement
The author(s) received no financial or material support for the research, authorship, and/or publication of this article.
ICMJE COI statement
D. M. Eastwood reports book royalties from Oxford University Press, lecture fees from Biomarin and Kyora Kirin, and being president elect to the British Orthopaedic Association and on the editorial board for The Bone & Joint Journal, all of which are unrelated to this article.
Ethical review statement
Consultation with the institutional R&D offices has deemed this project as a “service evaluation study” with no requirement for ethical approval. Registration and approval for audit and service evaluation was granted in both hospitals (St George’s University Hospitals NHS Foundation Trust and Great Ormond Street Hospital for Children NHS Foundation Trust, London, UK).
Open access funding
The authors report that the access funding for this manuscript was self-funded.
Follow D. G. Leo @DonatoGiuseppe8
Follow D. M. Eastwood @deboraheastwood
Follow Y. Gelfer @yaelgelfer
Supplementary material
Copy of the data extraction sheet.
© 2022 Author(s) et al. This is an open-access article distributed under the terms of the Creative Commons Attribution Non-Commercial No Derivatives (CC BY-NC-ND 4.0) licence, which permits the copying and redistribution of the work only, and provided the original author and source are credited. See https://creativecommons.org/licenses/by-nc-nd/4.0/.
