Study design

A multicentre prospective cross-sectional clinimetric study was performed between July 2012 and April 2013 at the orthopaedic and surgery departments of three participating hospitals. Ethics approval was obtained from the local medical ethical committee at all three Dutch hospitals (WO 12.064). Written informed consent was obtained from all patients.

Study patients

Patients were eligible for inclusion if they were 18 years of age or older, and were presenting in the emergency department with a displaced distal radial fracture that required reduction. Both conservatively and surgically treated patients were included. Patients were excluded if they 1) had a prior fracture or pathology of the ipsilateral distal radius, 2) had multiple fractures, 3) had cognitive impairment, or 4) were unable to understand the Dutch language.

We aimed to include at least 20 patients to assess the content validity, which is double the number that is required.¹⁷ This group was consulted at between eight and 12 weeks post-fracture.

In addition, we aimed to include at least 100 patients to assess the reliability domain, which is required in order to achieve a study status of high quality according to the COSMIN guidelines.¹⁷ These patients were assessed at between four and eight months post-fracture, and did not participate in the content validation. The type of fracture was scored on radiographs according to AO classification.¹⁸

The PRWE is a self-administered, patient-specific questionnaire that consists of 15 items. It was designed to measure wrist pain and disability in activities of daily living, and consists of two subscales: pain and function. The pain subscale comprises five items, while the subscale function is divided into six specific activities and four usual activities. Both subscales are summed and scored on a ten-point ordinal scale.^7,19 The ‘pain’ subscale score is the sum of the five items. The ‘function’ subscale score is calculated by the sum of the ten items divided by two. The total score of the PRWE is the sum of the scores of both subscales. A score of 100 represents the worst functional score, whereas 0 represents no disability.

In 2004, the PRWE was modified to become the PRWHE (patient-rated wrist/hand evaluation). The PRWHE consists of the same items and scoring system as the PRWE, with minor changes.²⁰ In the PRWHE, the term ‘wrist’ was replaced by ‘wrist/hand’. Also, two aesthetic items, that are not part of the scoring system, were added. Therefore, measurement properties of these two items were not assessed in this study. The PRWHE-Dutch Language Version (PRWHE-DLV) was used in this study.²¹

The DASH is a self-administered questionnaire, developed to evaluate symptoms and physical function of the whole upper limb. It is scored in two components: the main disability/symptom section and two optional sections. The main component of the DASH is a 30-item scale concerning the patient’s health status during the preceding week: 21 items regarding the degree of difficulty in performing certain physical activities; five items pertaining to the severity of pain, activity-related pain, tingling, weakness, and stiffness; and four items concerning the effect that the upper limb problem has had on social activities, work, sleep, and self-image. Each item is scored on a five-point ordinal scale. To calculate the main DASH score, all completed responses are summed and the mean is calculated. The total score is calculated by subtracting 1 from the mean and multiplying by 25 ((n-1)*25). This gives a total score ranging from best to worst on a scale of 0 to 100.²² At least 27 of the 30 items must be completed to calculate a score.

Both optional sections - high performance sport/music and work - consist of four items, scored on a five-point ordinal scale and calculated similarly. However, all five items must be answered, as the percentage of missing items must not exceed 10%. The DASH-Dutch Language Version (DASH-DLV) was used in this study.²³

Assessment of measurement properties: validity

Content validity examines the degree to which the content of a health-related-patient-reported outcome measurement (HR-PROM) is an adequate reflection of the construct to be measured. For all measurement instruments, it is important that the content validity is assessed by experts. For PROMs, patients - particularly representatives of the target population - are the experts. They are the most appropriate assessors of the relevance of the items of the questionnaire.²⁴

Content validity was assessed by phone interview. Eight weeks after incurring a distal radial fracture, patients were asked by phone to participate in the study. When patients agreed to participate, a semi-structured interview was conducted, in which the patient was asked about the functional problems that were the most limiting to them. Each interview started with the standard question: “What are your main limitations in both physical and social functioning after breaking your wrist?” If necessary, examples were given (e.g. carrying heavy objects, personal hygiene).

After the interview, a list was compiled of the functional problems named during the interview. Using the results of all of the interviews, a list of the 20 most common functional problems was created and compared with the items used in the PRWE and the DASH questionnaire.

Content validity is based on judgement and no statistical testing is involved.²⁴ We considered the content validity to be good if 75% of the items of the PRWE and DASH matched the problems mentioned in the interviews.

Assessment of measurement properties: reliability

All patients who were eligible for inclusion received an information letter, between four and eight months after sustaining a distal radial fracture, in which they were asked to participate in this study. Participants completed a web-based questionnaire containing the PRWE and the DASH at home. If participants did not have access to the internet, they could alternatively receive a paper version. Two weeks after completing the questionnaire for the first time, patients received an email or letter in which they were asked to complete the questionnaire for the second time at home. In this two-week interval, no major changes in the health status were expected and immediate recall would be unlikely.

The digital and paper versions of the DASH and PRWE were identical. All items on the web-based questionnaire required a response, thus no unanswered items were expected. If a patient who received a paper version had not answered all of the items, he/she was contacted by phone and asked to provide a response to the omitted items. If the patient did not want to answer any of these items, they were excluded from the study.

The internal consistency is the degree of inter-relatedness among items.³ If items in a scale are summarized in a total score, the items should be sufficiently correlated. This correlation is established by the internal consistency and indicates whether the items seem to measure the same construct.²⁵ If one item measures something else, it will have a lower item-total correlation than the other items. The internal consistency was assessed by using the first measurement of the test-retest reliability.

First, exploratory factor analysis was performed to determine whether the PROM consisted of only one overall dimension, or of more than one dimension. Factor analysis was assessed by calculating eigenvalues. An eigenvalue of one or higher indicated a dimension. The eigenvalues were presented in a scree plot. The relative contribution of different dimensions was judged based on the ‘elbow’ in the scree plot and the percentage of variance.²⁴ If the PROM consisted of two or more dimensions, factor loading was assessed. Factor loading represents the correlation between the items in the PROM and the underlying dimensions. We considered factor loadings of at least 0.50 to be meaningful.²⁶

Internal consistency was determined by calculating the Cronbach’s α for the dimensions found in the factor analysis. If the Cronbach’s α has a value of > 0.70, items are considered sufficiently correlated.²⁷ However, values > 0.95 can indicate that the instrument contains too many items that assess the same underlying construct.²⁸ The QuickDASH is a shortened version of the DASH outcome measure. Instead of 30 items, the QuickDASH uses only 11 items. The Cronbach’s α for the 11items of the QuickDASH was also determined.

The test-retest reliability is the proportion of the total variance in the measurements, due to true differences between patients over time.³ This refers to the degree to which the measurement instrument is free from measurement error. It also estimates the extent to which scores for patients who have not changed are the same for repeated measurements at different timepoints.^3,29 High reliability is important for discriminative purposes if one wants to distinguish outcomes among patients. The test-retest reliability was assessed by calculating the intraclass correlation coefficient (ICC) with a 95% confidence interval (CI). The ICC is a relative parameter and will always have a value between 0 and 1. Higher values represent higher reliability. An ICC of > 0.70 is considered acceptable, > 0.80 is considered good, and > 0.90 is considered excellent.²⁹ Only patients who completed the optional modules of the DASH twice were included for the test-retest assessment of these modules.

The measurement error is the systematic and random error of a patient’s score that is not attributed to true changes in the construct to be measured.³ When the measurement error is low or zero, the difference measured is due to true differences. To express the measurement error, the standard error of the measurement (sem) and the smallest detectable change (SDC) can be used. The sem represents the standard deviation of repeated measures of one individual. The SDC represents the minimal change that must occur in order for the scale to affirm that it is a real change, rather than a measurement error. The sem was calculated from the square root of the variance between the measurements and the error variance of the ICC. For a conventional confidence level of 95%, the SDC was calculated as 1.96 × √2 × sem.²⁴

The presence of floor or ceiling effects may have a negative effect on the quality of the instrument. If patients score primarily in the extremes, the responsiveness may be limited. Floor or ceiling effects were considered to be present if more than 15% of the respondents achieved the minimum or maximum possible score.³⁰ When taking the SDC into account, we should consider floor and ceiling effects more broadly. If a score is closer to the SDC than the maximum or minimum score, a change beyond the measurement error cannot be measured. We also assessed the percentage of patients within the SDC range from both extremes. Floor and ceiling effects were assessed by using the first measurement of the test-retest reliability.

Statistical analysis

Statistical analysis was conducted using the SPSS software, version 18 (IBM Corp., Armonk, New York).

Validity

In total, 35 patients met the inclusion criteria and received a phone call (Fig. 1). Two patients refused participation in the study and 13 patients did not respond. Consequently, 20 patients were included in the study to assess the content validity and were interviewed.

Fig. 1

Flow chart showing patient recruitment and exclusion for the assessment of content validity.

The mean age was 59.30 years (sd 13.61). Most included patients were women (80%). Half of the patients had an AO subtype C1 fracture and most of the patients (65%) received conservative treatment (Table I). The mean time between the fracture and the semi-structured interview was 9.85 weeks (sd 1.98).

A total of 74 problems were mentioned in the semi-structured interviews. All 15 items (100%) of the PRWHE were named in the interviews. Therefore, we considered the content validity of the PRWHE to be good. Only one DASH item, ‘gardening’, was not mentioned in the interviews. Ninety-six per cent of the DASH questions were mentioned in the interviews, which we considered to be good content validity. Both work and sport were mentioned in the semi-structured interview. Therefore, we consider the content validity of both optional modules of the DASH to be good.

Reliability

Of the 297 patients who entered the emergency room at one of the three participating hospitals during the study period with a displaced distal radial fracture requiring reduction, a total of 119 met the inclusion criteria and completed the questionnaire the first time after a mean of 6.15 months (sd 1.00). Most included patients were women (75%). Ten patients did not respond to the second measurement. Therefore, 119 patients were included for the assessment of internal consistency, and 109 patients were included to assess test-retest reliability and measurement error (Fig. 2). The mean time between the first and second assessments was 18.66 days (sd 7.27). No major change in health status took place in any of the patients between the measurement points, so no patients were excluded for that reason. Since not all patients were employed and/or played a sport or instrument, the optional work and sport modules of the DASH were assessed with fewer patients. The exact number of patients is described in Table I, as are their characteristics for each assessment.

Fig. 2

Flowchart showing patient recruitment and exclusion for the assessment of internal consistency, test-retest reliability and measurement error.

There were no missing items in the online completed questionnaires. In total, 23 patients completed the questionnaires on paper. There were no missing items in the paper PRWE. One item was left unanswered in the DASH in two instances. Even after a phone consult, the answer could not be obtained as these patients did not wish to answer the sexuality item of the DASH. Both patients were excluded from further assessment.

The factor analysis was performed for both the PRWE and the DASH. Only one dimension was extracted from the PRWE, which explained 66.26% of the total variance. The ‘elbow’ in the scree plot was seen at the second component (Fig. 3). The Cronbach’s α of the total PRWE was 0.96 (Table II), indicating excellent internal consistency and redundancy. Removing items from the questionnaire did not result in a higher Cronbach’s α. The Cronbach’s α for the 11 items of the QuickDASH when using our data would be 0.91, also indicating an excellent internal consistency and no redundancy.

Fig. 3

Screeplot PRWE. Only one dimension was extracted of the PRWE, which explained 66.26% of the total variance. The “elbow” in the scree plot was seen at the second component.

Five dimensions were extracted from the DASH. Item loading (Table III) showed that the first dimension consisted of items typically asking about strength. The second dimension is more specific on function. The third dimension consisted of items pertaining to pain and disabilities. The fourth and fifth dimensions consisted of two or more specific items.

The first dimension explained 55.71% of the total variance, and the scree plot showed an ‘elbow’ at component two, indicating only one overall meaningful dimension (Fig. 4). Therefore, despite extracting five dimensions, we assessed the Cronbach’s α only for the total DASH and the optional modules.

Fig. 4

Screeplot DASH. Five dimensions were extracted of the DASH. Dimension one explained 55.71% of the total variance and the “elbow” in the scree plot was seen at the second component, insinuating only one overall meaningful dimension.

The Cronbach’s α of the total DASH, the optional work section, and the sport/music section were 0.97, 0.94, and 0.96, respectively, indicating excellent internal consistency. Removing items from the questionnaire did not result in a higher Cronbach’s α.

Table II shows the distribution of the data at the first and second measurements. The ICC of the PRWE was 0.87, which indicates good reliability. The sem was 7.40, with a SDC of 20.51. The ICC of the total DASH, the optional work module, and the sport module were 0.91, 0.87, and 0.87, respectively, indicating excellent reliability for the total DASH and good reliability for the optional modules. The sems were 5.10, 5.08, and 11.18, respectively; the SDCs were 14.12, 14.08, and 30.99, respectively.

No floor or ceiling effects were found in the PRWE or total DASH questionnaires. Floor or ceiling effects were found in both optional modules of the DASH questionnaire (Table IV). When taking the SDC into consideration, more than 15% of the scores of both the PRWE and the total DASH were within the range of the SDC from the lowest possible score (45% and 46%), showing a clear floor effect. Only the score for the optional sport module of the DASH questionnaire was within the range of the SDC from the highest possible score, showing a ceiling effect.