Eligibility criteria

Within a test-retest design it is important to ensure participants’ disease status is clinically stable at two time points to avoid risk of bias [14]. Therefore a recent change in medication may result in changes of the participant’s hand function and/or physical and mental health status. Patients were screened at the rheumatology outpatient clinics by research nurses and occupational therapists using an eligibility checklist and excluded if they are about to or recently started (during the last 3-months) or increased dose of a biologic or disease modifying anti-rheumatic drugs (DMARDs), low dose oral steroids or received an intra-muscular steroid injection, had cognitive impairment affecting ability to understand and complete the study questionnaire; had another health condition(s) which is moderately to severely affecting their ability to participate in activities and/or hand function; had mental health problem(s) or terminal illness and it was inappropriate to request participation; or were unable to provide informed, written consent. However, people with Fibromyalgia, Osteoarthritis or other conditions that is secondary to RA (e.g. heart disease) were not excluded from the study. Those who were aged 18 years and above: able to read, write and understand English, and diagnosed with RA by a rheumatology consultant were included in the study providing written informed consent was obtained.

Procedures

Cognitive de-briefing interviews

A purposive sample of participants with wide range of demographic characteristics and health status were identified from the past participants of the EDAQ [Evaluation of Daily Activity Questionnaire] study [17, 18] within five rheumatology outpatient departments in the North West of England. They were mailed an invitation letter, participant information sheet, reply form and a FREEPOST envelope. Upon receipt of a positive reply, they were telephoned by an occupational therapist to go through the eligibility checklist, explain what the study involves and provide an opportunity to answer any questions they may have prior to deciding to take part. Consenting participants were booked in to partake in a cognitive de-briefing interview and mailed the Phase 1 questionnaire booklet to complete at home, in their own time no sooner than 1 week prior to the arranged telephone or face-to-face interview. These semi-structured interviews were aimed to ascertain whether the participant found the MAP-Hand items relevant, understandable and comprehensive (i.e. did they adequately reflect the most common daily activity difficulties experienced when using their hands and whether any additional items should be included). Such interviews are recommended during PROM development to ensure participants’ understanding of their content matches the intended use [12]. The interviewer used a five-point rating scale to assess the relevancy and ease of comprehension of each item in the MAP-Hand (relevancy was measured as 1=not relevant to 5=very relevant; and comprehension was measured as 1=very easy to understand to 5=very difficult to understand). Interviews were audio-recorded, transcribed and analysed to identify the need for recommended changes and/or the inclusion of new items. A summary of the findings was reviewed by the Expert Panel to decide whether further changes were required. Following this, a detailed report of the linguistic validation and the cross-cultural adaptation process taken place and the finalised British English MAP-Hand were submitted to the Norwegian developers for review and approval.

Participants

During Phase 2, participants were recruited from rheumatology outpatient clinics within 17 NHS hospitals across the UK. These included both rural and urban populations and a wide mix of socio-demographics.

Data collection

Participants were mailed a Test 1 questionnaire booklet, which included demographic and health data (e.g. age, gender, marital, educational and employment status, disease duration, medication) and following outcome measures: the (i) Health Assessment Questionnaire (HAQ) which includes ability to perform 20 daily activities rated on a 0–3 scale (0=not at all difficult; 3=unable to do) [19] and Upper Limb HAQ (ULHAQ)[7 Upper Limb HAQ items] [20]; the Medical Outcomes Survey 36 item Short-Form 36 (SF-36_v2) from which sub-scale of Physical Function was selected [21, 22]; British English Disabilities of the Arm Shoulder Hand (DASH) which consists of 30 items, measured using five-point Likert scales, 21 daily activity ability items, five symptom items, three participation items, and one of self-image [4]; and Symptom Numeric Rating Scales (NRS) from the EDAQ Part 1, rating hand and wrist pain and arthritis severity on a 10-point scale [17, 18].

Participants were mailed a repeat [Test 2] questionnaire booklet two to 3 weeks later to complete at home to conduct the test-retest reliability of the British English MAP-Hand. Test 2 questionnaire booklet only included brief items on basic demographics (i.e. date of birth, gender and postcode); two single items about current health status and functioning (i.e. “Considering all the ways that your condition affects you, how have you been over the past month?” and “Overall, how much your arthritis is troubling you now compared to when you last completed this questionnaire a few weeks ago?”) and the British English MAP-Hand for repeat testing (Additional file 2).

Statistical analysis

Mokken scaling

The Mokken scale is a non-parametric probabilistic model that utilises Loevinger’s H coefficient to determine the ‘scalability’ of a set of items. ‘H’ is a measure of the degree to which the score is able to discriminate between persons in the given sample [27]. It has been argued that Mokken scaling is a natural starting point for item analysis, and it is used here in that context, to identify if any items from the MAP-Hand display a level of discrimination inconsistent with the expectations of the Rasch model, as represented by low values (<0.3) of H. In the present study Mokken scaling is examined through the msp procedure in STATA 13 [28].

Construct validity

The Rasch model is widely applied to PROMs to ascertain if a quantitative structure is present for the domain(s) measured [29]. A practical realisation of additive conjoint measurement, where data are shown to meet the model expectations, it allows the transformation of ordinal data into an interval level latent estimate [30–34]. The model expectations are associated with a series of assumptions, or requirements, including the stochastic ordering of items (or fit), unidimensionality and local independence [35]. Fit is evaluated by several fit statistics, including chi-square statistics for items and in total (which should be non-significant, Bonferroni adjusted), standardised item and person residuals (within a range±2.5), and summary residuals with a mean of zero and standard deviation of one where data have perfect fit to the model. Local response dependency can be examined through the residual correlation matrix [36]. When the local independence assumption is violated, items can be grouped into ‘testlets’ which absorb the dependency [37]. When data are made into testlets, this delivers a bi-factor solution for the latent estimate, where any unique non-error variance is discarded [38].

A post-hoc test of unidimensionality was also undertaken following the approach described by Smith [39]. Finally, within the Rasch model framework, emphasis is placed upon the invariance of comparisons between groups, such that at the same level of the trait being measured (e.g. hand function), the probability of response to an item should be equal across groups, otherwise Differential Item Functioning (DIF) is present and will require adjustment [40, 41]. Consequently the process of fitting data to the Rasch model, widely referred to as Rasch analysis, consists of a series of tests related to the assumptions of the model, and adjustments to accommodate deviations from those expectations. This process, in relation to the measurement of health outcomes, is described in detail elsewhere [42].

In the current application, emphasis is placed upon fit to the model expectations, and invariance by contextual group, in this case by age, gender, employment and marital status, duration of disease and magnitude of disability as expressed by the HAQ. The analysis used the RUMM2030 software utilising the partial credit parameterisation of the Rasch model [43, 44].

Concurrent validity

The MAP-Hand scores were compared with comparative health measures, specifically the HAQ-20 and ULHAQ [19]; SF-36_v2 (Physical Functioning Score Norm-Based; General Health and Physical Component) [21, 22]; British English DASH [4]; numeric rating scales of hand and wrist pain (i.e. pain in the hand and wrist past week) and arthritis severity (i.e. effect of arthritis in the past month; pain when resting; pain when moving) [17]. Concurrent validity was measured using Spearman’s correlations between the MAP-Hand and these comparative health measures.

Reliability.

Internal consistency was measured using Cronbach’s Alpha (α) and the Person Separation Index (PSI) which, should the data have a normal distribution, is equivalent to Cronbach’s Alpha [45].

Phase 2: Psychometric testing

In Phase 2, 340 participants completed the Test 1 questionnaire and re-test was completed by the 80% (n =273) of the responders. The recruitment progress is summarised in Fig. 1 and the participants’ socio-demographic and health characteristics are detailed in Table ​Table11.

Open in a separate window

Fig. 1

MAPHAND Recruitment & Study Progress Flow Diagram (Phase 2)

Construct validity (Mokken and Rasch models)

A CFA failed to support the unidimensional structure of the 18-item set of the British English MAP-Hand (Chi-Square 236.0 (df 120; p <0.001). RMSEA was 0.53 (90%CI:0.43–0.63); CFI 0.995; TLI 0.994. Mokken scaling suggested that all 18 items showed a probabilistic ordering with a moderate scaling level of 0.61. Initial fit of the 18 items of the MAP-Hand to the Rasch model showed some misfit to the model (Table 2, Analysis 1), and a significant breach of the unidimensionality assumption. DIF was largely absent across all contextual factors, but was present for gender for the item 3 ‘ tying shoelaces’, the item 7 ‘opening screw top bottles’ and the item 18 ‘carrying heavy objects’. At any level of hand function, males were more likely to score higher (worse) than females for tying shoelaces, and females were more likely to score higher than males with opening screw top bottles. Reliability was high, but possibly inflated by the local dependency.

Table 2

Rasch Analysis of the MAP-HAND (n =340)

	Description	Chi- Square*	Df	P	Residual Item		Residual Person		Person Separation Index [PSI] Reliability	% tests >5%	95%CI	N
					Mean	SD	Mean	SD
1	MAP-HAND 18	113.17	72	0.001	−0.4073	1.4939	−0.2936	1.1725	0.95	13.4	11.0–15.7	340
2	Four testlet version	19.81	16	0.229	−0.197	2.6078	−0.4561	1.0624	0.92	5.36	3.0–7.8	340
3	Two testlet version	3.0	8	0.934	0.039	1.1792	−0.5832	0.8457	0.92	4.35	1.9–6.8	340
	Ideal Values			>0.05*		<1.4		<1.4	>0.70	<5.0	LCI<5.0

Open in a separate window

*Bonferroni Adjusted

Clusters of locally dependent items could be observed. For example, button, socks and laces, any form of opening jars or cans and carrying bags or heavy objects. Consequently four testlets were formed from the item set and the data refitted to the model. Here fit was much improved and the unidimensionality assumption held (Analysis 2). The average latent correlation between the four testlets was 0.91, and the amount of common non-error variance in the latent estimate was 0.97, meaning that just 3% of the non-error variance was discarded. This suggests the earlier breach of the unidimensionality assumption was caused by clusters of locally dependent items. Nevertheless, some gender DIF persisted. As earlier it was noted that some items favoured males, and others favoured females, the testlets were further merged in to pairs where opposite bias existed. This resulted in perfect fit to the model, and no DIF (Analysis 3). The scale and patients were slightly off-target in that the latter were more able (less difficulties) than the average of the scale (Fig. 2). However, the floor effect was minimal (5.6%). Table 3 provides the ordinal raw score- interval scale transformation.

Open in a separate window

Fig. 2

Distribution of persons-item threshold

A significant gradient of the transformed metric is seen across groups of functional limitation as defined by the HAQ (Table 4, Fig. 3) (ANOVA F=217.1; p <0.001). Females also showed more limitations in hand function than males (t-test; t =3.1; p =0.002). Duration of disease also showed a significant difference, mainly due to the group with duration over 21 years (ANOVA post-hoc tests). There was no significant difference by age group (ANOVA F=0.254; p 0.851).

Open in a separate window

Fig. 3

Boxplot of MAP-hand Metric for HAQ groups (n =340)

Concurrent validity

MAP-HAND correlated strongly with HAQ20 (r_s =.88), ULHAQ (r_s =.91), SF-36v2 (PF) Score (r_s =−.80) and DASH (r_s =.93), indicating strong concurrent validity (Table 5).

Internal consistency (reliability)

The Map-Hand showed a high Person Separation Index reliability (PSI), even after adjustment for local dependency (PSI range: 0.95–0.92). Reliability measured by Cronbach Alpha (α) was also excellent (α=0.96).

Test-retest reliability

Test-retest reliability was good: at item-level linear weighted kappa scores were good (range 0.61–0.75); at scale level, the ICC (2,1) score was 0.96 (95% CI 0.94, 0.97).

Measurement error

The SEM was 1.44 and the MDC₉₅ score was 3.99. There was no floor or ceiling effects present.

Implications for clinical and research practice

The British English MAP-Hand correlated highly with the HAQ20 and ULHAQ [4]. Although these outcomes are measuring similar constructs, the MAP-Hand was developed using patient generated items and this is reflected in the way in which the functional limitation is defined and scores are calculated. For instance, the hand activity performance is measured in the MAP-Hand at a person-level functioning (activity) [52] with or without the use of aids and adaptations (i.e. gadgets). This means that the MAP-Hand functional limitation score does not increase when the responder appraise their ability to do an activity such as “Writing by hand” (item 15) as “No difficulty” due to their use of pen grip to reduce pain and ease function. People with RA are increasingly encouraged to self-manage their condition, which includes adherence to joint protection advice, that may require behavioural modifications and the use of aids and environmental adaptations [53]. In the hands PIP, thumb interphalangeal (IP) and Distal Interphalangeal (DIP) joints are most involved in executing tasks, hence the use of gadgets such as adapted cutlery, can and bottle opener, zip puller and button hooks are recommended by occupational therapists to encourage joint protection and decrease dependency on others to help with such activities (i.e. the need for help and assistance), decrease pain and prevent joint damage. Therefore use of aids and environmental adaptations are viewed as enablers of function and independence, rather than a marker of disability. Nevertheless, the scoring system used within the HAQ [21, 22] increases with the use of special device by 1 point, if the patient ‘needs’ help from others to do an activity by 2 points and if the patient usually needs both a special device and help from another person by 3 points [54]. Moreover, permanent adaptations of the person’s environment (e.g. changing faucets in the bedroom or kitchen, using Velcro closures on clothing) are also counted as aids and devices [54]. This means, even if the responder has chosen to appraise their ability to do an activity as “without any difficulty” [scored 0] due to their use of aids and adaptations to enable function, their disability score will increase when they disclose the use of aids to help their functioning, unlike how the MAP-Hand is scored.

Another comparative scale that is commonly used in clinical and research practice and recently culturally validated for use in the UK is DASH [4]. DASH scores were also highly correlated with the MAP-Hand scores in this study (Table 6). DASH is a considerably larger, comprehensive scale of upper limb function, which also includes optional modules for assessing upper limb performance at work (WORKDASH) and measuring abilities and symptoms of athletes and performing artists (SPAMDASH). The optional modules are scored separately [4]. As the MAP-Hand, the DASH scores are calculated based on the responder’s ability, regardless of how they do the task. However, unlike the MAP-Hand the DASH includes items measuring both activity limitation (person-level) and participation restriction (societal-level) and as well as the activities performed using hands, the arm and shoulder function is also measured. Although DASH is a comprehensive assessment including 30 items, QUICKDASH is available as a shorter version and consists of 11 items (6 daily activity ability; two symptoms (pain and tingling); and three participation) [4, 55]. Therefore, although the MAP-Hand and the DASH questionnaires appear to measure the same construct at a first glance, their remits differ at conceptual and measurement level and clinicians and researchers should take these differences into consideration if they are having to choose the use of one measure over the other.

Statistical analysis

In this study the unidimensionality of the MAP-Hand was challenged by the CFA, but supported by the Rasch analysis. In both cases, substantial adjustments had to be made to accommodate the local dependency of the item set. The clusters of locally dependent items made clinical sense, grouping items with similar functional requirements, such as opening jars or cans.

Differential item function may also have contributed to the disturbance of dimensionality. The presence of DIF is not uncommon in health status measures of functioning [56]. The fact that, at any level of hand function males had more difficulty tying shoelaces may simply reflect that women are less likely to wear shoes with laces. Also that at any level of hand function, women have more problem opening jars may simply be a function of men having stronger grip. However at the scale level, the item DIF was cancelled out and, so as long as all 18 items are administered, the total score should remain unaffected.

In the analysis of the local independence, assumption took prominence, which was not reported in the original validation. This can inflate reliability, although in this case only marginally, and the level of reliability remained high, consistent with individual use. By using a testlet solution to absorb the local dependency, a satisfactory fit was achieved. The metric conversion that follows good fit will allow for an appropriate calculation of change scores, as well as aspects such as the minimal important difference, the calculations of which are invalid on ordinal scales [57].

The authors wish to thank all the study participants; Robert Peet and Kate Woodward-Nut, Centre for Health Sciences, University of Salford, for assistance with data collection and data entry; and all the Principal Investigators, rheumatology consultants, rheumatology and research nurses and occupational therapists assisting with participant recruitment and study support at the participating sites: Prof Terry O’Neill, Ann McGovern, Jennifer Green, Angharad Walker (Salford Royal Hospital); Prof Ian Bruce, Lindsey Barnes, Elizabeth Beswick, Sarah Evans (Manchester Royal Infirmary); Dr. Leena Dass, Dr. Sophia Naz, Lorraine Lock (North Manchester General Hospital); Dr. Chris Deighton, Alison Booth, Jo Morris (Royal Derby Hospital); Prof David Walsh, Debbie Wilson, Jayne Smith (Kings Mill Hospital, Sherwood Forest Hospitals NHS Foundation Trust); Dr. Chetan Mukhtyer, Loretta Dean, Susan Rowell (Norfolk and Norwich Hospitals); Dr. Bela Szenbenyi, Carol Gray (Diana Princess of Wales, Grimsby); Dr. Mike Green, Anne Gill, Lisa Carr (York Hospital); Dr. Kirsten Mackay, Julie Easterbrook, Liz Burnett (Torbay Hospital); Dr. Mike Green, Alison Miernik, Rachel Bailey-Hague (Harrogate District Hospital); Dr. Atheer Al-Ansari, Jayne Edwards, Julia Nicholas (Robert Jones & Agnes Hunt Hospital, Oswestry); Dr. Wendy Holden, Janet Cushnaghan, Angie Dempster, Hayley Paterson (Basingstoke and North Hampshire Hospital); Mr. David Johnson, Lindsey Barber, Jan Smith (Stepping Hill Hospital); Dr. Karen Douglas, Lucy Kadiki, Chitra Ramful, Daljit Kaur (Russell Hall Hospital, Dudley); Dr. Anca Ghiurlic, Christine Graver (Royal Hampshire Hospital, Winchester); Dr. Frank McKenna, Jane McConiffe (Trafford Hospitals); Dr. Sophia Naz and Lorraine Lock (Fairfield Hospital).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.