Validation of the Comprehensive Arm Prosthesis and Rehabilitation Outcomes Questionnaire

Purpose: To develop and psychometrically evaluate the Comprehensive Arm Prosthesis and Rehabilitation Outcomes Questionnaire (CAPROQ), a 28-item, self-report measure of three key facets associated with successful rehabilitation (perceived function, satisfaction, and pain) designed specifically for the adult upper limb loss (ULL) population. Materials and Methods: Using a national sample of adult ULL patients (N=240), factor structure, internal consistency, convergent/concurrent validity, and known group validity of the total CAPROQ score and three subscale scores were evaluated. Results: Confirmatory factor analysis indicated adequate-to-strong factor loading on each subscale: satisfaction (.623-.913), perceived function (.572-.860) and pain (.422-.834). Internal consistencies for the total measure and measure subscales were good-to-excellent (.89-.95) and convergent validity indicated moderate-to-strong statistically significant associations between the CAPROQ subscales and relevant measures. Concurrent validity showed moderate associations between CAPROQ total score, prosthetic wear time, and psychosocial adjustment scores. Known group validity indicated significant differences on CAPROQ total score between initial and definitive fitting stages (p=.012). Conclusion: Psychometric evaluation indicated that the CAPROQ and CAPROQ subscales were structurally sound, internally consistent, and demonstrated convergent validity with currently used assessments of perceived functioning, satisfaction, and pain. CAPROQ is needed for guiding individual patient care, improving care models and future prosthesis selection and development.


Introduction
Improving treatment outcomes for the upper limb loss (ULL) community, including reducing prosthesis abandonment, hinge on the development and use of reliable, valid measures that comprehensively assess prosthetic rehabilitation, with strong consideration given to factors identified as central to success for this population, namely comfortable wear time and positive psychosocial adjustment 1,2 . The need for better outcome measures to guide clinical practice and product development for the ULL community is well-documented [3][4][5][6][7][8] . Although some of the assessments for prosthesis wearers are limited for use only for specific ULL subpopulations (e.g.., pediatric), others have insufficient evidence of psychometric validity, or simply do not address the unique needs of those with ULL, as they were created for broader upper limb (UL) injury populations 5,6,[9][10][11] . This results in clinicians and researchers having limited options for evaluation to guide patient care and industry development, often forcing the utilization of measures not designed to address the distinctive issues of ULL and prosthesis function. Despite these known challenges, progress toward measure development has been modest 5,12,13 .
Generally, outcome measures for prosthesis users evaluate either performance with a prosthesis or prosthesis user's perceptions of factors related to their prosthesis and rehabilitation. Among others, the Assessment of Capacity for Myoelectric Control (ACMC) 14 , the Capacity Assessment of Prosthetic Performance for the Upper Limb (CAPPFUL) 15,16 , and the Activities Measure for Upper Limb Amputees (AM-ULA) 17 have been developed to assess performance. Patient report outcome measures provide a critical complement to performance measures by enabling insight into the individual's experience and perceptions. A few self-report outcome measures for adults who use UL prostheses have been reported in the literature; however, each has notable limitations that may undermine their utility within the ULL population. For example, the Orthotic and Prosthetic User Survey -Upper Extremity Functional Status (OPUS-UEFS) is a validated measure of prosthesis use and perceived function, evaluating the ease with which adults with ULL perform 28 activities of daily living and whether they use a prosthesis to perform each task 18 . However, prosthesis use is not incorporated into scoring and extant research has recommended item reduction, as well as need for further examination of validity 5,19 . In addition, the OPUS-UEFS does not address critical factors linked to primary metrics of successful rehabilitation such as ability to wear prosthesis comfortably as needed, satisfaction with UL prosthetic rehabilitation and post amputation pain 2,20 . Finally, because the 28 tasks selected focus mostly on one handed activities, which can be completed with the intact hand only, the OPUS-UEFS does not necessarily detect perceived ability with a prosthesis or a change in skill with use of a prosthesis.
Other patient report measures with validation evidence for use in UL prosthetic rehabilitation include the Trinity Amputation and Prosthesis Experience Scales--Revised (TAPES-R) [21][22][23] , the Disabilities of Arm, Shoulder and Hand (DASH) questionnaire [24][25][26] , as well as the QuickDASH 12,27 . However, neither the DASH nor QuickDASH are designed specifically for the ULL and therefore do not capture information salient to successful prosthetic rehabilitation. The TAPES-R is an amputation/prosthesis-specific measure but was primarily designed for use with lower limb amputees. While it examines satisfaction, pain, and psychosocial adjustment, it lacks a singular metric of successful prosthetic rehabilitation. Importantly, all require further psychometric validation for the ULL population, including the need for assessment in larger, representative samples with adequate statistical power 4,23,25,26 .

Comprehensive Arm Prosthesis and Rehabilitation Outcomes Questionnaire (CAPROQ) Background
Research points to three critical facets of successful prosthetic rehabilitation outcomes: satisfaction with the device and rehabilitation, perception of functional abilities with that prosthesis, and post amputation pain, including pain associated with utilization of the device 2,28,29 . Further corroboration comes from a recent study conducted with over 800 veterans which found that among reasons for abandonment, notable issues were (1) lack of function, (2) factors related to satisfaction including cosmesis, lack of intuitive function, lack of reliability, "too much fuss" and (3) factors indicative of pain including comfort, weight and fatigue with use 30 . Indeed, pain is very commonly reported post UL amputation 31 , yet no measure specifically assesses pain in UL prosthesis users and specifically, the difference in pain when wearing and when not wearing a device. The Brief Pain Inventory (BPI) though extensively used has not been validated nor designed for this population and does not inform about the impact of prosthesis wear on pain 32 . This is an important distinction for the ULL community as wear and use of a prosthesis may in itself impact pain 33 .
The Comprehensive Arm Prosthesis and Rehabilitation Outcomes Questionnaire (CAPROQ) was developed to address these three key factors, providing both usable subscales and a single patient report metric of prosthetic rehabilitation outcomes based on subsection scores. More specifically, CAPROQ distinguishes itself in evaluating not only satisfaction with a prosthesis, but also satisfaction with broader elements of rehabilitation, perceived function in tasks that require the affected limb (unilateral and bilateral), and pain -both when wearing and when not wearing a prosthesis. The CAPROQ, therefore, improves upon existing measures in two primary ways. First, the CAPROQ is exclusively designed for the adult ULL community and empirical evaluation of psychometric properties (as described in this paper) is conducted with a nationwide, large, and representative sample from this community [demographics are listed in table 1]. Second, the CAPROQ is designed to comprehensively evaluate three empirically-derived domains related to successful rehabilitation outcomes (i.e., length of time the prosthesis can be worn comfortably and positive psychosocial adjustment) : (1) satisfaction with their prosthesis and rehabilitation (e.g., comfort, appearance, knowledge of how to use it); (2) perceived function (e.g., ability to perform self-care tasks); and (3) pain both when wearing and when not wearing the prosthesis (e.g., residual limb pain, phantom limb pain). Description of all elements evaluated in these domains can be found in table 2. The CAPROQ aims to comprehensively, yet concisely, evaluate these key patient-reported factors within a single measure to effectively guide individual patient care, assess care model effectiveness, and inform component selection and development for the ULL community. This initial psychometric study will describe the development of the measure and evaluate the factor structure, reliability (i.e., internal consistency), and validity (i.e., concurrent, known group) of the CAPROQ.

Variables
Descriptive Notes. Data presented as n (%) and M ± SD. a Data were missing for 8 participants (3.3%). b Data were missing for 16 participants (6.7%).

Framework Conceptualization and Instrument Development
CAPROQ addresses activity and participation domains as described in the WHO ICF 34 . It also assesses environmental and personal factors related to the assessed health condition (ULL). These additional factors are important to address in this population to obtain a holistic assessment of patient function. The CAPROQ was developed using a cyclical and iterative process as described in the Food and Drug Administration's (FDA) Guidance for Industry on patient reported outcome measures 35 . Patient input and evaluation of the measure throughout its development were critical to this process. Additional important considerations included method of administration, appropriateness of the recall period, response options, patient burden, and administrator burden. The latter two factors are especially salient, as time and healthcare resources must be efficiently and effectively allocated.

Initial Development and Description of Domains
Development of the CAPROQ began with literature review and a conceptual framework generated by subject matter experts (i.e., occupational and physical therapists, UL prosthetic specialists). The need for concise, quantifiable subsections in the areas of satisfaction, perceived function in activities of daily living (ADL), and pain were identified from the extant prosthetic rehabilitation literature 2,28,29 , and questions were subsequently generated by occupational and physical therapists specializing in UL amputation and prosthetic rehabilitation, as well as psychometric researchers, using simple and easy-tounderstand wording (i.e., 8 th grade reading level.)

Revision of Measure
The initial set of 86 questions were administered to a group of 43 ULL patient volunteers. A series of item reduction analyses (e.g., exploratory factor analysis [EFA], reliability analyses evaluating Cronbach's alpha) and an expert panel of occupational and physical therapists aided in narrowing of items to 8 satisfaction, 10 perceived functioning, and 10 pain questions for inclusion in the validation study of the CAPROQ. Item removal was based primarily on two factors: representativeness of items (from EFA) and utility of items. For example, for satisfaction questions, if items were identified as redundant (e.g., satisfaction with "color" and satisfaction with total "appearance"), the broader, more general question was retained. For perceived functioning, items that correlated most strongly with the overall construct were retained, while those showing weaker association were eliminated. For pain, items pertaining primarily to the trunk and affected side were retained.

CAPROQ Overview and Scoring
Satisfaction. The satisfaction questions ask about elements of the prosthesis and rehabilitation. Questions seek to explore patient's perceptions on prosthesis fit, function and esthetics; the prosthetic training they received; and their knowledge of how to operate their device. Sample items include, "How satisfied are you with the functionality of your prosthesis?" and "How satisfied are you with your knowledge of how to use your prosthesis?" Responses to eight items are given on a 10-point, Likert-type scale with descriptive anchors at 0 ("Very Dissatisfied") and 10 ("Very Satisfied"). Higher scores therefore indicate higher levels of overall satisfaction.
Perceived functioning. The perceived functioning questions ask participants about their perceived ability to perform primarily bimanual tasks in various functional planes of movement -at transverse body midline, shoulder height, and foot level. Respondents are asked to rank how easy it is to complete a task based on their performance using only their upper limbs (with or without prosthesis, depending on fitting stage) over the past 30 days. Sample items include, "open a card size envelope" and "zip up a jacket." Responses to 10 items are given on a 10-point, Likert-type scale with descriptive anchors at 0 ("Unable") and 10 ("Very Easy"). Higher scores therefore indicate higher levels of overall perceived functioning.
Pain. The pain questions ask participants about phantom and residual limb pain, as well as neck, back and shoulder pain. Each question asks for pain level (a) when wearing a prosthesis and (b) when not wearing a prosthesis. Sample items include, "phantom pain: pain that originates in the part of the limb that is no longer remaining" and "numbness or pain in your back". Responses to 10 items are given on a 10-point, Likert-type scale with descriptive anchors at 0 ("No Pain") and 10 ("Extreme Pain"). To align with the directionality of scoring of the satisfaction and perceived functioning subscales, scoring is weighted in reverse (e.g., 0 = 10, 1 = 9, etc.). Thus, after reverse scoring, higher scores indicate less pain.
Total CAPROQ score. A singular total CAPROQ score is derived by standardizing each of the CAPROQ subscales (i.e., for equal weighting) and summing these standardized scores together. Higher total scores on the CAPROQ indicate better rehabilitation outcomes and reflect the individual's perceived success with their prosthesis and rehabilitation; specifically, satisfaction with their prosthesis and rehabilitation, greater functional performance, and lesser pain.

Procedures
Participants were recruited through a combination of three methods: national email outreaches to persons with ULL from the researcher's client data base; small group and individual email outreach initiated by multiple site researchers/clinicians across the United States including Southwest, Northwest, Northeast and Midwest regions; and personal discussion at regularly scheduled prosthetic rehabilitation appointments with patients meeting study criteria. The study was approved by the Western Institutional Review Board (WIRB). Research began in July 2018 and concluded in March 2019. Participants had to meet the following inclusion criteria: have an UL amputation at any level, be at any stage of the prosthesis fitting process (initial fitting, definitive fitting, post definitive fitting), have any type of prosthesis (passive, body powered, electric, hybrid and/or activity specific), present with any type of amputation/limb loss etiology (disease, trauma, congenital), be able to understand study directions and content, and be between the ages of 18 and 95. Exclusion criteria included not having an amputation, not having a prosthesis, having any type of cognitive impairment that would preclude understanding of the study directions or content, being under age 18 or over 95. A total of 240 participants meeting provisional eligibility criteria were consecutively recruited and provided with the online survey consisting of the CAPROQ, the DASH, the Brief Pain Inventory Short Form (BPI), and the TAPES-R, presented sequentially in this order. More detailed descriptions of each measure can be found in Appendix A. Sections within the CAPROQ were also presented sequentially, however, questions within each CAPROQ subsection were presented in a randomized order. Of those that initiated the study, 210 individuals meeting inclusion criteria completed the CAPROQ, 201 completed the DASH, 189 completed the BPI, and 169 completed the TAPES-R. Demographics describing the study population can be found in table 1.

Data Analytic Plan
The following goals were established to psychometrically assess the structure, reliability, and validity of the CAPROQ: (1) evaluate the CAPROQ higher-order structural model with confirmatory factor analysis (CFA); (2) evaluate internal consistency of the three CAPROQ subscales and total measure; (3) evaluate concurrent validity via zero-order correlations between the CAPROQ subscales and relevant measure, as well as between the total CAPROQ score and relevant prosthetic rehabilitation outcomes; and (4) known-group validity based on evaluation of the associations between the total CAPROQ score and two stages of prosthetic fitting (i.e., initial and definitive).

Confirming Factor Structure.
A CFA was fitted to confirm the 28-item, higher-order factor structure of the CAPROQ (i.e., item loading on three latent subscales, and those three latent subscales loading on singular, overarching CAPROQ factor). Given that measurement scales on each of the CAPROQ items comprised more than five categories, the data were considered intervally-scaled for analyses. According to Hu and Bentler 36 , models with excellent fit have the following fit statistics: CFI and TLI ≥ 0.95 (0.90-0.94) and RMSEA and SRMR ≤ 0.06 (0.07-0.08).
The parenthetical values listed above indicate adequate-togood model fit. Generally, RMSEA less than 0.06 indicate good fit, whereas RMSEA above that threshold, but less than 0.08, indicates medium but acceptable fit 37 .

Reliability and Validity.
Cronbach's alpha coefficients were used to assess the internal consistency of the CAPROQ subscales and CAPROQ total score. For convergent validity of the subscales, zero-order Pearson correlation coefficients were used to assess associations between the CAPROQ satisfaction subscale and the satisfaction subscales of the TAPES-R; the CAPROQ perceived disability subscale and the total score on the DASH; and the CAPROQ pain subscale and the five primary pain indices on the BPI. For concurrent validity of the total measure, zero-order Pearson correlation coefficient evaluated the association between the CAPROQ total score and wear time (hours of comfortable wear time), TAPES general and social adjustment and TAPES adjustment to limitations and activity restrictions subscales. For known-group validity, an independent-samples t-test evaluated differences between CAPROQ total scores at two distinct fitting stages (i.e., initial versus definitive).

Software and Data Management.
Analyses of factor structure were conducted using R (version 4.0.2) statistical software. Descriptive statistics, reliability, and validity analyses were conducted using SPSS (Version 26.0). Zero-order correlation coefficients (r) and p-values were interpreted for significance and effect size in the convergent and concurrent validity analyses. Assumptions for each statistical analysis were checked and met (e.g., normality). Less than 0.03% of the data were missing. Therefore, maximum likelihood was used to handle missing data.

Confirming Factor Structure
The CFA evidenced adequate-to-good fit to be retained as a reasonable model (CFI = .914; TFI = .905; RMSEA = .074; SRMR = .068). For the three CAPROQ subscales, all items indicated adequate-to-strong factor loading on each of the subscales: satisfaction (.623-.913); perceived functioning (.572-.860), and pain (.422-.834). Factor structure and loadings for the CFA model can be found in table 2. Correlations between subscales on the CAPROQ from the CFA are presented in table 3.

Reliability and Validity
Internal Consistency. Internal consistency was assessed using Cronbach's alpha for each of the three factors on the CAPROQ. Internal consistencies for the CAPROQ subscales were good-to-excellent, with alphas on the subscales ranging from .89 to .95. Internal consistency of the total measure was excellent with an alpha of .92. See table 3 for full results of internal consistency coefficients.

Convergent Validity (Subscales).
Results of the convergent validity analyses for the CAPROQ satisfaction subscale indicated moderate-to-strong statistically significant associations between the CAPROQ and all satisfaction subscales on the TAPES-R, including esthetic satisfaction (r = .68) and functional satisfaction (r = .80). Results of the convergent validity analyses for the CAPROQ perceived disability subscale indicated strong statistically significant association between the CAPROQ and the DASH total score (r = -.61). Lastly, results of the convergent validity analyses for the CAPROQ pain subscale indicated strong statistically significant association between the CAPROQ and all pain-related indices on the BPI, including worst pain in the past 24 hours (r = -.61), least pain in the past 24 hours (r = -.53), average overall pain (r = -.63), current pain (r = -.61) and perceived pain interference with daily functioning (r = -.63).

Concurrent Validity (Total).
Results of the concurrent validity analyses for the CAPROQ total score indicated a moderate, statistically significant association between the CAPROQ total score and how many hours per day patients reported comfortable wear (r = .31). Results also indicated moderate-to-strong statistically significant associations between the CAPROQ total score and general adjustment (r = .28), social adjustment (r = .27), adjustment to limitations (r = .51), and perceived activity restrictions (r = -.34) related to their prothesis.

Known-Group Validity (Total).
Result of the knowngroup validity analyses indicated that patients in initial stage of fitting (M = 68.08) score significantly lower in the CAPROQ than patients at the later, definitive stage of fitting (M = 78.94), t(45) = 2.62, p = .012).

Discussion
The adult ULL community is a unique population for whom outcome measure development continues to be needed. The importance of appropriate device selection, fit and training in reducing device abandonment and improving rehabilitation outcomes, necessitates developing reliable and valid measures that capture comprehensive data to guide clinical care as well as industry research 2 . Considering the complexity in adequately predicting these critical rehabilitation outcomes (i.e., considerations relating to multiple domains in the WHO classification of function, disability, and health 34 , complete yet concise instruments are required to adequately capture patient experiences 35 . Currently available self-report measures are lacking in either comprehensiveness (i.e., the OPUS-UEFS designed to only assess perceived function and prosthesis use) or were intended for broader patient populations (i.e., the DASH designed for various UL musculoskeletal disorders, the TAPES-R designed primarily for lower limb amputations). Furthermore, validation studies of these self-report outcome measures lack sufficient sample size for strong empirical validation with the ULL population 5,6,23,26 . The CAPROQ, having been studied with over 200 individuals nationwide with UL amputation, fills the need for a reliable and psychometrically valid, multi-faceted assessment that captures a comprehensive yet concise (i.e., 10 items or less for each subscale) reflection of the patient's perceived prosthetic and rehabilitation outcomes in a single metric.
Importantly, psychometric evaluation of the CAPROQ provides evidence for the higher-order (three-factor) structure of the measure; indicates good internal consistency across subscales; demonstrates good convergent validity between subscales and established measures of prosthetic satisfaction, perceived function, and several clinically relevant pain indices (e.g., general pain, pain interference). The total CAPROQ score demonstrates concurrent validity with the critical metric of rehabilitation success (i.e., prothesis-related wear time and psychosocial functioning); and indicates known-group validity between the total CAPROQ score and distinct prosthetic fitting stages.
There are two primary advantages of the CAPROQ over existing measures in the field. First, the CAPROQ was designed specifically for use among adults with ULL, regardless of the individual's prosthetic device fitting stage, UL prosthetic device(s) or level of UL amputation. Second, the CAPROQ is designed to assess three critical domains related to UL prosthetic use in a singular, comprehensive measure with easy-tounderstand questions and straight-forward scoring and interpretation of total scores. This information may be useful in highlighting areas that need improvement through prosthetic modification, prosthetic training, or therapeutic intervention. Furthermore, it may highlight strengths and weaknesses of various prostheses and prosthetic components for prescriptive decisions, provide prosthetic component justification and illuminate future development needs.

Limitations
Findings from the current study should be considered in light of several limitations. First, although items with factor loadings greater than .40 are generally retained [38][39][40] , the phantom and residual limb pain items did have lesser factor loadings than the other items on the pain domain. Future work is needed to replicate and confirm the validity of the CAPROQ factor structure. Second, despite the CAPROQ being designed to evaluate patients prior to prosthesis fitting in areas of perceived function and pain when not wearing a prosthesis, validation of these subsections at this stage of care was not included in this validation study. Lastly, the current study consists entirely of cross-sectional data. As such, further psychometric evaluation utilizing larger samples and longitudinal designs is needed in order to establish testretest reliability, minimal detectable/clinical change, and, eventually, to establish norms on the CAPROQ for different levels of amputation.

Conclusion
The CAPROQ is a patient reported outcome measure designed specifically for the adult ULL community and can be administered throughout the continuum of care to assess patient perception of outcomes related to post amputation rehabilitation and prosthetic intervention. The CAPROQ is designed to balance comprehensiveness (i.e., evaluation of three domains empirically-related to UL rehabilitation outcomes -satisfaction with the prosthesis, perception of function, and pain when wearing and not wearing a prosthesis) and conciseness; indeed, patient and administrator burden is relatively low with average administration time requiring approximately 30 minutes. Initial psychometric evaluation indicates good-to-excellent internal consistency, convergent validity of CAPROQ subscales with other established measures, concurrent validity of the CAPROQ total score with other established measures of successful prosthetic rehabilitation and known group validity between stage of prosthetic fitting. As such, the CAPROQ may provide an excellent complement to performance-based measures (e.g., CAPPFUL 15 ) for capturing a holistic assessment of patients presenting with ULL that may help improve patient outcomes: guide patient care, reduce device abandonment, inform future prosthesis selection and product development.

Acknowledgments
These data and findings have not been previous presented. This research did not receive any specific grant from funding agencies in the public, commercial, or notfor-profit sectors.

Disclosure of Interest
We certify that we have affiliations with or financial involvement (eg, employment, consultancies, honoraria, stock ownership or options, expert testimony, grants and patents received or pending, royalties) with an organization or entity with a financial interest in, or financial conflict with, the subject matter or materials discussed in the manuscript AND all such affiliations and involvements are disclosed. Nathan Kearns received personal fees from Advanced Arm Dynamics, both during the conduct of the study and outside the submitted work. Nathan Kearns was partially funded by the National Institute on Alcohol Abuse and Alcoholism (F31AA027142) and the National Institute on Drug Abuse (T32DA016184). These funding sources had no direct role/influence in the development, analysis, and/or dissemination of this paper. Kerstin Baun, Jennifer Peterson, John Miguelez are salaried employees of Advanced Arm Dynamics, outside the submitted work.