Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Effects of mode of administration (MOA) on the measurement properties of the EORTC QLQ-C30: a randomized study

Effects of mode of administration (MOA) on the measurement properties of the EORTC QLQ-C30: a... Background: While modern electronic data collection methods (e.g., computer touch-screen or web-based) hold much promise, most current studies continue to make use of more traditional data collection techniques, including paper-and-pencil administration and telephone interviews. The present randomized trial investigated the measurement properties of the EORTC QLQ-C30 under three different modes of administration (MOA’s). Methods: A heterogeneous sample of 314 cancer patients undergoing treatment at a specialized treatment center in Amsterdam were randomized to one of three MOA’s for the QLQ-C30: paper-and-pencil at home via the mail, telephone interview, and paper-and-pencil at the hospital clinic. Group differences in internal consistency reliabilities (Cronbach’s alpha coefficient) for the scale scores were compared. Differences in mean scale scores were also compared by means of ANOVA, with adjustment for potential confounders. Results: Only one statistically significant, yet minor, difference in Cronbach’s alpha between the MOA groups was observed for the Role Functioning scale (all 3 alphas >0.80). Significant differences in group means -after adjustment- were found for the Emotional Functioning (EF) scale. Patients completing the written questionnaire at home had significantly lower levels of EF as compared to those interviewed via the telephone; EF scores of those completing the questionnaire at the clinic fell in-between those of the other two groups. These differences, however, were small in magnitude. Conclusions: MOA had little effect on the reliability or the mean scores of the EORTC QLQ-C30, with the possible exception of the EF scale. Background computer skills may preclude the use of written ques- Health-related quality of life (HRQoL) questionnaires tionnaires, whether pencil and paper or computer-based. can be administered using a variety of methods, includ- It may also be sometimes necessary to combine multiple ing face-to-face or telephone interviews, pencil and modes of administration in the same study, for example paper, computer touch-screen, or web-based. However, when conducting longitudinal research or combining not all researchers may have equal access to all modes data from various sources. of administration (MOA). For example, despite the For these reasons, it is important to consider whether attractiveness of high-tech electronic methods, none of the measurement characteristics of various MOA’sare the 107 abstracts cited in PubMed for 2007 concerning equivalent, because, if this is not the case, then it would the EORTC QLC-C30 HRQoL questionnaire reported be difficult to compare outcomes across MOA’swithin having used a computer for data collection. or between studies. Many studies of varying designs, In addition, various MOA may not be equally practical sizes, populations, and instruments have considered this for all respondents. For example, lack of language or issue, with generally similar results [1-9]. Namely, the effects of MOA on questionnaire measurement charac- * Correspondence: n.aaronson@nki.nl teristics are generally not large. However, only two stu- † Contributed equally dies have investigated the effect of MOA on the EORTC Division of Psychosocial Research and Epidemiology, The Netherlands QLQ-C30, one of the most widely used HRQoL Cancer Institute, 121 Plesmanlaan, 1066 CX Amsterdam, The Netherlands © 2010 Gundy and Aaronson; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Gundy and Aaronson Health and Quality of Life Outcomes 2010, 8:35 Page 2 of 7 http://www.hqlo.com/content/8/1/35 questionnaires in oncology [10-15]. In a large (N = 855) characteristics, comorbidity, sociodemographic data, and observational study, Cheung et al. [14] investigated the the EORTC QLQ-C30 questionnaire data collected dur- effect of two MOA’s, in-clinic interview with in-clinic ing the previous (in-clinic) measurement wave at T2. paper-and-pencil, on the measurement properties on 4 multi-item scales of the QLQ-C30. Velikova et al. [15] Procedure used all15scalesofthe QLQ-C30inarandomized, To assess the impact of different MOA’sonthe mea- cross-over study of 149 patients, comparing in-clinic surement performance of the EORTC QLQ-C30, touch-screen with in-clinic paper & pencil administra- patients were randomly assigned (with equal probabil- tion. Despite their differences and limitations, these two ities), during the first measurement wave of the study at studies each found several small, yet statistically signifi- T1, to one of three groups during the third measure- cant, differences in scale mean scores as a function of ment wave at T3: in-clinic written self-administration; MOA’s (approximately 3-7 points on a 100 point scale). telephone-based interviewer-administration, or mailed Both studies flagged the Emotional Functioning Scale as written self-administration. being potentially problematical. The purpose of the current study was to investigate, in Health-related quality of life (HRQoL) assessment a controlled, randomized setting, the measurement char- HRQoL was assessed with the European Organization acteristics of the EORTC QLQ C-30 under a variety of for Research and Treatment of Cancer (EORTC) Quality different, conventional MOA’s. of Life Questionnaire (QLQ-C30 (version 2.0)) [10-13]. It includes 5 functional scales (physical, role, cognitive, Methods emotional, and social), 3 symptom scales (fatigue, nausea Study Sample and vomiting, and pain), 6 single items (dyspnea, insom- The study sample employed in the current analysis was nia, anorexia, constipation, diarrhea, and financial composed of participants in a study conducted by te impact), and 1 global quality of life scale. The question- Velde and colleagues that evaluated various instruments naire employs a one-week time frame and a mix of for HRQoL assessment in oncology [15,16]. dichotomous response categories ("yes/no”), 4-point Likert-type response scales (ranging from “not at all” to Patients “very much”), and 7-point response scales (numbered The patient sample was composed of individuals with a visual analogue scales). The scoring procedures recom- variety of cancer diagnoses (primarily breast, colorectal, mended by the EORTC [13] were used. All scale and and lung) with various disease stages (local, loco-regio- single item scores of the QLQ-C30 were linearly trans- nal, or metastasized) who attended the Netherlands formed to a 0 to 100 scale. For the functioning scales, Cancer Institute/Antoni van Leeuwenhoek Hospital for higher scores represent a better level of functioning; for treatment. The data used in the current analysis were the symptom measures, a higher score corresponds to a collected approximately 4 months after start of radio- or higher level of symptomology. chemotherapy, during the third measurement wave (T3) The QLQ-C30 has been shown to be reliable and valid in a longitudinal study. in a range of patient populations and treatment settings. Exclusion criteria includedalifeexpectancyofless Across a number of studies, internal consistency esti- than 4 months, too ill to participate, participation in a mates (Cronbach’s coefficient a) of the multi-item scales concurrent HRQoL study, less than 18 years of age, and exceeded or approached 0.70 [12]. Test-retest reliability a lack of basic proficiency inDutch.Norestrictions coefficients have been found to range between 0.80 and were made with regard to age or performance status. 0.90 for most multi-item scales and single items [18]. Eligible patients received a full, verbal and written expla- Testsofvalidityhaveshown theQLQ-C30 to be nation of the purpose and procedures of the study. The responsive to meaningful between-group differences study was approved by the local ethics committee, and (e.g., local vs. metastatic disease, active treatment vs. fol- written informed consent was obtained from all partici- low-up) and changes in clinical status over time [10,12]. pating patients. Patient Characteristics Statistical Methods A number of variables, which were possibly relevant for Mean scores and standard deviations for the QLQ-C30 the quality of patient ratings of HRQoL, were measured scales, as well as for the characteristics of the patients for the purpose of describing the sample of patients, as were calculated. The internal consistency of the multi- well as for assessing the quality of the randomization item scales of the QLQ-C30 was assessed by Cronbach’s into three groups. Characteristics of the patients coefficient alpha [19,20]. included: indicators of health (i.e., the Karnofsky Perfor- Differences in scale/item means were tested by means mance Status scale [17]), treatment and disease of analysis of co-variance (ANCOVA), which allowed Gundy and Aaronson Health and Quality of Life Outcomes 2010, 8:35 Page 3 of 7 http://www.hqlo.com/content/8/1/35 adjustment for possible confounders.Toexamine the confounders. For all tests, the type I error (alpha) signif- magnitude of any observed difference between MOA’s, icance level was set at 0.05. mean difference scores between groups were then stan- dardized by dividing them by the pooled standard devia- Results tion, in order to estimate an effect size [21]. Following Sample accrual (Figure 1) Cohen [21], effect sizes of 0.20, 0.50, and 0.80 were con- During the study period, 614 patients who met the elig- sidered small, medium, and large, respectively. Osoba et ibility criteria were invited to participate in the study, of al. [22] determined that a difference of 10 or fewer whom 483 (79%) accepted at T1. Reasons for declining points on the (re-scaled) QLQ-C30 scales could be study participation included: (a) the study was perceived viewed as being “small”. as too emotionally burdensome (n = 54); (b) perceived Levene’s test for the equality of variances between lack of time (n = 22); (c) lack of interest (n = 18); or (d) groups was also calculated. Finally, multiple analysis of being too ill (n = 10). The remaining 29 patients had a (co-)variance provided a multivariate test of differences variety of other reasons. Patients declining participation between groups, with adjustment for possible were, on average, older (mean age 65 years vs. 57 years), Met eligibility requirements (n = 614) Excluded (n = 133) Declined to participate (n =104) Other reasons (n =29) Randomized at T1 (n = 481) Allocated to pencil & Allocated to telephone Allocated to pencil & paper at home interview at home paper in clinic (n = 182) (n =159) (n = 140) Received allocated MOA Received allocated MOA Received allocated MOA at T3 (n =132) at T3 (n =121) at T3 (n = 61) Did not receive allocated Did not receive allocated Did not receive allocated MOA at T3 (n =1) MOA at T3 (n =10) MOA at T3 (n = 50) (declined) (declined) (did not return to clinic) Too ill, died, drop-out, etc. Too ill, died, drop-out, etc. Too ill, died, drop-out, etc. (n=49) (n=28) (n=29) Analyzed (n = 132) Analyzed (n = 121) Analyzed (n = 61) Excluded from analysis Excluded from analysis Excluded from analysis (n = 50) (n = 38) (n = 79) (No valid measurement) (No valid measurement) (No valid measurement) Figure 1 Results of Patient accrual and randomization. Analysis Allocation Enr ollment Gundy and Aaronson Health and Quality of Life Outcomes 2010, 8:35 Page 4 of 7 http://www.hqlo.com/content/8/1/35 were less frequently married (59% vs. 76%), and more Table 1 Patient sample characteristics (n = 314) for 3 MOA groups often had compulsory education only (91% vs. 82%), than those who participated. Individual Paper & pencil Telephone Paper & Characteristics at home (n = 121) pencil Of the 483 patients initially enrolled in the study, and (n = 132) in-clinic randomized at the first assessment point T1, 375 (78%) (n = 61) remained available for the actual measurement at T3, Means (s.d.) Means (s.d.) Means (s.d.) Sig. which was used for the present analysis. The primary Age 56.6(12.9) 57.0(12.0) 54.1(11.5) .30 reasons for patient attrition were severe illness (n =36) KPS 77.4(14.4) 78.1(13.7) 78.0(15.0) .93 or death (n = 35). Patients lost to follow-up were more likely to have metastatic disease, and their KPS was 10 N(%) N(%) N(%) to 30 points lower than patients who continued partici- Sex (%) pation. The average time between Tl and T3 was Male 48(36%) 47(39%) 16(27%) .23 128 days. However, after randomization, 11 patients declined to Marital Status participate in the MOA condition to which they were Single 11(8%) 9(8%) 6(10%) .92 assigned, and 50 patients randomized to the in-clinic Married 99(75%) 97(81%) 44(73%) condition did not attend the hospital for a follow-up Divorced 10(8%) 7(6%) 5(8%) visit that coincided with this -third- assessment point. Widowed 12(9%) 7(6%) 5(8%) These patients were also excluded from further analysis. Education Statistical Power <10 years 70(53%) 69(58%) 32(53%) .21 We determined that the present sample size would be 10-15 years 39(30%) 35(29%) 12(20%) able to detect a “medium” effect size for differences in >15 years 23(17%) 16(13%) 16(27%) means (d = 0.50) between two groups with a power exceeding 90%, (assuming a two-sided test with a signifi- Employed cance of 5%) [21]. Yes 50(38%) 41(35%) 19(32%) .70 Sample characteristics (Tables 1 and 2) Stage of Disease* Characteristics of the patients in each of the three MOA Local/regional 86(66%) 73(61%) 28(47%) .04 groups are presented in Table 1. Pre-test HRQoL mea- surements, taken at T2, are presented in Table 2. Of Treatment** those patients remaining in the study at T3, very few Chemotherapy 56(42%) 50(41%) 44(73%) <.00 data were missing, not exceeding 3% for any of the Radiotherapy 70(53%) 67(55%) 15(25%) QLQ-C30 scales for any of the three conditions (data RT+CT/other 6(5%) 4(3%) 1(2%) not shown). Mainly due to the loss of the 50 patients randomized to the in-clinic condition, there was an Comorbidity imbalance in the number of patients per group, and in Yes 74% 74% 69% .76 the distribution of stage of disease, type of treatment, and several previous QLQ-C30 scale scores between the Primary Site three groups. These 50 dropout-patients differed from Breast 48(36%) 49(41%) 32(53%) .11 the patients remaining in the in-clinic condition primar- Colorectal 38(29%) 31(26%) 10(17%) ily in terms of type of treatment (p < 0.05, after adjust- Lung 36(27%) 29(24%) 9(15%) ment for other predictors). Other 10(8%) 12(10%) 9(15%) *p < 0.05 **p < 0.01 Internal Consistency of the QoL proxy scales (Table 3) Cronbach’salpha’s for the multi-item scales for each Mean QLQ-C30 scale score differences (Table 4) group were generally adequate (i.e., > 0.70) in the large The adjusted means and standard errors of the three majority of cases. The consistent exception was the Cog- MOA groups for each of the 15 QLQ-C30 scales are nitive Functioning scale; something that has been presented in Table 4. After adjustment for the possible observed in many other studies. There was a significant confounders shown in Table 1 and 2, significant group difference between the in-clinic paper-and-pencil and differences were found only for Emotional Functioning the telephone conditions for the Role Functioning (RF) (EF). The telephone condition had the highest EF, and scale, even though this scale performed rather well the paper-and-pencil at-home condition the lowest. The (alpha > = 0.8) for all three conditions. Gundy and Aaronson Health and Quality of Life Outcomes 2010, 8:35 Page 5 of 7 http://www.hqlo.com/content/8/1/35 Table 2 Patient sample characteristics (n = 314) for 3 MOA groups at Pretest (T2) QLQ-C30 Paper & pencil at home Telephone Paper & pencil in-clinic at pre-test (T2) (n = 132) (n = 121) (n = 61) Mean (s.d.) % Mean (s.d.) % Mean (s.d.) % Sig. floor/ceiling floor/ceiling floor/ceiling Physical function 62.0(28.9) 2%/23% 69.9(23.0) 2%/26% 66.8(24.2) 2%/23% .052 Role function 59.2(31.3) 4%/27% 67.9(28.3) 5%/36% 66.9(28.6) 12%/35% .050 Cognitive function* 80.0(18.4) 0%/43% 82.5(18.9) 1%/50% 87.2(14.4) 0%/39% .038 Emotional function 74.4(21.0) 1%/20% 77.7(19.6) 1%/36% 78.6(18.7) 0%/26% .294 Social function 76.5(27.6) 2%/52% 82.8(20.5) 1%/60% 81.7(22.7) 2%/51% .103 GlobalHealth/QoL 61.5(22.0) 1%/6% 67.2(19.5) 1%/12% 66.1(19.6) 0%/8% .078 Fatigue 45.6(26.0) 13%/3% 38.2(23.5) 21%/3% 40.1(26.9) 15%/3% .060 Nausea/vomitig 14.1(21.3) 69%/1% 10.4(19.0) 76%/1% 13.4(18.5) 62%/2% .321 Pain** 30.5(29.9) 30%/3% 24.7(26.2) 45%/4% 14.8(20.4) 46%/0% .001 Dyspnea 20.1(27.9) 55%/4% 18.1(24.0) 55%/0% 18.0(24.0) 51%/2% .785 Insomnia 30.0(33.8) 52%/8% 29.4(31.8) 63%/2% 21.9(28.5) 57%/3% .226 Anorexia 24.2(29.8) 72%/2% 19.1(29.3) 72%/5% 22.2(29.9) 66%/2% .392 Constipation 11.8(23.8) 83%/2% 7.8(19.2) 87%/1% 12.0(21.1) 82%/0% .268 Diarrhea 10.8(21.7) 85%/1% 11.7(22.7) 85%/0% 4.4(13.0) 87%/0% .074 Financial 8.2(19.0) 82%/1% 5.6(13.9) 89%/2% 3.3(11.7) 90%/0% .117 *p < 0.05 **p < 0.01 questionnaire in the same manner as the “paper & pen- Table 3 Cronbach’s alpha’s for multi-item Scales for three MOA groups cil at home” condition. Results indicated that patients in this fourth group had significantly poorer scores for the EORTC QLQ-C30 Paper & pencil Telephone Paper & pencil Multi-item Scales at home (N = 121) in-clinic EF and SL scales as compared to the” telephone” condi- (N = 132) (N = 61) tion, and did not differ from the original paper & pencil Physical function 0.69 0.71 0.69 conditions (data not shown). Role function* 0.87 0.80 0.93 Cognitive function 0.57 0.64 0.57 Miscellaneous statistical tests Emotional function 0.86 0.86 0.84 A Levene test for difference in variances between the Social function 0.78 0.64 0.81 groups was significant for Pain, Appetite loss, and Global health/QoL 0.84 0.84 0.89 Financial Difficulties (p < 0.05). A multivariate analysis Fatigue 0.87 0.87 0.88 of variance (Pillai’s trace/Wilk’s lambda, with adjustment Nausea/vomiting 0.73 0.75 0.64 for confounders) found no significant difference (p = Pain 0.86 0.87 0.79 0.40) between the three groups. (Data not presented.) *p < 0.05 **p < 0.01 Discussion & The significant difference in this comparison is between the telephone In this study we investigated several measurement prop- versus the in-clinic Paper & Pencil condition erties of the EORTC QLQ-C30 questionnaire under var- un-adjusted mean difference between these two condi- ious MOA’s. Despite the widespread use of the EORTC tions was approximately 6 points, the adjusted mean dif- QLQ-C30, only two studies had previously investigated ference being only 5.4 points. The pair-wise Cohen’sd’s this matter. One large observational study considered 4 for the “telephone vs. paper & pencil at home”,the of the QLQ-C30 multi-item scales [14], while the other “paper & pencil at home vs. pencil & paper in-clinic”, study used a randomized, cross-over design, but with a and the “telephone vs. paper & pencil in-clinic” condi- much smaller sample size, and with only two (in-clinic) tions were 0.31, 0.14, 0.19, respectively. These results conditions [15]. qualify the MOA effect for the EF scale as being “small”. The present study of a heterogeneous population of An additional analysis was conducted, adding a fourth 375 cancer patients considered three conditions (at- group of patients to the above analyses of differences home as well as in-clinic) in a randomized, between- between means. This fourth group consisted of the subjects trial. patients who were not available for the in-clinic paper & Remarkably, all three of these studies flag the Emo- pencil condition because they did not return to the tional Functioning (EF) scale as yielding a small, yet sta- clinic at T3. These patients were invited to complete the tistically significant difference as a function of MOA, Gundy and Aaronson Health and Quality of Life Outcomes 2010, 8:35 Page 6 of 7 http://www.hqlo.com/content/8/1/35 Table 4 Adjusted# Means (+s.e.) for three MOA groups EORTC QLQ-C30 Paper & pencil at home Telephone Paper & Pencil in clinic All Scales (N = 132) (N = 121) (N = 61) Mean (s.e.) Mean (s.e.) Mean (s.e.) Sig.# Physical function 68.9(2.9) 67.1(3.0) 65.3(3.6) .47 Role function 61.4(3.3) 62.2(3.3) 60.4(3.9) .86 Cognitive function 86.2(2.7) 86.0(2.8) 84.8(3.3) .87 Emotional function* 74.1(2.7) 79.5(2.8) 75.0(3.3) .04 Social function 79.1(3.1) 79.2(3.2) 76.9(3.8) .76 Global Health/QoL 64.7(2.6) 66.0(2.7) 64.5(3.2) .79 Fatigue 41.0(3.2) 41.9(3.2) 45.3(3.8) .42 Nausea/vomiting 6.9(2.9) 6.2(3.0) 7.5(3.5) .90 Pain 33.6(3.6) 33.1(3.7) 26.0(4.4) .11 Dyspnea 24.4(3.4) 26.4(3.5) 28.6(4.2) .48 Insomnia 30.5(4.3) 22.3(4.4) 27.3(5.2) .06 Anorexia 8.7(3.9) 13.4(4.0) 13.0(4.7) .29 Constipation 4.2(3.1) 4.8(3.1) 1.7(3.7) .61 Diarrhea 6.2(2.5) 6.4(2.6) 5.7(3.1) .97 Financial 10.4(2.9) 8.7(2.9) 5.8(3.5) .30 *p < 0.05 **p < 0.01 # adjusted for covariates in Tables 1 and 2 & overall effect size for Emotional Function scale are 0.16 for overall effect (Cohen’s f, based on partial eta squared) and 0.31, 0.14, 0.19 (Cohen’s d) for the “telephone vs. paper & pencil at home”,the “paper & pencil at home vs. pencil & paper in-clinic”, and the “telephone vs. paper & pencil in-clinic” pairs of conditions, respectively. with patients in paper-and-pencil MOA’s reporting lower prior to assessment. This occurred primarily in the in- levels of emotional functioning. The present study also clinic condition. Almost 50% of the patients allocated to found a small, yet significant difference in Cronbach’s this condition did not return to the clinic in time for alpha for the Role Functioning scale; however, the RF the present study. This differential drop-out (apparently) scale performed quite adequately for all three conditions. lead to group differences in patient characteristics, such We suspect that the slightly lower EF scale scores in as treatment, stage of disease, and pre-randomization paper-and-pencil conditions may be related to the HRQoL measures. However, we believe that adjustment “demand characteristics” associated with different for these patient characteristics in the statistical analyses MOA’s. Specifically, patients, who are encouraged to was largely able to correct for these group differences. react quickly and/or who are required to interact with An additional, sensitivity analysis included these in- an interviewer, may be stimulated to present more clinic dropouts, who were approached via “pencil & socially desirable responses than those patients allowed paper at home”. This analysis re-flagged the EF scale, as to reflect on their level of emotional functioning and well as the SL scale, indicating that the “telephone” whose responses to the questions are not the subject of MOA yielded a more positive result than pencil & paper direct observation. For example, patients are asked in conditions (which did not differ from each other). These the QLQ-C30 whether they are depressed, which is not findings are commensurate with the finding reported a directly observable state, and whose admission might above. be felt as being potentially stigmatizing. A second limitation concerns the use of version 2.0 of Many studies of varying designs, sizes, populations, the EORTC QLQ-C30. There are, namely, slight differ- and instruments have considered the issue of measure- ences with the current version 3.0, involving the number ment characteristics of various MOA, with generally of response categories for the Physical Function scales. similar results. Namely, while various MOA may differ This might slightly limit the generalizability of these in costs, completion rates, etc., the effects of MOA on results to users of version 3.0. questionnaire measurement characteristics are generally of “small to medium” size, if found at all. This would Conclusions suggest that one should exercise caution when mixing In conclusion, the findings of this investigation indicate MOA’s while investigating effects of similar magnitudes. that the 3 modes of administration studied here have lit- A limitation associated with the present investigation tle effect on the internal consistency or the mean concerns the post-randomization dropout of patients responses on the EORTC QLQ-C30 scales. The Gundy and Aaronson Health and Quality of Life Outcomes 2010, 8:35 Page 7 of 7 http://www.hqlo.com/content/8/1/35 11. Osoba D, Aaronson NK, Zee B, et al: Modification of the EORTC QLQ-C30 exception to this generalization is the Emotional Func- (version 2.0) based on content validity and reliability testing in large tioning scale, which exhibited small, yet significant, dif- samples of patients with cancer. Qual Life Res 1997, 6:103-108. ferences between various administration modes. These 12. Aaronson NK, Cull A, Kaasa S, Sprangers MAG: The European Organization for Research and Treatment of Cancer (EORTC) modular approach to results suggest that, with the possible exception of quality of life assessment in oncology: an update. Quality of life and assessment of emotional functioning, there is little rea- pharmacoeconomics in clinical trials Philadelphia: Lippincott-Raven son for concern about the comparison of QLQ-C30 PublishersSpilker B, 2 1996, 179-189. 13. Fayers PM, Aaronson N, Bjordal K, Groenvold M, Curran D, Bottomley A, on results within or across studies as a function of mode of behalf of the EORTC Quality of Life Group: EORTC QLQ-C30 Scoring Manual administration. Brussels: European Organization for Research and Treatment of Cancer 14. Cheung YB, Goh C, Thumboo J, Khoo KS, Wee J: Quality of life scores Acknowledgements differed according to mode of administration in a review of three major The authors would like to thank A. te Velde, and M.A.G. Sprangers for oncology questionnaires. J Clin Epidemiol 2006, 59:185-191. providing access to the data used in the current analyses. The original data 15. Velikova G, Wright EP, Smith AB, Cull A, Gould A, Forman D, Perren T, collection was financially supported by a grant from the Dutch Cancer Sted M, Brown J, Selby PJ: Automated collection of quality-of-life data: a Society. The authors also wish to thank the patients for their willingness to comparisons of paper and computer touch-screen questionnaires. J Clin participate in the study. Some of the results of this study were presented at Oncol 1999, 17:998-1007. the Annual Conference of the International Society for Quality of Life 16. te Velde A, Sprangers M, Aaronson NK: Feasibility, psychometric Research, Montevideo, Uruguay, October 25th, 2008. performance, and stability across modes of administration of the CARES- SF. Annals of Oncology 1996, 7:381-390. Authors’ contributions 17. Karnofsky D, Burchenal J: The clinical evaluation of chemotherapeutic NA conceived of the study, and participated in its design and coordination agents in cancer. Evaluation of Chemotherapeutic Agents New York: and helped to draft the manuscript. CG participated in the design of the Columbia University PressMacLeod C 1949. study, performed the statistical analysis, and drafted the manuscript. All 18. Hjermstad MJ, Fossa SD, Bjordal K, Kaasa S: Test/retest study of the authors read and approved the final manuscript. European Organization for Research and Treatment of Cancer Core Quality-of-Life Questionnaire. J CLIN ONCOL 1995, 13:1249-1254. Competing interests 19. Cronbach LJ: Coefficient alpha and the internal structure of tests. The authors declare that they have no competing interests. Psychometrika 1951, 16:297-334. 20. Lautenschlager GJ: ALPHATST: Testing for differences in coefficient alpha. Received: 10 November 2009 Accepted: 30 March 2010 Appl Psychol Meas 1989, 13:284. Published: 30 March 2010 21. Cohen J: Statistical power analysis for the behavioral sciences Hillsdale, New Yersey: Lawrence Erlbaum Associates 1988. 22. Osoba D, Rodrigues G, Myles J, Zee B, Pater J: Interpreting the Significance References of Changes in Health-Related Quality-of-Life Scores. Journal of Clinical 1. Barry MJ, Fowler FJ, Chang Y, Liss CL, Wilson H, Stek M Jr: The American Oncology 1998, 16(1):139-144. Urological Association symptom index: does mode of administration affect its psychometric properties? J Urol 1995, 154:1056-1059. doi:10.1186/1477-7525-8-35 2. Weinberger M, Oddone EZ, Samsa GP, Landsman PB: Are health-related Cite this article as: Gundy and Aaronson: Effects of mode of quality-of-life measures affected by the mode of administration? J Clin administration (MOA) on the measurement properties of the EORTC Epidemiol 1996, 49:135-140. QLQ-C30: a randomized study. Health and Quality of Life Outcomes 2010 3. Vereecken CA, Maes L: Comparison of a computer-administered and 8:35. paper-and-pencil-administered questionnaire on health and lifestyle behaviors. J Adolesc Health 2006, 38:426-432. 4. Fouladi RT, McCarthy CJ, Moller NP: Paper-and-pencil or online? Evaluating mode effects on measures of emotional functioning and attachment. Assessment 2002, 9:204-215. 5. Rhodes T, Girman CJ, Jacobsen SJ, Guess HA, Hanson KA, Oesterling JE, Lieber MM: Does the mode of questionnaire administration affect the reporting of urinary symptoms? Urology 1995, 46:341-345. 6. Wu AW, Jacobson DL, Berzon RA, Revicki DA, Horst van der C, Fichtenbaum CJ, Saag MS, Lynn L, Hardy D, Feinberg J: The effect of mode of administration on Medical Outcomes Study health ratings and EuroQol scores in AIDS. Quality of Life Research 1997, 6:0. 7. Weinberger M, Nagle B, Hanlon JT, Samsa GP, Schmader K, Landsman PB, Uttech KM, Cowper PA, Cohen HJ, Feussner JR: Assessing health-related quality of life in elderly outpatients: telephone versus face-to-face administration. J Am Geriatr Soc 1994, 41:1295-1299. Submit your next manuscript to BioMed Central 8. Jorngarden A, Wettergen L, von Essen L: Measuring health-related quality of life in adolescents and young adults: Swedish normative data for the and take full advantage of: SF-36 and the HADS, and the influence of age, gender, and method of administration. Health Qual Life Outcomes 2006, 4:91. • Convenient online submission 9. Perkins JJ, Sanson-Fisher RW: An examination of self- and telephone- • Thorough peer review administered modes of administration for the Australian SF-36. J Clin Epidemiol 1998, 51:969-973. • No space constraints or color figure charges 10. Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, • Immediate publication on acceptance Filiberti A, Flechtner H, Fleishman SB, De Haes JC: The European Organization for Research and Treatment of Cancer QLQ-C30: a quality- • Inclusion in PubMed, CAS, Scopus and Google Scholar of-life instrument for use in international clinical trials in oncology. J Natl • Research which is freely available for redistribution Cancer Inst 1993, 85:365-376. Submit your manuscript at www.biomedcentral.com/submit http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Health and Quality of Life Outcomes Springer Journals

Effects of mode of administration (MOA) on the measurement properties of the EORTC QLQ-C30: a randomized study

Loading next page...
 
/lp/springer-journals/effects-of-mode-of-administration-moa-on-the-measurement-properties-of-ZROHfVD0kq

References (29)

Publisher
Springer Journals
Copyright
Copyright © 2010 by Gundy and Aaronson; licensee BioMed Central Ltd.
Subject
Medicine & Public Health; Quality of Life Research; Quality of Life Research
eISSN
1477-7525
DOI
10.1186/1477-7525-8-35
pmid
20353582
Publisher site
See Article on Publisher Site

Abstract

Background: While modern electronic data collection methods (e.g., computer touch-screen or web-based) hold much promise, most current studies continue to make use of more traditional data collection techniques, including paper-and-pencil administration and telephone interviews. The present randomized trial investigated the measurement properties of the EORTC QLQ-C30 under three different modes of administration (MOA’s). Methods: A heterogeneous sample of 314 cancer patients undergoing treatment at a specialized treatment center in Amsterdam were randomized to one of three MOA’s for the QLQ-C30: paper-and-pencil at home via the mail, telephone interview, and paper-and-pencil at the hospital clinic. Group differences in internal consistency reliabilities (Cronbach’s alpha coefficient) for the scale scores were compared. Differences in mean scale scores were also compared by means of ANOVA, with adjustment for potential confounders. Results: Only one statistically significant, yet minor, difference in Cronbach’s alpha between the MOA groups was observed for the Role Functioning scale (all 3 alphas >0.80). Significant differences in group means -after adjustment- were found for the Emotional Functioning (EF) scale. Patients completing the written questionnaire at home had significantly lower levels of EF as compared to those interviewed via the telephone; EF scores of those completing the questionnaire at the clinic fell in-between those of the other two groups. These differences, however, were small in magnitude. Conclusions: MOA had little effect on the reliability or the mean scores of the EORTC QLQ-C30, with the possible exception of the EF scale. Background computer skills may preclude the use of written ques- Health-related quality of life (HRQoL) questionnaires tionnaires, whether pencil and paper or computer-based. can be administered using a variety of methods, includ- It may also be sometimes necessary to combine multiple ing face-to-face or telephone interviews, pencil and modes of administration in the same study, for example paper, computer touch-screen, or web-based. However, when conducting longitudinal research or combining not all researchers may have equal access to all modes data from various sources. of administration (MOA). For example, despite the For these reasons, it is important to consider whether attractiveness of high-tech electronic methods, none of the measurement characteristics of various MOA’sare the 107 abstracts cited in PubMed for 2007 concerning equivalent, because, if this is not the case, then it would the EORTC QLC-C30 HRQoL questionnaire reported be difficult to compare outcomes across MOA’swithin having used a computer for data collection. or between studies. Many studies of varying designs, In addition, various MOA may not be equally practical sizes, populations, and instruments have considered this for all respondents. For example, lack of language or issue, with generally similar results [1-9]. Namely, the effects of MOA on questionnaire measurement charac- * Correspondence: n.aaronson@nki.nl teristics are generally not large. However, only two stu- † Contributed equally dies have investigated the effect of MOA on the EORTC Division of Psychosocial Research and Epidemiology, The Netherlands QLQ-C30, one of the most widely used HRQoL Cancer Institute, 121 Plesmanlaan, 1066 CX Amsterdam, The Netherlands © 2010 Gundy and Aaronson; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Gundy and Aaronson Health and Quality of Life Outcomes 2010, 8:35 Page 2 of 7 http://www.hqlo.com/content/8/1/35 questionnaires in oncology [10-15]. In a large (N = 855) characteristics, comorbidity, sociodemographic data, and observational study, Cheung et al. [14] investigated the the EORTC QLQ-C30 questionnaire data collected dur- effect of two MOA’s, in-clinic interview with in-clinic ing the previous (in-clinic) measurement wave at T2. paper-and-pencil, on the measurement properties on 4 multi-item scales of the QLQ-C30. Velikova et al. [15] Procedure used all15scalesofthe QLQ-C30inarandomized, To assess the impact of different MOA’sonthe mea- cross-over study of 149 patients, comparing in-clinic surement performance of the EORTC QLQ-C30, touch-screen with in-clinic paper & pencil administra- patients were randomly assigned (with equal probabil- tion. Despite their differences and limitations, these two ities), during the first measurement wave of the study at studies each found several small, yet statistically signifi- T1, to one of three groups during the third measure- cant, differences in scale mean scores as a function of ment wave at T3: in-clinic written self-administration; MOA’s (approximately 3-7 points on a 100 point scale). telephone-based interviewer-administration, or mailed Both studies flagged the Emotional Functioning Scale as written self-administration. being potentially problematical. The purpose of the current study was to investigate, in Health-related quality of life (HRQoL) assessment a controlled, randomized setting, the measurement char- HRQoL was assessed with the European Organization acteristics of the EORTC QLQ C-30 under a variety of for Research and Treatment of Cancer (EORTC) Quality different, conventional MOA’s. of Life Questionnaire (QLQ-C30 (version 2.0)) [10-13]. It includes 5 functional scales (physical, role, cognitive, Methods emotional, and social), 3 symptom scales (fatigue, nausea Study Sample and vomiting, and pain), 6 single items (dyspnea, insom- The study sample employed in the current analysis was nia, anorexia, constipation, diarrhea, and financial composed of participants in a study conducted by te impact), and 1 global quality of life scale. The question- Velde and colleagues that evaluated various instruments naire employs a one-week time frame and a mix of for HRQoL assessment in oncology [15,16]. dichotomous response categories ("yes/no”), 4-point Likert-type response scales (ranging from “not at all” to Patients “very much”), and 7-point response scales (numbered The patient sample was composed of individuals with a visual analogue scales). The scoring procedures recom- variety of cancer diagnoses (primarily breast, colorectal, mended by the EORTC [13] were used. All scale and and lung) with various disease stages (local, loco-regio- single item scores of the QLQ-C30 were linearly trans- nal, or metastasized) who attended the Netherlands formed to a 0 to 100 scale. For the functioning scales, Cancer Institute/Antoni van Leeuwenhoek Hospital for higher scores represent a better level of functioning; for treatment. The data used in the current analysis were the symptom measures, a higher score corresponds to a collected approximately 4 months after start of radio- or higher level of symptomology. chemotherapy, during the third measurement wave (T3) The QLQ-C30 has been shown to be reliable and valid in a longitudinal study. in a range of patient populations and treatment settings. Exclusion criteria includedalifeexpectancyofless Across a number of studies, internal consistency esti- than 4 months, too ill to participate, participation in a mates (Cronbach’s coefficient a) of the multi-item scales concurrent HRQoL study, less than 18 years of age, and exceeded or approached 0.70 [12]. Test-retest reliability a lack of basic proficiency inDutch.Norestrictions coefficients have been found to range between 0.80 and were made with regard to age or performance status. 0.90 for most multi-item scales and single items [18]. Eligible patients received a full, verbal and written expla- Testsofvalidityhaveshown theQLQ-C30 to be nation of the purpose and procedures of the study. The responsive to meaningful between-group differences study was approved by the local ethics committee, and (e.g., local vs. metastatic disease, active treatment vs. fol- written informed consent was obtained from all partici- low-up) and changes in clinical status over time [10,12]. pating patients. Patient Characteristics Statistical Methods A number of variables, which were possibly relevant for Mean scores and standard deviations for the QLQ-C30 the quality of patient ratings of HRQoL, were measured scales, as well as for the characteristics of the patients for the purpose of describing the sample of patients, as were calculated. The internal consistency of the multi- well as for assessing the quality of the randomization item scales of the QLQ-C30 was assessed by Cronbach’s into three groups. Characteristics of the patients coefficient alpha [19,20]. included: indicators of health (i.e., the Karnofsky Perfor- Differences in scale/item means were tested by means mance Status scale [17]), treatment and disease of analysis of co-variance (ANCOVA), which allowed Gundy and Aaronson Health and Quality of Life Outcomes 2010, 8:35 Page 3 of 7 http://www.hqlo.com/content/8/1/35 adjustment for possible confounders.Toexamine the confounders. For all tests, the type I error (alpha) signif- magnitude of any observed difference between MOA’s, icance level was set at 0.05. mean difference scores between groups were then stan- dardized by dividing them by the pooled standard devia- Results tion, in order to estimate an effect size [21]. Following Sample accrual (Figure 1) Cohen [21], effect sizes of 0.20, 0.50, and 0.80 were con- During the study period, 614 patients who met the elig- sidered small, medium, and large, respectively. Osoba et ibility criteria were invited to participate in the study, of al. [22] determined that a difference of 10 or fewer whom 483 (79%) accepted at T1. Reasons for declining points on the (re-scaled) QLQ-C30 scales could be study participation included: (a) the study was perceived viewed as being “small”. as too emotionally burdensome (n = 54); (b) perceived Levene’s test for the equality of variances between lack of time (n = 22); (c) lack of interest (n = 18); or (d) groups was also calculated. Finally, multiple analysis of being too ill (n = 10). The remaining 29 patients had a (co-)variance provided a multivariate test of differences variety of other reasons. Patients declining participation between groups, with adjustment for possible were, on average, older (mean age 65 years vs. 57 years), Met eligibility requirements (n = 614) Excluded (n = 133) Declined to participate (n =104) Other reasons (n =29) Randomized at T1 (n = 481) Allocated to pencil & Allocated to telephone Allocated to pencil & paper at home interview at home paper in clinic (n = 182) (n =159) (n = 140) Received allocated MOA Received allocated MOA Received allocated MOA at T3 (n =132) at T3 (n =121) at T3 (n = 61) Did not receive allocated Did not receive allocated Did not receive allocated MOA at T3 (n =1) MOA at T3 (n =10) MOA at T3 (n = 50) (declined) (declined) (did not return to clinic) Too ill, died, drop-out, etc. Too ill, died, drop-out, etc. Too ill, died, drop-out, etc. (n=49) (n=28) (n=29) Analyzed (n = 132) Analyzed (n = 121) Analyzed (n = 61) Excluded from analysis Excluded from analysis Excluded from analysis (n = 50) (n = 38) (n = 79) (No valid measurement) (No valid measurement) (No valid measurement) Figure 1 Results of Patient accrual and randomization. Analysis Allocation Enr ollment Gundy and Aaronson Health and Quality of Life Outcomes 2010, 8:35 Page 4 of 7 http://www.hqlo.com/content/8/1/35 were less frequently married (59% vs. 76%), and more Table 1 Patient sample characteristics (n = 314) for 3 MOA groups often had compulsory education only (91% vs. 82%), than those who participated. Individual Paper & pencil Telephone Paper & Characteristics at home (n = 121) pencil Of the 483 patients initially enrolled in the study, and (n = 132) in-clinic randomized at the first assessment point T1, 375 (78%) (n = 61) remained available for the actual measurement at T3, Means (s.d.) Means (s.d.) Means (s.d.) Sig. which was used for the present analysis. The primary Age 56.6(12.9) 57.0(12.0) 54.1(11.5) .30 reasons for patient attrition were severe illness (n =36) KPS 77.4(14.4) 78.1(13.7) 78.0(15.0) .93 or death (n = 35). Patients lost to follow-up were more likely to have metastatic disease, and their KPS was 10 N(%) N(%) N(%) to 30 points lower than patients who continued partici- Sex (%) pation. The average time between Tl and T3 was Male 48(36%) 47(39%) 16(27%) .23 128 days. However, after randomization, 11 patients declined to Marital Status participate in the MOA condition to which they were Single 11(8%) 9(8%) 6(10%) .92 assigned, and 50 patients randomized to the in-clinic Married 99(75%) 97(81%) 44(73%) condition did not attend the hospital for a follow-up Divorced 10(8%) 7(6%) 5(8%) visit that coincided with this -third- assessment point. Widowed 12(9%) 7(6%) 5(8%) These patients were also excluded from further analysis. Education Statistical Power <10 years 70(53%) 69(58%) 32(53%) .21 We determined that the present sample size would be 10-15 years 39(30%) 35(29%) 12(20%) able to detect a “medium” effect size for differences in >15 years 23(17%) 16(13%) 16(27%) means (d = 0.50) between two groups with a power exceeding 90%, (assuming a two-sided test with a signifi- Employed cance of 5%) [21]. Yes 50(38%) 41(35%) 19(32%) .70 Sample characteristics (Tables 1 and 2) Stage of Disease* Characteristics of the patients in each of the three MOA Local/regional 86(66%) 73(61%) 28(47%) .04 groups are presented in Table 1. Pre-test HRQoL mea- surements, taken at T2, are presented in Table 2. Of Treatment** those patients remaining in the study at T3, very few Chemotherapy 56(42%) 50(41%) 44(73%) <.00 data were missing, not exceeding 3% for any of the Radiotherapy 70(53%) 67(55%) 15(25%) QLQ-C30 scales for any of the three conditions (data RT+CT/other 6(5%) 4(3%) 1(2%) not shown). Mainly due to the loss of the 50 patients randomized to the in-clinic condition, there was an Comorbidity imbalance in the number of patients per group, and in Yes 74% 74% 69% .76 the distribution of stage of disease, type of treatment, and several previous QLQ-C30 scale scores between the Primary Site three groups. These 50 dropout-patients differed from Breast 48(36%) 49(41%) 32(53%) .11 the patients remaining in the in-clinic condition primar- Colorectal 38(29%) 31(26%) 10(17%) ily in terms of type of treatment (p < 0.05, after adjust- Lung 36(27%) 29(24%) 9(15%) ment for other predictors). Other 10(8%) 12(10%) 9(15%) *p < 0.05 **p < 0.01 Internal Consistency of the QoL proxy scales (Table 3) Cronbach’salpha’s for the multi-item scales for each Mean QLQ-C30 scale score differences (Table 4) group were generally adequate (i.e., > 0.70) in the large The adjusted means and standard errors of the three majority of cases. The consistent exception was the Cog- MOA groups for each of the 15 QLQ-C30 scales are nitive Functioning scale; something that has been presented in Table 4. After adjustment for the possible observed in many other studies. There was a significant confounders shown in Table 1 and 2, significant group difference between the in-clinic paper-and-pencil and differences were found only for Emotional Functioning the telephone conditions for the Role Functioning (RF) (EF). The telephone condition had the highest EF, and scale, even though this scale performed rather well the paper-and-pencil at-home condition the lowest. The (alpha > = 0.8) for all three conditions. Gundy and Aaronson Health and Quality of Life Outcomes 2010, 8:35 Page 5 of 7 http://www.hqlo.com/content/8/1/35 Table 2 Patient sample characteristics (n = 314) for 3 MOA groups at Pretest (T2) QLQ-C30 Paper & pencil at home Telephone Paper & pencil in-clinic at pre-test (T2) (n = 132) (n = 121) (n = 61) Mean (s.d.) % Mean (s.d.) % Mean (s.d.) % Sig. floor/ceiling floor/ceiling floor/ceiling Physical function 62.0(28.9) 2%/23% 69.9(23.0) 2%/26% 66.8(24.2) 2%/23% .052 Role function 59.2(31.3) 4%/27% 67.9(28.3) 5%/36% 66.9(28.6) 12%/35% .050 Cognitive function* 80.0(18.4) 0%/43% 82.5(18.9) 1%/50% 87.2(14.4) 0%/39% .038 Emotional function 74.4(21.0) 1%/20% 77.7(19.6) 1%/36% 78.6(18.7) 0%/26% .294 Social function 76.5(27.6) 2%/52% 82.8(20.5) 1%/60% 81.7(22.7) 2%/51% .103 GlobalHealth/QoL 61.5(22.0) 1%/6% 67.2(19.5) 1%/12% 66.1(19.6) 0%/8% .078 Fatigue 45.6(26.0) 13%/3% 38.2(23.5) 21%/3% 40.1(26.9) 15%/3% .060 Nausea/vomitig 14.1(21.3) 69%/1% 10.4(19.0) 76%/1% 13.4(18.5) 62%/2% .321 Pain** 30.5(29.9) 30%/3% 24.7(26.2) 45%/4% 14.8(20.4) 46%/0% .001 Dyspnea 20.1(27.9) 55%/4% 18.1(24.0) 55%/0% 18.0(24.0) 51%/2% .785 Insomnia 30.0(33.8) 52%/8% 29.4(31.8) 63%/2% 21.9(28.5) 57%/3% .226 Anorexia 24.2(29.8) 72%/2% 19.1(29.3) 72%/5% 22.2(29.9) 66%/2% .392 Constipation 11.8(23.8) 83%/2% 7.8(19.2) 87%/1% 12.0(21.1) 82%/0% .268 Diarrhea 10.8(21.7) 85%/1% 11.7(22.7) 85%/0% 4.4(13.0) 87%/0% .074 Financial 8.2(19.0) 82%/1% 5.6(13.9) 89%/2% 3.3(11.7) 90%/0% .117 *p < 0.05 **p < 0.01 questionnaire in the same manner as the “paper & pen- Table 3 Cronbach’s alpha’s for multi-item Scales for three MOA groups cil at home” condition. Results indicated that patients in this fourth group had significantly poorer scores for the EORTC QLQ-C30 Paper & pencil Telephone Paper & pencil Multi-item Scales at home (N = 121) in-clinic EF and SL scales as compared to the” telephone” condi- (N = 132) (N = 61) tion, and did not differ from the original paper & pencil Physical function 0.69 0.71 0.69 conditions (data not shown). Role function* 0.87 0.80 0.93 Cognitive function 0.57 0.64 0.57 Miscellaneous statistical tests Emotional function 0.86 0.86 0.84 A Levene test for difference in variances between the Social function 0.78 0.64 0.81 groups was significant for Pain, Appetite loss, and Global health/QoL 0.84 0.84 0.89 Financial Difficulties (p < 0.05). A multivariate analysis Fatigue 0.87 0.87 0.88 of variance (Pillai’s trace/Wilk’s lambda, with adjustment Nausea/vomiting 0.73 0.75 0.64 for confounders) found no significant difference (p = Pain 0.86 0.87 0.79 0.40) between the three groups. (Data not presented.) *p < 0.05 **p < 0.01 Discussion & The significant difference in this comparison is between the telephone In this study we investigated several measurement prop- versus the in-clinic Paper & Pencil condition erties of the EORTC QLQ-C30 questionnaire under var- un-adjusted mean difference between these two condi- ious MOA’s. Despite the widespread use of the EORTC tions was approximately 6 points, the adjusted mean dif- QLQ-C30, only two studies had previously investigated ference being only 5.4 points. The pair-wise Cohen’sd’s this matter. One large observational study considered 4 for the “telephone vs. paper & pencil at home”,the of the QLQ-C30 multi-item scales [14], while the other “paper & pencil at home vs. pencil & paper in-clinic”, study used a randomized, cross-over design, but with a and the “telephone vs. paper & pencil in-clinic” condi- much smaller sample size, and with only two (in-clinic) tions were 0.31, 0.14, 0.19, respectively. These results conditions [15]. qualify the MOA effect for the EF scale as being “small”. The present study of a heterogeneous population of An additional analysis was conducted, adding a fourth 375 cancer patients considered three conditions (at- group of patients to the above analyses of differences home as well as in-clinic) in a randomized, between- between means. This fourth group consisted of the subjects trial. patients who were not available for the in-clinic paper & Remarkably, all three of these studies flag the Emo- pencil condition because they did not return to the tional Functioning (EF) scale as yielding a small, yet sta- clinic at T3. These patients were invited to complete the tistically significant difference as a function of MOA, Gundy and Aaronson Health and Quality of Life Outcomes 2010, 8:35 Page 6 of 7 http://www.hqlo.com/content/8/1/35 Table 4 Adjusted# Means (+s.e.) for three MOA groups EORTC QLQ-C30 Paper & pencil at home Telephone Paper & Pencil in clinic All Scales (N = 132) (N = 121) (N = 61) Mean (s.e.) Mean (s.e.) Mean (s.e.) Sig.# Physical function 68.9(2.9) 67.1(3.0) 65.3(3.6) .47 Role function 61.4(3.3) 62.2(3.3) 60.4(3.9) .86 Cognitive function 86.2(2.7) 86.0(2.8) 84.8(3.3) .87 Emotional function* 74.1(2.7) 79.5(2.8) 75.0(3.3) .04 Social function 79.1(3.1) 79.2(3.2) 76.9(3.8) .76 Global Health/QoL 64.7(2.6) 66.0(2.7) 64.5(3.2) .79 Fatigue 41.0(3.2) 41.9(3.2) 45.3(3.8) .42 Nausea/vomiting 6.9(2.9) 6.2(3.0) 7.5(3.5) .90 Pain 33.6(3.6) 33.1(3.7) 26.0(4.4) .11 Dyspnea 24.4(3.4) 26.4(3.5) 28.6(4.2) .48 Insomnia 30.5(4.3) 22.3(4.4) 27.3(5.2) .06 Anorexia 8.7(3.9) 13.4(4.0) 13.0(4.7) .29 Constipation 4.2(3.1) 4.8(3.1) 1.7(3.7) .61 Diarrhea 6.2(2.5) 6.4(2.6) 5.7(3.1) .97 Financial 10.4(2.9) 8.7(2.9) 5.8(3.5) .30 *p < 0.05 **p < 0.01 # adjusted for covariates in Tables 1 and 2 & overall effect size for Emotional Function scale are 0.16 for overall effect (Cohen’s f, based on partial eta squared) and 0.31, 0.14, 0.19 (Cohen’s d) for the “telephone vs. paper & pencil at home”,the “paper & pencil at home vs. pencil & paper in-clinic”, and the “telephone vs. paper & pencil in-clinic” pairs of conditions, respectively. with patients in paper-and-pencil MOA’s reporting lower prior to assessment. This occurred primarily in the in- levels of emotional functioning. The present study also clinic condition. Almost 50% of the patients allocated to found a small, yet significant difference in Cronbach’s this condition did not return to the clinic in time for alpha for the Role Functioning scale; however, the RF the present study. This differential drop-out (apparently) scale performed quite adequately for all three conditions. lead to group differences in patient characteristics, such We suspect that the slightly lower EF scale scores in as treatment, stage of disease, and pre-randomization paper-and-pencil conditions may be related to the HRQoL measures. However, we believe that adjustment “demand characteristics” associated with different for these patient characteristics in the statistical analyses MOA’s. Specifically, patients, who are encouraged to was largely able to correct for these group differences. react quickly and/or who are required to interact with An additional, sensitivity analysis included these in- an interviewer, may be stimulated to present more clinic dropouts, who were approached via “pencil & socially desirable responses than those patients allowed paper at home”. This analysis re-flagged the EF scale, as to reflect on their level of emotional functioning and well as the SL scale, indicating that the “telephone” whose responses to the questions are not the subject of MOA yielded a more positive result than pencil & paper direct observation. For example, patients are asked in conditions (which did not differ from each other). These the QLQ-C30 whether they are depressed, which is not findings are commensurate with the finding reported a directly observable state, and whose admission might above. be felt as being potentially stigmatizing. A second limitation concerns the use of version 2.0 of Many studies of varying designs, sizes, populations, the EORTC QLQ-C30. There are, namely, slight differ- and instruments have considered the issue of measure- ences with the current version 3.0, involving the number ment characteristics of various MOA, with generally of response categories for the Physical Function scales. similar results. Namely, while various MOA may differ This might slightly limit the generalizability of these in costs, completion rates, etc., the effects of MOA on results to users of version 3.0. questionnaire measurement characteristics are generally of “small to medium” size, if found at all. This would Conclusions suggest that one should exercise caution when mixing In conclusion, the findings of this investigation indicate MOA’s while investigating effects of similar magnitudes. that the 3 modes of administration studied here have lit- A limitation associated with the present investigation tle effect on the internal consistency or the mean concerns the post-randomization dropout of patients responses on the EORTC QLQ-C30 scales. The Gundy and Aaronson Health and Quality of Life Outcomes 2010, 8:35 Page 7 of 7 http://www.hqlo.com/content/8/1/35 11. Osoba D, Aaronson NK, Zee B, et al: Modification of the EORTC QLQ-C30 exception to this generalization is the Emotional Func- (version 2.0) based on content validity and reliability testing in large tioning scale, which exhibited small, yet significant, dif- samples of patients with cancer. Qual Life Res 1997, 6:103-108. ferences between various administration modes. These 12. Aaronson NK, Cull A, Kaasa S, Sprangers MAG: The European Organization for Research and Treatment of Cancer (EORTC) modular approach to results suggest that, with the possible exception of quality of life assessment in oncology: an update. Quality of life and assessment of emotional functioning, there is little rea- pharmacoeconomics in clinical trials Philadelphia: Lippincott-Raven son for concern about the comparison of QLQ-C30 PublishersSpilker B, 2 1996, 179-189. 13. Fayers PM, Aaronson N, Bjordal K, Groenvold M, Curran D, Bottomley A, on results within or across studies as a function of mode of behalf of the EORTC Quality of Life Group: EORTC QLQ-C30 Scoring Manual administration. Brussels: European Organization for Research and Treatment of Cancer 14. Cheung YB, Goh C, Thumboo J, Khoo KS, Wee J: Quality of life scores Acknowledgements differed according to mode of administration in a review of three major The authors would like to thank A. te Velde, and M.A.G. Sprangers for oncology questionnaires. J Clin Epidemiol 2006, 59:185-191. providing access to the data used in the current analyses. The original data 15. Velikova G, Wright EP, Smith AB, Cull A, Gould A, Forman D, Perren T, collection was financially supported by a grant from the Dutch Cancer Sted M, Brown J, Selby PJ: Automated collection of quality-of-life data: a Society. The authors also wish to thank the patients for their willingness to comparisons of paper and computer touch-screen questionnaires. J Clin participate in the study. Some of the results of this study were presented at Oncol 1999, 17:998-1007. the Annual Conference of the International Society for Quality of Life 16. te Velde A, Sprangers M, Aaronson NK: Feasibility, psychometric Research, Montevideo, Uruguay, October 25th, 2008. performance, and stability across modes of administration of the CARES- SF. Annals of Oncology 1996, 7:381-390. Authors’ contributions 17. Karnofsky D, Burchenal J: The clinical evaluation of chemotherapeutic NA conceived of the study, and participated in its design and coordination agents in cancer. Evaluation of Chemotherapeutic Agents New York: and helped to draft the manuscript. CG participated in the design of the Columbia University PressMacLeod C 1949. study, performed the statistical analysis, and drafted the manuscript. All 18. Hjermstad MJ, Fossa SD, Bjordal K, Kaasa S: Test/retest study of the authors read and approved the final manuscript. European Organization for Research and Treatment of Cancer Core Quality-of-Life Questionnaire. J CLIN ONCOL 1995, 13:1249-1254. Competing interests 19. Cronbach LJ: Coefficient alpha and the internal structure of tests. The authors declare that they have no competing interests. Psychometrika 1951, 16:297-334. 20. Lautenschlager GJ: ALPHATST: Testing for differences in coefficient alpha. Received: 10 November 2009 Accepted: 30 March 2010 Appl Psychol Meas 1989, 13:284. Published: 30 March 2010 21. Cohen J: Statistical power analysis for the behavioral sciences Hillsdale, New Yersey: Lawrence Erlbaum Associates 1988. 22. Osoba D, Rodrigues G, Myles J, Zee B, Pater J: Interpreting the Significance References of Changes in Health-Related Quality-of-Life Scores. Journal of Clinical 1. Barry MJ, Fowler FJ, Chang Y, Liss CL, Wilson H, Stek M Jr: The American Oncology 1998, 16(1):139-144. Urological Association symptom index: does mode of administration affect its psychometric properties? J Urol 1995, 154:1056-1059. doi:10.1186/1477-7525-8-35 2. Weinberger M, Oddone EZ, Samsa GP, Landsman PB: Are health-related Cite this article as: Gundy and Aaronson: Effects of mode of quality-of-life measures affected by the mode of administration? J Clin administration (MOA) on the measurement properties of the EORTC Epidemiol 1996, 49:135-140. QLQ-C30: a randomized study. Health and Quality of Life Outcomes 2010 3. Vereecken CA, Maes L: Comparison of a computer-administered and 8:35. paper-and-pencil-administered questionnaire on health and lifestyle behaviors. J Adolesc Health 2006, 38:426-432. 4. Fouladi RT, McCarthy CJ, Moller NP: Paper-and-pencil or online? Evaluating mode effects on measures of emotional functioning and attachment. Assessment 2002, 9:204-215. 5. Rhodes T, Girman CJ, Jacobsen SJ, Guess HA, Hanson KA, Oesterling JE, Lieber MM: Does the mode of questionnaire administration affect the reporting of urinary symptoms? Urology 1995, 46:341-345. 6. Wu AW, Jacobson DL, Berzon RA, Revicki DA, Horst van der C, Fichtenbaum CJ, Saag MS, Lynn L, Hardy D, Feinberg J: The effect of mode of administration on Medical Outcomes Study health ratings and EuroQol scores in AIDS. Quality of Life Research 1997, 6:0. 7. Weinberger M, Nagle B, Hanlon JT, Samsa GP, Schmader K, Landsman PB, Uttech KM, Cowper PA, Cohen HJ, Feussner JR: Assessing health-related quality of life in elderly outpatients: telephone versus face-to-face administration. J Am Geriatr Soc 1994, 41:1295-1299. Submit your next manuscript to BioMed Central 8. Jorngarden A, Wettergen L, von Essen L: Measuring health-related quality of life in adolescents and young adults: Swedish normative data for the and take full advantage of: SF-36 and the HADS, and the influence of age, gender, and method of administration. Health Qual Life Outcomes 2006, 4:91. • Convenient online submission 9. Perkins JJ, Sanson-Fisher RW: An examination of self- and telephone- • Thorough peer review administered modes of administration for the Australian SF-36. J Clin Epidemiol 1998, 51:969-973. • No space constraints or color figure charges 10. Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, • Immediate publication on acceptance Filiberti A, Flechtner H, Fleishman SB, De Haes JC: The European Organization for Research and Treatment of Cancer QLQ-C30: a quality- • Inclusion in PubMed, CAS, Scopus and Google Scholar of-life instrument for use in international clinical trials in oncology. J Natl • Research which is freely available for redistribution Cancer Inst 1993, 85:365-376. Submit your manuscript at www.biomedcentral.com/submit

Journal

Health and Quality of Life OutcomesSpringer Journals

Published: Mar 30, 2010

There are no references for this article.