Get 20M+ Full-Text Papers For Less Than $1.50/day. Subscribe now for You or Your Team.

Learn More →

The COACH prompting system to assist older adults with dementia through handwashing: An efficacy study

The COACH prompting system to assist older adults with dementia through handwashing: An efficacy... Background: Many older adults with dementia require constant assistance from a caregiver when completing activities of daily living (ADL). This study examines the efficacy of a computerized device intended to assist people with dementia through ADL, while reducing caregiver burden. The device, called COACH, uses artificial intelligence to autonomously guide an older adult with dementia through the ADL using audio and/or audio-video prompts. Methods: Six older adults with moderate-to-severe dementia participated in this study. Handwashing was chosen as the target ADL. A single subject research design was used with two alternating baseline (COACH not used) and intervention (COACH used) phases. The data were analyzed to investigate the impact of COACH on the participants' independence and caregiver burden as well as COACH's overall performance for the activity of handwashing. Results: Participants with moderate-level dementia were able to complete an average of 11% more handwashing steps independently and required 60% fewer interactions with a human caregiver when COACH was in use. Four of the participants achieved complete or very close to complete independence. Interestingly, participants' MMSE scores did not appear to robustly coincide with handwashing performance and/or responsiveness to COACH; other idiosyncrasies of each individual seem to play a stronger role. While the majority (78%) of COACH's actions were considered clinically correct, areas for improvement were identified. Conclusion: The COACH system shows promise as a tool to help support older adults with moderate-levels of dementia and their caregivers. These findings reinforce the need for flexibility and dynamic personalization in devices designed to assist older adults with dementia. After addressing identified improvements, the authors plan to run clinical trials with a sample of community-dwelling older adults and caregivers. Background the oldest old (aged 80 years and over) after the year 2010 Globally, the number of individuals aged 65 years and [1]. This will result in an increase in the worldwide older is predicted to increase steadily, particularly among number of individuals diagnosed with dementia, particu- Page 1 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 larly Alzheimer's Disease (AD), from the current estimate uted throughout the house interact with applications run- of 24.3 million individuals in 2006 to 81.1 million by ning on computers to take into account context when 2040 [2]. performing actions. For example, if it is a sunny day out- side and the resident has the television on, the Gator Tech Older adults have a strong preference for aging-in-place Smart House [13] will automatically close the blinds to (i.e. remaining in their own homes and communities) reduce glare. Other features include medication reminders compared to other forms of care, such as nursing homes that can appear on the bathroom mirror and automatic and other long-term care facilities [3]. Additionally, vari- sensing and ordering for soap and toilet paper refills. ous studies have implied that older adults (particularly Pigot et al. [14] developed Archipel, a cognitive modeling those who have AD) benefit from aging in environments system for cooking tasks that recognizes the user's to which they are accustomed as familiar environments intended plan and adapts prompting to a pre-determined can provide memory and task cues [4-6]. However, this cognitive impairment level. Sensors, such as RFID tags shift from the hospital to home-based care means that and readers, in the kitchen environment detect which family members and other informal caregivers are being objects have been used and provide cues (audio, video increasingly depended upon to attend to the long-term and strategic lighting) to help users through each step in health-care needs of older adults with AD. Increased the task. As with Autominder, Archipel will not give dependence and changes in the relationship dynamic are reminders for tasks the user has already accomplished. difficult for both people with AD and their family caregiv- ers to accept [7]. The constant pressure to meet their rela- Research is increasingly emphasizing the importance of tive's needs for assistance and support can result in maintaining functional independence in older adults as a debilitating levels of stress for the caregiver, resulting in way of maintaining good health and wellness among the affected person's placement into long-term care. From older adults with dementia, while simultaneously reduc- a caregiver's perspective, decreasing the number of inter- ing medical expenditures [15,16]. However, the extent to actions required to complete an activity of daily living which CATs can aid an individual with AD depends on the (ADL) has a direct positive impact on caregiver burden. users' willingness to implement it, which in turn depends Even small decreases in caregiver burden have been found on whether the individual and/or his/her caregiver can to alleviate the prevalence of depressive symptoms in car- operate the device, feels that the device is useful, and egivers of individuals with AD [8]. This can lead to more whether the device supports or undermines the sense of successful informal care, resulting in lower medical costs personal identity [17]. To be useful to both a care recipi- and delayed long-term care placements. ent with dementia and his/her caregiver(s), a CAT must be autonomous, non-invasive, and must not require explicit To support aging-in-place, older adults and their caregiv- feedback (e.g. button presses), as this cannot reasonably ers are increasingly relying on the use of computerized be expected of either people with AD or overworked car- Cognitive Assistive Technologies (CATs) to complete ADL egivers. Cognitive assistance should be able to accommo- [6]. Often coupled with some form of artificial intelli- date high levels of customization as the more the gence (AI), CATs strive to support cognitive disorders assistance is personalized and appropriate to the deficits thereby enhancing the user's autonomy [9]. The mainte- in question, the more likely it will be adhered to and nance or increase of independence is coupled with a understood by the user [18]. Finally, assistance should reduction in the levels of caregiver assistance, and likely only be given on an "as needed" basis to minimize confu- caregiver burden, as well as a decrease in home heath care sion and to keep the user as cognitively involved in the costs [10]. task as possible. A significant amount of recent work in CATs for assisting The majority of currently available CATs require extensive people with cognitive impairments use probabilistic mod- sensor deployment and maintenance and/or input from a els to infer task and occupant status from sensors distrib- cognitively intact individual. Most likely the caregiver of uted throughout a person's living environment [11]. For the individual with dementia would have to learn how to example, Autominder, developed by Pollack et al. [12], operate and (to some degree) maintain a potentially com- uses artificial intelligence planning to schedule events plex planning system. As many caregivers are overbur- such as medication taking around a person's daily sched- dened as it is, two goals of the system described in this ule, such as favorite television programs or daily walks. paper were to minimize the amount of hardware that was Autominder uses environmental sensors to detect the sta- needed, and to have the system function without any tus of activities, and if required, will provide the user with explicit input from the user or the caregiver. context-aware reminders regarding unattended activities. The Gator Tech Smart House is an example of a smart The result was the COACH (Cognitive Orthosis for Assist- home designed with older adults in mind. Sensors distrib- ing aCtivities in the Home), a system that employs various Page 2 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 computer vision and artificial intelligence techniques to should be reflected in an increase of the number of steps autonomously provide the user with verbal and/or visual in the handwashing activity the older adult is able to com- reminders as necessary during ADL. Table 1 summarizes plete independently from a caregiver (i.e. with no assist- the progression of the systems used in the previous ver- ance from the caregiver). sions of COACH. The systems in each version of COACH represent significant advances in the sophistication and 2. Does this new version of COACH reduce caregiver versatility compared to those used in the previous version. workload? If the caregiver's workload is reduced, this The systems for the latest version of COACH (Version 3 in should be reflected in a decrease in the number of times a Table 1) are described in more detail in the Methods sec- caregiver interacts with his/her care recipient. tion below. 3. How will the COACH system perform with respect to its This paper presents results from an eight-week efficacy ability to correctly provide assistance to the user through- study of the COACH with older adults with dementia. out the ADL? To achieve a positive outcome, the system Methods and results are presented, followed by a discus- must be able to follow the older adult through the hand- sion regarding the potential clinical significance of the washing task, autonomously giving the correct prompt if participants' and device performances. While a brief (and only if) they are needed. description of the technology will be provided in this paper, the reader is referred to [19] for an in-depth Methods Device (COACH) design description of the COACH system and algorithms. In this work the authors extend upon the two previous versions of the COACH device (summarized in Table 1), Objective The objective of this study was to answer the following which both focused on the activity of handwashing [20- research questions: 22]. Handwashing was chosen as the model ADL because it is a relatively safe activity that older adults with demen- 1. Is the COACH system able to guide an older adult with tia have difficulties completing because of the required dementia through the handwashing ADL with less planning and initiation skills. dependence on a caregiver? If dependence decreases it Table 1: Summary of previous COACH systems. COACH Version Tracking System Decision-making Prompting system Number of Related system Subjects* Publications Version 1 Pattern wristband Neural networks Audio prompts, with 10 [22,33] worn by the user interacting with a one prompt for each hard-coded taxonomy step Version 2 Using background Fully observable Audio prompts with 4 [20,37-39] subtraction to isolate Markov decision three levels of the user's hands. process (MDP). This assistance (minimal, Tracking of hands and technique assumes the moderate, and task objects (i.e. soap world is fully maximal) for each and towel) using a observable; it does step. preset colour model. not take into account hidden variables, such as user responsiveness. Version 3 (system Colour based flocking. [Belief monitoring Audio and audio-video 6 [18,19,21,23,25] presented in this system & policy] prompts with three paper) Partially observable levels of assistance Markov decision (minimal, maximal, process (POMDP). and maximal + video This model takes into demonstration) for account hidden each step. variables and is able to Encouragement and a make decisions in reminder regarding conditions of the activity the user is uncertainty. attempting added. Professional actor recorded prompts. * Results from previous trials are not presented as they are not comparable because of the variance in the technologies and procedures that were used. Page 3 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 Handwashing was defined as having five essential steps icy into an action for COACH to take. Possible actions that must be accomplished for successful activity comple- available to the COACH are to give a low-guidance verbal tion, which are depicted in Figure 1. COACH guided users prompt, give a high-guidance verbal prompt, give an ver- through these steps using four integrated components: the bal prompt with a video demonstration of the action, call tracking system, belief monitoring system, policy, and the caregiver to intervene, or to do nothing (i.e., continue prompting system, as represented in Figure 2. to observe the user). The COACH's different levels of prompting assistance give COACH the ability to select the Images captured by a video camera are processed by the most appropriate support for each individuals' stage of tracking system and the hand and towel positions are AD and overall responsiveness. Thus the level of detail passed to the belief monitoring system. These data are played for the user is based on factors such as the error used by the belief monitoring system to compute the belief committed, sensory and cognitive status of the user, and state; a probabilistic estimation of the current state of the past responsiveness to the previous prompts. user and environment. The belief state is passed from the belief monitoring system to the policy, which is essen- The COACH system presented above had three significant tially a lookup table that denotes the best course of action changes from the previous versions: 1) The use of marker- for the system to take for every state that could be received less flocking to track the activity; 2) the use of a partially from the belief monitor. Each belief state that is received observable Markov decision process (POMDP) to model the from the belief monitoring system is translated by the pol- handwashing guidance problem; and 3) the refinement of audio prompts and the addition of video demonstrations. Tracking was accomplished using a computer vision tech- nique known as flocking, which was developed by Hoey et al. [23]. It uses models of skin and towel color com- bined with a Bayesian sequential estimation technique. This method of tracking is quite robust and able to dependably track the location of the user's hands and the position of the towel, even after occlusion by an object or after leaving and returning to the camera's field of view. A POMDP was chosen as the basis for the new planning sys- tem because of this model's ability to make good deci- sions in situations of uncertainty, as well as making intelligent inferences, and therefore decisions, about unobservable states (e.g. a user's level of dementia) [24]. This type of model allows the COACH system to autono- mously tailor itself to the individual needs of its users because it can estimate and use individual's traits (e.g. cognitive awareness and responsiveness levels) to dynam- ically adapt to daily and long term needs. Implementation of a POMDP is an important contribution to not only the field of artificial intelligence but to the usability concerns of users and their caregivers as it enables user-specific prompting strategies while remaining autonomous. Greater details regarding the technical nature of COACH, including system detailed descriptions and planning algo- rithms, can be found in [19]. Audio prompts were recorded using a professional male actor to emulate the cadence and tone of a professional Th Figure 1 e five essential steps of handwashing caregiver. A male voice was used (as opposed to a female The five essential steps of handwashing. Successful one) because previous research by this group and others activity completion was considered to be any sequence of suggests that male voices are easier to hear and under- steps that took the participant from "Start" to "Finish". As stand, possibly because the male voice has a lower pitch/ the long-term care facility's guidelines required the use of liq- frequency [22]. The wording used for the prompts is uid soap wetting one's hands before getting the soap was not shown in Table 2 and was similar to the wording used in considered an essential step in the activity, therefore the water on and soap used steps are interchangeable. previous studies, modified slightly according to recom- mendations from Wilson et al. [18]. Prompts included the Page 4 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 A Figure 2 flow diagram of COACH components A flow diagram of COACH components. Images from the camera are translated into hand and towel positions by the tracking system. These are passed to the belief monitor, which calculates the probability distribution over the possible states. This belief state is passed to the policy, which selects an action for COACH to take (i.e. prompt, observe user, or call car- egiver). participant's name at the beginning of each prompt to get pre-recorded in the same washroom that was used in the his/her attention. Previous studies with COACH have trials and combined with the maximal assistance verbal found that some users can get confused about which activ- prompts. A frame from one of the videos is shown on the ity they were asked to complete (e.g. previous participants monitor in Figure 3b. have been known to wash the towel in the sink, wash his/ her face, etc.), therefore the prompt often contained a Participants and Ethics/Consent Process reminder to help participants remember which activity This study was reviewed and approved by the Toronto they were attempting to complete. The prompt then gave Rehabilitation Institute's Research Ethics Board (REB). the participant guidance for the step in the activity s/he Potential participants were identified by the staff at the was attempting. The potential usefulness of adding video long-term care (LTC) facility in Toronto, Canada where demonstrating correct completion of the activity step was the study took place. Informed consent to participate was examined by Labelle and Mihailidis [25]. Results were obtained in writing (using a the consent form approved positive; therefore audio-video capabilities were added to by the REB) from the participants' substitute decision this version of COACH. The videos used in this study were makers, after the study was described to them using an shot from the perspective of the participant. They were information sheet and informal interview. Table 2: Wording for the prompts used by COACH. Step Minimal verbal assistance Maximum verbal/video assistance* Turn on the water [Name], you're washing your hands. Can you turn the [Name], try turning the silver knobs. water on? Use the soap [Name] you're washing your hands. Please use the soap. [Name] you're washing your hands. Try putting on some soap. Rinse hands [Name], you're washing your hands. Please rinse your [Name], you're washing your hands. Try putting your hands in hands. the water. Turn off the water [Name] you're doing great. Can you turn the water off? [Name], you're doing great. Twist the knobs to turn the water off. Dry hands [Name], you're doing great. Dry your hands now. [Name], try using the towel. * The same wording was used for the maximum verbal assistance and video assistance prompts. Page 5 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 E Figure 3 xample of the COACH system setup Example of the COACH system setup. (a) Example of the LCD screen, speakers (top-right), and video camera (circled) setup. (b) Example of a video-based prompt demonstrating how to turn on the water. Participants had to meet the following inclusion/exclu- Research DragonFly2), and a wall-mounted 21-inch LCD sion criteria: over the age of 65 years of age, no history of screen and desktop speakers (Figure 3a). A Dell Latitude violence, fluent in English, can hear normal levels of laptop computer (2 GHz processor, 2 Gb RAM) was used speech, exhibit no severe motor impairments, and have as the processing unit for the system software and hard- moderate-to-severe dementia. Level of dementia was ware, as well as the operator graphical user interface to determined through the administration of the Mini-Men- display information about the system variables (e.g. esti- tal State Examination (MMSE), an assessment instrument mated plan steps, system response, etc.), and the partici- that is commonly used to estimate the level of cognitive pant's progress through the task. The trials were also impairment in adults [26]. Typically, participants are sep- recorded using a camcorder positioned above the partici- arated into four categories of impairment based on his/ pant to capture video for post-trial evaluation by human her MMSE score: no impairment (30–26 points), mild raters (but was not used by the system during the trials). (25–20 points), moderate (19–10 points), and severe (9- Study design 0 points). Each participant's dementia level was scored using the MMSE before the start and upon completion of A single subject research design (SSRD) was used in this the trials study because of the difficulty in recruiting and maintain- ing an adequate sample size and the variability of the par- Apparatus set-up ticipants' health [27-31]. This research design has been The study was conducted in a retrofitted washroom and used in the authors' previous studies and has been found adjoining office that were dedicated to the project by the to be the most appropriate procedure for the evaluation of LTC facility. The washroom was fitted with a ceiling- this type of technology. The study consisted of two base- mounted IEEE-1394 digital video camera (Point Grey line phases, A and A (COACH not used), and two inter- 1 2 Page 6 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 vention phases, B and B , (COACH used), run in the would be technically incorrect, the participant would still 1 2 order A -B -A -B to identify any carry-over effects. Based score a five on independent step completion. Number of 1 1 2 2 on studies completed with previous versions of COACH, caregiver interactions was a count of the number of times 10 trials per phase were deemed to be sufficient for partic- the caregiver had to interact with the participant to get ipant performance to stabilize and for the desired changes him/her to complete a step. An interaction was considered in the dependent variables to be observed [22]. to be any exchange between the caregiver and the partici- pant that was related to activity completion, including Procedure verbal prompting, demonstration, and touching (either Trials consisted of one trial per day per participant, Mon- the participant or an object). The functional assessment days to Fridays, for eight weeks for a total of 40 trials each. score (FAS) is a modified version of the Functional Inde- To ensure uniformity and avoid any potential risk of pendence Measure (FIM™), which is a standardized assess- injury from falls, each participant was required to sit in a ment tool used to measure one's ability to function with wheelchair and was taken to the test washroom by a car- independence over 18 activities of daily living [32]. Partic- egiver who was hired for this study. The caregiver posi- ipants received an FAS for each step in the activity and tioned the participant in front of the sink in the test scores ranged from zero (no attempt/refusal) to seven washroom and asked the participant to wash his/her (complete independence), with an overall maximum of hands. 35. If the participant completed the step in response to prompts provided by the COACH, a score of seven was During the A-phases of the trials, the caregiver interacted given. A higher cumulative FAS is expected to correlate with the participants as she normally would, providing with higher levels of activity completion independence. any prompts and reminders she felt were necessary to The face validity of the FAS was demonstrated in previous complete handwashing. During the B-phases, COACH trials by Mihailidis et al. [22,33]. was started by a researcher (who was hidden from the user) as soon as the caregiver requested the participant to During the B-phases, data were collected regarding the wash his/her hands. The caregiver then left the participant system responses to participant performance during the alone in the test washroom and discreetly observed him/ handwashing activity. These data were collected based on her from the hallway. The caregiver provided assistance the basic principles of signal detection theory (SDT) [34], only if instructed to do so by COACH (i.e., the caregiver which can be used to measure four conditions describing was summoned by the device to intervene) or if the car- device performance with respect to: hits, false alarms, cor- egiver felt the need to intervene for the well being of the rect rejects, and misses. These conditions with respect to participant (e.g., the participant was attempting to stand the COACH system are outlined in Table 3. For each step up from the wheelchair or was becoming upset). in the activity, the system was rated as having at least one, and potentially more, of the four possible SDT condi- Data collection tools tions. For example, if the COACH gave three incorrect A score sheet was used to collect data required to evaluate prompts and one correct prompt for a step, three false the system's efficacy in terms of both user and system per- alarms and one hit would be scored. formance. The score sheet was the same one that was developed and used in studies that examined previous Analysis of participant and device performance Video of each trial was reviewed and scored by an experi- versions of the COACH (refer to [22]). enced rater using a multi-modal score sheet to collect the With respect to user performance, scales on the score sheet types of data described previously. An experienced rater measured the following for each step of the activity: 1) was a researcher who was trained on the scoring methods independent step completion; 2) number of caregiver and has had previous experience rating COACH trials. interactions; and 3) functional assessment scale (FAS). Independent step completion was scored for every trial. Participants scored one point for the first time s/he com- Table 3: The four possible conditions used to determine pleted a step in a trial without assistance of any kind from COACH's performance. a human caregiver. As there were five essential steps (Fig- COACH Response ure 1) the maximum score that could be attained was five, even if a participant completed more than five steps inde- Prompt No prompt pendently. For example, a participant could independ- ently turned the water on, wet her hands, turned off the Participant Action Error Hit Miss water, got some soap, turned the water on again, rinsed her hands, turned off the water, and finally dried her No error False Alarm Correct Reject hands. Although none of the actions in this sequence Page 7 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 Space on each score sheet was provided for any general Participants comments or observations. Eight participants were recruited for this study, however two were withdrawn; S2 developed unrelated health prob- Because of the small number of participants in the study, lems, and S7's aggressive behavior caused concerns for the visual analyses of the data were used to identify trends of wellbeing of both herself and the study caregiver. Demo- participant behaviors and abilities, and compare changes graphics for the remaining six participants are presented in variability between phases. Visual analysis is a com- in Table 4. Based on his/her initial MMSE scores, five par- monly used technique for single-subject research designs. ticipants (S3, S4, S5, S6 and S8) were classified as having Data were examined for all trials and overall trends of in- moderate-level dementia, and one participant (S1) was group performance between baseline (A) and interven- classified as having severe-level dementia. tion (B) phases, as well as for variations in participant per- formance. Observed participant behaviors and reactions Participant performance were used to aid in the analysis of the results. As S1 was the only participant in the severe-level group and noticeably different trends from the other partici- Analyses of the device performance data were achieved pants, this sub-section examines the moderate-level par- through the calculation of the number of hits, misses, ticipants (S3, S4, S5, S6 and S8) as a group. Table 5 false alarms, and correct rejects (described in Table 3) summarizes overall individual participant performance made by the system during the intervention (B) phases. per test phase, which shows improvements in all three These data were also used to calculate two types of error: areas, particularly in a reduction in the number of interac- E (Equation 1) which reflects COACH not detecting an tions with the caregiver. From Table 6 it can be seen that error when participants made one, thus not giving a four of the five participants were able to independently prompt, and E (Equation 2) which reflects COACH complete the activity. Table 7 shows the overall number of detecting an error when none occurred, thus erroneously interactions with the caregiver required by the participant giving a prompt. These equations were derived by Mihai- to successfully complete essential handwashing steps, lidis [33] for the analysis of previous research on COACH. which decreased by an average of 66% when the device was introduced. Table 8 shows the participants' FAS for the handwashing activity increased by a negligible 2% for Misses (1) E = × 100 W the group. Figures 4 to 6 depict the daily average perform- Hits+ Misses ance for the entire moderate-level participant group (n = 5) for the number of steps completed independently, the False Alarms (2) number of interactions with a caregiver, and FAS respec- E = × 100 False Alarms+Correct Rejects tively. Results Device performance Inter-rater agreement A summary of the data regarding COACH performance is To ensure data reliability, a second experienced rater presented in Table 9. It should be noted that the item Par- scored 20 percent of all data collected regarding partici- ticipant ignored prompt from COACH in Table 9 represents pant performance and an inter-rater agreement was calcu- the combined number of both ignored hits and ignored lated using Cohen's Kappa (using SPPS v15.0) [27]. The and E (described by Equa- false alarms. The error rates, E w c measures of agreement (K values) were K = 0.96 (p < tions 1 and 2), were found to be 10.9% and 26.0% respec- 0.0005) for independent step completion, K = 0.69 (p < tively. This can be interpreted as COACH not responding 0.0005) for number of caregiver interactions, and K = 0.63 to 10.9% of the errors made by participants (E ) and (p < 0.0005) for FAS. COACH making an error in 26% of the cases where the participant was completing the step correctly (E ). Table 4: Demographics of the participants. Participant Gender Age (years) Education MMSE: study start MMSE: study completion Average MMSE S1 F 88 High school 5 3 4 S3 F 73 Post-secondary 12 18 15 S4 F 92 Elementary school 10 13 12 S5 M 81 Post-secondary 19 20 20 S6 F 87 High school 12 14 13 S8 F 89 Post-secondary 11 10 11 Page 8 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 Table 5: Average participant performance for each trial phase and overall group performance. Participant [Average MMSE Phase Mean number of steps Mean number of interactions Mean FAS* (out of 35) score] completed independently with human caregiver (out of 5) Severe-level dementia** S1 [4] A1 0.1 14.1 2.4 B1 1.8 10.6 17.1 A2 0.6 16.2 7.3 B2 0.9 20.6 11.2 Moderate-level dementia S3 [15] A1 5.0 0.0 35.0 B1 5.0 0.0 35.0 A2 4.9 0.2 34.8 B2 5.0 0.0 35.0 S4 [12] A1 3.6 2.6 29.9 B1 5.0 0.0 34.7 A2 4.5 1.4 33.9 B2 5.0 0.0 34.5 S5 [20] A1 3.3 4.4 30.7 B1 4.1 2.6 31.6 A2 3.8 3.8 32.2 B2 4.9 0.3 33.2 S6 [13] A1 5.0 0.0 35.0 B1 5.0 0.0 34.9 A2 5.0 0.3 34.8 B2 5.0 0.0 34.9 S8 [11] A1 4.6 1.9 33.2 B1 5.0 1.3 33.1 A2 4.5 2.2 32.8 B2 5.0 2.6 32.3 Mean score over phases for the A1&A 4.4 1.7 33.2 moderate-level group 2 B1&B2 4.9 0.6 34.1 % change*** 11 -66 2.4 Functional assessment score ** See Tables 6 to 8 for mean scores for S1 *** Calculated by [(B +B )-(A +A )]/(A +A )*100 1 2 1 2 1 2 lower end of the agreement range. Although both raters Discussion Inter-rater agreement had previous experience and agreed on scoring conven- Altman [35] has indicated that values of K can be inter- tions, many instances were more difficult to score than preted as very good agreement if they are between 0.81 one might imagine. For example, 'guidance' was consid- and 1.00 and as good agreement if they are between 0.61 ered to be a prompt and could be a verbal cue, visual cue, and 0.80. While the K-value for independent step comple- tactile cue or any combination of the three. When com- tion and the p-values obtained for all three measures were bined, prompts were difficult to clearly distinguish, such good, the K-values for the observed number of caregiver as when verbal guidance was followed closely by visual interactions (K = 0.69) and FAS (K = 0.63) were at the guidance. In this particular example, it could be inter- Table 6: Average number of steps per trial completed independently without (Phase A) and with (Phase B) COACH Participant [MMSE] Mean number of steps completed Mean number of steps completed Change (%) independently in Phase A (out of 5) independently in Phase B (out of 5) S1 [4] 0.38 1.33 250 S3 [15] 4.95 5.00 1 S4 [12] 4.10 5.00 22 S5 [20] 3.60 4.53 26 S6 [13] 5.00 5.00 0 S8 [11] 4.57 5.00 9 Page 9 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 Table 7: Average number of interactions with the caregiver per trial without (Phase A) and with (Phase B) COACH Participant [MMSE] Mean number of interactions in Phase A Mean number of interactions in Phase B Change (%) S1 [4] 15.19 15.56 2 S3 [15] 0.11 0.00 -100 S4 [12] 1.95 0.00 -100 S5 [20] 4.10 1.33 -68 S6 [13] 0.15 0.00 -100 S8 [11] 2.05 1.94 -5 preted as a single prompt (a verbal plus visual cue) by one third of the original number of interactions to complete rater and as two (first a verbal prompt, followed by a sep- handwashing. The change in the amount of assistance the arate visual prompt) by the second rater. Similar scoring participants required was mildly reflected by their FASs difficulties were encountered with the FAS, where the rat- (presented in Table 8), although the change is not as obvi- ings one through four represent the rater's opinion of how ous as it is with independence and interaction with car- much of the step (0 to 25%, 25 to 50%, 50 to 75%, 75 to egiver measures (Tables 5 and 6). S4's independence from 100%) the participant completed independently after a human caregiver when COACH is shown by her increase being given a prompt by the caregiver. to a near perfect FAS. The reason S4 did not achieve a per- fect score is because she was a relatively slow hand washer Participant performance and the FAS dictates that a score of 6 is assigned when the The participants showed a general improvement in the participant "took more than reasonable time to complete handwashing task when COACH was used, reflected in step". Participants S3 and S6 were able to perform most of more steps completed independently, fewer interactions the steps in handwashing independently before the intro- with the caregiver, and higher FASs. Results presented in duction of COACH, therefore these participants had no Table 6 suggest that improvements in handwashing inde- change in their FAS because there was little opportunity pendence when COACH is used are inversely propor- for them to improve. tional to a person's baseline performance, with greater levels of improvement being seen by those who are less S8 was a noteworthy subject as she was highly independ- independent when washing his/her hands. Moreover, ent, but would consistently omit the soap application four moderately impaired participants who routinely step. Although COACH provided a prompt for her to do required some assistance (S3, S4, S6, and S8) competed all so at almost every instance, she ignored these prompts five handwashing steps completely independently the and would only respond to verbal prompts from a human majority of the time COACH was used. It appears that the caregiver. Thus, essentially no change was seen in S8's improvement is a result of device use rather than activity number of caregiver interactions or FAS. S8 provides a learning by the participants, supported by the fact that good example of a user (in terms of ADL and cognitive performance improved when COACH was used in B , fell abilities) who may not be a good candidate for this style when it was removed in the A phase, and was regained in of computer-based guidance as her idiosyncrasies resulted B when COACH was reintroduced (as seen in Table 5). in compliance with verbal cues only when they are given There was a noticeable trend towards a decrease in the by a human. number of interactions with the caregiver during the inter- vention phases, with three of the subjects (S3, S4 and S6) For the more independent participants (S3 and S6) requiring no human assistance when COACH was used. COACH appears to function more as a "maintenance" S5 also showed a considerable decrease, requiring only a tool, able to support the participant in the occurrence of Table 8: Average participant FAS scores per trial without (Phase A) and with (Phase B) COACH Participant [MMSE] Mean FAS* in Phase A Mean FAS* in Phase B Change (%) S1 [4] 4.95 14.14 186 S3 [15] 34.89 35.00 0 S4 [12] 32.00 34.60 8 S5 [20] 31.50 32.43 3 S6 [13] 34.90 34.89 0 S8 [11] 33.00 32.71 -1 * Functional assessment score Page 10 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 A A B B 1 2 1 2 Without COACH With COACH Without COACH With COACH 4.5 3.5 2.5 1.5 0.5 0 5 10 15 20 25 30 35 40 Trial Number Mean n Figure 4 umber of steps completed independently for all participants for each day of the trials Mean number of steps completed independently for all participants for each day of the trials. A and A are the 1 2 baseline phases (no use of COACH), B and B are the intervention phases (COACH used). 1 2 an occasional error. S4, who needed modest amounts of classified as having severe dementia. Moreover, she mostly verbal assistance from her caregiver, became essen- showed a notable decline in general abilities over the tially independent when COACH was in use. Results from course of the study, which is reflected in her low and trials involving S3 and S6 suggest that using COACH does decreasing MMSE scores (Table 4). Although there is not not have a detrimental effect on the performance of capa- enough data to support any significant conclusions, S1's ble individuals. This indicates that the device could poten- decline may explain why there is a slight increase in the tially be introduced in the early stages of dementia number of caregiver interactions when the device is used without any negative effects. An early introduction would (Table 7), even though her ability to complete steps inde- allow the system to learn about the user's preferred hand- pendently improved during the intervention phases washing regime, which in turn could enable the system to (Table 6). It is thought that while COACH helped to make better decisions when guiding the individual remind S1 which step came next (i.e., improving her inde- through the activity later on when s/he does require assist- pendence), she required more and more prompting as the ance. From this study, it appears that by providing study progressed and her dementia became more severe prompts to users with moderate-level dementia, COACH (i.e., increasing number of caregiver interactions). This successfully encouraged more independent behavior with supposition is supported by the prompting behavior greater relative results seen in individuals who required exhibited by COACH. In B , S1 completed a correct step higher levels of assistance to complete the task. as a result of five of COACH's 33 correct prompts while in B S1 did not complete any steps in response to COACH's S1 was an exceptional subject in the study. She had the 40 correct prompts. S1 had 10 trials that required an lowest MMSE score and was the only participant who was exceptionally high (for this sample group) 20 or more Page 11 of 18 (page number not for citation purposes) Mean ( ) Number of Steps Completed Independently BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 A A B B 1 2 1 2 Without COACH With COACH Without COACH With COACH 0 5 10 15 20 25 30 35 40 Trial Number Mean n Figure 5 umber of interactions with a human caregiver for all participants for each day of the trials Mean number of interactions with a human caregiver for all participants for each day of the trials. A and A are 1 2 the baseline phases (no use of COACH), B and B are the intervention phases (COACH used). 1 2 interactions with the caregiver to complete the handwash- ing the task were the ones who were most likely to become ing activity. S1 showed the most relative improvement in independent when the device was used. These findings self-sufficiency (particularly with rinsing off soap and agree with studies conducted with previous versions of the hand drying) and this is reflected by her FAS. However, COACH, and suggest that COACH has the potential to dependent people such as S1 may require the device to be increase the independence and autonomy of individuals a constant presence and they will likely need physical suffering from dementia. Ideally, a system like COACH assistance from a caregiver with some steps (in particular, will enable caregivers to perform other tasks while the getting soap). completion of ADL are supervised by the system because the system would bring any difficulties regarding task The results presented here support the use of this type of completion to the attention of the caregiver. Therefore, technology to increase user independence while decreas- while use of COACH would not eliminate the need for a ing number of interactions with the caregiver, with four caregiver (as the caregiver would have to still be present in out of five of the moderate-level subjects able to complete the home to provide assistance that is beyond the capabil- the activity without any assistance whatsoever from the ities of a computerised reminder system), it could poten- caregiver when the device was in use. The authors feel that tially augment the burden of constant supervision of his/ research questions one and two have been adequately her loved one. This would allow more free time for the addressed and that the use of COACH results in increased caregiver and more privacy for the person with dementia, independence from a caregiver as well as a reduction in which in turn would hopefully improve quality of life for caregiver burden for people with moderate-level demen- the dyad and delay long-term care placement. tia. Participants who had only minor difficulties complet- Page 12 of 18 (page number not for citation purposes) Mean ( ) Number of Interactions with a Caregiver BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 A A B B 1 2 1 2 Without COACH With COACH Without COACH With COACH 0 5 10 15 20 25 30 35 40 Trial Number Mean Fu Figure 6 nctional Assessment Score (FAS) for all participants for each day of the trials Mean Functional Assessment Score (FAS) for all participants for each day of the trials. A and A are the baseline 1 2 phases (no use of COACH), B and B are the intervention phases (COACH used). 1 2 Device performance and its functioning as an assistive technology can be As device performance examines the interactions between measured primarily by the ability of the COACH to iden- the device, the user, and their environment, the data from tify an error by the participant and provide correct assist- all six participants has been grouped and is presented in ance in response. During this study, COACH had a total the discussion below (as opposed to the participants' per- of 750 observed conditions, which are summarized in formance data that were separated according to cognitive Table 9 and Figure 7. Of these, 170 (23%) were errors (i.e., impairment). The overall efficacy of the COACH system a false alarm or a miss, as defined in Table 3). Misses and Table 9: Device performance with regards to COACH's response to participants' actions and participants' reactions to prompts given by COACH. Opportunity outcome* Water On Use Soap Rinse Soap Water Off Use Towel TOTAL Hit 16 62 15 48 6 147 Miss 1 11 1 3 2 18 False alarm 7 107 18 18 2 152 Correct reject 117 69 87 60 100 433 Steps completed correctly in response to a prompt from 14 6 18 1 30 COACH Prompts from the COACH ignored by participants 20 159 24 43 7 253 *Hits, misses, false alarms and correct rejects are defined in Table 3. Page 13 of 18 (page number not for citation purposes) Mean ( ) Functional Assessment Score BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 false alarms by the COACH occurred because the system or frustrated were less likely to complete the remainder of misinterpreted a step in the handwashing task. For exam- the task if subsequent correct cues were given. Three par- ple, an ambiguous user action such as touching the taps ticipants verbally responded to the device when a false may have caused the system to incorrectly presume that alarm was given, expressing that they had already com- the water had been turned on. Wrong assumptions by pleted the step they were being prompting to do. For COACH reduce the probability that the correct course of example, when prompted incorrectly by COACH, S6 action will be taken. If COACH is not able to correct itself would often ask "What [did] I do?" then in frustration through other observations, a prompt may then be either would tell the COACH to "Shut up". When misses missed or provided for the wrong step. occurred the participant was usually unable to complete the rest of the task on his/her own, even if COACH recov- From a clinical point of view, the impact of the device's ered from the miss and prompted the participant cor- errors on participant performance should be used to rectly, hence trials where misses occurred often had determine if the device error rate was acceptable. False COACH summoning the caregiver to intervene (which alarms sometimes irritated and confused the participants, was the correct action to be taken by the system by this and it was observed that participants who were confused time). Although this resulted in an increase in the amount Hit & Completed 4% Hit & Ignored 16% False Alarm & Responded 2% Correct Reject False Alarm & Ignored 58% 18% Miss 2% Obser Figure 7 ved COACH conditions with corresponding number of observations and percentage of the all observed conditions Observed COACH conditions with corresponding number of observations and percentage of the all observed conditions. Note that correct prompts (hits) and incorrect prompts (false alarms) are separated into the two observed par- ticipant reactions to the prompts: completed/responded or ignored. Page 14 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 of interactions between the participant and the caregiver issued because the participant did not need assistance. Of and a decrease in the number of steps the participant was the correct prompts given to participants, 30 (20% of all able complete on their own, this tended to result in less hits) resulted in the participant complying with the cue frustration for the participant than false alarms. Thus and completing the step accordingly (Figure 7). The while misses and false alarms do not seem to cause signif- number of correct prompts ignored by participants ranged icant upset in the participants, it is very likely the irritation dramatically from one (by S6) to 68 (by S1). Notably, the and confusion resulting from mistakes by COACH hin- number of correct prompts ignored by S1 (severely- dered participant performance. Reducing the number of impaired) account for over half (58%) of the total correct these types of system errors is the primary priority of prompts ignored by all participants, with only five of the future work on the COACH. correct prompts given to S1 resulting in completion of the step. In contrast, S4 was the most responsive to hits, com- This study uncovered sequences and scenarios that led to pleting a step in response to the COACH 60% of time she false alarm prompts in this particular COACH model. was provided with support. In the more severe stages of Analysis of the data revealed that 107 (70%) of all false AD individuals tend to lose the ability to respond to his/ alarm prompts issued were to "use the soap", equating to her environment, the ability to speak and, ultimately, the 36% of all prompts given by the device or 8% of all ability to control movement [36]. This may explain S1's observed conditions. Clearly, incorrect prompting for lack of response to the system; however, it does not soap use constitutes the majority of the errors committed explain why S5, who has the highest MMSE, had a lower by COACH. Further investigation of sequences where "use handwashing performance than any of the participants the soap" false alarm errors occurred revealed that 59 with moderate dementia. S5 responded well to COACH, (55%) of the "use the soap" false alarms took place after completing a step in 43% of the cases when a correct the completion of the handwashing task (i.e. after all of prompt (hit) was given. the five essential steps were completed), mostly when the participant was drying his/her hands. Closer examination Sixteen of the 152 false alarms that were given (11% of all of these trials revealed that the tracking, policy and false alarms) elicited reactions from the participants (Fig- prompting modules appeared to be working well, there- ure 7). It should be noted that a response to a prompt is fore the authors conclude the problem is likely in the not considered to be the same as the completion of a step, belief state monitor. For many of these trials, when the as an unsuccessful attempt to complete a step is also con- user reaches the end of the handwashing activity and is sidered a response. For instance, if COACH (correctly or drying his/her hands, the COACH correctly shows the incorrectly) prompted a participant to "turn the water on" activity as being complete. However, after several seconds and the participant touched the taps without altering the belief state monitor slowly changes its belief distribu- water flow, this would be considered a reaction to the tion from a strong belief that handwashing is complete to prompt, although no progress was made in the step itself. the belief that the participant's hands are not washed. Therefore, the response rate to false alarms of 11% is con- When the belief that the participant's hands were not siderably lower than the 20% completion rate for correct washed reached a sufficient level, a prompt to "use the prompts when considering the compliance rate with soap" occurs. This "drift" back to the beginning of the task prompts in terms of step completion. When participants was intentionally designed to give the system an opportu- did perform an incorrect action because of a false alarm nity to correct itself if it incorrectly believed that the user from COACH (such as using the soap when the partici- had completed the task. However, the clinical trials pant's hands were already washed), this often resulted in showed that the usefulness of this functionality may not backtracking in the activity, adding to the number of steps outweigh the clinical costs associated with the resulting required for activity completion. It was observed that the erroneous prompts, particularly if the participant is some- extra steps and confusion resulting from backtracking one who takes a long time to dry his/her hands. This prob- offered a greater opportunity for errors to occur. As such, lem is being corrected in the next version of COACH by assistance from the caregiver was usually needed to com- implementing a "cutoff" where the system terminates plete the task. At 34%, S5 had the highest response rate to guidance once the system's belief that the user's hands are system prompts regardless of whether they were a hit or a clean and dry reaches an empirically determined thresh- false alarm. S5's high compliance rate may explain why old, at which point the caregiver will be called to escort his FAS did not reflect his other improvements as much as the participant from the washroom. they did for the other participants in the study (see Tables 5, 6, and 7). When S5 responded to a false alarm, this As presented in Table 9, COACH had 580 correct condi- often led to backtracking in the activity and participant tions (77% of overall conditions), composed of 147 cor- confusion, requiring the caregiver to be summoned, who rect prompts (or hits) when participants required then had to use greater levels of assistance to re-orient S5, assistance and 443 correct rejects, where no prompt was which would result in lower FAS results. Page 15 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 COACH autonomously used observations of each partici- COACH waited a pre-determined amount of time before pant's actions and responses to prompts to estimate his/ giving a prompt to the participant if s/he was "stuck" on a her level of dementia (a parameter that changed slowly step, with COACH giving the participant ample time to over a course of days) and responsiveness (a parameter attempt the step on his/her own before prompting. that changed on a day-to-day basis). These parameters COACH's "patience" may have played a role in the greater played a large role in dictating the level of detail of the levels of independence seen when the device is used, as prompts given by COACH, which would select prompts participants would sometimes correctly resume the task that were appropriate for the individual's levels of respon- after a pause of a few seconds; a pause which may be siveness and dementia. From the technical results pre- longer than most human caregivers would care to wait. sented in Hoey et al. [19], COACH autonomously However, COACH would sometimes be too patient. assigned a low dementia level (i.e. more impaired) to all but Prompts to correct a participant when they had confused one participant, S5, who had a rating of medium level the ordering of steps were often given several seconds after dementia. It is interesting to note that these ratings coin- they should have been. There were several instances where cide with the participant's MMSE scores rather than partic- the participant had completed at least one, and some- ipant performance during the handwashing activity. times several, incorrect step(s) before COACH gave the These results suggest an interesting possible additional appropriate corrective prompt. As such, it is quite possible application of COACH as a diagnostic tool; by watching that the participants would ignore fewer prompts from older adults perform ADL over a series of weeks or COACH if the timing of the prompts was improved. The months, a future version COACH may be used to detect authors are implementing the use of automatically deter- changes in the users' abilities, and consequently, level of mined, participant specific pauses between the delivery of dementia. COACH's ordering of the prompts was also prompts, which will be dynamically adjusted in response appropriate. The majority of instances showed the system to the individual's dementia and responsiveness levels. prompting the participant using a well-timed progression of prompting strategies from audio (only audio), video While there have been areas of the system identified for (audio and video), and finally to summoning the car- improvement, based on these and previous COACH trials, egiver to intervene if the participant was unresponsive to the authors speculate that participant responsiveness is the prompts. COACH would autonomously carry esti- dependant on several traits of the particular individual, mates about the user's dementia level from one trial to the such as cognitive abilities, hearing, vision, mood, compli- next, so that the system would not have to relearn partici- ance, and general attitude. Using a POMDP as the plan- pant behaviors for each trial. ning agent for this model allowed the system to estimate unobservable user traits such as responsiveness and Using the participant's name at the beginning of each cue dementia level to help tailor timing and level of prompts was a successful technique to gain his/her attention. For to the abilities of the individual, however, the ultimate the majority of the prompts, regardless of whether they usefulness of this type of assistive technology is very much were hits or false alarms, the participant would look up at dependant on the traits of the person who is using it. It is the video screen in anticipation of further instruction of significant interest that the level of dementia (as deter- when s/he heard his/her name. If a video was played, mined by the MMSE) alone does not appear to be an indi- most participants would watch the video to its comple- cator of an individual's success with this type of tion, regardless of whether or not they responded to the technology. prompt. The wording of the prompts was appropriate for this population as participants often spoke back to the Limitations prompts with a reply that showed they understood what There are several limitations regarding this study that was being asked, even if they did not comply. For exam- must be acknowledged. While the COACH shows promis- ple, when prompted by COACH to use the soap, S8 would ing results, the sample size was too small to draw any sig- often say "No thank you, I don't want to". As participants nificant conclusions about wide-scale applicability or usually distinctly looked up at the screen when they heard performance. The study presented here focused on a mod- their name, (when played) watched video cues to comple- erate-to-severe level dementia, with the majority of the tion, and provided coherent responses to prompts, it can participants being from the moderate group. Although the be concluded that the audio and audio-video cuing tech- researchers believe the trends seen here would also be niques used in this study are successful at getting the seen in mild and severe populations, to what extent attention of most older adults with dementia, although remains unknown until the device can be tested with a attention span and compliance is dependant on the indi- larger group that contains a diversity of dementia levels. vidual traits of the participant. More testing with a broader population, including infor- mal caregivers, must be done before any significant con- clusions regarding this device can be made. Page 16 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 Future Work that COACH could be useful to the caregiving dyads of The results gathered here are convincing enough that the individuals who respond well to prompting without tac- authors are preparing the device for the next set of clinical tile cues. These findings also support the importance of trials, which will include more participants and will be understanding the special, diverse and dynamic needs of conducted within homes in the community. Some this target user group to ensure that appropriate, custom- improvements that the researchers hope to include are the izable assistance is available in assistive technologies to adaptation of the system to distinguish between multiple help support people with dementia and their caregivers. washroom ADL such as tooth brushing and, eventually, toileting. The team plans to examine the implementation Abbreviations of speech recognition to allow the system to recognize AD: Alzheimer's disease; ADL: Activity of daily living; AI: other types of implicit user feedback. Multiple camera Artificial Intelligence; CAT: Cognitive assistive orthosis; input (vision/tracking systems) gained by placing cameras COACH: Cognitive Orthosis for Assisting with aCtivities in various positions throughout the washroom could pro- in the Home; FAS: Functional assessment score; MMSE: vide greater user observation accuracy and versatility and Mini Mental State Examination. would enable 3D observations. It is hoped that the next set of trials can be performed over a longer period of time Competing interests with a larger sample size to decrease the effects of the nat- The authors declare that they have no competing interests. ural performance variability that is found in this popula- tion. Authors' contributions AM supervised the project, developed the study design, and assisted in the preparation and editing of the manu- Conclusion This paper presents the results from clinical trials with a script. JB participated in the study design, test area set-up, small group of potential users of the COACH, a cognitive second rating of the trials, and drafting the manuscript. TC assistive technology designed to assist older adults with assisted in running trials, was the primary rater, and par- moderate to severe dementia through ADL. This study ticipated in drafting the manuscript. JH designed the soft- aimed to determine whether or not the POMDP-based ware algorithms used in this study. All authors COACH system was capable of: 1) reducing user depend- participated in the preparation of the final manuscript. ence on a caregiver, 2) decreasing caregiver workload, and/or 3) providing correct guidance through the hand- Acknowledgements The authors would like to acknowledge the invaluable contributions of Axel washing task. When COACH was used, the participants von Bertoldi (who assisted in the development, implementation, and testing appeared to show an increase in the number of hand- of the software and hardware), Kate Fenton (who was our caregiver for the washing steps they were able to complete without assist- study), and David Giesbrecht (who ran the technical side of the trials). This ance from the caregiver as well as the decrease in number research was generously supported by a grant from the American Alzhe- of times they required assistance from the caregiver during imer's Association (ETAC Program). the activity. Four of the five moderate-level participants were independent from a human caregiver during hand- References washing when COACH was used. Based on these results, 1. U.S. Census Bureau: Global Population Profile: 2002 Washington D.C.: U.S. Government Printing Office; 2004. this study has affirmatively answered the first two research 2. Ferri C, Prince M, Brayne C, Brodaty H, Fratiglioni L, Ganguli M, Hall questions. Through these clinical trials, the POMDP- K, Hasegawa K, Hendrie H, Huang Y, Jorm A, Mathers C, Menezes P, based planning system shows promise as a possible plan- Rimmer E, Scazufca M: Global prevalence of dementia: a Delphi consensus study. The Lancet 2006, 366:2112-2117. ning algorithm for guiding older adults with dementia 3. Shenk D, Kuwahara K, Zablotsky D: Older women's attachments through handwashing, albeit several areas in need of to their home and possessions. Journal of Aging Studies 2004, 18:157-169. improvement have been identified. These improvements 4. Bryant L: In their own words: A model of healthy aging. Social will be made and tested before the next set of clinical trials Science and Medicine 2001, 53:927-941. begin, which are planned to be supervised community- 5. Cutchin M: The process of mediated aging-in-place: a theoret- ically and empirically based model. Social Science and Medicine based (as opposed to long-term care facility-based) trials 2003, 57:1077-1090. starting in 2009. It is hoped that the next set of trials will 6. Intille SS: A New Research Challenge: Persuasive Technology allow the authors to answer these research questions more to Motivate Healthy Aging. IEEE Transactions on Information Tech- nology in Biomedicine 2004, 8:235-237. definitively with a lager sample size that includes a greater 7. Dura J, Stukenberg K, Kiecolt-Glaser J: Anxiety and depressive diversity in dementia levels. disorders in adult children caring for demented parents. Pscy- hology and Aging 1991, 6:467-473. 8. Aguglia E, Onor ML, Trevisiol M: Stress in the caregivers of In general the participants were less dependent on a Alzheimer's patients: An experimental investigation in Italy. human caregiver when COACH was used. As the effective- American Journal of Alzheimer's Disease and Other Dementias 2004, 4:248-252. ness varied considerably and seemed to be dependent on each individual's idiosyncrasies, these findings suggest Page 17 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 9. Pollack ME: Planning Technology for Intelligent Cognitive 31. Wolery M, Harris SR: Interpreting Results of Single-Subject Orthotics. 6th International Conference on AI Planning and Scheduling Research Designs. Physical Therapy 1982, 62:445-452. 2002. 32. Granger CV, Linn RT: Biologic Patterns of Disability. Journal of 10. Mann WC, Ottenbacher KJ, Fraas L, Tomita M, Granger CV: Effec- Outcome Measurement 2000, 4:595-614. tiveness of assistive technology and environmental interven- 33. Mihailidis A: The development of an intelligent cognitive tions in maintaining independence and reducing home care orthosis to facilitate handwashing for persons with moder- costs for the frail elderly. Archives of Family Medicine 1999, ate-to-severe dementia. In PhD University of Strathclyde, Bioen- 8:210-217. gineering Unit; 2001. 11. Pentney W, Philipose M, Bilmes J, Kautz H: Learning Large Scale 34. Wickens CD: Signal Detection, Information Theory, and Common Sense Models of Everyday Life. 27th Annual Confer- Absolute Judgement. In Engineering Psychology and Human Perform- ence of AAAI; Vancouver, BC 2007. ance 2nd edition. Edited by: Pearson LC, Claire M. New York: Harper 12. Pollack ME: Autominder: A Case Study of Assistive Technol- Collins Publishers Inc; 1992:24-73. ogy for Elders with Cognitive Impairment. Generations 2006, 35. Altman DG: Practical statistics for medical research London: Chapman 30:67-79. and Hall; 1991. 13. Helal S, Mann W, El-Zabadani H, King J, Kaddoura Y, Jansen E: The 36. Reisberg B, Ferris SH, de Leon MJ, Crook T: The Global Deterio- Gator Tech Smart House: A Programmable Pervasive ration Scale for assessment of primary degenerative demen- Space. IEEE Computer 2005, 38:50-60. tia. American Journal of Psychiatry 1982, 139:1136-1139. 14. Pigot H, Lussier-Desrochers D, Bauchet J, Giroux S, Lachapelle Y, 37. Boger J, Hoey J, Poupart P, Boutilier C, Fernie G, Mihailidis A: A plan- (Eds): A Smart Home to Assist in Recipe Completion. ning system based on Markov decision processes to guide Amsterdam, The Netherlands: IOS Press; 2008. people with dementia through activities of daily living. IEEE 15. Lindsay J, Anderson L: Dementia/Alzheimer's Disease. In BMC Transactions on Information Technology in Biomedicine 2006, 10:323-333. Women's Health Ottawa: University of Ottawa; 2004. 38. Mihailidis A, Boger J, Canido M, Hoey J: The use of an intelligent 16. Mathieson K, Kronenfeld J, Keith V: Maintaining Functional Inde- prompting system for people with dementia: A case study. pendence in Eaderly Adults: The roles of Health Status and ACM Interactions (Special issue on Designing for seniors: innovations for Financial Resources in Predicting Home Modifications and graying times) 2007, 14:34-37. Use of Mobility Equipment. The Gerontologist 2002, 42:24-31. 39. Mihailidis A, Carmichael B, Boger J: The use of computer vision in 17. McCreadie C, Tinker A: The acceptability of assistive technol- an intelligent environment to support aging-in-place, safety, ogy to older people. Ageing and Society 2005, 25:91-110. and independence in the home. IEEE Transactions on Information 18. Wilson R, Rochon E, Mihailidis A, Leonard C, Lim M, Cole A: Exam- Technology in Biomedicine 2004, 8:238-247. ining effective communication strategies used by formal car- egivers when interacting with Alzheimer's disease residents Pre-publication history during an activity of daily living (ADL). Brain and Language 2007, The pre-publication history for this paper can be accessed 103:199-200. 19. Hoey J, Poupart P, von Bertoldi A, Craig T, Boutilier C, Mihailidis A: here: Automated Handwashing Assistance For Persons With Dementia Using Video and A Partially Observable Markov Decision Process. Computer Vision and Image Understanding – Spe- http://www.biomedcentral.com/1471-2318/8/28/prepub cial Issue on Computer Vision Systems . 20. Boger J, Poupart P, Hoey J, Boutilier C, Fernie G, Mihailidis A: A Decision-Theoretic Approach to Task Assistance for Per- sons with Dementia. International Joint Conference on Artificial Intel- ligence (IJCAI); Edinburgh 2005:1293-1299. 21. Hoey J, von Bertoldi A, Poupart P, Mihailidis A: Assisting persons with dementia during handwashing using a partially observ- able Markov decision process. The 5th International Conference on Computer Vision Systems. Germany 2007. 22. Mihailidis A, Barbenel JC, Fernie GR: The efficacy of an intelligent cognitive orthosis to facilitate handwashing by persons with moderate-to-severe dementia. Neuropsychological Rehabilitation 2004, 14:135-171. 23. Hoey J: Tracking using flocks of features, with application to assisted handwashing. British Machine Vision Conference (BMVC) Edinburgh, Scotland 2006. 24. Kaelbling LP, Littman ML, Cassandra AR: Planning and acting in partially observable stochastic domains. Artificial Intelligence 1998, 101:99-134. 25. Labelle K, Mihailidis A: Facilitating handwashing in persons with moderate-to-severe dementia: Comparing the efficacy of verbal and visual automated prompting. American Journal of Occupational Therapy 2006, 60:442-450. 26. Folstein MF, Folstein SE, McHugh PR: Mini-Mental State: A prac- Publish with Bio Med Central and every tical method for grading the cognitive state of patients for scientist can read your work free of charge the clinician. Journal of Psychiatric Research 1975, 12:189-198. 27. Franklin RD, Allison DB, Gorman BS: Design and Analysis of Single-Case "BioMed Central will be the most significant development for Research Mahwah, New Jersey: Lawrence Erlbaum Associates; 1996. disseminating the results of biomedical researc h in our lifetime." 28. Harris S, Brooks D: N of One: Single Case Research Design for Sir Paul Nurse, Cancer Research UK the Practising Clinician. CPA Research Division Newsletter; 1992:10-13. Your research papers will be: 29. Kaewtrakulpong P, Bowden R: An improved adaptive back- available free of charge to the entire biomedical community ground mixture model for real time tracking with shadow detection. In Proceedings of 2nd European Workshop on Advanced peer reviewed and published immediately upon acceptance Video-Based Surveillance Systems; September 4, 2001; London, U.K Klu- cited in PubMed and archived on PubMed Central wer Academic Publishers; 2001. 30. Portney LG, Wathins MP: Chapter 12: Single-Subject Designs. yours — you keep the copyright In Foundations of Clinical Research: Applications to Practice Upper Saddle BioMedcentral River, NJ: Prentice Hall Health; 2000:223-264. Submit your manuscript here: http://www.biomedcentral.com/info/publishing_adv.asp Page 18 of 18 (page number not for citation purposes) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png BMC Geriatrics Springer Journals

The COACH prompting system to assist older adults with dementia through handwashing: An efficacy study

Loading next page...
 
/lp/springer-journals/the-coach-prompting-system-to-assist-older-adults-with-dementia-Lc5N1SscJ7

References (56)

Publisher
Springer Journals
Copyright
Copyright © 2008 by Mihailidis et al; licensee BioMed Central Ltd.
Subject
Medicine & Public Health; Geriatrics/Gerontology; Aging; Rehabilitation
eISSN
1471-2318
DOI
10.1186/1471-2318-8-28
pmid
18992135
Publisher site
See Article on Publisher Site

Abstract

Background: Many older adults with dementia require constant assistance from a caregiver when completing activities of daily living (ADL). This study examines the efficacy of a computerized device intended to assist people with dementia through ADL, while reducing caregiver burden. The device, called COACH, uses artificial intelligence to autonomously guide an older adult with dementia through the ADL using audio and/or audio-video prompts. Methods: Six older adults with moderate-to-severe dementia participated in this study. Handwashing was chosen as the target ADL. A single subject research design was used with two alternating baseline (COACH not used) and intervention (COACH used) phases. The data were analyzed to investigate the impact of COACH on the participants' independence and caregiver burden as well as COACH's overall performance for the activity of handwashing. Results: Participants with moderate-level dementia were able to complete an average of 11% more handwashing steps independently and required 60% fewer interactions with a human caregiver when COACH was in use. Four of the participants achieved complete or very close to complete independence. Interestingly, participants' MMSE scores did not appear to robustly coincide with handwashing performance and/or responsiveness to COACH; other idiosyncrasies of each individual seem to play a stronger role. While the majority (78%) of COACH's actions were considered clinically correct, areas for improvement were identified. Conclusion: The COACH system shows promise as a tool to help support older adults with moderate-levels of dementia and their caregivers. These findings reinforce the need for flexibility and dynamic personalization in devices designed to assist older adults with dementia. After addressing identified improvements, the authors plan to run clinical trials with a sample of community-dwelling older adults and caregivers. Background the oldest old (aged 80 years and over) after the year 2010 Globally, the number of individuals aged 65 years and [1]. This will result in an increase in the worldwide older is predicted to increase steadily, particularly among number of individuals diagnosed with dementia, particu- Page 1 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 larly Alzheimer's Disease (AD), from the current estimate uted throughout the house interact with applications run- of 24.3 million individuals in 2006 to 81.1 million by ning on computers to take into account context when 2040 [2]. performing actions. For example, if it is a sunny day out- side and the resident has the television on, the Gator Tech Older adults have a strong preference for aging-in-place Smart House [13] will automatically close the blinds to (i.e. remaining in their own homes and communities) reduce glare. Other features include medication reminders compared to other forms of care, such as nursing homes that can appear on the bathroom mirror and automatic and other long-term care facilities [3]. Additionally, vari- sensing and ordering for soap and toilet paper refills. ous studies have implied that older adults (particularly Pigot et al. [14] developed Archipel, a cognitive modeling those who have AD) benefit from aging in environments system for cooking tasks that recognizes the user's to which they are accustomed as familiar environments intended plan and adapts prompting to a pre-determined can provide memory and task cues [4-6]. However, this cognitive impairment level. Sensors, such as RFID tags shift from the hospital to home-based care means that and readers, in the kitchen environment detect which family members and other informal caregivers are being objects have been used and provide cues (audio, video increasingly depended upon to attend to the long-term and strategic lighting) to help users through each step in health-care needs of older adults with AD. Increased the task. As with Autominder, Archipel will not give dependence and changes in the relationship dynamic are reminders for tasks the user has already accomplished. difficult for both people with AD and their family caregiv- ers to accept [7]. The constant pressure to meet their rela- Research is increasingly emphasizing the importance of tive's needs for assistance and support can result in maintaining functional independence in older adults as a debilitating levels of stress for the caregiver, resulting in way of maintaining good health and wellness among the affected person's placement into long-term care. From older adults with dementia, while simultaneously reduc- a caregiver's perspective, decreasing the number of inter- ing medical expenditures [15,16]. However, the extent to actions required to complete an activity of daily living which CATs can aid an individual with AD depends on the (ADL) has a direct positive impact on caregiver burden. users' willingness to implement it, which in turn depends Even small decreases in caregiver burden have been found on whether the individual and/or his/her caregiver can to alleviate the prevalence of depressive symptoms in car- operate the device, feels that the device is useful, and egivers of individuals with AD [8]. This can lead to more whether the device supports or undermines the sense of successful informal care, resulting in lower medical costs personal identity [17]. To be useful to both a care recipi- and delayed long-term care placements. ent with dementia and his/her caregiver(s), a CAT must be autonomous, non-invasive, and must not require explicit To support aging-in-place, older adults and their caregiv- feedback (e.g. button presses), as this cannot reasonably ers are increasingly relying on the use of computerized be expected of either people with AD or overworked car- Cognitive Assistive Technologies (CATs) to complete ADL egivers. Cognitive assistance should be able to accommo- [6]. Often coupled with some form of artificial intelli- date high levels of customization as the more the gence (AI), CATs strive to support cognitive disorders assistance is personalized and appropriate to the deficits thereby enhancing the user's autonomy [9]. The mainte- in question, the more likely it will be adhered to and nance or increase of independence is coupled with a understood by the user [18]. Finally, assistance should reduction in the levels of caregiver assistance, and likely only be given on an "as needed" basis to minimize confu- caregiver burden, as well as a decrease in home heath care sion and to keep the user as cognitively involved in the costs [10]. task as possible. A significant amount of recent work in CATs for assisting The majority of currently available CATs require extensive people with cognitive impairments use probabilistic mod- sensor deployment and maintenance and/or input from a els to infer task and occupant status from sensors distrib- cognitively intact individual. Most likely the caregiver of uted throughout a person's living environment [11]. For the individual with dementia would have to learn how to example, Autominder, developed by Pollack et al. [12], operate and (to some degree) maintain a potentially com- uses artificial intelligence planning to schedule events plex planning system. As many caregivers are overbur- such as medication taking around a person's daily sched- dened as it is, two goals of the system described in this ule, such as favorite television programs or daily walks. paper were to minimize the amount of hardware that was Autominder uses environmental sensors to detect the sta- needed, and to have the system function without any tus of activities, and if required, will provide the user with explicit input from the user or the caregiver. context-aware reminders regarding unattended activities. The Gator Tech Smart House is an example of a smart The result was the COACH (Cognitive Orthosis for Assist- home designed with older adults in mind. Sensors distrib- ing aCtivities in the Home), a system that employs various Page 2 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 computer vision and artificial intelligence techniques to should be reflected in an increase of the number of steps autonomously provide the user with verbal and/or visual in the handwashing activity the older adult is able to com- reminders as necessary during ADL. Table 1 summarizes plete independently from a caregiver (i.e. with no assist- the progression of the systems used in the previous ver- ance from the caregiver). sions of COACH. The systems in each version of COACH represent significant advances in the sophistication and 2. Does this new version of COACH reduce caregiver versatility compared to those used in the previous version. workload? If the caregiver's workload is reduced, this The systems for the latest version of COACH (Version 3 in should be reflected in a decrease in the number of times a Table 1) are described in more detail in the Methods sec- caregiver interacts with his/her care recipient. tion below. 3. How will the COACH system perform with respect to its This paper presents results from an eight-week efficacy ability to correctly provide assistance to the user through- study of the COACH with older adults with dementia. out the ADL? To achieve a positive outcome, the system Methods and results are presented, followed by a discus- must be able to follow the older adult through the hand- sion regarding the potential clinical significance of the washing task, autonomously giving the correct prompt if participants' and device performances. While a brief (and only if) they are needed. description of the technology will be provided in this paper, the reader is referred to [19] for an in-depth Methods Device (COACH) design description of the COACH system and algorithms. In this work the authors extend upon the two previous versions of the COACH device (summarized in Table 1), Objective The objective of this study was to answer the following which both focused on the activity of handwashing [20- research questions: 22]. Handwashing was chosen as the model ADL because it is a relatively safe activity that older adults with demen- 1. Is the COACH system able to guide an older adult with tia have difficulties completing because of the required dementia through the handwashing ADL with less planning and initiation skills. dependence on a caregiver? If dependence decreases it Table 1: Summary of previous COACH systems. COACH Version Tracking System Decision-making Prompting system Number of Related system Subjects* Publications Version 1 Pattern wristband Neural networks Audio prompts, with 10 [22,33] worn by the user interacting with a one prompt for each hard-coded taxonomy step Version 2 Using background Fully observable Audio prompts with 4 [20,37-39] subtraction to isolate Markov decision three levels of the user's hands. process (MDP). This assistance (minimal, Tracking of hands and technique assumes the moderate, and task objects (i.e. soap world is fully maximal) for each and towel) using a observable; it does step. preset colour model. not take into account hidden variables, such as user responsiveness. Version 3 (system Colour based flocking. [Belief monitoring Audio and audio-video 6 [18,19,21,23,25] presented in this system & policy] prompts with three paper) Partially observable levels of assistance Markov decision (minimal, maximal, process (POMDP). and maximal + video This model takes into demonstration) for account hidden each step. variables and is able to Encouragement and a make decisions in reminder regarding conditions of the activity the user is uncertainty. attempting added. Professional actor recorded prompts. * Results from previous trials are not presented as they are not comparable because of the variance in the technologies and procedures that were used. Page 3 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 Handwashing was defined as having five essential steps icy into an action for COACH to take. Possible actions that must be accomplished for successful activity comple- available to the COACH are to give a low-guidance verbal tion, which are depicted in Figure 1. COACH guided users prompt, give a high-guidance verbal prompt, give an ver- through these steps using four integrated components: the bal prompt with a video demonstration of the action, call tracking system, belief monitoring system, policy, and the caregiver to intervene, or to do nothing (i.e., continue prompting system, as represented in Figure 2. to observe the user). The COACH's different levels of prompting assistance give COACH the ability to select the Images captured by a video camera are processed by the most appropriate support for each individuals' stage of tracking system and the hand and towel positions are AD and overall responsiveness. Thus the level of detail passed to the belief monitoring system. These data are played for the user is based on factors such as the error used by the belief monitoring system to compute the belief committed, sensory and cognitive status of the user, and state; a probabilistic estimation of the current state of the past responsiveness to the previous prompts. user and environment. The belief state is passed from the belief monitoring system to the policy, which is essen- The COACH system presented above had three significant tially a lookup table that denotes the best course of action changes from the previous versions: 1) The use of marker- for the system to take for every state that could be received less flocking to track the activity; 2) the use of a partially from the belief monitor. Each belief state that is received observable Markov decision process (POMDP) to model the from the belief monitoring system is translated by the pol- handwashing guidance problem; and 3) the refinement of audio prompts and the addition of video demonstrations. Tracking was accomplished using a computer vision tech- nique known as flocking, which was developed by Hoey et al. [23]. It uses models of skin and towel color com- bined with a Bayesian sequential estimation technique. This method of tracking is quite robust and able to dependably track the location of the user's hands and the position of the towel, even after occlusion by an object or after leaving and returning to the camera's field of view. A POMDP was chosen as the basis for the new planning sys- tem because of this model's ability to make good deci- sions in situations of uncertainty, as well as making intelligent inferences, and therefore decisions, about unobservable states (e.g. a user's level of dementia) [24]. This type of model allows the COACH system to autono- mously tailor itself to the individual needs of its users because it can estimate and use individual's traits (e.g. cognitive awareness and responsiveness levels) to dynam- ically adapt to daily and long term needs. Implementation of a POMDP is an important contribution to not only the field of artificial intelligence but to the usability concerns of users and their caregivers as it enables user-specific prompting strategies while remaining autonomous. Greater details regarding the technical nature of COACH, including system detailed descriptions and planning algo- rithms, can be found in [19]. Audio prompts were recorded using a professional male actor to emulate the cadence and tone of a professional Th Figure 1 e five essential steps of handwashing caregiver. A male voice was used (as opposed to a female The five essential steps of handwashing. Successful one) because previous research by this group and others activity completion was considered to be any sequence of suggests that male voices are easier to hear and under- steps that took the participant from "Start" to "Finish". As stand, possibly because the male voice has a lower pitch/ the long-term care facility's guidelines required the use of liq- frequency [22]. The wording used for the prompts is uid soap wetting one's hands before getting the soap was not shown in Table 2 and was similar to the wording used in considered an essential step in the activity, therefore the water on and soap used steps are interchangeable. previous studies, modified slightly according to recom- mendations from Wilson et al. [18]. Prompts included the Page 4 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 A Figure 2 flow diagram of COACH components A flow diagram of COACH components. Images from the camera are translated into hand and towel positions by the tracking system. These are passed to the belief monitor, which calculates the probability distribution over the possible states. This belief state is passed to the policy, which selects an action for COACH to take (i.e. prompt, observe user, or call car- egiver). participant's name at the beginning of each prompt to get pre-recorded in the same washroom that was used in the his/her attention. Previous studies with COACH have trials and combined with the maximal assistance verbal found that some users can get confused about which activ- prompts. A frame from one of the videos is shown on the ity they were asked to complete (e.g. previous participants monitor in Figure 3b. have been known to wash the towel in the sink, wash his/ her face, etc.), therefore the prompt often contained a Participants and Ethics/Consent Process reminder to help participants remember which activity This study was reviewed and approved by the Toronto they were attempting to complete. The prompt then gave Rehabilitation Institute's Research Ethics Board (REB). the participant guidance for the step in the activity s/he Potential participants were identified by the staff at the was attempting. The potential usefulness of adding video long-term care (LTC) facility in Toronto, Canada where demonstrating correct completion of the activity step was the study took place. Informed consent to participate was examined by Labelle and Mihailidis [25]. Results were obtained in writing (using a the consent form approved positive; therefore audio-video capabilities were added to by the REB) from the participants' substitute decision this version of COACH. The videos used in this study were makers, after the study was described to them using an shot from the perspective of the participant. They were information sheet and informal interview. Table 2: Wording for the prompts used by COACH. Step Minimal verbal assistance Maximum verbal/video assistance* Turn on the water [Name], you're washing your hands. Can you turn the [Name], try turning the silver knobs. water on? Use the soap [Name] you're washing your hands. Please use the soap. [Name] you're washing your hands. Try putting on some soap. Rinse hands [Name], you're washing your hands. Please rinse your [Name], you're washing your hands. Try putting your hands in hands. the water. Turn off the water [Name] you're doing great. Can you turn the water off? [Name], you're doing great. Twist the knobs to turn the water off. Dry hands [Name], you're doing great. Dry your hands now. [Name], try using the towel. * The same wording was used for the maximum verbal assistance and video assistance prompts. Page 5 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 E Figure 3 xample of the COACH system setup Example of the COACH system setup. (a) Example of the LCD screen, speakers (top-right), and video camera (circled) setup. (b) Example of a video-based prompt demonstrating how to turn on the water. Participants had to meet the following inclusion/exclu- Research DragonFly2), and a wall-mounted 21-inch LCD sion criteria: over the age of 65 years of age, no history of screen and desktop speakers (Figure 3a). A Dell Latitude violence, fluent in English, can hear normal levels of laptop computer (2 GHz processor, 2 Gb RAM) was used speech, exhibit no severe motor impairments, and have as the processing unit for the system software and hard- moderate-to-severe dementia. Level of dementia was ware, as well as the operator graphical user interface to determined through the administration of the Mini-Men- display information about the system variables (e.g. esti- tal State Examination (MMSE), an assessment instrument mated plan steps, system response, etc.), and the partici- that is commonly used to estimate the level of cognitive pant's progress through the task. The trials were also impairment in adults [26]. Typically, participants are sep- recorded using a camcorder positioned above the partici- arated into four categories of impairment based on his/ pant to capture video for post-trial evaluation by human her MMSE score: no impairment (30–26 points), mild raters (but was not used by the system during the trials). (25–20 points), moderate (19–10 points), and severe (9- Study design 0 points). Each participant's dementia level was scored using the MMSE before the start and upon completion of A single subject research design (SSRD) was used in this the trials study because of the difficulty in recruiting and maintain- ing an adequate sample size and the variability of the par- Apparatus set-up ticipants' health [27-31]. This research design has been The study was conducted in a retrofitted washroom and used in the authors' previous studies and has been found adjoining office that were dedicated to the project by the to be the most appropriate procedure for the evaluation of LTC facility. The washroom was fitted with a ceiling- this type of technology. The study consisted of two base- mounted IEEE-1394 digital video camera (Point Grey line phases, A and A (COACH not used), and two inter- 1 2 Page 6 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 vention phases, B and B , (COACH used), run in the would be technically incorrect, the participant would still 1 2 order A -B -A -B to identify any carry-over effects. Based score a five on independent step completion. Number of 1 1 2 2 on studies completed with previous versions of COACH, caregiver interactions was a count of the number of times 10 trials per phase were deemed to be sufficient for partic- the caregiver had to interact with the participant to get ipant performance to stabilize and for the desired changes him/her to complete a step. An interaction was considered in the dependent variables to be observed [22]. to be any exchange between the caregiver and the partici- pant that was related to activity completion, including Procedure verbal prompting, demonstration, and touching (either Trials consisted of one trial per day per participant, Mon- the participant or an object). The functional assessment days to Fridays, for eight weeks for a total of 40 trials each. score (FAS) is a modified version of the Functional Inde- To ensure uniformity and avoid any potential risk of pendence Measure (FIM™), which is a standardized assess- injury from falls, each participant was required to sit in a ment tool used to measure one's ability to function with wheelchair and was taken to the test washroom by a car- independence over 18 activities of daily living [32]. Partic- egiver who was hired for this study. The caregiver posi- ipants received an FAS for each step in the activity and tioned the participant in front of the sink in the test scores ranged from zero (no attempt/refusal) to seven washroom and asked the participant to wash his/her (complete independence), with an overall maximum of hands. 35. If the participant completed the step in response to prompts provided by the COACH, a score of seven was During the A-phases of the trials, the caregiver interacted given. A higher cumulative FAS is expected to correlate with the participants as she normally would, providing with higher levels of activity completion independence. any prompts and reminders she felt were necessary to The face validity of the FAS was demonstrated in previous complete handwashing. During the B-phases, COACH trials by Mihailidis et al. [22,33]. was started by a researcher (who was hidden from the user) as soon as the caregiver requested the participant to During the B-phases, data were collected regarding the wash his/her hands. The caregiver then left the participant system responses to participant performance during the alone in the test washroom and discreetly observed him/ handwashing activity. These data were collected based on her from the hallway. The caregiver provided assistance the basic principles of signal detection theory (SDT) [34], only if instructed to do so by COACH (i.e., the caregiver which can be used to measure four conditions describing was summoned by the device to intervene) or if the car- device performance with respect to: hits, false alarms, cor- egiver felt the need to intervene for the well being of the rect rejects, and misses. These conditions with respect to participant (e.g., the participant was attempting to stand the COACH system are outlined in Table 3. For each step up from the wheelchair or was becoming upset). in the activity, the system was rated as having at least one, and potentially more, of the four possible SDT condi- Data collection tools tions. For example, if the COACH gave three incorrect A score sheet was used to collect data required to evaluate prompts and one correct prompt for a step, three false the system's efficacy in terms of both user and system per- alarms and one hit would be scored. formance. The score sheet was the same one that was developed and used in studies that examined previous Analysis of participant and device performance Video of each trial was reviewed and scored by an experi- versions of the COACH (refer to [22]). enced rater using a multi-modal score sheet to collect the With respect to user performance, scales on the score sheet types of data described previously. An experienced rater measured the following for each step of the activity: 1) was a researcher who was trained on the scoring methods independent step completion; 2) number of caregiver and has had previous experience rating COACH trials. interactions; and 3) functional assessment scale (FAS). Independent step completion was scored for every trial. Participants scored one point for the first time s/he com- Table 3: The four possible conditions used to determine pleted a step in a trial without assistance of any kind from COACH's performance. a human caregiver. As there were five essential steps (Fig- COACH Response ure 1) the maximum score that could be attained was five, even if a participant completed more than five steps inde- Prompt No prompt pendently. For example, a participant could independ- ently turned the water on, wet her hands, turned off the Participant Action Error Hit Miss water, got some soap, turned the water on again, rinsed her hands, turned off the water, and finally dried her No error False Alarm Correct Reject hands. Although none of the actions in this sequence Page 7 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 Space on each score sheet was provided for any general Participants comments or observations. Eight participants were recruited for this study, however two were withdrawn; S2 developed unrelated health prob- Because of the small number of participants in the study, lems, and S7's aggressive behavior caused concerns for the visual analyses of the data were used to identify trends of wellbeing of both herself and the study caregiver. Demo- participant behaviors and abilities, and compare changes graphics for the remaining six participants are presented in variability between phases. Visual analysis is a com- in Table 4. Based on his/her initial MMSE scores, five par- monly used technique for single-subject research designs. ticipants (S3, S4, S5, S6 and S8) were classified as having Data were examined for all trials and overall trends of in- moderate-level dementia, and one participant (S1) was group performance between baseline (A) and interven- classified as having severe-level dementia. tion (B) phases, as well as for variations in participant per- formance. Observed participant behaviors and reactions Participant performance were used to aid in the analysis of the results. As S1 was the only participant in the severe-level group and noticeably different trends from the other partici- Analyses of the device performance data were achieved pants, this sub-section examines the moderate-level par- through the calculation of the number of hits, misses, ticipants (S3, S4, S5, S6 and S8) as a group. Table 5 false alarms, and correct rejects (described in Table 3) summarizes overall individual participant performance made by the system during the intervention (B) phases. per test phase, which shows improvements in all three These data were also used to calculate two types of error: areas, particularly in a reduction in the number of interac- E (Equation 1) which reflects COACH not detecting an tions with the caregiver. From Table 6 it can be seen that error when participants made one, thus not giving a four of the five participants were able to independently prompt, and E (Equation 2) which reflects COACH complete the activity. Table 7 shows the overall number of detecting an error when none occurred, thus erroneously interactions with the caregiver required by the participant giving a prompt. These equations were derived by Mihai- to successfully complete essential handwashing steps, lidis [33] for the analysis of previous research on COACH. which decreased by an average of 66% when the device was introduced. Table 8 shows the participants' FAS for the handwashing activity increased by a negligible 2% for Misses (1) E = × 100 W the group. Figures 4 to 6 depict the daily average perform- Hits+ Misses ance for the entire moderate-level participant group (n = 5) for the number of steps completed independently, the False Alarms (2) number of interactions with a caregiver, and FAS respec- E = × 100 False Alarms+Correct Rejects tively. Results Device performance Inter-rater agreement A summary of the data regarding COACH performance is To ensure data reliability, a second experienced rater presented in Table 9. It should be noted that the item Par- scored 20 percent of all data collected regarding partici- ticipant ignored prompt from COACH in Table 9 represents pant performance and an inter-rater agreement was calcu- the combined number of both ignored hits and ignored lated using Cohen's Kappa (using SPPS v15.0) [27]. The and E (described by Equa- false alarms. The error rates, E w c measures of agreement (K values) were K = 0.96 (p < tions 1 and 2), were found to be 10.9% and 26.0% respec- 0.0005) for independent step completion, K = 0.69 (p < tively. This can be interpreted as COACH not responding 0.0005) for number of caregiver interactions, and K = 0.63 to 10.9% of the errors made by participants (E ) and (p < 0.0005) for FAS. COACH making an error in 26% of the cases where the participant was completing the step correctly (E ). Table 4: Demographics of the participants. Participant Gender Age (years) Education MMSE: study start MMSE: study completion Average MMSE S1 F 88 High school 5 3 4 S3 F 73 Post-secondary 12 18 15 S4 F 92 Elementary school 10 13 12 S5 M 81 Post-secondary 19 20 20 S6 F 87 High school 12 14 13 S8 F 89 Post-secondary 11 10 11 Page 8 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 Table 5: Average participant performance for each trial phase and overall group performance. Participant [Average MMSE Phase Mean number of steps Mean number of interactions Mean FAS* (out of 35) score] completed independently with human caregiver (out of 5) Severe-level dementia** S1 [4] A1 0.1 14.1 2.4 B1 1.8 10.6 17.1 A2 0.6 16.2 7.3 B2 0.9 20.6 11.2 Moderate-level dementia S3 [15] A1 5.0 0.0 35.0 B1 5.0 0.0 35.0 A2 4.9 0.2 34.8 B2 5.0 0.0 35.0 S4 [12] A1 3.6 2.6 29.9 B1 5.0 0.0 34.7 A2 4.5 1.4 33.9 B2 5.0 0.0 34.5 S5 [20] A1 3.3 4.4 30.7 B1 4.1 2.6 31.6 A2 3.8 3.8 32.2 B2 4.9 0.3 33.2 S6 [13] A1 5.0 0.0 35.0 B1 5.0 0.0 34.9 A2 5.0 0.3 34.8 B2 5.0 0.0 34.9 S8 [11] A1 4.6 1.9 33.2 B1 5.0 1.3 33.1 A2 4.5 2.2 32.8 B2 5.0 2.6 32.3 Mean score over phases for the A1&A 4.4 1.7 33.2 moderate-level group 2 B1&B2 4.9 0.6 34.1 % change*** 11 -66 2.4 Functional assessment score ** See Tables 6 to 8 for mean scores for S1 *** Calculated by [(B +B )-(A +A )]/(A +A )*100 1 2 1 2 1 2 lower end of the agreement range. Although both raters Discussion Inter-rater agreement had previous experience and agreed on scoring conven- Altman [35] has indicated that values of K can be inter- tions, many instances were more difficult to score than preted as very good agreement if they are between 0.81 one might imagine. For example, 'guidance' was consid- and 1.00 and as good agreement if they are between 0.61 ered to be a prompt and could be a verbal cue, visual cue, and 0.80. While the K-value for independent step comple- tactile cue or any combination of the three. When com- tion and the p-values obtained for all three measures were bined, prompts were difficult to clearly distinguish, such good, the K-values for the observed number of caregiver as when verbal guidance was followed closely by visual interactions (K = 0.69) and FAS (K = 0.63) were at the guidance. In this particular example, it could be inter- Table 6: Average number of steps per trial completed independently without (Phase A) and with (Phase B) COACH Participant [MMSE] Mean number of steps completed Mean number of steps completed Change (%) independently in Phase A (out of 5) independently in Phase B (out of 5) S1 [4] 0.38 1.33 250 S3 [15] 4.95 5.00 1 S4 [12] 4.10 5.00 22 S5 [20] 3.60 4.53 26 S6 [13] 5.00 5.00 0 S8 [11] 4.57 5.00 9 Page 9 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 Table 7: Average number of interactions with the caregiver per trial without (Phase A) and with (Phase B) COACH Participant [MMSE] Mean number of interactions in Phase A Mean number of interactions in Phase B Change (%) S1 [4] 15.19 15.56 2 S3 [15] 0.11 0.00 -100 S4 [12] 1.95 0.00 -100 S5 [20] 4.10 1.33 -68 S6 [13] 0.15 0.00 -100 S8 [11] 2.05 1.94 -5 preted as a single prompt (a verbal plus visual cue) by one third of the original number of interactions to complete rater and as two (first a verbal prompt, followed by a sep- handwashing. The change in the amount of assistance the arate visual prompt) by the second rater. Similar scoring participants required was mildly reflected by their FASs difficulties were encountered with the FAS, where the rat- (presented in Table 8), although the change is not as obvi- ings one through four represent the rater's opinion of how ous as it is with independence and interaction with car- much of the step (0 to 25%, 25 to 50%, 50 to 75%, 75 to egiver measures (Tables 5 and 6). S4's independence from 100%) the participant completed independently after a human caregiver when COACH is shown by her increase being given a prompt by the caregiver. to a near perfect FAS. The reason S4 did not achieve a per- fect score is because she was a relatively slow hand washer Participant performance and the FAS dictates that a score of 6 is assigned when the The participants showed a general improvement in the participant "took more than reasonable time to complete handwashing task when COACH was used, reflected in step". Participants S3 and S6 were able to perform most of more steps completed independently, fewer interactions the steps in handwashing independently before the intro- with the caregiver, and higher FASs. Results presented in duction of COACH, therefore these participants had no Table 6 suggest that improvements in handwashing inde- change in their FAS because there was little opportunity pendence when COACH is used are inversely propor- for them to improve. tional to a person's baseline performance, with greater levels of improvement being seen by those who are less S8 was a noteworthy subject as she was highly independ- independent when washing his/her hands. Moreover, ent, but would consistently omit the soap application four moderately impaired participants who routinely step. Although COACH provided a prompt for her to do required some assistance (S3, S4, S6, and S8) competed all so at almost every instance, she ignored these prompts five handwashing steps completely independently the and would only respond to verbal prompts from a human majority of the time COACH was used. It appears that the caregiver. Thus, essentially no change was seen in S8's improvement is a result of device use rather than activity number of caregiver interactions or FAS. S8 provides a learning by the participants, supported by the fact that good example of a user (in terms of ADL and cognitive performance improved when COACH was used in B , fell abilities) who may not be a good candidate for this style when it was removed in the A phase, and was regained in of computer-based guidance as her idiosyncrasies resulted B when COACH was reintroduced (as seen in Table 5). in compliance with verbal cues only when they are given There was a noticeable trend towards a decrease in the by a human. number of interactions with the caregiver during the inter- vention phases, with three of the subjects (S3, S4 and S6) For the more independent participants (S3 and S6) requiring no human assistance when COACH was used. COACH appears to function more as a "maintenance" S5 also showed a considerable decrease, requiring only a tool, able to support the participant in the occurrence of Table 8: Average participant FAS scores per trial without (Phase A) and with (Phase B) COACH Participant [MMSE] Mean FAS* in Phase A Mean FAS* in Phase B Change (%) S1 [4] 4.95 14.14 186 S3 [15] 34.89 35.00 0 S4 [12] 32.00 34.60 8 S5 [20] 31.50 32.43 3 S6 [13] 34.90 34.89 0 S8 [11] 33.00 32.71 -1 * Functional assessment score Page 10 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 A A B B 1 2 1 2 Without COACH With COACH Without COACH With COACH 4.5 3.5 2.5 1.5 0.5 0 5 10 15 20 25 30 35 40 Trial Number Mean n Figure 4 umber of steps completed independently for all participants for each day of the trials Mean number of steps completed independently for all participants for each day of the trials. A and A are the 1 2 baseline phases (no use of COACH), B and B are the intervention phases (COACH used). 1 2 an occasional error. S4, who needed modest amounts of classified as having severe dementia. Moreover, she mostly verbal assistance from her caregiver, became essen- showed a notable decline in general abilities over the tially independent when COACH was in use. Results from course of the study, which is reflected in her low and trials involving S3 and S6 suggest that using COACH does decreasing MMSE scores (Table 4). Although there is not not have a detrimental effect on the performance of capa- enough data to support any significant conclusions, S1's ble individuals. This indicates that the device could poten- decline may explain why there is a slight increase in the tially be introduced in the early stages of dementia number of caregiver interactions when the device is used without any negative effects. An early introduction would (Table 7), even though her ability to complete steps inde- allow the system to learn about the user's preferred hand- pendently improved during the intervention phases washing regime, which in turn could enable the system to (Table 6). It is thought that while COACH helped to make better decisions when guiding the individual remind S1 which step came next (i.e., improving her inde- through the activity later on when s/he does require assist- pendence), she required more and more prompting as the ance. From this study, it appears that by providing study progressed and her dementia became more severe prompts to users with moderate-level dementia, COACH (i.e., increasing number of caregiver interactions). This successfully encouraged more independent behavior with supposition is supported by the prompting behavior greater relative results seen in individuals who required exhibited by COACH. In B , S1 completed a correct step higher levels of assistance to complete the task. as a result of five of COACH's 33 correct prompts while in B S1 did not complete any steps in response to COACH's S1 was an exceptional subject in the study. She had the 40 correct prompts. S1 had 10 trials that required an lowest MMSE score and was the only participant who was exceptionally high (for this sample group) 20 or more Page 11 of 18 (page number not for citation purposes) Mean ( ) Number of Steps Completed Independently BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 A A B B 1 2 1 2 Without COACH With COACH Without COACH With COACH 0 5 10 15 20 25 30 35 40 Trial Number Mean n Figure 5 umber of interactions with a human caregiver for all participants for each day of the trials Mean number of interactions with a human caregiver for all participants for each day of the trials. A and A are 1 2 the baseline phases (no use of COACH), B and B are the intervention phases (COACH used). 1 2 interactions with the caregiver to complete the handwash- ing the task were the ones who were most likely to become ing activity. S1 showed the most relative improvement in independent when the device was used. These findings self-sufficiency (particularly with rinsing off soap and agree with studies conducted with previous versions of the hand drying) and this is reflected by her FAS. However, COACH, and suggest that COACH has the potential to dependent people such as S1 may require the device to be increase the independence and autonomy of individuals a constant presence and they will likely need physical suffering from dementia. Ideally, a system like COACH assistance from a caregiver with some steps (in particular, will enable caregivers to perform other tasks while the getting soap). completion of ADL are supervised by the system because the system would bring any difficulties regarding task The results presented here support the use of this type of completion to the attention of the caregiver. Therefore, technology to increase user independence while decreas- while use of COACH would not eliminate the need for a ing number of interactions with the caregiver, with four caregiver (as the caregiver would have to still be present in out of five of the moderate-level subjects able to complete the home to provide assistance that is beyond the capabil- the activity without any assistance whatsoever from the ities of a computerised reminder system), it could poten- caregiver when the device was in use. The authors feel that tially augment the burden of constant supervision of his/ research questions one and two have been adequately her loved one. This would allow more free time for the addressed and that the use of COACH results in increased caregiver and more privacy for the person with dementia, independence from a caregiver as well as a reduction in which in turn would hopefully improve quality of life for caregiver burden for people with moderate-level demen- the dyad and delay long-term care placement. tia. Participants who had only minor difficulties complet- Page 12 of 18 (page number not for citation purposes) Mean ( ) Number of Interactions with a Caregiver BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 A A B B 1 2 1 2 Without COACH With COACH Without COACH With COACH 0 5 10 15 20 25 30 35 40 Trial Number Mean Fu Figure 6 nctional Assessment Score (FAS) for all participants for each day of the trials Mean Functional Assessment Score (FAS) for all participants for each day of the trials. A and A are the baseline 1 2 phases (no use of COACH), B and B are the intervention phases (COACH used). 1 2 Device performance and its functioning as an assistive technology can be As device performance examines the interactions between measured primarily by the ability of the COACH to iden- the device, the user, and their environment, the data from tify an error by the participant and provide correct assist- all six participants has been grouped and is presented in ance in response. During this study, COACH had a total the discussion below (as opposed to the participants' per- of 750 observed conditions, which are summarized in formance data that were separated according to cognitive Table 9 and Figure 7. Of these, 170 (23%) were errors (i.e., impairment). The overall efficacy of the COACH system a false alarm or a miss, as defined in Table 3). Misses and Table 9: Device performance with regards to COACH's response to participants' actions and participants' reactions to prompts given by COACH. Opportunity outcome* Water On Use Soap Rinse Soap Water Off Use Towel TOTAL Hit 16 62 15 48 6 147 Miss 1 11 1 3 2 18 False alarm 7 107 18 18 2 152 Correct reject 117 69 87 60 100 433 Steps completed correctly in response to a prompt from 14 6 18 1 30 COACH Prompts from the COACH ignored by participants 20 159 24 43 7 253 *Hits, misses, false alarms and correct rejects are defined in Table 3. Page 13 of 18 (page number not for citation purposes) Mean ( ) Functional Assessment Score BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 false alarms by the COACH occurred because the system or frustrated were less likely to complete the remainder of misinterpreted a step in the handwashing task. For exam- the task if subsequent correct cues were given. Three par- ple, an ambiguous user action such as touching the taps ticipants verbally responded to the device when a false may have caused the system to incorrectly presume that alarm was given, expressing that they had already com- the water had been turned on. Wrong assumptions by pleted the step they were being prompting to do. For COACH reduce the probability that the correct course of example, when prompted incorrectly by COACH, S6 action will be taken. If COACH is not able to correct itself would often ask "What [did] I do?" then in frustration through other observations, a prompt may then be either would tell the COACH to "Shut up". When misses missed or provided for the wrong step. occurred the participant was usually unable to complete the rest of the task on his/her own, even if COACH recov- From a clinical point of view, the impact of the device's ered from the miss and prompted the participant cor- errors on participant performance should be used to rectly, hence trials where misses occurred often had determine if the device error rate was acceptable. False COACH summoning the caregiver to intervene (which alarms sometimes irritated and confused the participants, was the correct action to be taken by the system by this and it was observed that participants who were confused time). Although this resulted in an increase in the amount Hit & Completed 4% Hit & Ignored 16% False Alarm & Responded 2% Correct Reject False Alarm & Ignored 58% 18% Miss 2% Obser Figure 7 ved COACH conditions with corresponding number of observations and percentage of the all observed conditions Observed COACH conditions with corresponding number of observations and percentage of the all observed conditions. Note that correct prompts (hits) and incorrect prompts (false alarms) are separated into the two observed par- ticipant reactions to the prompts: completed/responded or ignored. Page 14 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 of interactions between the participant and the caregiver issued because the participant did not need assistance. Of and a decrease in the number of steps the participant was the correct prompts given to participants, 30 (20% of all able complete on their own, this tended to result in less hits) resulted in the participant complying with the cue frustration for the participant than false alarms. Thus and completing the step accordingly (Figure 7). The while misses and false alarms do not seem to cause signif- number of correct prompts ignored by participants ranged icant upset in the participants, it is very likely the irritation dramatically from one (by S6) to 68 (by S1). Notably, the and confusion resulting from mistakes by COACH hin- number of correct prompts ignored by S1 (severely- dered participant performance. Reducing the number of impaired) account for over half (58%) of the total correct these types of system errors is the primary priority of prompts ignored by all participants, with only five of the future work on the COACH. correct prompts given to S1 resulting in completion of the step. In contrast, S4 was the most responsive to hits, com- This study uncovered sequences and scenarios that led to pleting a step in response to the COACH 60% of time she false alarm prompts in this particular COACH model. was provided with support. In the more severe stages of Analysis of the data revealed that 107 (70%) of all false AD individuals tend to lose the ability to respond to his/ alarm prompts issued were to "use the soap", equating to her environment, the ability to speak and, ultimately, the 36% of all prompts given by the device or 8% of all ability to control movement [36]. This may explain S1's observed conditions. Clearly, incorrect prompting for lack of response to the system; however, it does not soap use constitutes the majority of the errors committed explain why S5, who has the highest MMSE, had a lower by COACH. Further investigation of sequences where "use handwashing performance than any of the participants the soap" false alarm errors occurred revealed that 59 with moderate dementia. S5 responded well to COACH, (55%) of the "use the soap" false alarms took place after completing a step in 43% of the cases when a correct the completion of the handwashing task (i.e. after all of prompt (hit) was given. the five essential steps were completed), mostly when the participant was drying his/her hands. Closer examination Sixteen of the 152 false alarms that were given (11% of all of these trials revealed that the tracking, policy and false alarms) elicited reactions from the participants (Fig- prompting modules appeared to be working well, there- ure 7). It should be noted that a response to a prompt is fore the authors conclude the problem is likely in the not considered to be the same as the completion of a step, belief state monitor. For many of these trials, when the as an unsuccessful attempt to complete a step is also con- user reaches the end of the handwashing activity and is sidered a response. For instance, if COACH (correctly or drying his/her hands, the COACH correctly shows the incorrectly) prompted a participant to "turn the water on" activity as being complete. However, after several seconds and the participant touched the taps without altering the belief state monitor slowly changes its belief distribu- water flow, this would be considered a reaction to the tion from a strong belief that handwashing is complete to prompt, although no progress was made in the step itself. the belief that the participant's hands are not washed. Therefore, the response rate to false alarms of 11% is con- When the belief that the participant's hands were not siderably lower than the 20% completion rate for correct washed reached a sufficient level, a prompt to "use the prompts when considering the compliance rate with soap" occurs. This "drift" back to the beginning of the task prompts in terms of step completion. When participants was intentionally designed to give the system an opportu- did perform an incorrect action because of a false alarm nity to correct itself if it incorrectly believed that the user from COACH (such as using the soap when the partici- had completed the task. However, the clinical trials pant's hands were already washed), this often resulted in showed that the usefulness of this functionality may not backtracking in the activity, adding to the number of steps outweigh the clinical costs associated with the resulting required for activity completion. It was observed that the erroneous prompts, particularly if the participant is some- extra steps and confusion resulting from backtracking one who takes a long time to dry his/her hands. This prob- offered a greater opportunity for errors to occur. As such, lem is being corrected in the next version of COACH by assistance from the caregiver was usually needed to com- implementing a "cutoff" where the system terminates plete the task. At 34%, S5 had the highest response rate to guidance once the system's belief that the user's hands are system prompts regardless of whether they were a hit or a clean and dry reaches an empirically determined thresh- false alarm. S5's high compliance rate may explain why old, at which point the caregiver will be called to escort his FAS did not reflect his other improvements as much as the participant from the washroom. they did for the other participants in the study (see Tables 5, 6, and 7). When S5 responded to a false alarm, this As presented in Table 9, COACH had 580 correct condi- often led to backtracking in the activity and participant tions (77% of overall conditions), composed of 147 cor- confusion, requiring the caregiver to be summoned, who rect prompts (or hits) when participants required then had to use greater levels of assistance to re-orient S5, assistance and 443 correct rejects, where no prompt was which would result in lower FAS results. Page 15 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 COACH autonomously used observations of each partici- COACH waited a pre-determined amount of time before pant's actions and responses to prompts to estimate his/ giving a prompt to the participant if s/he was "stuck" on a her level of dementia (a parameter that changed slowly step, with COACH giving the participant ample time to over a course of days) and responsiveness (a parameter attempt the step on his/her own before prompting. that changed on a day-to-day basis). These parameters COACH's "patience" may have played a role in the greater played a large role in dictating the level of detail of the levels of independence seen when the device is used, as prompts given by COACH, which would select prompts participants would sometimes correctly resume the task that were appropriate for the individual's levels of respon- after a pause of a few seconds; a pause which may be siveness and dementia. From the technical results pre- longer than most human caregivers would care to wait. sented in Hoey et al. [19], COACH autonomously However, COACH would sometimes be too patient. assigned a low dementia level (i.e. more impaired) to all but Prompts to correct a participant when they had confused one participant, S5, who had a rating of medium level the ordering of steps were often given several seconds after dementia. It is interesting to note that these ratings coin- they should have been. There were several instances where cide with the participant's MMSE scores rather than partic- the participant had completed at least one, and some- ipant performance during the handwashing activity. times several, incorrect step(s) before COACH gave the These results suggest an interesting possible additional appropriate corrective prompt. As such, it is quite possible application of COACH as a diagnostic tool; by watching that the participants would ignore fewer prompts from older adults perform ADL over a series of weeks or COACH if the timing of the prompts was improved. The months, a future version COACH may be used to detect authors are implementing the use of automatically deter- changes in the users' abilities, and consequently, level of mined, participant specific pauses between the delivery of dementia. COACH's ordering of the prompts was also prompts, which will be dynamically adjusted in response appropriate. The majority of instances showed the system to the individual's dementia and responsiveness levels. prompting the participant using a well-timed progression of prompting strategies from audio (only audio), video While there have been areas of the system identified for (audio and video), and finally to summoning the car- improvement, based on these and previous COACH trials, egiver to intervene if the participant was unresponsive to the authors speculate that participant responsiveness is the prompts. COACH would autonomously carry esti- dependant on several traits of the particular individual, mates about the user's dementia level from one trial to the such as cognitive abilities, hearing, vision, mood, compli- next, so that the system would not have to relearn partici- ance, and general attitude. Using a POMDP as the plan- pant behaviors for each trial. ning agent for this model allowed the system to estimate unobservable user traits such as responsiveness and Using the participant's name at the beginning of each cue dementia level to help tailor timing and level of prompts was a successful technique to gain his/her attention. For to the abilities of the individual, however, the ultimate the majority of the prompts, regardless of whether they usefulness of this type of assistive technology is very much were hits or false alarms, the participant would look up at dependant on the traits of the person who is using it. It is the video screen in anticipation of further instruction of significant interest that the level of dementia (as deter- when s/he heard his/her name. If a video was played, mined by the MMSE) alone does not appear to be an indi- most participants would watch the video to its comple- cator of an individual's success with this type of tion, regardless of whether or not they responded to the technology. prompt. The wording of the prompts was appropriate for this population as participants often spoke back to the Limitations prompts with a reply that showed they understood what There are several limitations regarding this study that was being asked, even if they did not comply. For exam- must be acknowledged. While the COACH shows promis- ple, when prompted by COACH to use the soap, S8 would ing results, the sample size was too small to draw any sig- often say "No thank you, I don't want to". As participants nificant conclusions about wide-scale applicability or usually distinctly looked up at the screen when they heard performance. The study presented here focused on a mod- their name, (when played) watched video cues to comple- erate-to-severe level dementia, with the majority of the tion, and provided coherent responses to prompts, it can participants being from the moderate group. Although the be concluded that the audio and audio-video cuing tech- researchers believe the trends seen here would also be niques used in this study are successful at getting the seen in mild and severe populations, to what extent attention of most older adults with dementia, although remains unknown until the device can be tested with a attention span and compliance is dependant on the indi- larger group that contains a diversity of dementia levels. vidual traits of the participant. More testing with a broader population, including infor- mal caregivers, must be done before any significant con- clusions regarding this device can be made. Page 16 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 Future Work that COACH could be useful to the caregiving dyads of The results gathered here are convincing enough that the individuals who respond well to prompting without tac- authors are preparing the device for the next set of clinical tile cues. These findings also support the importance of trials, which will include more participants and will be understanding the special, diverse and dynamic needs of conducted within homes in the community. Some this target user group to ensure that appropriate, custom- improvements that the researchers hope to include are the izable assistance is available in assistive technologies to adaptation of the system to distinguish between multiple help support people with dementia and their caregivers. washroom ADL such as tooth brushing and, eventually, toileting. The team plans to examine the implementation Abbreviations of speech recognition to allow the system to recognize AD: Alzheimer's disease; ADL: Activity of daily living; AI: other types of implicit user feedback. Multiple camera Artificial Intelligence; CAT: Cognitive assistive orthosis; input (vision/tracking systems) gained by placing cameras COACH: Cognitive Orthosis for Assisting with aCtivities in various positions throughout the washroom could pro- in the Home; FAS: Functional assessment score; MMSE: vide greater user observation accuracy and versatility and Mini Mental State Examination. would enable 3D observations. It is hoped that the next set of trials can be performed over a longer period of time Competing interests with a larger sample size to decrease the effects of the nat- The authors declare that they have no competing interests. ural performance variability that is found in this popula- tion. Authors' contributions AM supervised the project, developed the study design, and assisted in the preparation and editing of the manu- Conclusion This paper presents the results from clinical trials with a script. JB participated in the study design, test area set-up, small group of potential users of the COACH, a cognitive second rating of the trials, and drafting the manuscript. TC assistive technology designed to assist older adults with assisted in running trials, was the primary rater, and par- moderate to severe dementia through ADL. This study ticipated in drafting the manuscript. JH designed the soft- aimed to determine whether or not the POMDP-based ware algorithms used in this study. All authors COACH system was capable of: 1) reducing user depend- participated in the preparation of the final manuscript. ence on a caregiver, 2) decreasing caregiver workload, and/or 3) providing correct guidance through the hand- Acknowledgements The authors would like to acknowledge the invaluable contributions of Axel washing task. When COACH was used, the participants von Bertoldi (who assisted in the development, implementation, and testing appeared to show an increase in the number of hand- of the software and hardware), Kate Fenton (who was our caregiver for the washing steps they were able to complete without assist- study), and David Giesbrecht (who ran the technical side of the trials). This ance from the caregiver as well as the decrease in number research was generously supported by a grant from the American Alzhe- of times they required assistance from the caregiver during imer's Association (ETAC Program). the activity. Four of the five moderate-level participants were independent from a human caregiver during hand- References washing when COACH was used. Based on these results, 1. U.S. Census Bureau: Global Population Profile: 2002 Washington D.C.: U.S. Government Printing Office; 2004. this study has affirmatively answered the first two research 2. Ferri C, Prince M, Brayne C, Brodaty H, Fratiglioni L, Ganguli M, Hall questions. Through these clinical trials, the POMDP- K, Hasegawa K, Hendrie H, Huang Y, Jorm A, Mathers C, Menezes P, based planning system shows promise as a possible plan- Rimmer E, Scazufca M: Global prevalence of dementia: a Delphi consensus study. The Lancet 2006, 366:2112-2117. ning algorithm for guiding older adults with dementia 3. Shenk D, Kuwahara K, Zablotsky D: Older women's attachments through handwashing, albeit several areas in need of to their home and possessions. Journal of Aging Studies 2004, 18:157-169. improvement have been identified. These improvements 4. Bryant L: In their own words: A model of healthy aging. Social will be made and tested before the next set of clinical trials Science and Medicine 2001, 53:927-941. begin, which are planned to be supervised community- 5. Cutchin M: The process of mediated aging-in-place: a theoret- ically and empirically based model. Social Science and Medicine based (as opposed to long-term care facility-based) trials 2003, 57:1077-1090. starting in 2009. It is hoped that the next set of trials will 6. Intille SS: A New Research Challenge: Persuasive Technology allow the authors to answer these research questions more to Motivate Healthy Aging. IEEE Transactions on Information Tech- nology in Biomedicine 2004, 8:235-237. definitively with a lager sample size that includes a greater 7. Dura J, Stukenberg K, Kiecolt-Glaser J: Anxiety and depressive diversity in dementia levels. disorders in adult children caring for demented parents. Pscy- hology and Aging 1991, 6:467-473. 8. Aguglia E, Onor ML, Trevisiol M: Stress in the caregivers of In general the participants were less dependent on a Alzheimer's patients: An experimental investigation in Italy. human caregiver when COACH was used. As the effective- American Journal of Alzheimer's Disease and Other Dementias 2004, 4:248-252. ness varied considerably and seemed to be dependent on each individual's idiosyncrasies, these findings suggest Page 17 of 18 (page number not for citation purposes) BMC Geriatrics 2008, 8:28 http://www.biomedcentral.com/1471-2318/8/28 9. Pollack ME: Planning Technology for Intelligent Cognitive 31. Wolery M, Harris SR: Interpreting Results of Single-Subject Orthotics. 6th International Conference on AI Planning and Scheduling Research Designs. Physical Therapy 1982, 62:445-452. 2002. 32. Granger CV, Linn RT: Biologic Patterns of Disability. Journal of 10. Mann WC, Ottenbacher KJ, Fraas L, Tomita M, Granger CV: Effec- Outcome Measurement 2000, 4:595-614. tiveness of assistive technology and environmental interven- 33. Mihailidis A: The development of an intelligent cognitive tions in maintaining independence and reducing home care orthosis to facilitate handwashing for persons with moder- costs for the frail elderly. Archives of Family Medicine 1999, ate-to-severe dementia. In PhD University of Strathclyde, Bioen- 8:210-217. gineering Unit; 2001. 11. Pentney W, Philipose M, Bilmes J, Kautz H: Learning Large Scale 34. Wickens CD: Signal Detection, Information Theory, and Common Sense Models of Everyday Life. 27th Annual Confer- Absolute Judgement. In Engineering Psychology and Human Perform- ence of AAAI; Vancouver, BC 2007. ance 2nd edition. Edited by: Pearson LC, Claire M. New York: Harper 12. Pollack ME: Autominder: A Case Study of Assistive Technol- Collins Publishers Inc; 1992:24-73. ogy for Elders with Cognitive Impairment. Generations 2006, 35. Altman DG: Practical statistics for medical research London: Chapman 30:67-79. and Hall; 1991. 13. Helal S, Mann W, El-Zabadani H, King J, Kaddoura Y, Jansen E: The 36. Reisberg B, Ferris SH, de Leon MJ, Crook T: The Global Deterio- Gator Tech Smart House: A Programmable Pervasive ration Scale for assessment of primary degenerative demen- Space. IEEE Computer 2005, 38:50-60. tia. American Journal of Psychiatry 1982, 139:1136-1139. 14. Pigot H, Lussier-Desrochers D, Bauchet J, Giroux S, Lachapelle Y, 37. Boger J, Hoey J, Poupart P, Boutilier C, Fernie G, Mihailidis A: A plan- (Eds): A Smart Home to Assist in Recipe Completion. ning system based on Markov decision processes to guide Amsterdam, The Netherlands: IOS Press; 2008. people with dementia through activities of daily living. IEEE 15. Lindsay J, Anderson L: Dementia/Alzheimer's Disease. In BMC Transactions on Information Technology in Biomedicine 2006, 10:323-333. Women's Health Ottawa: University of Ottawa; 2004. 38. Mihailidis A, Boger J, Canido M, Hoey J: The use of an intelligent 16. Mathieson K, Kronenfeld J, Keith V: Maintaining Functional Inde- prompting system for people with dementia: A case study. pendence in Eaderly Adults: The roles of Health Status and ACM Interactions (Special issue on Designing for seniors: innovations for Financial Resources in Predicting Home Modifications and graying times) 2007, 14:34-37. Use of Mobility Equipment. The Gerontologist 2002, 42:24-31. 39. Mihailidis A, Carmichael B, Boger J: The use of computer vision in 17. McCreadie C, Tinker A: The acceptability of assistive technol- an intelligent environment to support aging-in-place, safety, ogy to older people. Ageing and Society 2005, 25:91-110. and independence in the home. IEEE Transactions on Information 18. Wilson R, Rochon E, Mihailidis A, Leonard C, Lim M, Cole A: Exam- Technology in Biomedicine 2004, 8:238-247. ining effective communication strategies used by formal car- egivers when interacting with Alzheimer's disease residents Pre-publication history during an activity of daily living (ADL). Brain and Language 2007, The pre-publication history for this paper can be accessed 103:199-200. 19. Hoey J, Poupart P, von Bertoldi A, Craig T, Boutilier C, Mihailidis A: here: Automated Handwashing Assistance For Persons With Dementia Using Video and A Partially Observable Markov Decision Process. Computer Vision and Image Understanding – Spe- http://www.biomedcentral.com/1471-2318/8/28/prepub cial Issue on Computer Vision Systems . 20. Boger J, Poupart P, Hoey J, Boutilier C, Fernie G, Mihailidis A: A Decision-Theoretic Approach to Task Assistance for Per- sons with Dementia. International Joint Conference on Artificial Intel- ligence (IJCAI); Edinburgh 2005:1293-1299. 21. Hoey J, von Bertoldi A, Poupart P, Mihailidis A: Assisting persons with dementia during handwashing using a partially observ- able Markov decision process. The 5th International Conference on Computer Vision Systems. Germany 2007. 22. Mihailidis A, Barbenel JC, Fernie GR: The efficacy of an intelligent cognitive orthosis to facilitate handwashing by persons with moderate-to-severe dementia. Neuropsychological Rehabilitation 2004, 14:135-171. 23. Hoey J: Tracking using flocks of features, with application to assisted handwashing. British Machine Vision Conference (BMVC) Edinburgh, Scotland 2006. 24. Kaelbling LP, Littman ML, Cassandra AR: Planning and acting in partially observable stochastic domains. Artificial Intelligence 1998, 101:99-134. 25. Labelle K, Mihailidis A: Facilitating handwashing in persons with moderate-to-severe dementia: Comparing the efficacy of verbal and visual automated prompting. American Journal of Occupational Therapy 2006, 60:442-450. 26. Folstein MF, Folstein SE, McHugh PR: Mini-Mental State: A prac- Publish with Bio Med Central and every tical method for grading the cognitive state of patients for scientist can read your work free of charge the clinician. Journal of Psychiatric Research 1975, 12:189-198. 27. Franklin RD, Allison DB, Gorman BS: Design and Analysis of Single-Case "BioMed Central will be the most significant development for Research Mahwah, New Jersey: Lawrence Erlbaum Associates; 1996. disseminating the results of biomedical researc h in our lifetime." 28. Harris S, Brooks D: N of One: Single Case Research Design for Sir Paul Nurse, Cancer Research UK the Practising Clinician. CPA Research Division Newsletter; 1992:10-13. Your research papers will be: 29. Kaewtrakulpong P, Bowden R: An improved adaptive back- available free of charge to the entire biomedical community ground mixture model for real time tracking with shadow detection. In Proceedings of 2nd European Workshop on Advanced peer reviewed and published immediately upon acceptance Video-Based Surveillance Systems; September 4, 2001; London, U.K Klu- cited in PubMed and archived on PubMed Central wer Academic Publishers; 2001. 30. Portney LG, Wathins MP: Chapter 12: Single-Subject Designs. yours — you keep the copyright In Foundations of Clinical Research: Applications to Practice Upper Saddle BioMedcentral River, NJ: Prentice Hall Health; 2000:223-264. Submit your manuscript here: http://www.biomedcentral.com/info/publishing_adv.asp Page 18 of 18 (page number not for citation purposes)

Journal

BMC GeriatricsSpringer Journals

Published: Nov 7, 2008

There are no references for this article.