Pilot Evaluation of "KeepWell" Using a Hybrid Effectiveness-Implementation Pragmatic Randomized Controlled Trial

Effectiveness of an eHealth Tool for Older Adults With Multimorbidity ("KeepWell"): Protocol for a Hybrid Effectiveness-implementation, Pragmatic Randomized Controlled Trial

The burden of chronic disease is a global phenomenon, particularly amongst seniors. Aging is an expensive process. In Canada, 10% of seniors who have the most complex health needs account for 60% of the total annual health care spending in many provinces. Given these projections, we need to adapt our current models of care. In response, different chronic disease management tools have been created with a central aim to facilitate ongoing, proactive and preventive support for optimal chronic disease management. In particular, self-management tools have been acknowledged as an effective way to optimize disease management and are easily scalable and can reach a broader population of older people with chronic diseases. In fact, online self-management tools are particularly relevant for supporting seniors with complex care needs in their homes; and they are interested in using the Internet and social media. However, interventions and tools seldom consider all aspects of disease management, are not usually developed specifically for seniors or created for sustained use and are primarily focused on a single disease. As such, the projected health outcomes of seniors continue to remain poor, and the quality and efficiency of care remain sub-optimal. To respond to these challenges, we created an eHealth self-management application called "KeepWell" that supports seniors with complex care needs in their homes. It is a patient-centered, multi-chronic disease management tool that incorporates the care for two or more chronic conditions from among the top high-burden chronic diseases. KeepWell was built on a strong evidentiary base including several knowledge syntheses, a co-design process with our integrated knowledge translation team involving patients, researchers, clinicians and developers; and a usability and pilot evaluation. The objectives of our study will be to evaluate the effectiveness, economic impact and uptake of KeepWell in a 6-month, pragmatic, hybrid effectiveness-implementation randomized controlled trial (RCT).

Study Overview

• Status: Completed
• Conditions: Chronic Disease
• Intervention / Treatment: Other: KeepWell tool; Other: Control - Usual care

Detailed Description

Methods

Design We will evaluate the effectiveness and uptake of the KeepWell solution in a 6-month, pragmatic, hybrid effectiveness-implementation RCT for optimizing and sustaining the self-management of older adults with multimorbidity. We will use the type 2 design, which facilitates the simultaneously investigation of the effectiveness of an intervention while rigorously testing the implementation strategy. The implementation evaluation will be guided by the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework.

Theoretical basis of our work We used the Knowledge-to-Action (KTA) model to guide our methods. Our team has experience applying the KTA framework in creating technology-based interventions [refs]. We have also adapted an integrated KT (IKT) strategy whereby all our stakeholder team members who helped us to create KeepWell and prepare this protocol are going to continue to steer the execution of our study. This team helped to develop the objectives, project plan and timelines; discussed potential barriers and mitigating strategies, and monitored progress.

Population and recruitment Older adults will be identified with support from: (i) the University of Toronto Primary Care Research Network (UTOPIAN); (ii) the Retired Teachers of Ontario (RTO); and (iii) our clinician (geriatricians and family physicians) partnering sites: North York General Hospital, St. Michael's hospital of Unity Health, Sunnybrook healthcentre, and St. Peter's hospital. All these recruitment sources have a roster of older adults with multiple chronic conditions via their affiliated primary care and geriatric clinics. We have created a recruitment poster that will be administered at each site according to a procedure most feasible and practical for their practice flow and resources. For example, the poster may be administered directly to older adults by family physicians and geriatricians during a visit or through their clinic staff upon registering for a visit or posted on walls. In addition, the poster will be distributed via an email listserv from our patient partner organization, the RTO with a membership of more than 70,000 older adults. Study eligibility criteria are: (i) age 65 years; (ii) have one or more of the following chronic conditions: diabetes, heart failure, cardiovascular disease, dementia, chronic kidney disease, osteoporosis, osteoarthritis, rheumatoid arthritis, Chronic Obstructive Pulmonary Disease (COPD), depression, urinary incontinence, stroke; (iii) English speaking; (iv) access to a computer or tablet device; (v) have an email address; and (vi) able to consent. Consent-giving capacity will be assessed using the validated "MacCAT-CR" tool.

The recruitment poster includes a phone number and email address dedicated to this research project. Potential participants will self-refer after reviewing the poster, by contacting the research coordinator using one of two strategies: (1) if their mode of reply is by phone, the research coordinator will review the study information sheet and consent process document with potential participants and answer any questions. Study eligibility will also be assessed. Participants will have the option to consent over the phone or via an online survey (SurveyMonkey). Non-consenting respondents will be asked to provide a reason for their decision; (2) if the mode of reply is email, an automated, email response system will be initiated. The email will include the same study and consent process information as described for the phone option and a link to a survey (via SurveyMonkey) that will include questions to assess their eligibility and to obtain conent. Regardless of participants' mode of reply, they will be enrolled and assigned a unique identification number and randomization sequence as soon as the research coordinator receives confirmation of eligibility and consent. Once they do, participants will receive an email with a link to KeepWell and their unique login details (intervention condition) or to a link to an online survey to complete a risk assessment questionnaire (usual care control condition).

Randomization and blinding Consenting participants will be randomly allocated to the intervention (KeepWell) or control (usual care) using a random number generator. Consenting participants will be assigned a login and password by the research coordinator, which they can use to access the KepWell tool. Each individual login and password will be randomly allocated into intervention or control using a 1:1 ratio; participants will be the unit of randomization. The research coordinator will have the master list of logins and allocation. Allocation will thus be concealed because this list will not be shared with the research team. To protect against sources of bias, investigators, outcome assessors and data analysts will be blinded to the randomization sequence. Blinding of older adults will not be possible given that the intervention is a standalone, web-based tool and the control condition is usual care.

Intervention and Control KeepWell is a fully functional, standalone eHealth application aimed at supporting the self-management of older adults with multimorbidity. KeepWell can be used on any electronic device and has innovative features that most other chronic disease solutions don't have: (i) a multi-disease focus (it can generate lifestyle advice for any combination of the top 10 chronic conditions affecting older adults); (ii) an avatar health coach that walks users through a health prioritization and goal setting exercise; (iii) a health risk questionnaire (HRQ) covering three risk dimensions: health (chronic diseases), lifestyle (physical activity, diet, smoking, alcohol, caffeine, bladder health), and social and emotional well-being (social frailty, isolation, loneliness); (iv) an evidence-based Action plan with lifestyle advice customized to user's risks; and (v) other eHealth self-management tactics that have been shown to improve outcomes (interactive lifestyle tracker, journaling. A health coach avatar leads users through a health priority and goal setting exercise that allows them to create a customized action plan based on guideline recommendations for lifestyle changes. Users can then put their plan into action through a lifestyle tracking feature that allows them to track their progress in their identified lifestyle areas. KeepWell also has an extensive resource library, which has links to additional high-quality health and lifestyle information across topics of interest to older adults. Participants allocated to the control condition will receive care as usual but will be asked to complete the health risk questionnaire at baseline, 3- and 6-month follow-up via an online survey to collect outcomes data. The control group will receive full access to KeepWell at the conclusion of the study.

Outcomes Primary outcome: perceived self-efficacy for managing identified chronic diseases or risks measured at 6-months follow-up using a validated 6-item, self-efficacy scale, which is embedded within the health risk questionnaire of the KeepWell platform. We selected this outcome because increasing self-efficacy is a prerequisite for behaviour change which, through improved self-management may influence health and healthcare use. All participants will complete this outcome assessment via the health risk questionnaire of KeepWell at baseline, 3- and 6-month follow-up.

Secondary outcomes: (i) self-efficacy at 3-months follow-up; (ii) cost description analysis to assess the total cost of implementing the KeepWell overall and by stage (e.g., one-time costs vs. ongoing costs); (iii) implementation outcomes according to Proctor et al: reach (access), adoption (uptake, utilization); fidelity; maintenance; acceptability (satisfaction), appropriateness (relevance), feasibility; (iv) patient reported outcomes (self-reported disease management, perceived health and functional status, emotional well-being, QOL); (v) patient-reported experiences (self-reported patient activation, empowerment, patient-practitioner communication, shared decision-making). Outcomes will be measured using the health risk questionnaire embedded within KeepWell and an online survey at final follow-up.

Sample size calculation An increase in self efficacy of 40% is considered significant and has been achieved in other studies of self-management interventions including one of the highest quality studies of a chronic disease self-management program. In their Cochrane review of lay-led self-management interventions, Foster et al found 10 studies that looked at self efficacy and these interventions showed a small, statistically-significant improvement (standardized mean difference -0.30, 95% confidence interval (CI) -0.41 to -0.19). Using these estimates to calculate our sample size, targeting a power of 0.8 and assuming a dropout rate of 25%, we estimate that 220 older adults are needed per group for a total of 440 participants. In a previous interrupted time series study, an Osteoporosis self-management tool housed in a touch-screen laptop computer was implemented across 3 primary care practices for 12 months, with a total of 350 patients who completed the risk assessment questionnaire, so our proposed sample size of 440 patients is feasible, given our hospital and primary care partnerships.

Analysis We will perform descriptive statistics (binary outcomes) and means and standard deviations (continuous outcomes). For the primary outcome, a general linear model will be fit to investigate differences between groups in self-efficacy at 6 months. We will adjust for potential confounders (including baseline self-efficacy) and perform subgroup analyses (patient chronic conditions, risk factors, mode of KeepWell use [Tablet/Laptop], age group [65-74, 75-84, 85+], sex and gender). We will use the chi-squared and Fisher's exact test to compare binary outcomes and independent two sample t-tests for continuous outcomes. We will adjust for potential confounders in the final analysis (e.g., receipt of education about chronic diseases and risks or specialist referrals); and perform subgroup analyses (e.g., setting, gender, and age: 65-74 and 75+). We will perform visual investigation (through scatter plot displays) and analytical outcomes along with logistic regression analyses to determine whether unique patient characteristics predict thresholds of use for different KeepWell components. To examine the change in self-efficacy over time between groups (incorporating self-efficacy scores at baseline, 3- and 6- months), we will use a linear mixed-effects model. For the cost description analysis, we will estimate the cost to delivering KeepWell from the public healthcare payer perspective, including out-of-pocket costs for patients, technical costs for the clinic, efficiencies gained, estimated health care utilization increases / reductions; and exploration resource costs required (e.g., online data collection / analysis, web system testing / hosting). The KeepWell platform collects user data for these measures, so we will be able to track them longitudinally to observe how the solution impacts on outcomes. The findings from the economic analysis will represent the potential preliminary economic impact of KeepWell to the healthcare system. All outcomes will be measured at baseline, 3- and 6-months follow-up and assessed according to intention to treat. All statistical analyses will be carried out using the R statistical software.

• Study Type: Interventional
• Enrollment (Actual): 456
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Canada
  • Ontario
    • Toronto, Ontario, Canada, M2K 1E1
      • North York General Hospital

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 65 years and older (Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

Inclusion criteria:

(i) age 65 years; (ii) have one or more of the following chronic conditions: diabetes, heart failure, cardiovascular disease, dementia, chronic kidney disease, osteoporosis, osteoarthritis, rheumatoid arthritis, COPD, depression, urinary incontinence, stroke; (iii) English speaking; (iv) access to a computer or tablet device; (v) have an email address; and (vi) able to consent.

Exclusion criteria:

(i) Non-English speaking; (ii) no access to a computer or tablet device; (ii) no email address; and (iii) unable to consent (as assessed by the MacCAT-CR tool)

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Single Group Assignment
• Masking: Double

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Intervention - KeepWell tool; KeepWell is standalone eHealth application aimed at supporting the self-management of older adults with multimorbidity, and it has the following features: (i) lifestyle advice for any combination of the top 10 chronic conditions affecting older adults); (ii) an avatar health coach that walks users through a health prioritization and goal setting exercise; (iii) a health risk questionnaire (HRQ) covering health (chronic diseases), lifestyle (physical activity, diet, smoking, alcohol, caffeine, bladder health), and social and emotional well-being (social frailty, isolation, loneliness) dimensions; (iv) an evidence-based, customized Action plan; (v) an interactive lifestyle tracker; (vi) journaling; (vii) and a health resources library. A health coach avatar leads users through a health priority and goal setting exercise that allows them to create a customized action plan based on guideline recommendations for lifestyle changes.	Other: KeepWell tool; KeepWell is standalone eHealth application aimed at supporting the self-management of older adults with multimorbidity, and it has the following features: (i) lifestyle advice for any combination of the top 10 chronic conditions affecting older adults); (ii) an avatar health coach that walks users through a health prioritization and goal setting exercise; (iii) a health risk questionnaire (HRQ) covering health (chronic diseases), lifestyle (physical activity, diet, smoking, alcohol, caffeine, bladder health), and social and emotional well-being (social frailty, isolation, loneliness) dimensions; (iv) an evidence-based, customized Action plan; (v) an interactive lifestyle tracker; (vi) journaling; (vii) and a health resources library. A health coach avatar leads users through a health priority and goal setting exercise that allows them to create a customized action plan based on guideline recommendations for lifestyle changes.
Placebo Comparator: Control; Participants allocated to the control condition will receive care as usual but will be asked to complete the health risk questionnaire at baseline, 3- and 6-month follow-up via an online survey to collect outcomes data. The control group will receive full access to KeepWell at the conclusion of the study.	Other: Control - Usual care; Participants allocated to the control condition will receive care as usual.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Self-efficacy	Self-efficacy for managing chronic disease (6-item scale)	Measured at baseline and 6 months of follow-up

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Implementation	Measurement of change between 3 and 6 months follow-up in implementation outcomes (reach, adoption, fidelity, maintenance, acceptability, appropriateness, feasibility)	Measured at 3 and 6 months of follow-up
Patient reported outcomes	Change between 3 and 6 months of follow-up in self-reported disease management, perceived health and functional status, emotional well-being, quality of life	Measured at 3 and 6 months of follow-up
Change between 3 and 6 months of follow-up in patient reported experiences	Self-reported patient activation, empowerment, patient-practitioner communication, shared decision-making.	Measured at 3 and 6-months of followup

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Self-efficacy	Self-efficacy for managing chronic disease (6-item scale)	Measured at 3 months of follow-up
The total cost of implementing KeepWell	Cost description analysis to assess the total cost of implementing KeepWell	Cumulative, reported at 6 months of follow-up

Collaborators and Investigators

• Sponsor: North York General Hospital
• Collaborators: Sunnybrook Health Sciences Centre; Unity Health Toronto
• Investigators: Principal Investigator: Monika Kastner, PhD, North York General Hospital

Publications and helpful links

General Publications

• Kastner M, Makarski J, Hayden L, Hamid JS, Holroyd-Leduc J, Twohig M, Macfarlane C, Hynes MT, Prasaud L, Sklar B, Honsberger J, Wang M, Kramer G, Hobden G, Armson H, Ivers N, Leung FH, Liu B, Marr S, Greiver M, Desroches S, Sibley K, Saunders H, Isaranuwatchai W, McArthur E, Harvey S, Manawadu K, Petricca K, Straus SE. Effectiveness of an eHealth self-management tool for older adults with multimorbidity (KeepWell): protocol for a hybrid effectiveness-implementation randomised controlled trial. BMJ Open. 2021 Feb 17;11(2):e048350. doi: 10.1136/bmjopen-2020-048350.

Study record dates

Study Major Dates

• Study Start (Actual): November 1, 2020
• Primary Completion (Actual): March 31, 2022
• Study Completion (Actual): April 12, 2022

Study Registration Dates

• First Submitted: March 12, 2020
• First Submitted That Met QC Criteria: June 16, 2020
• First Posted (Actual): June 18, 2020

Study Record Updates

• Last Update Posted (Actual): April 13, 2022
• Last Update Submitted That Met QC Criteria: April 12, 2022
• Last Verified: April 1, 2022

More Information

Terms related to this study

• Keywords: Web-based intervention; Older adults; Multimorbidity
• Additional Relevant MeSH Terms: Pathologic Processes; Disease Attributes; Chronic Disease
• Other Study ID Numbers: KeepWell RCT

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: We will share de-identified participant level data with our bio-statistician. We will share this information through encrypted, secured hospital email. We will also share all IPD that underlie results in a publication.
• IPD Sharing Time Frame: We will share supporting information at the beginning of the study for the purposes of sharing the plan with the research team and providing them with the opportunity to provide feedback on supporting information (2-4 weeks), and IPD at the conclusion of our study for the purposes of data interpretation and analysis (2-4 weeks).

IPD Sharing Access Criteria

The biostatistician and health economist will have access to PID to perform the analysis of primary and secondary outcomes including the cost analysis. They will have access to the data via an encrypted file via secured hospital email for 2-4 weeks or longer if needed (no more than 8 weeks).

The biostatistician will perform descriptive statistics (binary outcomes) and means and standard deviations (continuous outcomes). For the primary outcome, a general linear model will be fit to investigate differences between groups in self-efficacy at 6 months. We will adjust for potential confounders and perform subgroup analyses. For the cost description analysis, the health economist will estimate the cost to delivering KeepWell, including out-of-pocket costs for patients, technical costs for the clinic, efficiencies gained, estimated health care utilization increases / reductions; and exploration resource costs required. All outcomes will be assessed according to intention to treat.

• IPD Sharing Supporting Information Type: STUDY_PROTOCOL; SAP; ANALYTIC_CODE; CSR

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No