ImpleMentAll - Towards Evidence-based Tailored Implementation Strategies for eHealth (ImpleMentAll)

The ImpleMentAll (IMA) project aims to examine the effectiveness of tailored implementation compared to usual implementation of Internet-based Cognitive Behavioural Therapy (iCBT) for patients suffering from common mental disorders in routine practice. Common mental health disorders account for an alarming proportion of the global burden of disease. Being regarded as an evidence-based psychotherapeutic eHealth intervention, Internet- based Cognitive Behavioural Therapy (iCBT), has the potential to answer to this societal challenge by providing an efficacious and efficient treatment from which more people can benefit. ImpleMentAll will develop, apply, and evaluate tailored implementation strategies in the context of on-going eHealth implementation initiatives in the EU and beyond. The objectives are:

1. To develop a generic Integrated Theory-based Framework for Intervention Tailoring Strategies (the ItFits-toolkit) for data-driven tailored implementation of evidence-based eHealth services.
2. To demonstrate the impact of the ItFits toolkit on the implementation of eHealth for common mental disorders.
3. To disseminate the validated toolkit in various healthcare contexts across Europe.

Following a stepped-wedge trial design, the ItFits-toolkit will be introduced in twelve implementation sites in nine countries, and evaluated for its effectiveness in obtaining implementation success. An in-depth process evaluation using a realist evaluation methodology will provide information about the particularities of tailored implementation and the application of the ItFits-toolkit in real implementation work. The resulting ItFits-toolkit will enable data driven evaluation of eHealth implementation projects and its methods, materials, and strategies will provide concrete guidance on tuning implementation interventions to local determinant of practice across a variety of health care systems.

Study Overview

• Status: Completed
• Conditions: Tailored Implementation: Online ItFits-toolkit; Implementation as Usual
• Intervention / Treatment: Other: ItFits-toolkit; Other: Implementation as Usual

Detailed Description

ImpleMentAll aims to examine the effectiveness of tailored implementation (i.e. the ItFits-toolkit) compared to usual implementation of Internet-based Cognitive Behavioural Therapy (iCBT) in routine practice in twelve implementation sites in nine countries. The project has a total duration of 51 months divided in three phases: 1) preparation (Jan 2017 - May 2018), 2) trial period (June 2018 - September 2020), and 3) analysis and reporting (October 2020 - March 2021). The preparation phase includes localising the generic study protocol to local settings and for obtaining the required ethical clearances. In the trial period, the ItFits-toolkit will be put into practice and data will be collected in each implementation site. In the final phase, the data will be cleaned and analysed. Depending on the results and conclusions of the study, the ItFits-toolkit will be further improved by adapting it to the findings.

iCBT, an innovative psychotherapeutic intervention, has been found to be an efficacious treatment alternative for a range of common mental health disorders. The iCBT service aims to combine innovative Internet technology with a broad evidence-based definition of CBT. The iCBT services that are being implemented apply different delivery formats ranging from self-help, to therapist guided and blended with face-to-face therapy for depression and/or anxiety disorders and/or in comorbidity with somatic symptom disorder. iCBT is currently being implemented in twelve different sites in primary and specialised care in Italy, Spain, Germany, France, the Netherlands, Denmark, Kosovo, Albania and Australia. All these sites are in different phases providing a natural laboratory for the IMA study.

A generic 'Integrated Theory-based Framework for Implementation Tailoring Strategies' toolkit (the ItFits-toolkit) is developed and will in introduced in these 12 implementation sites at randomly chosen moments. ItFits functions as an online self-help toolkit by which users are guided through the process of tailoring site-specific implementation strategies. The ItFits-toolkit includes four modules that implementers need to work through: 1) identifying and prioritising implementation goals and determinants of practices, 2) matching up implementation determinants to strategies, 3) designing a plan for carrying out strategies in a local context, and 4) applying strategies, and reviewing progress. In each of these four modules, evidence-informed materials such as iCBT relevant determinants of practices and implementation strategies, are included as well as methods for engaging with stakeholders.

By introducing the ItFits-toolkit in settings with ongoing implementation of iCBT services, a comparison of the effectiveness and efficiency of the ItFits-toolkit with Implementation-as-Usual (IAU) is possible. IAU refers to any existing efforts to embed and integrate iCBT within an organisation. The main research hypothesis is that the Itfits-toolkit will lead to better implementation outcomes than IAU does. Reaching better outcomes can be expressed in two dimensions: the capability to establish the desired effects (i.e. implementation effectiveness) and its efficiency (i.e. implementation costs in relation to uptake and normalisation outcomes) by which the effects came about. Implementation effectiveness is defined in terms of uptake and as the extent to which a service is regarded as being a normal part of practice, i.e. normalisation degree. The question is whether the ItFits-toolkit leads to a positive impact on uptake and normalisation. Parallel to the main objective, the efficiency of the ItFits-toolkit in reaching the implementation outcomes as compared to IAU will be explored. Organisational readiness for change will be investigated to corroborate the uptake and normalisation outcomes adding to the development of measurement instruments for implementation processes. In addition, the perceived satisfaction and usability of the ItFits-toolkit will be explored for potential improvement of the toolkit. An in-depth qualitative process evaluation using Realist Evaluation methods will be conducted alongside this implementation effectiveness study.

A stepped-wedge trial (SWT) design is applied by which the time points at which implementation sites receive the ItFits-toolkit are randomised. All implementation sites have received the ItFits-toolkit sequentially acting as control and intervention condition at the end of the trial. There are 6 groups and 2 implementation sites per group. Sites will be informed about the group they are allocated to three months prior to cross-over. Every three months, a new group crosses over from IAU to using the ItFits-toolkit.

There are two types of study participants: 1) implementers, and 2) staff involved in iCBT service delivery. Implementers are directly involved in the development, coordination and execution of implementation activities. Implementers are represented in the study by an Implementation Lead (IL) who has a coordinating role in the implementation activities. Implementers are first engaged with the Implementation-As-Usual activities and become users of the ItFits-toolkit when the implementation site crosses-over to the ItFits-toolkit condition (i.e. when they receive the ItFits-toolkit). When using the ItFits-toolkit, the IL is supported by an implementation core team (1-3 persons), and a number of relevant stakeholders providing feedback and advice on the implementation work through a process of co-creation. The IL engages with the ItFits-toolkit user-interface directly and collects and provides the data on implementation site level. Service delivery staff are the 'receivers' of the implementation activities and are subject to changes in their daily work to implement iCBT. Staff within the site are eligible for partaking in the study when they are involved in iCBT delivery. Involvement refers to a distinct role in delivering iCBT: mental health specialists, referrers, administration, and ICT support. Next to the study participants, each site has a study team responsible for the preparation, coordination, and execution of the local trials. These persons do not provide data themselves, but are involved in localising the current protocol, obtaining ethical approvals, recruitment of study participants, data collection, analysis, and reporting.

Implementation success is defined as a multidimensional concept that includes increases in uptake of the iCBT service on organisational level, normalisation process in staff, and efficiency outcomes on organisational level. Uptake is defined as a combination of referral rates and the extent to which the service completed by patients; normalisation process as the extent to which the new service is considered by staff to be a normal part of daily work; and efficiency as costs of implementation effort per uptake and costs per degree of normalisation. To be able to relate the effects of the ItFits-toolkit to these primary outcomes, Implementation Leads' exposure to, perceived satisfaction with, and usability of the ItFits-toolkit will be assessed. A measure for organisational readiness for change will be used to corroborate the normalisation instrument.

The instruments used to measure the primary and exploratory outcomes are based on brief and validated questionnaires. In addition, a number of newly developed measures are constructed for measuring uptake (referral rate, completeness of use), implementation costs (effort, other costs, and overheads), and exposure to the ItFits-toolkit (event-based log files). Uptake, degree of normalisation, implementation costs, and organisational readiness for change are measured at baseline and every three months for the total duration of the study. Data on exposure to the ItFits-toolkit will be collected continuously, whereas satisfaction and usability of the ItFits-toolkit will be assessed at the end of the exposure period in which an implementation site has received and used the toolkit.

At local implementation sites, questionnaire data will be gathered using web-based surveys in the ItFits-toolkit platform. Questionnaire measures for staff will be in local language. Quantitative data that is collected will be anonymized at local research sites, and checked, pruned, pooled, and packaged on a secure data server at Central Data Management. No data will be collected or used without the explicit informed consent of the staff that are participating in the IMA study. Study participants can withdraw at any point without disadvantage. As part of the informed consent procedure, informed consent for data transfer to other parties within the IMA consortium, and long-term preservation will also be asked.

The main aim of the analysis is to test whether the introduction of the ItFits-toolkit will induce positive chances in the uptake and normalisation of iCBT, and in the efficiency of this process (i.e. uptake and normalisation divided by implementation costs), across and within sites. Generalized linear mixed models (GLMM) will be used to determine the extent to which the total variation observed in uptake and normalisation is a result of the introduction and use of the ItFits-toolkit. To analyse efficiency in terms of cost-effectiveness, the costs for implementing iCBT using the ItFits-toolkit will be compared to the costs for Implementation-as-Usual in relation to the achieved outcomes (i.e. uptake and normalisation). In the analyses, all observed data will be included, in an effort to implement the intention-to-treat principle.

• Study Type: Interventional
• Enrollment (Actual): 456
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Albania
  • Tirana, Albania
    • CMHTIR
• Australia
  • Camberra, Australia
    • ANU
  • Sydney, Australia
    • BDI
• Denmark
  • Odense, Denmark
    • Internet Psychiatrien
• France
  • Toulouse, France
    • FDM
• Germany
  • Erlangen, Germany
    • Get.On
  • Leipzig, Germany
    • DF
• Italy
  • Turin, Italy
    • ASLTO3
• Kosovo
  • Prizren, Kosovo
    • MHCPPriz
• Netherlands
  • Amsterdam, Netherlands
    • GGz inGeest
  • Groningen, Netherlands
    • UMCG
• Spain
  • Badalona, Spain
    • BSA

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

There are two types of study participants, i.e. persons providing data: 1) implementers, and 2) staff involved in iCBT service delivery at the local implementation sites.

Implementers Eligible implementers are individuals who are directly involved in the development, coordination and execution of implementation activities within an implementation site. The implementation sites are the organizations engaged in the implementation processes as well as in iCBT service delivery. The implementation site started the implementation processes at least three months prior to the baseline measurement, and is committed to ongoing implementation of the iCBT service delivery. Implementation sites have been selected and enrolment is closed. Implementers are represented in the local trial by an Implementation Lead (IL) who has a coordinating role in the implementation activities. The IL will collect and provide the data on implementation site level and need to sign an informed consent form. Enrolment of implementers is considered to be open due to possible staff-changes within the participating implementation sites.

Inclusion criteria implementers and IL:

• Individuals that are directly involved in the development, coordination and execution of implementation activities within an implementation site;
• is a reasonably profound user of the English language;
• the IL has a coordinating role in implementing the iCBT service.

Exclusion criteria implementers and IL - Involved in trial management.

Staff involved in iCBT service delivery

Staff involved in service delivery are individuals who engage in the commissioning the iCBT service to clients. Staff members can have different roles in the service delivery:

• As therapists such as psychologists, psychiatrists, or mental health nurses.
• As referrers such as GPs, pharmacists, community workers, or case managers.
• As, administrators such as clerical workers, or secretariats.
• As ICT support such as security officers, maintenance officers, or helpdesk staff.

Inclusion criteria service delivery staff:

• Involved in the delivery of the iCBT service
• In a process of adapting their current way of working in order to deliver iCBT service to patients in routine care.

Exclusion criteria service delivery staff:

- Enrolled in the trial as implementer.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Crossover Assignment
• Masking: Triple

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: ItFits-toolkit; A generic 'Integrated Theory-based Framework for Implementation Tailoring Strategies' toolkit (the ItFits-toolkit) functions as an online self-help toolkit by which users are guided through the process of tailoring site-specific implementation strategies. The ItFits-toolkit includes four modules that implementers need to work through: 1) identifying and prioritising implementation goals and determinants of practices, 2) matching up implementation determinants to strategies, 3) designing a plan for carrying out strategies in a local context, and 4) applying strategies, and reviewing progress. In each of these four modules, evidence-informed materials such as iCBT relevant determinants of practices and implementation strategies, are included as well as methods for engaging with stakeholders.	Other: ItFits-toolkit; The ItFits-toolkit provides evidence-informed methods, materials, knowledge on determinants and implementation strategies, and concrete guidance on tailoring implementation strategies to local determinants of practices, apply them and evaluate their impact. It applies a standardised four-step approach to iteratively develop evidence-informed implementation strategies. The ItFits-toolkit is based on the Normalisation Process Theory (NPT), which suggests that lasting changes in practice only happen through people working together. The ItFits toolkit functions as an online self-help toolkit with minimal support. The toolkit will provide instructions for the IL to establish a core team and to create a sounding board consisting to enable co-creation.; Other Names: ItFits
Active Comparator: Implementation as Usual; Implementation-as-Usual (IAU) refers to any existing approaches and efforts to embed and integrate iCBT within an organisation. All implementation sites included in IMA are engaged in and conducting IAU. IAU activities can be, but are not necessarily planned or guided by scientific evidence and often emerge from practice experiences and other sources of information. No standardisation in IAU across the sites is applied except for the implementation objective. That is, all implementation sites pursue the goal of increasing the number of patients treated by the iCBT service.	Other: Implementation as Usual; IAU refers to any existing efforts to embed and integrate iCBT within an organisation.; Other Names: IAU

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Change in Uptake - referral rate (organisation level)	Referral rate: Patients can be referred to the iCBT service in two ways: 1) self-referral or 2) referral by health care professionals. Only eligible patients are referred to the services and eligibility criteria follow routine practice guidelines and procedures for the specific iCBT service.	Month 0, repeated every three months until month 27.
Change in Uptake - completion rate (organisation level)	Completion rate refers to the extent to which the treatment was actually consumed by patients after receiving access to the iCBT service on the platform. Four categories of completion will be assessed: 1. Not started, 2. In treatment, 3. Drop-out, 4. Completed All iCBT services that are implemented include the four generic therapeutic CBT. As the length and format of the specific iCBT service varies greatly between the sites, categories 3 and 4 will be measured in accordance with a service and site-specific definition of treatment completion.	Month 0, repeated every three months until month 27.
Change in Normalisation (staff level)	Normalisation refers to the actions people do to embed and integrate an innovation in routine practice. The Normalisation Assessment Development (NoMAD) is a 23-item self-report questionnaire that taps the four core concepts (Coherence, Cognitive Participation, Collective Action, Reflexive Monitoring) of the NPT. NoMAD was validated in UK, Australian, and Dutch samples of healthcare staff involved in various implementation projects.	Month 0, repeated every three months until month 27.
Change in Efficiency (organisation level)	Efficiency refers to implementation costs divided by outcomes (uptake and normalisation). Implementation costs are defined as the cost impact of an implementation effort. Measures of the cost of implementation effort allows for estimating the monetary efficiency of the ItFits-toolkit in relation to uptake and normalisation as compared to IAU. The following cost-indicators will be collected during the study: 1) Effort, 2) Consumables, equipment, and services, and 3) Overheads. Effort is a function of hours times hourly staff rates (wages). All costs will be collected in Euros.	Month 0, repeated every three months until month 27.

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Exposure to the ItFits-toolkit (organisation level)	Exposure to the ItFits-toolkit will be measured by logging use of individual ItFits-modules (use) and a binary confirmation of the existence of output of the module (result). Measurements are automated and follow platform use.	Month 0, continuous log until month 27.
Satisfaction with the ItFits-toolkit (organisation level)	Satisfaction will be assessed with the eight item Client Satisfaction Questionnaire (CSQ-3) (Larsen, Attkisson, Hargreaves, & Nguyen, 1979). CSQ-3 is a Likert scale with three items (short version of the CSQ-8) and a very brief instrument to investigate client satisfaction with the delivered services.	Month 27
Usability of the ItFits-toolkit (organisation level)	Usability will be measured with the System Usability Scale (SUS) (Brooke, 1996). SUS is a 10-item Likert scale and a brief instrument to measure perceived system usability.	Month 27
Organisational Readiness for Implementing Change (Staff level)	To assess organizational readiness for change we will use the 'Organizational Readiness for Implementing Change' questionnaire (ORIC, Shea et al., 2014). Organizational readiness for change is measured using 12 items. Ratings of the items are done by a 5-Point-Likert scale. Shea et al. (2014) investigated the ORIC for different psychometric properties.	Month 0, repeated every three months until month 27.
Perceived impact on site specific implementation goals and determinants (organisational level)	The impact of the ItFits-toolkit on site specific implementation goals will be explored. The overarching goal in all implementation sites is to increase the uptake of iCBT in routine practice. As part of module 1 of the ItFits-toolkit, implementation sites will tailor this goal towards objectives that are specific to the local circumstances and context. The perceived impact of the ItFits-toolkit in addressing these implementation objectives will be assessed by means of a Visual Analogue Scale (VAS; range 0-10).	End of exposure period to the ItFits-toolkit (i.e. 6 months after crossing over from IAU to using the ItFits-toolkit).

Collaborators and Investigators

• Sponsor: VU University of Amsterdam
• Collaborators: London School of Hygiene and Tropical Medicine; Northumbria University; Region of Southern Denmark; Australian National University; Azienda Sanitaria Locale 3, Torino; Fondation FondaMental; Academisch Ziekenhuis Groningen; Badalona Serveis Assistencials; GGZ inGeest; Get.On; European Alliance against Depression EV; Zyra Per Shendet Mendor; Qendres se Shendetit the Mireqenies Komunitare; Global alliance of mental illness advocacy networks Europe AISBL; Black Dog Institute; European Commission (funder)
• Investigators: Study Director: Christiaan Vis, MA, VU Amsterdam; Study Chair: Heleen Riper, PhD, VU Amsterdam

Publications and helpful links

General Publications

• Doukani A, Cerga Pashoja A, Fanaj N, Qirjako G, Meksi A, Mustafa S, Vis C, Hug J. Organizational Readiness for Implementing an Internet-Based Cognitive Behavioral Therapy Intervention for Depression Across Community Mental Health Services in Albania and Kosovo: Directed Qualitative Content Analysis. JMIR Form Res. 2021 Nov 1;5(11):e29280. doi: 10.2196/29280.
• Buhrmann L, Schuurmans J, Ruwaard J, Fleuren M, Etzelmuller A, Piera-Jimenez J, Finch T, Rapley T, Potthoff S, Aouizerate B, Batterham PJ, Calear A, Christensen H, Pedersen CD, Ebert DD, Van der Eycken E, Fanaj N, van Genugten C, Hanssen D, Hegerl U, Hug J, Kleiboer A, Mathiasen K, May C, Mustafa S, Oehler C, Cerga-Pashoja A, Pope C, Qirjako G, Rosmalen J, Sacco Y, Samalin L, Skjoth MM, Tarp K, Titzler I, Zanalda E, Zbukvic I, Smit JH, Riper H, Vis C; ImpleMentAll consortium. Tailored implementation of internet-based cognitive behavioural therapy in the multinational context of the ImpleMentAll project: a study protocol for a stepped wedge cluster randomized trial. Trials. 2020 Oct 28;21(1):893. doi: 10.1186/s13063-020-04686-4.

Helpful Links

• main project website
• Itfits toolkit

Study record dates

Study Major Dates

• Study Start (Actual): June 1, 2018
• Primary Completion (Actual): December 31, 2020
• Study Completion (Actual): December 31, 2020

Study Registration Dates

• First Submitted: August 20, 2018
• First Submitted That Met QC Criteria: August 24, 2018
• First Posted (Actual): August 29, 2018

Study Record Updates

• Last Update Posted (Actual): March 11, 2021
• Last Update Submitted That Met QC Criteria: March 10, 2021
• Last Verified: March 1, 2021

More Information

Terms related to this study

• Keywords: Implementation; iCBT; Tailored implementation strategies; Common mental disorders
• Other Study ID Numbers: 1

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: Yes
• IPD Plan Description: IPD will be available on request following a standardised data accession form on relevant participant data. Note this data does not contain clinical outcomes.
• IPD Sharing Time Frame: Late 2021-mid 2022
• IPD Sharing Access Criteria: Accession will be granted through standardised accession form following review by project's dedicated data accession and publication committee.
• IPD Sharing Supporting Information Type: Study Protocol; Statistical Analysis Plan (SAP); Informed Consent Form (ICF); Analytic Code

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No