Development and Evaluation of the Electronic Frailty Index+ (eFI+)

Development and Evaluation of the Electronic Frailty Index+ (eFI+) Tool: Integrated Prognostic-decision Modelling to Target Interventions for Older People With Moderate or Severe Frailty

Research questions

i) How should electronic frailty index (eFI) components be combined with additional routine primary care data to develop prognostic models for predicting key outcomes of requirement for home care, falls/fractures, nursing home admission and mortality in older people with moderate or severe frailty?

ii) Can model predictive performance be improved through addition of data from measures that are practical for primary care use, but not available in routine data?

iii) How should risk predictions from the prognostic models be translated into a decision analytic model (DAM) to guide clinical management?

iv) What is the potential cost-effectiveness of implementing interventions targeted at subgroups of older people with frailty in routine NHS care?

Background

Lead applicant Clegg led the eFI development, validation and national implementation. This has been translated into major UK health policy change through inclusion in the 2017/18 GP contract, which supports frailty stratification using the eFI, and UK National Health Service Long Term Plan.

Aim

To develop and evaluate the eFI+, a prognostic tool supplementing the original eFI including 4 integrated prognostic-decision models. The eFI+ will stratify older people with moderate or severe frailty into subgroups most likely to benefit from key interventions (community rehabilitation; falls prevention; comprehensive geriatric assessment; advance care planning).

Methods

Design

Prognostic model development, internal validation and external validation using large datasets (ResearchOne, SAIL databank, Leeds Data Model) and cohort study data (CARE75+), with linked DAM and health economic analysis.

Population

Patients ≥65 with moderate or severe frailty, defined by the existing eFI.

Key outcomes

12-month outcomes for prognostic models:

• New/increased home care package
• Emergency Department (ED) attendance/hospitalisation with fall/fracture
• Nursing home admission
• All-cause mortality

Statistical methods

i) Prognostic modelling

The investigators will build 4 separate prognostic models for our 4 key outcomes by combining the eFI with additional individual-level routine data, informed by reviews to identify prognostic factors. Each model will be developed and internally validated in one large dataset, to adjust for potential overfitting, with subsequent external validation of predictive performance in a second large dataset.

Separately, the investigators will use CARE75+ (n≈1,200) to investigate additional predictive value of clinical measures practical for primary care (e.g. gait speed, activities of daily living, loneliness).

ii) Decision analytic model (DAM)

The investigators will translate the prognostic models into a framework to support clinical decision-making, in co-production with stakeholders/PPI. The investigators will integrate prognostic models with effect size estimates from systematic reviews/meta-analyses to identify relevant thresholds of predicted risk, above which implementation of our key interventions would be warranted.

iii) Health economic evaluation

12-month and long-term cost effectiveness models will be developed, informed by the DAM.

Study Overview

• Status: Unknown
• Conditions: Frailty
• Intervention / Treatment: Other: Decision Analytic Modelling

Detailed Description

Health technologies being assessed

The eFI+ will be developed using components of the original eFI, supplemented with additional routine primary care EHR data, and guidance on the added benefits of implementing simple clinical measures in routine primary care practice. The eFI+ will be suitable for rapid implementation in UK primary care EHR systems, building on existing close links with system suppliers (SystmOne/EMISWeb/Vision/Microtest).

The investigators will develop, then internally and externally validate the eFI+ using the Secure Anonymised Information Linkage (SAIL) databank, the ResearchOne database, and the Leeds Data Model (LDM).

In addition, the investigators will analyse Community Ageing Research 75+ (CARE75+) cohort study data (CI Clegg, n≈1,200) as the only national cohort study to include eFI scores, to investigate how simple measures that can be assessed in primary care, but are not available in routine EHR data (e.g. gait speed, timed-up-and-go test; activities of daily living; loneliness) may improve prediction.

Study design

Prognostic model development, internal validation and external validation using routine primary care research data (ResearchOne), linked datasets (SAIL databank and LDM) and cohort study data (CARE75+), with integrated Decision Analytic Modelling, including health economic analysis.

Databases

1. Secure Anonymised Information Linkage (SAIL) databank

   Anonymised records from around 5 million people in Wales, with linked primary care, ED attendance, hospital admissions, outpatient data, social care, Welsh Care Homes Dataset, and ONS mortality data. SAIL includes eFI summary scores and individual components.

2. ResearchOne

   Nationally representative, de-identified data from around 6 million UK primary care electronic health records on the TPP SystmOne clinical system. ResearchOne includes eFI summary scores and individual components.

3. Leeds Data Model (LDM)

   Anonymised, linked primary, secondary, community and social care data from 810,000 patients across 108 practices in Leeds, including eFI summary scores and individual components.

4. Community Ageing Research 75+ (CARE75+) cohort

National prospective cohort study (n≈1,200) collecting detailed sociodemographic information, frailty measures (including eFI scores), simple instruments suitable for use in primary care (e.g. gait speed, timed-up-and-go test; activities of daily living; informal care; loneliness), and key outcomes at six, 12, 24 and 48 months. CARE75+ is a very rich dataset that provides a highly efficient method to investigate how simple instruments might augment eFI performance.

Eligible population

Patients ≥65 years with moderate frailty (eFI score 0.24 to 0.36) or severe frailty (eFI score >0.36) and registered with a ResearchOne, SAIL or LDM practice on 1st April 2018.

All CARE75+ participants with moderate frailty (eFI score 0.24 to 0.36) or severe frailty (eFI score >0.36) will be eligible.

Outcomes for risk prediction (all 12 months)

• New or increased home care package
• ED attendance/hospitalisation with fall or fracture
• Nursing home admission
• All-cause mortality

Predictors

Components of the eFI, supplemented with variables available within routine primary care EHR data and clinical assessment measures practical for use in primary care.

Prognostic models

Each prognostic model will be developed and internally validated in just one of the databases, and then externally validated in a second database

Sample size for prognostic model development

SAIL and ResearchOne extracts will each include ≈600,000 patients aged 65 or over, with an estimated 72,000 having moderate frailty, and 24,000 severe frailty. LDM extract will include ≈150,000 patients aged 65 or over, with an estimated 18,000 having moderate frailty and 6,000 severe frailty.

For model development, a key indicator of the effective sample size is the number of outcome events. Previous research into the outcomes of interest, and feasibility estimates using CARE75+, ResearchOne and SAIL, inform estimates for anticipated number of events within 12 months.

• New or increased home care package: Anticipated 15,864 events in SAIL, based on 14.9% 12 month incidence in moderate frailty group (10,080 events), and 24.1% 12 month incidence in severe frailty group (5,784 events).
• ED attendance/hospitalisation with fall or fracture: Anticipated 8,064 events in SAIL, based on 7.4% 12 month incidence in moderate frailty group (5,328 events) and 11.4% incidence in severe frailty (2,736 events).
• Nursing home admission: Anticipated 2,160 events in ResearchOne, based on 2.0% 12 month incidence in moderate frailty group (1,440 events) and 3.8% 12 month incidence in severe frailty group (720 events).
• All-cause mortality: Anticipated 12,216 events in ResearchOne, based on 10.6% 12 month incidence in moderate frailty group (7,632 events) and 19.1% 12 month incidence severe frailty group (4,584 events).

Therefore, even when taking the lowest estimate of incident events by 12 months (for nursing home admission), for each outcome the investigators would expect at least 2,160 events in each of ResearchOne or SAIL. This enables us to robustly estimate a prognostic model for each outcome even with up to 108 predictor parameters, corresponding to 20 events per potential predictor parameter (2160/20). This exceeds 'rule-of-thumb' recommendations of 10 or 15 events per predictor parameter.

Furthermore, conservatively assuming the new models will have a Nagelkerke R-squared of 15%, Riley's sample size formula suggests that at least 7.5 events for each predictor parameter will ensure overfitting and optimism are minimized, when the outcome proportion is 3%. When increasing outcome proportion to 20% (home care package), 9% (fall/fracture), or 15% (mortality), the minimum sample size required is 18, 11.5 and 15 events per predictor parameter, respectively. The investigators exceed all these, due to the large datasets available.

Sample size for external validation

Current recommendations are that at least 100 events and 100 non-events (ideally 200) are required for prognostic model external validation. Our estimates indicate considerably more than this, such as 2160 events for the least prevalent outcome of care home admission in SAIL and ResearchOne, and 540 in LDM (which will only be used for external validation of models).

Missing data

Handled using multiple imputation and Rubin's rules, under a missing at random assumption, including outcome in the imputation model, accounting for practice clustering.

Analysis plan

i) Prognostic modelling

The investigators will build 4 separate prognostic models within the development datasets to predict risk of our 4 key stated outcomes in individuals with moderate or severe frailty as the startpoint.

For each outcome, for those with moderate frailty (eFI score 0.24 to 0.36) or severe frailty (eFI score >0.36) the investigators will develop and internally validate a prognostic model containing just eFI (as a whole as it currently stands) and then containing components of eFI (included as predictors) along with additional routine primary care EHR data. The regression model will be logistic regression or flexible parametric survival, for binary or time-to-event outcomes (as appropriate when the investigators observe the database coding and censoring etc.), to produce outcome risks by 12 months.

Due to the large sample size, overfitting is expected to be small, but the investigators will adjust for it using penalisation via a global shrinkage factor estimated via bootstrapping. Where variable selection is considered important for parsimony, the investigators will rather use penalisation via elastic net. Internal validation will use bootstrapping of the entire development dataset, and optimism-adjusted estimates of predictive performance produced for calibration (e.g. calibration-in-the-large, calibration slope, Observed/Expected), discrimination (e.g. C-statistic) and overall (e.g. Nagelkerke R2) performance of predicted risks. Continuous variables will not be categorised and potential non-linear effects examined using splines or fractional polynomials. Non-proportional hazards for predictors will also be examined with interaction terms with time.

All models will be externally validated in a different database. Predictive performance statistics will be derived as described above (e.g. C-statistic, calibration slope), alongside calibration plots showing agreement between observed and predicted risks, across the spectrum of predicted risks, using a loess non-parameter smoother.

Separately, the investigators will use the CARE75+ dataset to investigate the additional predictive power of clinical assessment measures of prognostic factors identified from the reviews that are practical for use in routine primary care.

ii) Decision modelling

Prognostic models will be translated into a framework to guide clinical decision making by identifying relevant thresholds of predicted risk, above which implementation of our stated interventions is warranted. This will allow us to generate a decision analytic model (DAM), which will be examined using decision curves and net benefit in the external validation datasets.

iii) Health economic evaluation

The health economic evaluation will be conducted in two stages. The objective of the first stage is to provide a short-term, 12-month comparison of the cost-effectiveness of the scenarios identified by the DAM. For the second stage, the investigators will extend our analysis to a long-term cost-effectiveness evaluation of these scenarios.

• Study Type: Observational
• Enrollment (Anticipated): 1000000

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 61 years and older (Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

• Sampling Method: Non-Probability Sample

Study Population

SAIL databank:

Anonymised records from around 5 million people in Wales, with linked primary care, ED attendance, hospital admissions, outpatient data, social care, Welsh Care Homes Dataset, and ONS mortality data.

ResearchOne:

Nationally representative, de-identified data from around 6 million primary care electronic health records on the TPP SystmOne clinical system.

Leeds Data Model:

Anonymised, linked primary, secondary, community and social care data from 810,000 patients across 108 practices in Leeds, UK.

CARE75+:

National UK prospective cohort study (n≈1,200), with key outcomes at six, 12, 24 and 48 months.

Description

Inclusion Criteria:

• Age ≥65 years
• Moderate frailty (eFI score 0.24 to 0.36) or severe frailty (eFI score >0.36)
• Registered with a ResearchOne, SAIL or LDM practice on 1st April 2018
• CARE75+ participants with moderate frailty (eFI score 0.24 to 0.36) or severe frailty (eFI score >0.36)

Exclusion Criteria:

• Age <65 years
• Fit/mild frailty

Study Plan

How is the study designed?

Design Details

• Observational Models: Cohort
• Time Perspectives: Prospective

Number of groups / cohorts

4

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
SAIL databank; Anonymised records from around 5 million people in Wales, with linked primary care, ED attendance, hospital admissions, outpatient data, social care, Welsh Care Homes Dataset, and ONS mortality data. SAIL includes eFI summary scores and individual components.	Other: Decision Analytic Modelling; Prognostic models will be translated into a framework to guide clinical decision making by identifying relevant thresholds of predicted risk, above which implementation of our stated interventions is warranted.
ResearchOne; Nationally representative, de-identified data from around 6 million primary care electronic health records on the TPP SystmOne clinical system. ResearchOne includes eFI summary scores and individual components.	Other: Decision Analytic Modelling; Prognostic models will be translated into a framework to guide clinical decision making by identifying relevant thresholds of predicted risk, above which implementation of our stated interventions is warranted.
Leeds Data Model; Anonymised, linked primary, secondary, community and social care data from 810,000 patients across 108 practices in Leeds, including eFI summary scores and individual components.	Other: Decision Analytic Modelling; Prognostic models will be translated into a framework to guide clinical decision making by identifying relevant thresholds of predicted risk, above which implementation of our stated interventions is warranted.
CARE75+; National prospective cohort study (n≈1,200) collecting detailed sociodemographic information, frailty measures (including eFI scores), simple instruments suitable for use in primary care (e.g. gait speed, timed-up-and-go test; activities of daily living; informal care; loneliness), and key outcomes at six, 12, 24 and 48 months. CARE75+ is a very rich dataset that provides a highly efficient method to investigate how simple instruments might augment eFI performance.	Other: Decision Analytic Modelling; Prognostic models will be translated into a framework to guide clinical decision making by identifying relevant thresholds of predicted risk, above which implementation of our stated interventions is warranted.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Number of participants admitted to nursing homes	Incidence of new admission to a nursing home, identified by new nursing home residence in the routine dataset, based on address data	12 months
Number of participants requiring new/increased home care package	Incidence of new or increased home care services, identified by coded evidence of new or increased use of home care services in the routine dataset	12 months
Number of participants experiencing emergency department (ED) attendance/hospitalisation with fall/fracture	Incidence of emergency department (ED) attendance or hospitalisation with fall or fracture, identified using coded evidence of ED attendeance/hospitalisation with fall/fracture in the routine dataset	12 months
All-cause mortality	Incidence of all-cause mortality, defined using Office for National Statistics death data, or coded evidence of death in the routine dataset	12 months

Collaborators and Investigators

• Sponsor: University of Leeds
• Collaborators: University of Nottingham; University College, London; Swansea University; National Institute for Health Research, United Kingdom; University of Leicester; University of Exeter; Keele University; Bradford Teaching Hospitals NHS Foundation Trust; NHS Bradford Districts Clinical Commissioning Group
• Investigators: Principal Investigator: Andrew Clegg, MD, University of Leeds

Study record dates

Study Major Dates

• Study Start (Actual): January 1, 2015
• Primary Completion (Anticipated): May 1, 2022
• Study Completion (Anticipated): May 1, 2022

Study Registration Dates

• First Submitted: September 2, 2019
• First Submitted That Met QC Criteria: October 1, 2019
• First Posted (Actual): October 2, 2019

Study Record Updates

• Last Update Posted (Actual): October 25, 2019
• Last Update Submitted That Met QC Criteria: October 23, 2019
• Last Verified: October 1, 2019

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Pathologic Processes; Frailty
• Other Study ID Numbers: NIHR127905

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: No

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No