Military Service Identification Tool (MSIT)

Validating the Military Service Identification Tool in Its Ability to Correctly Identify Civilians and Those Who Have Served in the Military

Estimates of the UK's military veteran population, defined by the British Government as those who have served in the military for at least one day, is approximately 2.5 million, equivalent to around 5% of household residents aged 16 years or over in the UK. UK military veterans receive healthcare provision from the National Health Service (NHS), with care recorded in local, regional and national EHRs. EHRs - structured and unstructured (i.e. free text) - can be used to evaluate disease prevalence, surveillance, to perform epidemiological analyses and investigate quality of care and to improve clinical decision-making.

There is no national marker in UK EHRs to identify veterans, nor is there a requirement for healthcare professionals to record it, making it difficult to evaluate the unique healthcare needs of those who have served in the UK Armed Forces. This study, funded by Forces in Mind Trust, seeks to validate the Military Service Identification Tool, an open-source computer program that searches through free-text clinical notes to make a prediction on a person's military status. It is in the public interest to know the health of our Armed Forces. The Tool has been validated using manually annotated datasets, but we now need to valid an individual's military status by contacting them via post or telephone and asking, "Have you ever served in the Armed Forces". The research team will work closely with the CRIS Patient Advisory Group and local healthcare professionals.

Study Overview

• Status: Recruiting
• Conditions: Mental Health Disorder
• Intervention / Treatment: Other: Validating the Military Service Identification Tool

Detailed Description

Currently, there is no military service marker in electronic healthcare records to identify who is a veteran in England and Wales [3]. This makes it difficult to evaluate the unique healthcare needs of those who have served in the UK military. In a previous study, the investigators set out to investigate whether it was feasible to identify veterans who accessed secondary mental healthcare services via their anonymized electronic healthcare records using a manual, text search based, method. To do this, the investigators used the South London and Maudsley (SLaM) Biomedical Research Centre Case Register. This is a novel data resource, derived directly from the routine electronic healthcare records of the SLaM NHS Foundation Trust. SLaM is one of Europe's largest mental health providers, serving over 1.2 million residents in four South London boroughs. This Case Register holds patient's electronic healthcare records for all secondary mental healthcare provisions within SLaM. All these electronic healthcare records have been anonymized and a system has been developed to enable researchers to access and search through these records called the clinical record interactive system (CRIS).

As part of the first study the investigators developed a manual, text search based, method using commonly used terms and phrases found in the free text clinical notes to identify veterans. Running this manual method resulted in the identification of n = 6,039 potential veteran records, of which n = 1,600 were selected to scrutinize in more detail, whilst considering time and manpower restrictions. This resulted in n = 693 veteran records, suggesting an identification rate of 43% when using the manual method. Therefore, it was concluded that it was feasible to identify veterans in electronic healthcare records, but time consuming. Each potential veteran record was manually verified by the research team by reading through each patient's notes. This took on average 11 minutes per record.

To address the time it took to identify veterans and improve sensitivity and reach, he investigators developed the Military Service Identification Tool (MSIT), and it proved to be quicker, more accurate and cheaper than the manual method. The Tool was designed to detect military service, not just veteran status. 1the investigators took a systematic approach to developing and testing the MSIT. Then a different subsets of all the electronic healthcare records available to us, to ensure the MSIT was developed and trained on a different set of electronic healthcare records (named the training dataset). Subsequently the MSIT was verified on another subset of the data (named the gold standard dataset) to ensure it would still be able to identify veterans if a different set of electronic healthcare records would be used.

The investigators managed to identify 2,922 veterans in the electronic healthcare records when applying MSIT to inspect 150,000 individual records. The MSIT took only 20 minutes to go through all these records and had an identification rate of 88% when compared to the manual approach. The investigators also matched these veterans to 2,922 non-veterans based on age and gender to compare their mental health treatment pathways.

Despite the success of our initial developments, the investigators have faced a significant barrier that may prevent the widespread, dissemination, roll-out and implementation of the MSIT. Various academics and stakeholders (e.g. NHS England, Cobseo) are interested in the MSIT. However, it has been highlighted an important limitation, namely it is still unclear whether the veterans identified by the MSIT are 'actual' veterans. MSIT identifies veterans based on the notes provided by the clinician in the electronic healthcare record. It is also important to understand if higher rates of physical and mental health are represented in a veteran population compared to civilians. By including both samples - civilian and veteran - this ensures that fair comparisons are able to be made. This There is currently no opportunity to verify their veteran status. For example, no service number is provided in their medical records.

Therefore, to overcome these limitations and to demonstrate that the MSIT is correctly identifying military service (serving personnel, veterans or civilian) it is important to reach out to those who have been identified to ask about their previous Service.

• Study Type: Observational
• Enrollment (Anticipated): 5844

Contacts and Locations

Study Locations

• United Kingdom
  • Greater Londn
    • London, Greater Londn, United Kingdom, SE58AZ
      • Recruiting
      • South London and Maudsley NHS Foundation Trust
      • Contact: Dr Carol Cooley; Phone Number: 020 7848 0339; Email: slam-ioppn.research@kcl.ac.uk
      • Principal Investigator: Dr Sharon AM Stevelink, PhD
      • Sub-Investigator: Prof Nicola T. Fear, PhD

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: N/A
• Genders Eligible for Study: All

• Sampling Method: Non-Probability Sample

Study Population

Participants have already been identified using the Military Service Identification Tool. In a previous research project, the Tool was run on the records of 150,000 patients extracted from the Clinical Record Interactive Search, which is managed by NIHR Maudsley Biomedical Research Centre (BRC). A copy of the article describing the methodology has recently been accepted for publication in JMIR Medical Informatics (DOI: 10.2196/15852). In total 2,922 probable veterans were identified and matched to 2,922 civilians on age and gender. The dataset has a total sample size of 5,844 individuals. Once approvals are in place, we will identify those who have given consent to be contacted and meet our eligibility criteria (described in other parts of our application).

Description

Only those participants who formed part of our original study will be screened for eligibility. This corresponds to 2,922 probable veterans and 2,922 civilians. The following inclusion criteria will apply:

1. Participants are listed as 'Alive' in the SLaM BRC Clinical Record Interactive Search database;
2. Participants will be aged 18 years and old;
3. Participant has given Consent for Contact;
4. Participant has a valid postal or email address or telephone number;
5. Participants have capacity to consent and deemed fit to participate in research by a clinician.
6. Participants will be able to speak English.

Participants will be excluded if a clinician is required to be notified of the research prior to contact, as this may indicate at risk participants. Further participants who are not fluent in spoken English will be unable to take part in the study due to inability to understand the participant information and consent form. Therefore, non-English speaking participants will not be included in this study.

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Probable Civilian; This cohort represents those in which the Military Service Identification Tool has determined are a civilian and have not served in the Armed Forces.	Other: Validating the Military Service Identification Tool; This study seeks to validate the Military Service Identification Tool in its ability to accurately determine military service status.
Probable Veteran; This cohort represents those in which the Military Service Identification Tool has determined are a military veteran and have served in the Armed Forces.	Other: Validating the Military Service Identification Tool; This study seeks to validate the Military Service Identification Tool in its ability to accurately determine military service status.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Accuracy of the tool	The primary outcome of this study is to provide descriptive statistics on the accuracy of the Military Service Identification Tool in correctly identifying those who have served in the Armed Forces.	Analysis will be conducted within 6 months after data collection.

Collaborators and Investigators

• Sponsor: King's College London
• Collaborators: Combat Stress; Forces in Mind Trust

Publications and helpful links

General Publications

• Mark KM, Leightley D, Pernet D, Murphy D, Stevelink SAM, Fear NT. Identifying Veterans Using Electronic Health Records in the United Kingdom: A Feasibility Study. Healthcare (Basel). 2019 Dec 19;8(1):1. doi: 10.3390/healthcare8010001.

Helpful Links

• Pre-print of the Military Service Identification Tool.

Study record dates

Study Major Dates

• Study Start (Actual): September 1, 2020
• Primary Completion (Anticipated): June 1, 2022
• Study Completion (Anticipated): September 1, 2022

Study Registration Dates

• First Submitted: March 5, 2020
• First Submitted That Met QC Criteria: March 6, 2020
• First Posted (Actual): March 9, 2020

Study Record Updates

• Last Update Posted (Actual): September 2, 2021
• Last Update Submitted That Met QC Criteria: September 1, 2021
• Last Verified: August 1, 2021

More Information

Terms related to this study

• Keywords: mental health; natural language processing
• Additional Relevant MeSH Terms: Mental Disorders
• Other Study ID Numbers: 278987

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: We will not be sharing any data collected with other researchers as data will be deleted within 6 months of the study finishing.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No