Development of a Model for Digital Monitoring of the Mental State of the Hospitalized Patient

Development and Validation of an Experience Sampling Method Questionnaire (ESM) for Digital Monitoring of Mental State in Psychiatric Hospitalization: Convergence of ESM Reports, Standard Clinical Assessments, and Smartwatch Data

This study presents the development and validation of a unique Digital Experience Sampling Method (ESM) questionnaire specifically adapted for monitoring changes in the mental state of patients during psychiatric hospitalization. The questionnaire was carefully crafted through focus groups involving patients and clinical staff, ensuring its relevance and applicability to the unique characteristics of mental state changes in a hospitalization setting.

To evaluate the validity of the ESM questionnaire, symptom severity trends obtained from the questionnaire will be compared with estimates derived from the Positive and Negative Syndrome Scale (PANSS) assessment. Data will be collected from 100 subjects over a 14-day psychiatric hospitalization period. In addition to the ESM questionnaire, smartwatch sensors will monitor physiological indicators.

Feasibility and patient compliance will be assessed by examining patients' willingness to use the digital ESM questionnaires and the smartwatch sensors. The study will also cross-reference self-reported sleep quality and activity levels captured in the ESM questionnaires with objective physiological indicators and nursing staff reports, providing insights into the reliability of the patient-reported data.

Furthermore, the study will evaluate the impact of the ESM data on clinical decision-making by physicians throughout the 14-day psychiatric hospitalization period. Patient satisfaction and satisfaction among the multidisciplinary team with the monitoring model will also be assessed.

This research underscores the potential of digital technologies to enhance patient-centered care and facilitate informed treatment decisions in psychiatric hospitalization settings.

Study Overview

• Status: Recruiting
• Conditions: Personality Disorders; Depressive Disorder; Mood Disorders; Schizophrenia; Schizoaffective Disorder; Anxiety Disorders; Bipolar Disorder; Severe Mental Disorder; Psychotic Episode

Detailed Description

The research will be carried out in two phases:

1. Development of the monitoring questionnaire - Content validity phase - This phase has been fully completed, and its results will be presented in the preliminary results chapter. At this research stage, the core content domains of the monitoring questionnaire will be developed. The research process in this phase includes the following:

1. Examining the content of scales from the Positive and Negative Syndrome Scale (PANSS) for schizophrenia and The Symptom Checklist-90-R (SCL-90-R) questionnaire: At this stage, the psychiatric symptoms and the main criteria for assessing a mental state as emerged from those questionnaires were examined for discussion in the focus groups.

2. Focus group No. 1: The first focus group included patients hospitalized in the psychiatric ward. The purpose of the focus group is to formulate the ESM core and additional content domains and the questionnaire items and to discuss the optimal measurement frequency according to the hospitalized patients' opinions. The focus group will also examine the wording that best suits how the patients describe their mental state.

   Indices The subjects' point of view regarding the psychiatric symptoms they suffered from during their hospitalization was examined during a free discussion. Emphasis was placed on the symptoms that change daily throughout the hospitalization and how much they indicate a change in their mental state. The investigators also investigated what information they would like to report to the treatment team about their mental state throughout the hospitalization. After the free discussion, the position of the subjects was examined regarding the criteria for assessing mental state as they emerged from the monitoring scales of the PROM questionnaires.

   After a qualitative analysis of the information collected at this stage, the main themes of the ESM questionnaire were formulated. According to the themes found, suggestions were developed for the questionnaire items while noting how the patient's symptoms were worded. A professional transcriber transcribed the discussion.

3. Focus group No. 2: The second focus group includes a multi-disciplinary team working in the psychiatric inpatient wards. In a free discussion, the multi-disciplinary team members examined the core issues that should appear in the questionnaire for monitoring the mental state of hospitalized patients, then explored and expressed their opinion regarding the issues raised by the patients in the first focus group. The result of this step will be an ESM questionnaire adapted to the needs of the patients and the clinical staff in the inpatient wards.

   Indices In the discussion, the participants floated important themes for monitoring according to their points of view. After a free discussion, the participants were presented with the themes raised by the patients in the previous focus group. They were also presented with the possible questionnaire items formulated after the last stage, and their position was examined regarding the appropriateness of fitting these items to the questionnaire. In addition, the discussion dealt with the optimal sampling frequency, in their opinion.

   The initial formulation of the ESM questionnaire items. The questionnaire items were formulated considering the information collected in the focus groups. An effort was made to use the patients' words to ask about their mental state. In addition, existing and valid self-report items marked as necessary by the patients and staff in the focus groups discussion were selected. Items from these questionnaires were reformulated according to the ESM approach that examines the current state. At the end of this stage, an ESM questionnaire was developed, which consists of 15 items on a five levels Likert scale. In the questionnaire, there was another item of a Visual Analogue Scale type where the subject had to mark his mood on a scale ranging from 1-100.

4. Focus group No.3: The third focus group included hospitalized inpatients for cognitive debriefing - Cognitive debriefing is the process by which a patient questionnaire is actively tested among representatives of the target population and target language group to determine whether respondents understand the questionnaire as the authors intended it to be understood.

Indices For each item in the developed ESM questionnaire, subjects were asked to describe freely what they thought the item was testing. In addition, to each item, the investigators discussed how appropriate the question was in their opinion to describe their mental state throughout the hospitalization and whether they would change anything in the wording. At the end of this part of the discussion, the subjects were asked how many times a day they would be willing to report their mental state, how many items they would be ready to answer, and what additional questions they would like to be asked to learn about their condition. In addition, the investigators examined the patients' attitudes regarding their agreement to answer questions via mobile phone and their agreement to continue answering a similar questionnaire after their discharge from the hospital.

Research Process The focus groups were held at Sheba Medical Center on 5/18/22, 08/9/22, and 11/09/22. The groups were guided by the researcher and another facilitator, who work as senior clinical psychologists in the psychiatric inpatient wards at the Sheba Medical Center. A professional transcriber attended each group and typed the conversation throughout the focus groups. The transcripts of the discussions in the focus groups were qualitatively analyzed. After collecting and processing the data from the focus group, the ESM questionnaire was improved and refined, considering the findings.

2. Running the monitoring model - feasibility phase - this phase will begin in the middle of 2023 and is expected to continue for 18 months.

Participants

To conduct the monitoring model, the investigators will recruit 100 subjects hospitalized in the inpatient wards. To receive feedback on implementing the monitoring model from the multi-disciplinary care team working in the wards, the investigators will recruit 30 multi-disciplinary team members. The running will be carried out in two stages:

Pre-Test - 20 patients and feedback from 5 multi-disciplinary - estimated time - 3 months.

Field Test - 80 additional patients and feedback from 30 multi-disciplinary staff members - estimated 12 months.

Indices Pre-test indices -

1. Selection of the ESM questionnaire items with the best distribution indices and improving the wording of items with low variability scores within and between subjects. Feedback from staff? Field test indices -
2. The effect of exposing the psychiatrists to the patient's monitoring indicators on their clinical decisions regarding him will be examined in a short questionnaire that the physician will fill out after each medication change and mark on a Likert scale of 1 5 - how much the exposure to the monitoring indicators influenced the clinical decision.

3. Feasibility and compliance:

   1. The responsiveness to participation in the monitoring model - The number of subjects who agreed to participate in the study out of the subjects the investigators offered to participate.
   2. The extent of responding to the self-report questionnaires over 14 days multiplied by three times a day.
   3. The degree of responsiveness to wearing and charging the smartwatch for 14 days.
   4. The degree of difference between subjects with different psychiatric diagnoses (psychotic spectrum, depression, mania, and personality disorders) in their adherence and responsiveness to agree to participate in the study, answer the self-report ESM questionnaires and wear and charge the smartwatch.

4. The validity of the subject's self-reports on sleep quality and activity level:

   1. The validity of the subject's self-reports on sleep quality -The subject's self-reported data about his sleep quality and number of awakenings will be compared to physiological indicators that the smartwatch will collect (number of hours of sleep, number of awakenings) and will be equalized in addition to the medical reports of the nursing staff in the ward.

   2. The validity of the subject's self-reports on his activity level:

      The subject's self-reported data about his energy and activity level will be compared to physiological indicators that the smartwatch will collect (number of steps, accelerations) and will be equalized in addition to the reports of the nursing staff in the ward.

   3. The degree of difference between subjects with different psychiatric diagnoses (psychotic spectrum, depression, mania, and personality disorders) in correspondence between the self-report and the physiological indices and the reports of the nursing staff.
5. Changes in the hospitalized patient's condition over the timeline: Changes within 14 days of hospitalization will be presented through the patient's self-report indicators, the physiological indicators collected by the sensor, and the reports of the multi-disciplinary team in the ward.
6. Patient satisfaction: After 14 days of monitoring, the subjects will report their satisfaction with using the monitoring model throughout the hospitalization through questionnaires.
7. The satisfaction of the multi-disciplinary teams: At the end of the study, the teams will report through questionnaires on their satisfaction with using this monitoring model and its influence on the team's clinical decision-making.

Research Process After recruiting the patient for the study and signing informed consent, the monitoring application (Datos Health) and the wearable smartwatch application (Garmin) will be installed on the patient's smartphone. The patient will receive an extensive explanation from the research team's representative on the self-reporting method and will experience using the application.

From this stage, the hospitalized subject will receive ongoing clinical care, alongside participation in monitoring indicators by self-reporting in the ESM questionnaires using the Datos Health application and physiological indices monitoring by using the Garmin smartwatch. The trial will end after 14 days or until the patient's discharge (whichever comes first). The ESM self-report sampling will be conducted a frequency of 3 times a day in a semi-random manner:

• The first sample will be randomly sampled between 8-10 am.
• A second sample will be randomly sampled between 1-3 pm.
• A third sample will be randomly sampled between 7-10 pm. In each clinical discussion of the subject's mental state, the ward staff will watch the dashboard of the Datos Health application, which will display the subject's self-report measures and physiological indicators. For each change in the patient's clinical status, determined by the physician (change of medication, vacation, preparation for discharge, discharge), the physician will answer a questionnaire in which he will document how much (between 1-5) the clinical decision was influenced by the monitoring indicators he observed.

The feasibility and compliance, Validity, and changes in the patient's condition over timeline indicators will be collected for 14 days and analyzed at the study's end. The patient satisfaction indicators will be tested after 14 days of monitoring through a questionnaire sent to each subject. The satisfaction indicators of the clinical staff in the ward will be tested through questionnaires sent to the clinical staff at the end of the study.

• Study Type: Observational
• Enrollment (Estimated): 130

Contacts and Locations

• Study Contact: Name: Mark Weiser, MD; Phone Number: +97235303773; Email: mweiser@netvision.net.il
• Study Contact Backup: Name: Hilla Mordechai, MA; Phone Number: +972507863471; Email: hillamor1@gmail.com

Study Locations

• Israel
  • Ramat Gan, Israel
    • Recruiting
    • Chaim Sheba Medical Center
    • Contact: Hilla Mordechai, MA; Phone Number: +972507863471; Email: hillamor1@gmail.com

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

Focus group No. 1: The first focus group included patients hospitalized in the psychiatric ward. Recruited nine subjects over age 18 (6 women and three men).

Focus group No. 2: The second focus group includes a multi-disciplinary team working in the psychiatric inpatient wards. Recruited seven staff members working in the psychiatric hospitalization wards in Sheba Medical Center (3 women and four men), participated in the focus group.

Focus group No.3: The third focus group included hospitalized inpatients for cognitive debriefing. Recruited six subjects over 18 (two women and four men).

To conduct the monitoring model, we will recruit 100 subjects hospitalized in the inpatient wards. To receive feedback on implementing the monitoring model from the multi-disciplinary care team working in the wards, we will recruit 30 multi-disciplinary team members.

Description

Focus group no.1 :

Inclustion Criteria-

• Hospitalized by consent in the psychiatric wards at the Sheba Medical Center.
• Signed informed consent to participate in the study. Exclusion Criteria-
• Patients with a low level of compliance to treatment.
• Patients suffering from a significant active physical illness, intellectual disability, or complex organic conditions including dementia conditions
• Patients who cannot read and write in the Hebrew language
• Patients who are not qualified to give informed consent.

Focus group no.2:

Inclusion Criteria:

• Staff members work in the inpatient wards and have a valid Israeli license.
• Signed informed consent to participate in the study.

No Exclusion Criteria.

Focus group no.3-

Inclustion Criteria:

• Hospitalized by consent in the psychiatric wards at the Sheba Medical Center.
• Signed informed consent to participate in the study.

Exclusion Criteria:

• Patients with a low level of compliance to treatment.
• Patients suffering from a significant active physical illness, intellectual disability, or complex organic conditions including dementia conditions
• Patients who cannot read and write in the Hebrew language
• Patients who are not qualified to give informed consent.

Monitoring model- The investigators will recruit 100 subjects hospitalized in the inpatient wards.

Inclusion criteria:

• Adults are consenting hospitalized in a psychiatric hospital at the Sheba Medical Center.
• Staying in the open part of the ward.
• Owns a mobile smartphone.

Exclusion criteria:

• Patients with a low level of treatment compliance.
• Patients with a developmental or intellectual disability, complex organic conditions including dementia.
• Patients who cannot read and write in the Hebrew language
• Patients who are incapable of informed consent.

The study inclusion criteria for the multi-disciplinary team:

• Multi-disciplinary team members who work in the psychiatric inpatient wards at the Sheba Medical Center.
• The staff members will have a valid Israeli license (from the professions of medicine, nursing, psychology, social work, and occupational therapy).

Study Plan

How is the study designed?

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
The primary objective of this study is to validate the ESM questionnaire by comparing the trends in symptom severity as reported through the ESM questionnaire with the PANSS assessment of symptom severity.	The positive and negative syndrome scale (PANSS) for schizophrenia is measuring the severity of symptoms in patients with schizophrenia spectrum disorders. It is conducted through a brief interview rather than a self-report. The ESM (Experience Sampling Method) is a method Structured based on a diary, in which the subjects are asked to report their thoughts, feelings, and symptoms during their daily life, as well as to describe the context in which the report takes place. The validation of the ESM questionnaire will be conducted by comparing the change trends in the questionnaire indices during a 14-day psychiatric hospitalization period to the change trends in four different measurements of psychiatric symptoms assessed by the PANSS diagnostic tool, which is widely recognized as the gold standard in psychiatric symptom assessment. The subject's assessment using PANSS will be conducted at three different time points.	14 days.

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Development of the monitoring questionnaire - Content validity phase.	The subjects' point of view regarding the psychiatric symptoms they suffered from during their hospitalization was examined during a free discussion. Emphasis was placed on the symptoms that change daily throughout the hospitalization and how much they indicate a change in their mental state. We also investigated what information they would like to report to the treatment team about their mental state throughout the hospitalization. After the free discussion, the position of the subjects was examined regarding the criteria for assessing mental state as they emerged from the monitoring scales of the PROM questionnaires. After a qualitative analysis of the information collected at this stage, the main themes of the ESM questionnaire were formulated. According to the themes found, suggestions were developed for the questionnaire items while noting how the patient's symptoms were worded. A professional transcriber transcribed the discussion.	14 days

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Investigation of the feasibility and compliance of patients in responding to self-report questionnaires.	The responsiveness to participation in the monitoring model - The extent of responding to the self-report questionnaires over 14 days multiplied by three times a day.	14 days
Investigation of the feasibility of patients in participating in the study.	The responsiveness to participation in the study - The number of subjects who agreed to participate in the study out of the subjects we offered to participate.	14 days
Investigation of the feasibility and compliance of patients with different diagnoses in participating in the study.	The responsiveness to participation in the monitoring model - The number of subjects who agreed to participate in the study out of the subjects we offered to participate. The degree of difference between subjects with different psychiatric diagnoses (psychotic spectrum, depression, mania, and personality disorders) in their adherence and responsiveness to agree to participate in the study, answer the self-report ESM questionnaires and wear and charge the smartwatch.	14 days
The validity of subjects' self-reports on sleep quality and activity level will be assessed by comparing the self-report ESM data with the physiological indicators collected by the smartwatch.	The ESM indices will be compared to reports from the nursing staff on the ward. Furthermore, we will explore whether these comparisons yield different outcomes among subjects with different psychiatric diagnoses.	14 days
Testing the effectiveness of the monitoring model.	The effectiveness will be assessed by examining the impact of exposing the psychiatrists to ESM data, as well as the physiological indicators collected by the smartwatch, on the decision-making process regarding changes in the patient's medication plan during psychiatric hospitalization.	14 days
Assessing patient satisfaction with the monitoring model used throughout their 14-day hospitalization.	Following the monitoring period, subjects will be asked to provide feedback on their satisfaction levels through questionnaires.	14 days
Assessing the satisfaction of the multi-disciplinary teams.	At the end of the study, the teams will report through questionnaires on their satisfaction with using this monitoring model and its influence on the team's clinical decision-making.	14 days
Investigation of the feasibility and compliance of patients to instructions of smartwatch use.	The degree of responsiveness to instructions regarding the use of the smartwatch for 14 days.	14 days

Collaborators and Investigators

• Sponsor: Sheba Medical Center

Publications and helpful links

General Publications

• Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006 May 22;166(10):1092-7. doi: 10.1001/archinte.166.10.1092.
• Verhagen SJ, Hasmi L, Drukker M, van Os J, Delespaul PA. Use of the experience sampling method in the context of clinical trials. Evid Based Ment Health. 2016 Aug;19(3):86-9. doi: 10.1136/ebmental-2016-102418.
• Myin-Germeys I, Oorschot M, Collip D, Lataster J, Delespaul P, van Os J. Experience sampling research in psychopathology: opening the black box of daily life. Psychol Med. 2009 Sep;39(9):1533-47. doi: 10.1017/S0033291708004947. Epub 2009 Feb 12.
• Pilkonis PA, Choi SW, Reise SP, Stover AM, Riley WT, Cella D; PROMIS Cooperative Group. Item banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS(R)): depression, anxiety, and anger. Assessment. 2011 Sep;18(3):263-83. doi: 10.1177/1073191111411667. Epub 2011 Jun 21.
• Johnson EI, Grondin O, Barrault M, Faytout M, Helbig S, Husky M, Granholm EL, Loh C, Nadeau L, Wittchen HU, Swendsen J. Computerized ambulatory monitoring in psychiatry: a multi-site collaborative study of acceptability, compliance, and reactivity. Int J Methods Psychiatr Res. 2009;18(1):48-57. doi: 10.1002/mpr.276.
• 1. Derogatis, L. R. (1992). SCL-90-R: Administration, scoring and procedures manual for the R(evised) version and other instruments of the psychopathology rating scale series.
• 2. Hektner, J. M., Schmidt, J. A., & Csikszentmihalyi, M. (2007). Experience sampling method: Measuring the quality of everyday life. Sage.
• 6. Kroenke, K., & Spitzer, R. L. (2002). The PHQ-9: a new depression diagnostic and severity measure. Psychiatric annals, 32(9), 509-515.
• 7. Stone, A., Shiffman, S., Atienza, A., & Nebeling, L. (2007). The science of real-time data capture: Self-reports in health research. Oxford University Press.

Study record dates

Study Major Dates

• Study Start (Actual): May 18, 2022
• Primary Completion (Estimated): January 19, 2024
• Study Completion (Estimated): January 19, 2024

Study Registration Dates

• First Submitted: August 6, 2023
• First Submitted That Met QC Criteria: December 21, 2023
• First Posted (Estimated): December 27, 2023

Study Record Updates

• Last Update Posted (Estimated): December 27, 2023
• Last Update Submitted That Met QC Criteria: December 21, 2023
• Last Verified: October 1, 2023

More Information

Terms related to this study

• Keywords: Feasibility; Validity; Psychiatry; Compliance; Inpatients; Development; Smartwatch; Hospitalization; PANSS; EMA; ESM; Digital Monitoring; Mental State
• Additional Relevant MeSH Terms: Schizophrenia Spectrum and Other Psychotic Disorders; Bipolar and Related Disorders; Personality Disorders; Depressive Disorder; Schizophrenia; Psychotic Disorders; Anxiety Disorders; Mental Disorders; Bipolar Disorder; Mood Disorders
• Other Study ID Numbers: SHEBA-21-8403-MW-CTIL

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: UNDECIDED

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No