A Study to the Impact of Accuracy Problem Lists in Electronic Health Records on Correctness and Speed of Clinical Decision-making Performed by Dutch Healthcare Providers (ADAM's APPLE)

December 21, 2022 updated by: Eva Klappe

a Randomized Controlled Trial Study to Determine the Impact of Accuracy of Problem Lists in Electronic Health Records on Clinical Decision-making

The primary objective of this study is to determine whether patient records with complete, structured and up-to-date problem lists ('accurate problem lists'), result in better clinical decision-making, compared to patient records that convey the same information in a less structured way where the problem list has missing and/or duplicate diagnoses ('inaccurate problem lists'). The secondary objective is to determine whether the time required to make a correct decision is less for patient records with accurate problem lists compared to patient records with inaccurate problem lists.

Study Overview

Status

Status

Indicates the current recruitment status or the expanded access status.
Completed

Conditions

Conditions

The disease, disorder, syndrome, illness, or injury that is being studied. On ClinicalTrials.gov, conditions may also include other health-related issues, such as lifespan, quality of life, and health risks.

Intervention / Treatment

Intervention / Treatment

A process or action that is the focus of a clinical study. Interventions include drugs, medical devices, procedures, vaccines, and other products that are either investigational or already available. Interventions can also include noninvasive approaches, such as education or modifying diet and exercise.

Detailed Description

A problem list in Electronic Health Records (EHRs) is considered an essential feature in the collection of structured data. The problem list provides a centralized summary of each patient's medical problems and these problems or diagnoses are selected from the terminology underlying the problem list, such as SNOMED CT. If well maintained and structured, the problem list is a valuable tool for reviewing records of (unfamiliar) patients as it quickly shows the required information when needed. While studies have shown that the use of structured formats can serve as prompt for extra details, greater consistency of information and clinical decision-making there is little evidence whether a patient record with complete and structured problem lists results in more accurate and faster clinical decision making.

In the study (ADAM's APPLE: Adequate Data registration And Monitoring, subproject: Accurate Presentation of Problem List Elements), the investigators will perform a crossover randomized controlled trial in which a laboratory experiment will be performed among individual healthcare professionals to assess the impact of patient records with accurate and inaccurate problem lists on clinical decision-making. The participants will be presented with two records of two different patients in a training environment of the software system EPIC, one of them with an accurate problem list and the other that conveys the complete information in the patient record (in free text notes) but with an inaccurate problem list with missing diagnoses and duplicate information. The participants do not know which of the two records includes the accurate problem list and which record includes the inaccurate problem list. Participants are asked to decide whether or not to prescribe two medications for those two patients. One medication is not allowed per patient because the patient is allergic to that medication, which is documented on the allergy list. For the first patient record, the other medication is not allowed, because of a contraindicated diagnosis and for the second patient record the other medication is not allowed, because of a side effect that has occurred using that medication in the medical history. Based on the correctness of the motivation for correct answers and the time to the right answers, the research question if accurate problem lists in patient records lead to better and faster decision-making is answered.

Prior to this study, two healthcare professionals in the research team determined suitable use cases and questions for this study. These use cases were based on real-world unstructured versions of patient records. Two optimized accurate problem lists were also created for both patient records, which was defined according to the problem list policy at our institution (i.e. all current active problems and relevant medical history should be documented on the problem list).

Study Type

Study Type

Describes the nature of a clinical study. Study types include interventional studies (also called clinical trials), observational studies (including patient registries), and expanded access.
Interventional

Enrollment (Actual)

Enrollment

The number of participants in a clinical study. The "anticipated" enrollment is the target number of participants that the researchers need for the study.
160

Phase

Phase

The stage of a clinical trial studying a drug or biological product, based on definitions developed by the U.S. Food and Drug Administration (FDA). The phase is based on the study's objective, the number of participants, and other characteristics. There are five phases: Early Phase 1 (formerly listed as Phase 0), Phase 1, Phase 2, Phase 3, and Phase 4. Not Applicable is used to describe trials without FDA-defined phases, including trials of devices or behavioral interventions.
  • Not Applicable

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Contact

Study Contact

The name and contact information for the person who can answer enrollment questions for a clinical study. Each location where the study is being conducted may also have a specific contact, who may be better able to answer those questions.

Study Contact Backup

Study Locations

  • Netherlands
    • Noord-Holland
      • Amsterdam, Noord-Holland, Netherlands, 1105AZ
        • Amsterdam UMC, location AMC

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Eligibility Criteria

The key requirements that people who want to participate in a clinical study must meet or the characteristics they must have. Eligibility criteria consist of both inclusion criteria (which are required for a person to participate in the study) and exclusion criteria (which prevent a person from participating). Types of eligibility criteria include whether a study accepts healthy volunteers, has age or age group requirements, or is limited by sex.

Ages Eligible for Study

  • Child
  • Adult
  • Older Adult

Accepts Healthy Volunteers

No

Genders Eligible for Study

All

Description

Inclusion Criteria:

  • Healthcare professionals who are allowed to prescribe medication, thus hold a position as: medical specialist, medical resident, nurse specialist or physician assistant, research-specialists
  • Healthcare professionals must have followed at least the 'basic EHR Epic course'. This electronic health record course lasts for three days and includes how to send letters, register diagnoses in a record, request testing, all in the software system EPIC, which concludes with an exam on the theory.

Exclusion Criteria:

  • Non-Dutch speaking employees as the patient cases and the exercises are described in Dutch

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

  • Primary Purpose: Health Services Research
  • Allocation: Randomized
  • Interventional Model: Crossover Assignment
  • Masking: Single

Arms and Interventions

Participant Group / Arm

Participant Group / Arm

A group or subgroup of participants in a clinical trial that receives a specific intervention/treatment, or no intervention, according to the trial's protocol.
Intervention / Treatment

Intervention / Treatment

A process or action that is the focus of a clinical study. Interventions include drugs, medical devices, procedures, vaccines, and other products that are either investigational or already available. Interventions can also include noninvasive approaches, such as education or modifying diet and exercise.
Active Comparator: accurate problem list, then inaccurate problem list

in round 1, the participants will use the patient record of patient A, with an accurate problem list and answer the question: can the patient be prescribed Medication X and Y where medication X is a control question and medication Y is related to a contraindicated diagnosis (on problem list)

In round 2, the participants will use the patient record of patient B, with an inaccurate problem list and answer the question: can the patient be prescribed Medication X and Y where medication X is a control question and medication Y is related to medical history (not on problem list)
Other: patient A with accurate problem list
A problem list that contains a diagnosis code that is contraindicated with a type of medication (Y). Also, all other relevant diagnoses and medical history for the patient are up-to-date on the problem list, which was defined according to the problem list policy at our institution (i.e. all current active problems and relevant medical history should be documented on the problem list). Additionally, the problem list is included in eight out of thirteen notes using so-called smart phrases that can automatically import a (part of a) problem list. One note includes the problem list with the diagnosis relevant for the question asked.
Other: patient B with inaccurate problem list

A problem list that does not contain the diagnosis code and corresponding details explaining medical history of this diagnosis caused by a type of medication (Y). Additionally, the problem list is included in three out of thirteen notes using so-called smart phrases that can automatically import a (part of a) problem list. The relevant diagnosis is not documented on the problem list and hence is not included in the imported problem list in the notes.

The expert panel provided and anonymized two real-world representative examples of hematology patient records that included inaccurate problem lists and that had many free-text notes. An 'inaccurate problem list' is defined as a problem list where diagnoses are missing resulting in missed trigger medication or order-alerts, where diagnoses are 'active' although they should be closed or removed and/or where the problem list contained duplicated diagnoses.
Active Comparator: inaccurate problem list, then accurate problem list

in round 1, the participants will use the patient record of patient A, with an inaccurate problem list and answer the question: can the patient be prescribed Medication X and Y where medication X is a control question and medication Y is related to a contraindicated diagnosis (not on problem list)

In round 2, the participants will use the patient record of patient B, with an accurate problem list and answer the question: can the patient be prescribed Medication X and Y where medication X is a control question and medication Y is related to medical history (on problem list)
Other: patient A with inaccurate problem list

A problem list that does not contain the diagnosis code that is contraindicated with the type of medication (Y). Additionally, the problem list is included in eight out of thirteen notes using so-called smart phrases that can automatically import a (part of a) problem list. The relevant diagnosis is not documented on the problem list and hence is not included in the imported problem list in the notes.

The expert panel provided and anonymized two real-world representative examples of hematology patient records that included inaccurate problem lists and that had many free-text notes. An 'inaccurate problem list' is defined as a problem list where diagnoses are missing resulting in missed trigger medication or order-alerts, where diagnoses are 'active' although they should be closed or removed and/or where the problem list contained duplicated diagnoses.

Other: patient B with accurate problem list
A problem list that contains the diagnosis code and corresponding details explaining medical history of this diagnosis caused by a type of medication (Y). Also, all other relevant diagnoses and medical history for the patient are up-to-date on the problem list, which was defined according to the problem list policy at our institution (i.e. all current active problems and relevant medical history should be documented on the problem list). Additionally, the problem list is included in three out of thirteen notes using so-called smart phrases that can automatically import a (part of a) problem list. One note includes the problem list with the diagnosis and details relevant for the question asked.

What is the study measuring?

Primary Outcome Measures

Primary Outcome Measures

In a clinical study's protocol, the planned outcome measure that is the most important for evaluating the effect of an intervention/treatment. Most clinical studies have one primary outcome measure, but some have more than one.
Outcome Measure
Measure Description
Time Frame
the correctness of the answer of medication B including the right motivation
Time Frame: during the experiment/questionnaire
Measured using a questionnaire showing the two question(s) per patient record on a separate tablet during the experiment. Two Yes/No questions per patient record (so a total of four questions per participant) are answered. Answers are stored using an automatically assigned unique anonymized identifier. The question for both patient records is: "can the patient be prescribed medication A and/or B?". Medications A and B can both not be prescribed, but medication B is related to the problem list diagnoses and medication A is related to the allergy list which is the same for both versions of the patient records. A motivation is required per Yes/No answer to determine the correctness of the answer and prevent from a chance of gambling. An independent researcher from the research team who did not perform the experiments will categorize the motivation of the answers for medication B.
during the experiment/questionnaire

Secondary Outcome Measures

Secondary Outcome Measures

In a clinical study's protocol, a planned outcome measure that is not as important as the primary outcome measure for evaluating the effect of an intervention but is still of interest. Most clinical studies have more than one secondary outcome measure.
Outcome Measure
Measure Description
Time Frame
the total time to answer the two questions correctly, where the answer of medication B also includes the right motivation
Time Frame: during the experiment/questionnaire
The time to enter the motivation was not measured. However, the participants might give the right answer but the wrong motivation. For this secondary outcome measure, only the answers that have the right motivation are considered as 'right answer'.
during the experiment/questionnaire
the total time to answer the two questions correctly including the right motivations for medication A and B
Time Frame: during the experiment/questionnaire
A time stamp is registered when the participant opens the question, and a time stamp is registered when the participant confirms the answers. A time stamp is therefore registered for the total of two Yes/No answers per patient record.
during the experiment/questionnaire
the correctness of both the answer for medication A and B including the right motivations for A and B
Time Frame: during the experiment/questionnaire
Measured using a questionnaire showing the two question(s) per patient record on a separate tablet during the experiment. Two Yes/No questions per patient record (so a total of four questions per participant) are answered. Answers are stored using an automatically assigned unique anonymized identifier. The question for both patient records is: "can the patient be prescribed medication A and/or B?". Medications A and B can both not be prescribed, but medication B is related to the problem list diagnoses and medication A is related to the allergy list which is the same for both versions of the patient records. A motivation is required per Yes/No answer to determine the correctness of the answer and prevent from a chance of gambling. An independent researcher from the research team who did not perform the experiments will categorize the motivation of the answers for medication A and B.
during the experiment/questionnaire
the correctness of the answer for medication A including the right explanation
Time Frame: during the experiment/questionnaire
Measured using a questionnaire showing the two question(s) per patient record on a separate tablet during the experiment. Two Yes/No questions per patient record (so a total of four questions per participant) are answered. Answers are stored using an automatically assigned unique anonymized identifier. The question for both patient records is: "can the patient be prescribed medication A and/or B?". Medications A and B can both not be prescribed, but medication B is related to the problem list diagnoses and medication A is related to the allergy list which is the same for both versions of the patient records. A motivation is required per Yes/No answer to determine the correctness of the answer and prevent from a chance of gambling. An independent researcher from the research team who did not perform the experiments will categorize the motivation of the answers for medication A.
during the experiment/questionnaire

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Sponsor

The organization or person who initiates the study and who has authority and control over the study.
Eva Klappe

Investigators

Investigators

A researcher involved in a clinical study. Related terms include site principal investigator, site sub-investigator, study chair, study director, and study principal investigator.
  • Principal Investigator: Eva Klappe, MSc, Academisch Medisch Centrum - Universiteit van Amsterdam (AMC-UvA)

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Actual)

Study Start

The actual date on which the first participant was enrolled in a clinical study. The "anticipated" study start date is the date that the researchers think will be the study start date.
December 1, 2022

Primary Completion (Actual)

Primary Completion

The date on which the last participant in a clinical study was examined or received an intervention to collect final data for the primary outcome measure. Whether the clinical study ended according to the protocol or was terminated does not affect this date. For clinical studies with more than one primary outcome measure with different completion dates, this term refers to the date on which data collection is completed for all the primary outcome measures. The "anticipated" primary completion date is the date that the researchers think will be the primary completion date for the study.
December 21, 2022

Study Completion (Actual)

Study Completion

The date on which the last participant in a clinical study was examined or received an intervention/treatment to collect final data for the primary outcome measures, secondary outcome measures, and adverse events (that is, the last participant's last visit). The "anticipated" study completion date is the date that the researchers think will be the study completion date.
December 21, 2022

Study Registration Dates

First Submitted

First Submitted

The date on which the study sponsor or investigator first submitted a study record to ClinicalTrials.gov. There is typically a delay of a few days between the first submitted date and the record's availability on ClinicalTrials.gov (the first posted date).
November 22, 2022

First Submitted That Met QC Criteria

First Submitted That Met QC Criteria

The date on which the study sponsor or investigator first submits a study record that is consistent with National Library of Medicine (NLM) quality control (QC) review criteria. The sponsor or investigator may need to revise and submit a study record one or more times before NLM's QC review criteria are met. It is the responsibility of the sponsor or investigator to ensure that the study record is consistent with the NLM QC review criteria.
December 10, 2022

First Posted (Actual)

First Posted

The date on which the study record was first available on ClinicalTrials.gov after National Library of Medicine (NLM) quality control (QC) review has concluded. There is typically a delay of a few days between the date the study sponsor or investigator submitted the study record and the first posted date.
December 20, 2022

Study Record Updates

Last Update Posted (Estimate)

Last Update Posted

The most recent date on which changes to a study record were made available on ClinicalTrials.gov. There may be a delay between when the changes were submitted to ClinicalTrials.gov by the study's sponsor or investigator (the last update submitted date) and the last update posted date.
December 23, 2022

Last Update Submitted That Met QC Criteria

Last Update Submitted That Met QC Criteria

The most recent date on which the study sponsor or investigator submitted changes to a study record that are consistent with National Library of Medicine (NLM) quality control (QC) review criteria. It is the responsibility of the sponsor or investigator to ensure that the study record is consistent with the NLM QC review criteria.
December 21, 2022

Last Verified

Last Verified

The most recent date on which the study sponsor or investigator confirmed the information about a clinical study on ClinicalTrials.gov as accurate and current. If a study with a recruitment status of recruiting; not yet recruiting; or active, not recruiting has not been confirmed within the past 2 years, the study's recruitment status is shown as unknown.
December 1, 2022

More Information

Terms related to this study

Other Study ID Numbers

Other Study ID Numbers

Identifiers or ID numbers other than the NCT number that are assigned to a clinical study by the study's sponsor, funders, or others. These numbers may include unique identifiers from other trial registries and National Institutes of Health grant numbers.
  • Amsterdam UMC 2019-AMC-JK-7
  • 2019-AMC-JK-7 (Other Grant/Funding Number: Amsterdam UMC)

Plan for Individual participant data (IPD)

Plan to Share Individual Participant Data (IPD)?

No

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Clinical Trials on Quality of Health Care

Clinical Trials on patient A with accurate problem list

Search Similar Trials

Sponsors and Collaborators

Medical Conditions

Drug Interventions

Conditions

Rare Diseases

Drug Interventions

Dietary Supplements

Sponsor/Collaborators

Locations

Cookie Settings

We use cookies to ensure that we give you the best experience on our website. For more details carefully read our Privacy and Cookies Policy. ...>>>You can change your preferences or withdraw your consent at any time by deleting the cookies from your website or computer as described in the policy. Please accept it and choose your preferences by ticking the corresponding boxes in the "Manage Settings" section below. Please carefully read our Terms of Service before you proceed with using any part of this website.
«Manage Settings