Preventing Medication Dispensing Errors in Pharmacy Practice with Interpretable Machine Intelligence: Wave 2

Pharmacists currently perform an independent double-check to identify drug-selection errors before they can reach the patient. However, the use of machine intelligence (MI) to support this cognitive decision-making work by pharmacists does not exist in practice. This research is being conducted to examine the effectiveness machine intelligence (MI) advice on to determine if its impact on pharmacists' work performance and cognitive demand.

Study Overview

• Status: Completed
• Conditions: Machine Intelligence in the Pharmacy
• Intervention / Treatment: Behavioral: No MI Help; Behavioral: Interpretable MI; Behavioral: Uninterpretable MI

• Study Type: Interventional
• Enrollment (Actual): 30
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Michigan
    • Ann Arbor, Michigan, United States, 48109
      • University of Michigan

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

1. Licensed pharmacist in the United States
2. Age 18 years and older at screening
3. PC/Laptop with Microsoft Windows 10 or Mac (Macbook, iMac) with MacOS with Google Chrome or Firefox web browser installed on the device
4. Screen resolution of 1024x968 pixels or more
5. A laptop integrated webcam or USB webcam is also required for the eye tracking purpose.

Exclusion Criteria:

1. Eyeglasses with more than one power (bifocals, trifocals, progressives, layered lenses, or regression lenses)
2. Cataracts, intraocular implants, glaucoma, or permanently dilated pupil
3. Require a screen reader/magnifier or other assistive technology to use the computer
4. Eye surgery (e.g., corneal)
5. Eye movement or alignment abnormalities (lazy eye, strabismus, nystagmus)

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Interpretable MI; Participants receive interpretable machine intelligence to complete the medication verification task.	Behavioral: No MI Help; Participants will complete the medication verification task without any MI help; Behavioral: Interpretable MI; Participants receive interpretable machine intelligence assistance to complete the medication verification tasks.
Experimental: Uninterpretable MI; Participants receive uninterpretable (i.e., black-box) machine intelligence to complete the medication verification task.	Behavioral: No MI Help; Participants will complete the medication verification task without any MI help; Behavioral: Uninterpretable MI; Participants receive uninterpretable (i.e., black-box) machine intelligence assistance to complete the medication verification tasks.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Decision accuracy	Difference in detection rate measured by number of medication verification errors	1 day - Single study visit
Trust change	Difference in trust as measured by visual analog scale will be calculated based on AI advice accuracy. Participants will indicate their level of trust in the AI advice after every trial on a scale from 1-100, with higher scores indicating greater levels of trust.	1 day - Single study visit
Cognitive effort	Difference in cognitive effort measured by duration of fixation and fixation count	1 day - Single study visit

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Reaction time	Difference in task time measured by the number of seconds from starting the task to accepting or rejecting a medication image	1 day - Single study visit

Collaborators and Investigators

• Sponsor: Corey Lester
• Collaborators: National Library of Medicine (NLM)

Study record dates

Study Major Dates

• Study Start (Actual): February 1, 2023
• Primary Completion (Actual): May 12, 2023
• Study Completion (Actual): May 12, 2023

Study Registration Dates

• First Submitted: January 21, 2025
• First Submitted That Met QC Criteria: January 21, 2025
• First Posted (Actual): March 25, 2025

Study Record Updates

• Last Update Posted (Actual): March 25, 2025
• Last Update Submitted That Met QC Criteria: January 21, 2025
• Last Verified: January 1, 2025

More Information

Terms related to this study

• Other Study ID Numbers: HUM00213493; 5R01LM013624 (U.S. NIH Grant/Contract)

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No