Idea Density in Exam Performance (IDEP)

Impact of Reduced Idea Density on Pharmacy Students' Attainment in Pharmaceutical Calculations: a Single-blind Multicentre Randomised Controlled Trial

Text can be written in multiple ways to mean the same thing; changing how a text is written can make it easier or harder to understand. How many concepts or ideas there are in a text, divided by the total number of words, is one possible way to determine how easy or hard it is to understand. This ratio is called idea density (ID).

Varying ID has been shown to affect the speed at which a reader understands; it impacts certain people more than others, such as second language speakers. This effect may be of particular importance in an exam, where understanding a question in a limited time is key. In the UK, pharmacy students must undertake an exam set by the General Pharmaceutical Council (GPhC) to be registered as pharmacists. The exam involves pharmaceutical calculations and shows variable pass rates. This study aims to evaluate the impact of reducing ID in a pharmaceutical calculation test and will be conducted in 14 schools of pharmacy in the UK. All participants will take a GPhC style test. Then, participants will be divided into two groups of equal size; one group will undertake a second test with the same ID as the first, while the second group will undertake a test with a lower ID. Finally, the investigators will compare the second test scores between the two groups as cohorts and question by question, evaluating whether lowering ID has increased students' scores.

If and effect is seen, ensuring that questions are written with a controlled ID may help ensure we are examining more fairly and allowing students with the requisite knowledge to pass.

Study Overview

• Status: Completed
• Conditions: Medical Education; Health Education; Educational Assessment
• Intervention / Treatment: Other: Reduced Idea Density

Detailed Description

Pharmaceutical Council must be undertaken. It involves pharmaceutical calculations and shows variable pass rates. Linguistic factors, such as idea density, affect and predict comprehension time. This trial will evaluate the effect of lowering question idea density on attainment in a pharmaceutical calculations exam aligned to that of the General Pharmaceutical Council.

Methods: This is a single-blind, parallel 2-arm multicentre randomised controlled trial conducted in fourteen Universities across the United Kingdom. A 1:1 randomisation and a sample size of 188 pharmacy students will be sufficient to detect a 1-point difference in the mean scores between the intervention and control group during a pharmacy calculation test with two-tails, 80% power and 5% significance level. Each school will recruit a minimum of 14/15 students. Participants will sit two 12-question pharmaceutical calculation tests. All participants will take the same baseline test; then, will be randomised and undertake a second test 2-week after, with standard idea density for the control group and lower idea density for the intervention. Primary outcome: the scores obtained by the participants undertaking the second calculation test 2-week after the baseline. Secondary outcomes: percentage of students achieving a pass during the second test; effect of demographic characteristics (first or not-first English language speakers, age, ethnicity, year of study, specific learning disability) on participants' attainment when lowering idea density.

Conclusion: Results could inform the development of new standards in the pharmaceutical calculations exam.

• Study Type: Interventional
• Enrollment (Actual): 204
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Mandy Williams; Phone Number: 6467 01519046467; Email: A.F.Williams@ljmu.ac.uk

Study Locations

• United Kingdom
  • Merseyside
    • Liverpool, Merseyside, United Kingdom, L3 3AF
      • Liverpool John Moores University

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• Over 18 years of age
• Registered as a student on an MPharm course in the UK
• Be in years 1-4 of the course (levels 1-7)

Exclusion Criteria:

• Under 18
• Not registered on an MPharm course in the UK
• Are currently undertaking a foundation year (level 3)

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Basic Science
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Single

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Control; Participants in the control group will receive an initial 12 question baseline test (same as intervention). They will after two weeks receive a second test that is linguistically the same as the national GPhC exam.
Experimental: Intervention; Participants in the intervention group will receive an initial 12 question baseline test (same as control). They will after two weeks receive a second test that is reduced in idea density by around 10% overall.	Other: Reduced Idea Density; Question idea density reduced (ratio of propositions to total words used).

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Overall performance on test 2	The primary outcome is the performance in the second calculation test 2. Overall score per student can be between 0-12, and performance of all students on each question a decimal percent between 0-1, with 0 meaning 0% of students answered correctly and 1 meaning 100% of students answered correctly.	1 month

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Pass rate	The percentage of students achieving a pass (equivalent to 70% or above) during the second test.	1 month
Demographic	The effect of each demographic characteristic on students' attainment in pharmacy calculation when lowering idea density?; First language speaker (English as the first language) or not; Age; Ethnicity; Year of study; Specific learning disability	1 month

Collaborators and Investigators

• Sponsor: Liverpool John Moores University
• Collaborators: University of Central Lancashire; Queen's University, Belfast; Newcastle University; University of Ulster; University of Birmingham; University of Wolverhampton; Aston University; University of Greenwich; University of Huddersfield; Robert Gordon University; Kingston University; University of Bradford; De Montfort University

Study record dates

Study Major Dates

• Study Start (Actual): October 3, 2022
• Primary Completion (Actual): November 6, 2022
• Study Completion (Actual): December 6, 2022

Study Registration Dates

• First Submitted: August 31, 2022
• First Submitted That Met QC Criteria: September 1, 2022
• First Posted (Actual): September 2, 2022

Study Record Updates

• Last Update Posted (Estimated): October 10, 2023
• Last Update Submitted That Met QC Criteria: October 9, 2023
• Last Verified: October 1, 2023

More Information

Terms related to this study

• Keywords: Medical education; Assessment; Pedagogy; Linguistics; Idea density
• Other Study ID Numbers: 22/PBS/004

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: Study protocol to be published Original (anonymised) data to be placed on university repository
• IPD Sharing Time Frame: Within 1 year of study (when published), indefinitely available on university repository
• IPD Sharing Supporting Information Type: STUDY_PROTOCOL

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No