Defining Decision Thresholds for Judgments on Health Benefits and Harms: Study Protocol
Defining Decision Thresholds for Judgments on Health Benefits and Harms Using the GRADE Evidence to Decision (EtD) Frameworks: Protocol for a Randomized Methodological Study (GRADE-THRESHOLD)
Study Overview
Status
Status
Conditions
Conditions
Intervention / Treatment
Intervention / Treatment
Detailed Description
The objective of this study is to derive decision-thresholds (DTs) for Evidence to Decision (EtD) judgments on the magnitude of health benefits and harms. The study hypothesis is that DTs could discriminate between the four categories for EtD judgments. Explicit DTs, providing an indication for which could be the appropriate judgment for a given scenario, might have the potential to support panels of decision-makers in their work, facilitate a common understanding, and promote consistency and transparency in judgments.
Study investigators will conduct a methodological randomized controlled trial to collect the data that allow deriving the decision-thresholds. They will invite clinicians, epidemiologists, decision scientists, health research methodologists, experts in Health Technology Assessment (HTA), members of guideline development groups and the public to participate in the trial. Then, investigators will investigate the validity of decision-threshold by measuring the agreement between judgments that were made in the past by guideline panels and the judgments that the DTs approach would suggest if applied on the same guideline data.
Study Type
Study Type
Enrollment (Anticipated)
Enrollment
Phase
Phase
- Not Applicable
Contacts and Locations
Study Contact
Study Contact
- Name: Holger J Schunemann, MD, PhD
- Phone Number: 24699 +19055259140
- Email: schuneh@mcmaster.ca
Study Contact Backup
- Name: Gian Paolo Morgano, PhD
- Email: morganog@mcmaster.ca
Study Locations
-
-
Ontario
-
Hamilton, Ontario, Canada, L8S 4K1
- Recruiting
- McMaster University
-
Contact:
- Holger J Schunemann, MD, PhD
- Phone Number: 24699 +19055259140
- Email: schuneh@mcmaster.ca
-
Contact:
- Gian Paolo Morgano, PhD
- Phone Number: +393498125719
- Email: morganog@mcmaster.ca
-
Principal Investigator:
- Holger J Schunemann, MD, PhD
-
Sub-Investigator:
- Gian Paolo Morgano, PhD
-
Sub-Investigator:
- Wojtek Wiercioch, PhD
-
-
Participation Criteria
Eligibility Criteria
Eligibility Criteria
Ages Eligible for Study
Accepts Healthy Volunteers
Genders Eligible for Study
Description
Inclusion Criteria:
The target population of the survey will include:
- clinicians
- epidemiologist
- decision scientists
- health research methodologists
- experts in health technology assessment (HTA)
- and members of guideline working groups
- members from the general public
Exclusion Criteria:
• Prior participation in the survey
Study Plan
How is the study designed?
Design Details
- Primary Purpose: Other
- Allocation: Randomized
- Interventional Model: Single Group Assignment
- Masking: Triple
Number of Arms
Arms and Interventions
Participant Group / ArmParticipant Group / Arm |
Intervention / TreatmentIntervention / Treatment |
|---|---|
|
Experimental: Threshold estimates
Descriptive case-scenarios
|
Each case-scenario will include: (1) a GRADE Summary of Finding (SoF) table providing information about the Population, Intervention, Comparator, Outcome (PICO), the relative and absolute anticipated effects of the intervention, and the certainty in the evidence; (2) a Health Outcome Descriptor describing key attributes of the outcome under consideration including symptoms, time horizon, testing and treatment, and consequences; a measure of the impact on health of the outcome (also known as 'value' of the outcome or 'health utility' in health economics).
|
|
Active Comparator: Alternative threshold estimates
Descriptive case-scenarios
|
Each case-scenario will include: (1) a GRADE Summary of Finding (SoF) table providing information about the Population, Intervention, Comparator, Outcome (PICO), the relative and absolute anticipated effects of the intervention, and the certainty in the evidence; (2) a Health Outcome Descriptor describing key attributes of the outcome under consideration including symptoms, time horizon, testing and treatment, and consequences; a measure of the impact on health of the outcome (also known as 'value' of the outcome or 'health utility' in health economics).
|
What is the study measuring?
Primary Outcome Measures
Primary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Small effect of an intervention
Time Frame: immediate after completion of study
|
Questionnaire: Participants will be asked to indicate which decision threshold for discriminating between EtD judgments of 'Trivial or None' and 'Small' they felt appropriate at the time of participation in the survey.
|
immediate after completion of study
|
|
Moderate effect of an intervention
Time Frame: immediate after completion of study
|
Questionnaire:Participants will be asked to indicate which decision threshold for discriminating between EtD judgments of 'Small' and 'Moderate' they felt appropriate at the time of participation in the survey.
|
immediate after completion of study
|
|
Large effect of an intervention
Time Frame: immediate after completion of study
|
Questionnaire:Participants will be asked to indicate which decision threshold for discriminating between EtD judgments of 'Moderate' and Large' they felt appropriate at the time of participation in the survey.
|
immediate after completion of study
|
Collaborators and Investigators
Sponsor
Sponsor
Publications and helpful links
General Publications
- Chan AW, Tetzlaff JM, Gotzsche PC, Altman DG, Mann H, Berlin JA, Dickersin K, Hrobjartsson A, Schulz KF, Parulekar WR, Krleza-Jeric K, Laupacis A, Moher D. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013 Jan 8;346:e7586. doi: 10.1136/bmj.e7586.
- Abramson JH. WINPEPI updated: computer programs for epidemiologists, and their teaching potential. Epidemiol Perspect Innov. 2011 Feb 2;8(1):1. doi: 10.1186/1742-5573-8-1.
- Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines; Graham R, Mancher M, Miller Wolman D, Greenfield S, Steinberg E, editors. Clinical Practice Guidelines We Can Trust. Washington (DC): National Academies Press (US); 2011. Available from http://www.ncbi.nlm.nih.gov/books/NBK209539/
- Schunemann HJ. Guidelines 2.0: do no net harm-the future of practice guideline development in asthma and other diseases. Curr Allergy Asthma Rep. 2011 Jun;11(3):261-8. doi: 10.1007/s11882-011-0185-8.
- Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, Liberati A, O'Connell D, Oxman AD, Phillips B, Schunemann H, Edejer TT, Vist GE, Williams JW Jr; GRADE Working Group. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res. 2004 Dec 22;4(1):38. doi: 10.1186/1472-6963-4-38.
- Alonso-Coello P, Schunemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, Treweek S, Mustafa RA, Rada G, Rosenbaum S, Morelli A, Guyatt GH, Oxman AD; GRADE Working Group. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016 Jun 28;353:i2016. doi: 10.1136/bmj.i2016. No abstract available.
- Alonso-Coello P, Oxman AD, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, Treweek S, Mustafa RA, Vandvik PO, Meerpohl J, Guyatt GH, Schunemann HJ; GRADE Working Group. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ. 2016 Jun 30;353:i2089. doi: 10.1136/bmj.i2089. No abstract available.
- Marcucci M, Sinclair JC. A generalised model for individualising a treatment recommendation based on group-level evidence from randomised clinical trials. BMJ Open. 2013 Aug 13;3(8):e003143. doi: 10.1136/bmjopen-2013-003143.
- McCabe C, Claxton K, Culyer AJ. The NICE cost-effectiveness threshold: what it is and what that means. Pharmacoeconomics. 2008;26(9):733-44. doi: 10.2165/00019053-200826090-00004.
- Pauker SG, Kassirer JP. Therapeutic decision making: a cost-benefit analysis. N Engl J Med. 1975 Jul 31;293(5):229-34. doi: 10.1056/NEJM197507312930505.
- Hultcrantz M, Mustafa RA, Leeflang MMG, Lavergne V, Estrada-Orozco K, Ansari MT, Izcovich A, Singh J, Chong LY, Rutjes A, Steingart K, Stein A, Sekercioglu N, Arevalo-Rodriguez I, Morgan RL, Guyatt G, Bossuyt P, Langendam MW, Schunemann HJ. Defining ranges for certainty ratings of diagnostic accuracy: a GRADE concept paper. J Clin Epidemiol. 2020 Jan;117:138-148. doi: 10.1016/j.jclinepi.2019.05.002. Epub 2019 May 18.
- Hultcrantz M, Rind D, Akl EA, Treweek S, Mustafa RA, Iorio A, Alper BS, Meerpohl JJ, Murad MH, Ansari MT, Katikireddi SV, Ostlund P, Tranaeus S, Christensen R, Gartlehner G, Brozek J, Izcovich A, Schunemann H, Guyatt G. The GRADE Working Group clarifies the construct of certainty of evidence. J Clin Epidemiol. 2017 Jul;87:4-13. doi: 10.1016/j.jclinepi.2017.05.006. Epub 2017 May 18.
- Faraone SV. Interpreting estimates of treatment effects: implications for managed care. P T. 2008 Dec;33(12):700-11. No abstract available.
- Johnston BC, Ebrahim S, Carrasco-Labra A, Furukawa TA, Patrick DL, Crawford MW, Hemmelgarn BR, Schunemann HJ, Guyatt GH, Nesrallah G. Minimally important difference estimates and methods: a protocol. BMJ Open. 2015 Oct 1;5(10):e007953. doi: 10.1136/bmjopen-2015-007953.
- Guyatt GH, Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, Johnston BC, Karanicolas P, Akl EA, Vist G, Kunz R, Brozek J, Kupper LL, Martin SL, Meerpohl JJ, Alonso-Coello P, Christensen R, Schunemann HJ. GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles-continuous outcomes. J Clin Epidemiol. 2013 Feb;66(2):173-83. doi: 10.1016/j.jclinepi.2012.08.001. Epub 2012 Oct 30. Erratum In: J Clin Epidemiol. 2015 Apr;68(4):475.
- Baldeh T, Saz-Parkinson Z, Muti P, Santesso N, Morgano GP, Wiercioch W, Nieuwlaat R, Grawingholt A, Broeders M, Duffy S, Hofvind S, Nystrom L, Ioannidou-Mouzaka L, Warman S, McGarrigle H, Knox S, Fitzpatrick P, Rossi PG, Quinn C, Borisch B, Lebeau A, de Wolf C, Langendam M, Piggott T, Giordano L, van Landsveld-Verhoeven C, Bernier J, Rabe P, Schunemann HJ. Development and use of health outcome descriptors: a guideline development case study. Health Qual Life Outcomes. 2020 Jun 5;18(1):167. doi: 10.1186/s12955-020-01338-8.
- Alper BS, Oettgen P, Kunnamo I, Iorio A, Ansari MT, Murad MH, Meerpohl JJ, Qaseem A, Hultcrantz M, Schunemann HJ, Guyatt G; GRADE Working Group. Defining certainty of net benefit: a GRADE concept paper. BMJ Open. 2019 Jun 4;9(6):e027445. doi: 10.1136/bmjopen-2018-027445.
- Puhan MA, Singh S, Weiss CO, Varadhan R, Boyd CM. A framework for organizing and selecting quantitative approaches for benefit-harm assessment. BMC Med Res Methodol. 2012 Nov 19;12:173. doi: 10.1186/1471-2288-12-173.
- Weinstein MC, Stason WB. Foundations of cost-effectiveness analysis for health and medical practices. N Engl J Med. 1977 Mar 31;296(13):716-21. doi: 10.1056/NEJM197703312961304.
- Morgano GP, Mbuagbaw L, Santesso N, Xie F, Brozek JL, Siebert U, Bognanni A, Wiercioch W, Piggott T, Darzi AJ, Akl EA, Verstijnen IM, Parmelli E, Saz-Parkinson Z, Alonso-Coello P, Schünemann HJ. Defining decision thresholds for judgments on health benefits and harms using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) frameworks: a protocol for a randomised methodological study (GRADE-THRESHOLD). BMJ Open. 2022 Mar 10;12(3):e053246. doi: 10.1136/bmjopen-2021-053246.
Study record dates
Study Major Dates
Study Start (Actual)
Study Start
Primary Completion (Anticipated)
Primary Completion
Study Completion (Anticipated)
Study Completion
Study Registration Dates
First Submitted
First Submitted
First Submitted That Met QC Criteria
First Submitted That Met QC Criteria
First Posted (Actual)
First Posted
Study Record Updates
Last Update Posted (Actual)
Last Update Posted
Last Update Submitted That Met QC Criteria
Last Update Submitted That Met QC Criteria
Last Verified
Last Verified
More Information
Terms related to this study
Other Study ID Numbers
Other Study ID Numbers
- GRADETHRESHLD
Plan for Individual participant data (IPD)
Plan to Share Individual Participant Data (IPD)?
Drug and device information, study documents
Studies a U.S. FDA-regulated drug product
Studies a U.S. FDA-regulated device product
This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.
Clinical Trials on Decision Support Techniques
-
NCT04380428CompletedMedical Education | Decision Support Techniques
-
NCT00325923CompletedDecision Making | Decision Support Techniques | Decision Aids | Choice Behavior
-
NCT01190488CompletedPalliative Care | Decision Support Systems, Clinical | Decision Making | Decision Support Techniques | Hospices
-
NCT04103151CompletedTriage | Heart Rate | Decision Support Techniques | Variability
-
NCT05463887CompletedQuality of Life | Healthy Aging | Primary Prevention | Decision Support Techniques
-
NCT07019688CompletedAnxiety | Health Communication | Decision Support Techniques | Information Dissemination | Artifical Intelligence
-
NCT03327103UnknownDecision Support Techniques
-
NCT03629509UnknownBreast Neoplasm Female | Young Adult | Decision Support Techniques | Evaluation Studies
-
NCT07623304Recruiting
-
NCT05670613Completed
Clinical Trials on descriptive case-scenarios
-
NCT06369740Completed
-
NCT07319819Not yet recruiting
-
NCT02754310Completed
-
NCT05841602CompletedHospitalism | Children, Only | COVID-19 Pandemic | Nurse's Role | Feelings
-
NCT05372341CompletedLong-term Consequences of COVID-19 Infection
-
NCT04221841CompletedExecutive Dysfunction | Obsessive-Compulsive Symptom
-
NCT03814486CompletedPalliative Care | Platelet Transfusion
-
NCT05778019Recruiting
-
NCT01657279CompletedStroke | Ischemic Stroke | Cerebrovascular Disease