Consensus And Norms for Medical Visualization Academic Standards Study - A Modified Delphi Study (CANVAS)

The CANVAS (Consensus And Norms for Medical Visualization Academic Standards) Study: Developing Professional Standards for Effective Medical Visualization - A Modified Delphi Study

The goal of this observational study (modified Delphi consensus study) is to establish an international expert consensus on professional standards for medical visualization in medical visualization expertsfrom North America, Europe, and South Korea, each with a minimum of five years of relevant professional experience. The specific objectives are:

1. to develop and validate core consensus statements on professional standards for medical visualization (consensus threshold ≥75%).
2. to quantify the distribution of divergence in professional opinion.

Participants will:

Fill out a short background survey about their job title, training, work setting, location, and experience with AI image tools.

Rate 10 statements about medical visualization standards on a scale from 1 (Strongly Disagree) to 5 (Strongly Agree). These statements cover three topics: accuracy, accessibility, and ethics. Participants will complete up to 3 rounds of anonymous online surveys.

Write comments to explain why they disagree with any statement they rate 1 or 2.

Between rounds, review a summary showing their own earlier ratings and the group's overall results (with no names attached). Then, re-rate any statements the group has not yet agreed on.

Study Overview

• Status: Not yet recruiting
• Conditions: Medical Visualization; Medical Illustration
• Intervention / Treatment: Other: Delphi procedure

Detailed Description

This study uses a modified Delphi design to develop consensus-based professional standards for medical visualization. The standard Delphi method typically begins with open-ended questions to generate initial statements. In this modified version, a systematic literature review and a focus group replace that first open-ended round, reducing participant burden and producing a more structured starting questionnaire. The study follows the Conducting and Reporting of Delphi Studies (CREDES) framework and reports results according to the Accurate Consensus Reporting Document (ACCORD) checklist. All activities are conducted online. There is no physical study site.

Questionnaire Development The initial questionnaire was developed in two stages. In the first stage, a systematic literature review identified existing frameworks, guidelines, and evidence related to professional standards in medical visualization. Three databases were searched: PubMed (title and abstract fields), Embase (title field), and the Cochrane Library (title field). The search terms were "medical illustration" OR "medical visualization," limited to clinical studies and review articles published between May 2016 and May 2026. The research team also hand-searched professional guidelines from the Association of Medical Illustrators (AMI) and the Institute of Medical Illustrators (IMI), as well as all issues of The Journal of Biocommunication (JBC). A total of 20 articles were included for synthesis.

In the second stage, the principal investigator convened a focus group of domain experts, including clinicians and three practicing medical illustrators. The group held three online meetings to discuss the literature review findings and draft the initial statements. To protect the independence of the Delphi process, no focus group member serves on the Delphi expert panel. This process produced a 10-item questionnaire organized into three domains: (i) Accuracy and Scientific Rigor (4 items), (ii) Accessibility and Diversity (3 items), and (iii) Disclosure and Ethics (3 items).

Content Validity Assessment Before distribution to the Delphi panel, 10 cross-disciplinary experts rated the relevance of each item on a 4-point scale (1 = not relevant; 4 = highly relevant). Items with an item-level Content Validity Index (I-CVI) of 0.83 or higher were retained. Items scoring between 0.70 and 0.83 were revised. Items below 0.70 were deleted. None of these 10 validators serve on the Delphi expert panel.

Delphi Round Procedures The study includes up to three sequential rounds of anonymous rating, all conducted via the REDCap platform deployed by the Information Technology Office of National Taiwan University Hospital.

In Round 1, all panel members rate the 10 statements on a 5-point Likert scale (1 = Strongly Disagree to 5 = Strongly Agree). For any item rated 1 or 2, respondents provide an open-ended comment explaining their reasoning. Between rounds, the research team compiles descriptive statistics for each item. Each panel member receives (a) their own prior ratings, (b) anonymized group statistics (mean, median, interquartile range, and rating distribution), and (c) a summary of anonymized qualitative comments. Two research team members independently review qualitative comments using content analysis and draft revision recommendations. Items reaching consensus are finalized and removed from subsequent rounds. In Rounds 2 and 3, panel members re-rate only those items that have not yet reached consensus, in light of group feedback. No real-time interaction or group discussion takes place at any stage.

Consensus Definition An item reaches inclusion consensus when all three of the following criteria are met simultaneously: (1) 75% or more of panel members rate the item 4 or 5; (2) the group median is 4 or higher; and (3) the interquartile range (IQR) is 1 or less. An item reaches exclusion consensus if 75% or more of panel members rate it 1 or 2. Items that do not meet either threshold after three rounds are classified as non-consensus and reported descriptively.

Stopping Criteria The consultation process ends after a maximum of three rounds, regardless of item-level consensus status. If all items reach consensus before the third round, the process ends early. If the number of continuously participating members falls below 15 in any round, that round is considered unable to guarantee representativeness, and the Delphi process is terminated early.

Statistical Analysis Plan Descriptive statistics summarize expert ratings for each item, reported as mean ± standard deviation (SD), supplemented by median and IQR. All tests are two-tailed, with p < 0.05 considered statistically significant. Inter-rater agreement is assessed using Kendall's coefficient of concordance (W), interpreted as weak (W < 0.3), moderate (0.3 ≤ W ≤ 0.7), or strong (W > 0.7). All analyses are performed in IBM SPSS Statistics; graphical visualizations are produced in GraphPad Prism.

• Study Type: Observational
• Enrollment (Estimated): 50

Contacts and Locations

• Study Contact: Name: Monal, Yu-Hsuan Chang, M.D.; Phone Number: 886-935891029; Email: monal@ntuh.gov.tw

Study Locations

• Taiwan
  • Taipei
    • Taipei, Taipei, Taiwan, 100
      • National Taiwan University Hospital
      • Contact: Monal, Yu-Hsuan Chang, M.D.; Phone Number: 251742 886-2-2312-3456; Email: monal@ntuh.gov.tw
      • Principal Investigator: CHENG-JUNG LI
      • Principal Investigator: Monal, Yu-Hsuan Chang, M.D.

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

The target expert panel size for this study was set at 35 to 50 members. The Delphi method is a structured consensus-building process rather than a hypothesis-testing framework, and no formal statistical power calculation method applicable to Delphi studies currently exists in the literature.

The recruitment channel involved the principal investigator compiling a roster of domain experts based on program directors or departmental contacts at the eight accredited training institutions, with the principal investigator issuing personalized invitations directly. The invitation process followed a standardized email procedure in three stages: (1) Initial invitation: the principal investigator sent a personalized email explaining the study objectives, expected time commitment (approximately 15 minutes per round), the anonymity mechanism, and data protection measures; (2) Follow-up reminder: if the invitee did not respond within 14 days, a first reminder was sent; if there was still no respon

Description

Experts were invited from medical illustration training programs or professional associations in North America, Europe, and South Korea. Inclusion criteria required that invitees meet at least one of the following conditions:

Inclusion Criteria:

• Current faculty or staff at one of the accredited degree-granting institutions in medical and scientific illustration
• Current member of the Korean Association of Medical Visualization Artists (KAMVA);
• Holder of the Certified Medical Illustrator (CMI) credential issued by the Board of Certification of Medical Illustrators
• Current visualization editor or supervisor at a high-impact medical journal.
• Required to possess at least five years of professional experience in medical visualization or a related discipline
• Willing to complete the Delphi questionnaire, having provided informed consent prior to questionnaire commencement.

Exclusion Criteria:

• Prior participation in the focus group or validity assessment stages of this study's initial questionnaire development
• Direct financial interest in commercial AI medical imaging products or related services
• Inability to complete the questionnaire in the designated language (English) in the absence of a corresponding language version.

Individuals meeting any of the above criteria were excluded to preserve the independence of the consensus process.

Study Plan

How is the study designed?

Number of groups / cohorts

1

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Delphi Panel; Experts were invited from medical illustration training programs or professional associations in North America, Europe, and South Korea.	Other: Delphi procedure; The study involves generating consensus statements by experts on medical visualization from North America, Europe, and South Korea on professional standards for medical illustration and visualization - covering accuracy and scientific rigor, accessibility and diversity, and disclosure and ethics - using a modified Delphi process.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Proportion of Statements Reaching Expert Consensus	The proportion of the 10 initial professional standard statements (across three domains: Accuracy & Scientific Rigor; Accessibility & Diversity; Disclosure & Ethics) that reach the predefined consensus threshold, defined as the simultaneous fulfillment of three criteria: (1) ≥75% of panel members assigning a rating of 4 or 5 on a 5-point Likert scale (1 = Strongly Disagree to 5 = Strongly Agree); (2) group median ≥4; and (3) interquartile range (IQR) ≤1. Items meeting all three criteria are classified as "inclusion consensus." Items rated 1 or 2 by ≥75% of panel members are classified as "exclusion consensus." Items meeting neither threshold after three rounds are classified as "non-consensus."	Through study completion, approximately 12 months
Agreement Rate for Each Consensus Statement	The percentage of expert panel members rating each of the 10 professional standard statements as 4 (Agree) or 5 (Strongly Agree) on a 5-point Likert scale, reported per round. This is the principal component of the tripartite consensus definition.	Through study completion, approximately 12 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Inter-Rater Reliability	Kendall's coefficient of concordance (W) measuring the degree of agreement among expert panel members across all rated items. Interpretive thresholds: weak (W < 0.3), moderate (0.3 ≤ W ≤ 0.7), strong (W > 0.7).	Through study completion, approximately 12 months
Demographic and Professional Composition of the Expert Panel	Descriptive summary of the expert panel composition, including distribution of primary professional discipline (e.g., surgical illustration, anatomical/educational illustration, molecular/cellular visualization, 3D modeling, animation/interactive media, patient education, science communication), geographic region (North America, Europe, East Asia), highest educational degree, years of professional experience, and experience with generative AI imaging tools.	Through study completion, approximately 12 months
Number of Delphi Rounds Required to Reach Consensus per Statement	The number of Delphi rounds (out of a maximum of 3) required for each individual statement to meet the predefined consensus threshold. Statements not reaching consensus after 3 rounds are classified as "non-consensus" and reported descriptively.	Through study completion, approximately 12 months
Qualitative Themes of Expert Disagreement	Thematic categories identified through content analysis of open-ended written comments provided by panel members.	Through study completion, approximately 12 months

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Panel Retention Rate Across Delphi Rounds	The number and percentage of expert panel members who complete each sequential Delphi round, relative to the initial enrolled panel. If continuously participating members fall below 15 in any round, the round is considered unable to guarantee representativeness and the Delphi process is terminated early per the predefined stopping criteria.	Through study completion, approximately 12 months

Collaborators and Investigators

• Sponsor: National Taiwan University Hospital
• Collaborators: Johns Hopkins University
• Investigators: Principal Investigator: Monal, Yu-Hsuan Chang, M.D., National Taiwan University Hospital

Publications and helpful links

General Publications

• Junger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliat Med. 2017 Sep;31(8):684-706. doi: 10.1177/0269216317690685. Epub 2017 Feb 13.
• Polit DF, Beck CT. The content validity index: are you sure you know what's being reported? Critique and recommendations. Res Nurs Health. 2006 Oct;29(5):489-97. doi: 10.1002/nur.20147.
• Gattrell WT, Logullo P, van Zuuren EJ, Price A, Hughes EL, Blazey P, Winchester CC, Tovey D, Goldman K, Hungin AP, Harrison N. ACCORD (ACcurate COnsensus Reporting Document): A reporting guideline for consensus methods in biomedicine developed via a modified Delphi. PLoS Med. 2024 Jan 23;21(1):e1004326. doi: 10.1371/journal.pmed.1004326. eCollection 2024 Jan.
• Alon L, Shoval DH, Levkovich I. Bias, representation, and clinical fidelity in AI-generated images for medical education: a systematic literature review. NPJ Digit Med. 2026 Apr 18;9(1):469. doi: 10.1038/s41746-026-02608-3.
• Manyara AM, Purvis A, Ciani O, Collins GS, Taylor RS. Sample size in multistakeholder Delphi surveys: at what minimum sample size do replicability of results stabilize? J Clin Epidemiol. 2024 Oct;174:111485. doi: 10.1016/j.jclinepi.2024.111485. Epub 2024 Jul 26.
• Young JQ, Van Merrienboer J, Durning S, Ten Cate O. Cognitive Load Theory: implications for medical education: AMEE Guide No. 86. Med Teach. 2014 May;36(5):371-84. doi: 10.3109/0142159X.2014.889290. Epub 2014 Mar 4.
• Calkins CM, Franciosi JP, Kolesari GL. Human anatomical science and illustration: the origin of two inseparable disciplines. Clin Anat. 1999;12(2):120-9. doi: 10.1002/(SICI)1098-2353(1999)12:23.0.CO;2-V.
• Hajar R. Medical illustration: art in medical education. Heart Views. 2011 Apr;12(2):83-91. doi: 10.4103/1995-705X.86023. No abstract available.

Helpful Links

• 23. Accreditation Review Committee for the Medical Illustrator (ARC-MI). Standards and Guidelines for the Accreditation of Educational Programs in Medical Illustration. Chicago, IL: CAAHEP; 2014.

Study record dates

Study Major Dates

• Study Start (Estimated): July 1, 2026
• Primary Completion (Estimated): April 30, 2027
• Study Completion (Estimated): December 31, 2027

Study Registration Dates

• First Submitted: June 29, 2026
• First Submitted That Met QC Criteria: June 29, 2026
• First Posted (Actual): July 6, 2026

Study Record Updates

• Last Update Posted (Actual): July 6, 2026
• Last Update Submitted That Met QC Criteria: June 29, 2026
• Last Verified: June 1, 2026

More Information

Terms related to this study

• Keywords: Expert consensus; Medical Visualization; Medical illustration; Modified Delphi method
• Other Study ID Numbers: 202606124RIND

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: This study employs the Delphi consensus methodology, in which data are derived from expert panelists' professional opinions rather than from patient encounters, medical records, biological specimens, or identifiable clinical information. As such, the study does not generate individual participant data (IPD) in the conventional sense. Furthermore, respondent anonymity is a fundamental methodological requirement of the Delphi technique, as it minimizes the influence of authority bias and preserves the independence of each panelist's judgment across iterative survey rounds. Disclosing identifiable expert-level responses would compromise this core methodological principle. Aggregated, de-identified consensus results will be reported in peer-reviewed publications.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No