Realistic in Generation of HEp-2 Cell Images Using Latent Diffusion Models: a Multi-center Visual Turing Test

Evaluating the Realism of ANA HEp-2 Cell Images Synthesized Using Latent Diffusion Models: A Multi-center Visual Turing Test

The objective of this prospective observational study is to rigorously examine the feasibility and efficacy of utilizing latent diffusion models for data augmentation in anti-nuclear antibody (ANA) Hep-2 cell immunofluorescence images. The main question it aims to answer is:

Can the application of such models potentially enhance the data quality, increase sample diversity, or improve the accuracy and efficiency of subsequent analytical processes (like disease diagnosis and classification) when utilized with ANA-related images?

Study Overview

• Status: Not yet recruiting
• Conditions: Anti-nuclear Antibody; Visual Turing Tests; Artifical Intelligence
• Intervention / Treatment: Behavioral: referring to the results of AI model output

Detailed Description

A fundamental problem in biomedical research is the low number of observations available, mostly due to a lack of available biosamples, prohibitive costs, or ethical reasons. Augmenting few real observations with generated in silico samples could lead to more robust analysis results and a higher reproducibility rate. Here, The investigators propose to use unsupervised learning with latent diffusion models for the realistic generation of ANA-IIF image data.

The investigators hypothesize that the the generation of ANA-IIF image will be realistic if it is hard to differentiate them (fake) from real (true) . To test this hypothesis, the investigators present a Multi-center Visual Turing tests (https://turing.rednoble.net/) in order to evaluate the quality of the generated (fake) images.

This experimental setup allows the investigators to validate the overall quality of the generated ANA-IIF images, which can then be used to (1) train cytopathologists for educational purposes, and (2) generate realistic samples to train deep networks with big data.

• Study Type: Observational
• Enrollment (Estimated): 300

Contacts and Locations

• Study Contact: Name: Junxiang Zeng, Dr; Phone Number: +8613162232879; Email: zengjunxiang@xinhuamed.com.cn

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

We are recruiting cytopathologists from clinical laboratories in multiple medical institutions worldwide who specialize in interpreting anti-nuclear antibody (ANA) patterns to participate in a visual Turing test.

Description

Inclusion Criteria:

• Originating from reputable medical institutions
• Possessing relevant certification and qualifications
• Having over one year of experience in interpreting anti-nuclear antibody (ANA) patterns within a laboratory setting

Exclusion Criteria:

• Lacking relevant professional certification and qualifications
• Without experience in interpreting ANA patterns
• Unwilling to accept the rules and informed consent of the visual Turing test

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
experts; with over 20 years of experience in ANA-IIF reading	Behavioral: referring to the results of AI model output; determining the ANA pattern type with or without referring to the results of AI model output.
junior cytopathologists; less than 5 years of academic medical experience	Behavioral: referring to the results of AI model output; determining the ANA pattern type with or without referring to the results of AI model output.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
The realistic of images synthesized by diffusion models	The investigators conducted a study using the visual Turing test method, measuring through a questionnaire format, and assessed the measurement results using a 5-point Likert Scale. The 5-point Likert Scale assesses participants' opinions on the quality of images through five response options: Real, Much like, Uncertain, Not quite like, Fake. It calculates scores by assigning numbers (e.g., 5 to 1) to these options, summing up scores for each participant. Results are evaluated by analyzing the distribution of scores, including mean scores, and assessing their reliability and validity. Additionally, the investigator calculated a range of parameters utilized for internal model assessment, including: including precision, recall, F1 score, and mean average precision (mAP).	Baseline

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
The impact of the AI model's output on the participants	The investigator evaluated the change in the accuracy rate of participants' interpretations before and after being assisted by AI model, investigators will conduct a comparative analysis. Additionally, the investigator calculate the Kappa coefficient of agreement between human interpretations and the model, and evaluate whether there are differences in accuracy among cytopathologists with varying levels of experience when assisted by AI.	Baseline
The time taken of ANA pattern interpretation	The investigator compare the time taken of participant to complete interpretations before and after the AI model's intervention, assessing whether there is a reduction in average interpretation time per case, from X minutes pre-AI assistance to Y minutes post-AI.	Baseline

Collaborators and Investigators

• Sponsor: Xinhua Hospital, Shanghai Jiao Tong University School of Medicine
• Investigators: Study Director: Guangyu Chen, PhD, Xinhua Hospital Affiliated to Shanghai Jiaotong University School of Medicine

Publications and helpful links

General Publications

• Zeng J, Gao X, Gao L, Yu Y, Shen L, Pan X. Recognition of rare antinuclear antibody patterns based on a novel attention-based enhancement framework. Brief Bioinform. 2024 Jan 22;25(2):bbad531. doi: 10.1093/bib/bbad531.
• Rahman S, Wang L, Sun C, Zhou L. Deep learning based HEp-2 image classification: A comprehensive review. Med Image Anal. 2020 Oct;65:101764. doi: 10.1016/j.media.2020.101764. Epub 2020 Jul 7.
• Hobson P, Lovell BC, Percannella G, Vento M, Wiliem A. Benchmarking human epithelial type 2 interphase cells classification methods on a very large dataset. Artif Intell Med. 2015 Nov;65(3):239-50. doi: 10.1016/j.artmed.2015.08.001. Epub 2015 Aug 13.
• Niehues JM, Muller-Franzes G, Schirris Y, Wagner SJ, Jendrusch M, Kloor M, Pearson AT, Muti HS, Hewitt KJ, Veldhuizen GP, Zigutyte L, Truhn D, Kather JN. Using histopathology latent diffusion models as privacy-preserving dataset augmenters improves downstream classification performance. Comput Biol Med. 2024 Jun;175:108410. doi: 10.1016/j.compbiomed.2024.108410. Epub 2024 Apr 4.
• Selim M, Zhang J, Brooks MA, Wang G, Chen J. DiffusionCT: Latent Diffusion Model for CT Image Standardization. AMIA Annu Symp Proc. 2024 Jan 11;2023:624-633. eCollection 2023.
• Marouf M, Machart P, Bansal V, Kilian C, Magruder DS, Krebs CF, Bonn S. Realistic in silico generation and augmentation of single-cell RNA-seq data using generative adversarial networks. Nat Commun. 2020 Jan 9;11(1):166. doi: 10.1038/s41467-019-14018-z.

Helpful Links

• Our platform for the Visual Turing Test

Study record dates

Study Major Dates

• Study Start (Estimated): September 1, 2024
• Primary Completion (Estimated): December 1, 2025
• Study Completion (Estimated): June 1, 2026

Study Registration Dates

• First Submitted: July 23, 2024
• First Submitted That Met QC Criteria: August 2, 2024
• First Posted (Actual): August 7, 2024

Study Record Updates

• Last Update Posted (Actual): August 7, 2024
• Last Update Submitted That Met QC Criteria: August 2, 2024
• Last Verified: July 1, 2024

More Information

Terms related to this study

• Keywords: anti-nuclear antibody; latent diffusion models; Visual Turing tests
• Other Study ID Numbers: XH-24-007

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No