- ICH GCP
- USA klinikai vizsgálatok nyilvántartása
- Klinikai vizsgálat NCT07810491
Clinical Validation of an Artificial Intelligence System for the Interpretation of Laboratory Test Results Combined With a Structured Medical History (LTC-VALID)
Telemedicine Technology Supporting the Diagnostic Process Based on Automated Analysis of Laboratory Test Results With a Structured Medical History: A Prospective, Two-Center, Two-Phase Clinical Validation Study
This study evaluates a certified artificial intelligence (AI) based software system that automatically interprets laboratory test results in combination with a structured, dynamically generated medical history questionnaire.
The purpose of the study is to determine how accurately and how safely the system assigns a patient to a category of urgency of medical contact, and how closely the interpretations produced by the system correspond to the assessment of an expert physician.
Adults treated at two university hospitals in Katowice, Poland, who are referred for laboratory testing, complete an electronic medical history questionnaire after their laboratory results become available. The system then generates an interpretation for each laboratory result, including a category of urgency of medical contact, a suggested medical specialty and suggested further laboratory tests. The attending physician independently records a clinical assessment of the same laboratory results and medical history without access to the output of the system. An independent expert physician subsequently establishes the reference assessment, blinded to the authorship of the assessments being compared. The interpretation report is released to the participant only after the assessment of the attending physician has been recorded and locked.
The study is conducted in two phases. The first phase (609 participants) is exploratory and uses the initial version of the system. The second phase (290 participants) is confirmatory and uses the final, frozen version of the system. All outcome measures are assessed in both phases; the pre-specified confirmatory hypothesis test applies to the second phase.
The endpoints of this study are properties of the software system, namely the concordance of its output with an expert reference assessment. Health outcomes of participants are not measured.
A tanulmány áttekintése
Állapot
Beavatkozás / kezelés
Részletes leírás
Design. LTC-VALID is a prospective, two-center, single-arm, two-phase clinical validation study of a CE-marked class IIa medical device software intended for the automated interpretation of laboratory test results.
Phase structure. Phase I (609 participants, months 1 to 9) is exploratory and serves algorithm development and gap identification using version 1.0 of the software. An interposed optimization stage (months 9 to 10) produces a frozen version 2.0. Phase II (290 participants, months 11 to 13) is confirmatory. All outcome measures listed below are collected identically in both phases and are reported separately by phase. Results from the two phases are not pooled, because the two phases evaluate different versions of the software. The pre-specified confirmatory hypothesis test for the co-primary measures is applied to Phase II data; Phase I results for the same measures are exploratory and are reported descriptively.
Procedures. Eligible participants provide written informed consent, are referred for a mandatory basic laboratory panel and one or two of 29 specialist laboratory panels, and provide a single blood draw at a certified laboratory collection point. After the results become available, the participant completes a dynamically generated electronic medical history questionnaire. The software produces one interpretation per laboratory result; interpretations are not aggregated by the software. The attending physician records an independent clinical assessment of the same data while blinded to the software output; this assessment is locked before the interpretation report is released to the participant. Participants complete a questionnaire evaluating the report, and physicians complete a form evaluating the completeness and relevance of the automated medical history.
Reference standard and comparators. An independent expert physician receives the complete documentation and establishes an own reference assessment before reviewing the assessments to be compared. The assessments of the software, of the attending physician and of large language models are presented in random order and blinded as to authorship. The comparators are comparators of assessment, not study arms; the study is single-arm and no randomization or control group is used.
Reporting standard. The primary analysis follows the Standards for Reporting of Diagnostic Accuracy Studies (STARD). The study is a diagnostic accuracy study and not a study of clinical effectiveness.
Statistical approach. The two primary outcome measures are co-primary and are evaluated using an intersection-union test; both must meet their pre-specified criteria. The pre-specified criteria apply to Phase II: for the safety measure, the lower bound of the one-sided 95 percent confidence interval is at or above 95 percent; for the accuracy measure, at or above 90 percent. Confidence intervals for proportions are calculated using exact methods (Clopper-Pearson). Phase I is exploratory and its data are reported descriptively. Sensitivity for the rare urgency categories is reported with confidence intervals and event counts as a secondary, non-confirmatory measure, because its denominator is not sufficient for formal hypothesis testing at the planned sample size.
Tanulmány típusa
Beiratkozás (Becsült)
Fázis
- Nem alkalmazható
Kapcsolatok és helyek
Tanulmányi kapcsolat
- Név: Jerzy Chudek, MD, PhD
- Telefonszám: +48322591202
- E-mail: clinicaltrials@labplus.pl
Tanulmányi helyek
-
-
-
Katowice, Lengyelország
- Independent Public Clinical Hospital named after Andrzej Mielecki, Medical University of Silesia
-
Kapcsolatba lépni:
- Jerzy Chudek
- Telefonszám: +48322591202
- E-mail: clinicaltrials@labplus.pl
-
Katowice, Lengyelország
- Prof. K. Gibinski University Clinical Center, Medical University of Silesia
-
Kapcsolatba lépni:
- Michal Holecki, MD, PhD
- Telefonszám: +48322591202
- E-mail: clinicaltrials@labplus.pl
-
-
Részvételi kritériumok
Jogosultsági kritériumok
Tanulmányozható életkorok
- Felnőtt
- Idősebb felnőtt
Egészséges önkénteseket fogad
Leírás
Inclusion Criteria:
- Treated at one of the two participating clinical centers of the Medical University of Silesia
- Age 18 years or older and under 80 years
- Presence of symptoms justifying the initiation of a diagnostic work-up
- Meets the criteria for ordering at least one of the 29 specialist laboratory panels included in the study
- Able to complete an electronic questionnaire in Polish independently, using a smartphone or a personal computer
- Holds a Polish national identification number (PESEL)
- Written informed consent covering all three components of the study
Exclusion Criteria:
- Pregnancy
- Age 80 years or older
- Inability to provide informed consent, including cognitive impairment or a language barrier
- Participation in another clinical study that could affect the results
- Refusal of consent to any of the three components of the study
Tanulási terv
Hogyan készül a tanulmány?
Tervezési részletek
- Elsődleges cél: Diagnosztikai
- Kiosztás: N/A
- Beavatkozó modell: Egyetlen csoportos hozzárendelés
- Maszkolás: Nincs (Open Label)
Fegyverek és beavatkozások
Résztvevő csoport / kar |
Beavatkozás / kezelés |
|---|---|
|
Kísérleti: AI-Based Interpretation of Laboratory Test Results
Single arm.
All participants complete an electronic, dynamically generated medical history questionnaire administered by the software after their laboratory results become available, and subsequently receive a software-generated interpretation report.
The report is released to the participant only after the independent clinical assessment of the attending physician has been recorded and locked.
No control group is used.
The assessments of the attending physician, of an independent expert physician and of large language models are comparators of assessment and do not constitute study arms.
|
A CE-marked class IIa medical device software for the automated interpretation of laboratory test results, operated within its intended purpose.
The software administers a structured medical history questionnaire in which subsequent questions are selected dynamically on the basis of the laboratory results and of previous answers.
For each laboratory result the software generates one interpretation, comprising a category of urgency of medical contact (immediate, urgent, routine, or no need for medical contact), a suggested medical specialty, a list of suggested further laboratory tests, and an explanatory text addressed to the patient.
Interpretations are not aggregated by the software.
The interpretation report is generated once per participant, after completion of the questionnaire, and is released to the participant after the assessment of the attending physician has been recorded and locked.
The generated report is not used to direct the clinical management of the participant.
Más nevek:
|
Mit mér a tanulmány?
Elsődleges eredményintézkedések
Eredménymérő |
Intézkedés leírása |
Időkeret |
|---|---|---|
|
Percentage of Participants for Whom the Urgency Category Assigned by the Software Was Not Lower Than the Category Assigned by the Expert Physician (Patient Triage Safety Indicator)
Időkeret: Through completion of the expert reference assessment for all enrolled participants, up to 16 months after study start
|
The urgency category assigned by the software is compared with the urgency category assigned by an expert physician serving as the reference standard.
The urgency categories, ordered from highest to lowest, are: immediate medical contact, urgent medical contact, routine medical contact, and no need for medical contact.
A participant is counted in the numerator when the category assigned by the software is the same as or higher than the category assigned by the expert physician.
The measure is the percentage of participants counted in the numerator.
Range: 0 to 100 percent; higher values indicate better performance.
The measure is assessed separately in each study phase and data from the two phases are not combined.
Each comparison is based on data locked at the time of enrollment; the expert assessment is performed retrospectively, in batches, on a locked dataset, and the calendar timing of the batch review does not affect the measured quantity.
|
Through completion of the expert reference assessment for all enrolled participants, up to 16 months after study start
|
|
Percentage of Participants for Whom the Urgency Category Assigned by the Software Was Identical to the Category Assigned by the Expert Physician (Patient Triage Accuracy)
Időkeret: Through completion of the expert reference assessment for all enrolled participants, up to 16 months after study start
|
The urgency category assigned by the software is compared with the urgency category assigned by an expert physician serving as the reference standard.
The urgency categories, ordered from highest to lowest, are: immediate medical contact, urgent medical contact, routine medical contact, and no need for medical contact.
A participant is counted in the numerator when the category assigned by the software is identical to the category assigned by the expert physician.
The measure is the percentage of participants counted in the numerator.
Range: 0 to 100 percent; higher values indicate better performance.
The measure is assessed separately in each study phase and data from the two phases are not combined.
Each comparison is based on data locked at the time of enrollment; the expert assessment is performed retrospectively, in batches, on a locked dataset, and the calendar timing of the batch review does not affect the measured quantity.
|
Through completion of the expert reference assessment for all enrolled participants, up to 16 months after study start
|
Másodlagos eredményintézkedések
Eredménymérő |
Intézkedés leírása |
Időkeret |
|---|---|---|
|
Sensitivity for the Immediate Medical Contact Category
Időkeret: Through completion of the expert reference assessment for all enrolled participants, up to 16 months after study start
|
Proportion of participants assigned by the expert physician to the immediate medical contact category who were assigned to the same category by the software.
Range: 0 to 100 percent; higher values indicate better performance.
Reported with a 95 percent confidence interval and the number of events.
No formal hypothesis test is performed, because the expected denominator at the planned sample size is not sufficient for confirmatory testing.
Assessed separately in both study phases.
|
Through completion of the expert reference assessment for all enrolled participants, up to 16 months after study start
|
|
Sensitivity for the Urgent Medical Contact Category
Időkeret: Through completion of the expert reference assessment for all enrolled participants, up to 16 months after study start
|
Proportion of participants assigned by the expert physician to the urgent medical contact category who were assigned to the same category by the software.
Range: 0 to 100 percent; higher values indicate better performance.
Reported with a 95 percent confidence interval and the number of events.
No formal hypothesis test is performed, because the expected denominator at the planned sample size is not sufficient for confirmatory testing.
Assessed separately in both study phases.
|
Through completion of the expert reference assessment for all enrolled participants, up to 16 months after study start
|
|
Agreement on Urgency Category Measured by Quadratic Weighted Cohen's Kappa
Időkeret: Through completion of the expert reference assessment for all enrolled participants, up to 16 months after study start
|
Agreement between the urgency category assigned by the software and by the expert physician, expressed as Cohen's kappa with quadratic weights applied to the four ordered urgency categories.
Range: -1 to 1; higher values indicate better agreement.
Reported with a 95 percent confidence interval.
Supplementary to the co-primary measure of accuracy.
Assessed separately in both study phases.
|
Through completion of the expert reference assessment for all enrolled participants, up to 16 months after study start
|
|
Per-Result Concordance on Recommended Medical Specialty
Időkeret: Through completion of the expert reference assessment for all enrolled participants, up to 16 months after study start
|
Concordance between the medical specialty recommended by the software and by the expert physician, evaluated at the level of a single laboratory result.
Reported as precision and macro-averaged recall.
Range: 0 to 1; higher values indicate better performance.
Assessed separately in both study phases.
|
Through completion of the expert reference assessment for all enrolled participants, up to 16 months after study start
|
|
Per-Result Concordance on Recommended Additional Laboratory Tests
Időkeret: Through completion of the expert reference assessment for all enrolled participants, up to 16 months after study start
|
Concordance between the additional laboratory tests recommended by the software and by the expert physician, evaluated at the level of a single laboratory result.
Reported as precision and macro-averaged recall.
Range: 0 to 1; higher values indicate better performance.
Assessed separately in both study phases.
|
Through completion of the expert reference assessment for all enrolled participants, up to 16 months after study start
|
|
Comparative Concordance of the Software, the Attending Physician and Large Language Models Against the Reference Standard
Időkeret: Months 15 to 18 after study start
|
Concordance of three sources of assessment - the software, the attending physician and large language models - with the same expert reference assessment, using the measures defined above.
Assessments are compared retrospectively on a locked dataset.
Reported descriptively for each source; no ranking test is performed.
|
Months 15 to 18 after study start
|
|
Proportion of Participants Rating the Interpretation Report as Comprehensible
Időkeret: Immediately after release of the report
|
Proportion of participants who, in a study-specific questionnaire completed after release of the interpretation report, indicate that the content of the report was comprehensible.
Range: 0 to 100 percent; higher values indicate better performance.
Assessed separately in both study phases.
|
Immediately after release of the report
|
|
Proportion of Automated Medical History Records Assessed by the Attending Physician as Complete and Relevant
Időkeret: Immediately after the assessment of the attending physician
|
Proportion of automated medical history records for which the attending physician reports no objections regarding completeness or relevance, recorded in a study-specific form.
Range: 0 to 100 percent; higher values indicate better performance.
Assessed separately in both study phases.
|
Immediately after the assessment of the attending physician
|
Együttműködők és nyomozók
Szponzor
Együttműködők
Nyomozók
- Kutatásvezető: Jerzy Chudek, MD, PhD, Medical University of Silesia in Katowice
Publikációk és hasznos linkek
Tanulmányi rekorddátumok
Tanulmány főbb dátumok
Tanulmány kezdete (Becsült)
Elsődleges befejezés (Becsült)
A tanulmány befejezése (Becsült)
Tanulmányi regisztráció dátumai
Először benyújtva
Először nyújtották be, amely megfelel a minőségbiztosítási kritériumoknak
Első közzététel (Tényleges)
Tanulmányi rekordok frissítései
Utolsó frissítés közzétéve (Tényleges)
Az utolsó frissítés elküldve, amely megfelel a minőségbiztosítási kritériumoknak
Utolsó ellenőrzés
Több információ
A tanulmányhoz kapcsolódó kifejezések
Kulcsszavak
Egyéb vizsgálati azonosító számok
- LP004
- FEDS.01.02-IP.01-0078/24 (Egyéb támogatási/finanszírozási szám: European Funds for Lower Silesia 2021-2027 (FEDS))
- BNW/NWN/0052/KB1/34/26 (Egyéb azonosító: Bioethics Committee, Medical University of Silesia in Katowice)
Terv az egyéni résztvevői adatokhoz (IPD)
Tervezi megosztani az egyéni résztvevői adatokat (IPD)?
IPD terv leírása
Gyógyszer- és eszközinformációk, tanulmányi dokumentumok
Egy amerikai FDA által szabályozott gyógyszerkészítményt tanulmányoz
Egy amerikai FDA által szabályozott eszközterméket tanulmányoz
Ezt az információt közvetlenül a clinicaltrials.gov webhelyről szereztük be, változtatás nélkül. Ha bármilyen kérése van vizsgálati adatainak módosítására, eltávolítására vagy frissítésére, kérjük, írjon a következő címre: register@clinicaltrials.gov. Amint a változás bevezetésre kerül a clinicaltrials.gov oldalon, ez a webhelyünkön is automatikusan frissül. .