Large Language Models for Stuttering Assessment and Therapy
Alignment of Large Language Model Responses on Stuttering Definition, Assessment, and Therapy With Clinical Practice Guidelines: A Multi-Model Comparison
This observational study aims to compare the quality of responses generated by four large language models (ChatGPT, Claude, Google Gemini, and Grok-4.20) to questions about stuttering. The study focuses on three main areas: general information about stuttering, clinical assessment, and therapy. A total of nine questions were developed based on evidence-based clinical practice guidance, including the American Speech-Language-Hearing Association (ASHA) Practice Portal. Each question is presented to each language model in separate sessions, and the generated responses are recorded for evaluation.
Five speech-language therapists with clinical experience in stuttering independently evaluate the model-generated responses. Each response is rated for relevance, accuracy, clarity, completeness, and consistency using a 5-point Likert scale. The responses are also compared with guideline-based reference information. The study does not involve any clinical intervention or treatment of patients. The aim is to determine how closely large language model responses align with current clinical practice guidance and to identify their potential strengths and limitations when used as informational or clinical support tools in the field of stuttering.
調査の概要
状態
条件
研究の種類
入学 (実際)
連絡先と場所
研究場所
-
-
Istanbul
-
Istanbul、Istanbul、トルコ（Türkiye）、34310
- Istanbul Gelisim University
-
-
参加基準
適格基準
就学可能な年齢
健康ボランティアの受け入れ
受講資格のある性別
サンプリング方法
調査対象母集団
The study population consisted of adult speech-language therapists in Türkiye who were actively working in the field of stuttering and fluency disorders. Five speech-language therapists with clinical experience in stuttering were purposively selected to independently evaluate responses generated by four large language models to standardized questions concerning stuttering information, assessment, and therapy.
説明
Inclusion Criteria:
- Having an undergraduate or postgraduate degree in speech and language therapy.
- Actively practicing clinically in the field of stuttering/fluency disorders.
- Voluntarily agreeing to participate in the study.
- Completing the evaluation form in full.
Exclusion Criteria:
- Not having active clinical experience in the field of stuttering.
- Incomplete completion of the evaluation form.
- Withdrawal of voluntary participation during the evaluation process.
研究計画
研究はどのように設計されていますか？
デザインの詳細
- 観測モデル：OTHER
- 時間の展望：CROSS_SECTIONAL
コホートと介入
グループ/コホート |
介入・治療 |
|---|---|
|
Expert Speech-Language Therapists
Five speech-language therapists with clinical experience in stuttering independently evaluated responses generated by four large language models (ChatGPT, Claude, Google Gemini, and Grok-4.20). The responses addressed nine questions covering general information about stuttering, clinical assessment, and therapy. Each response was rated for relevance, accuracy, clarity, completeness, and consistency using a 5-point Likert scale. |
ChatGPT responses to nine standardized questions on stuttering information, assessment, and therapy were generated in separate sessions and recorded for independent expert evaluation. Questions were repeated under comparable conditions to allow assessment of response consistency. Claude responses to nine standardized questions on stuttering information, assessment, and therapy were generated in separate sessions and recorded for independent expert evaluation. Questions were repeated under comparable conditions to allow assessment of response consistency. Google Gemini responses to nine standardized questions on stuttering information, assessment, and therapy were generated in separate sessions and recorded for independent expert evaluation. Questions were repeated under comparable conditions to allow assessment of response consistency. Grok-4.20 responses to nine standardized questions on stuttering information, assessment, and therapy were generated in separate sessions and recorded for independent expert evaluation. Questions were repeated under comparable conditions to allow assessment of response consistency. |
この研究は何を測定していますか？
主要な結果の測定
結果測定 |
メジャーの説明 |
時間枠 |
|---|---|---|
|
Expert-Rated Quality of Large Language Model Responses
時間枠：During the single cross-sectional expert evaluation conducted over approximately 1 month
|
Responses generated by four large language models (ChatGPT, Claude, Google Gemini, and Grok-4.20) to nine standardized questions about stuttering were independently evaluated by five speech-language therapists with clinical experience in stuttering. Each response was rated using a 5-point Likert scale (1 = strongly disagree to 5 = strongly agree) across five criteria: relevance, accuracy, clarity, completeness, and consistency. Responses were evaluated against guideline-based reference information derived from evidence-based clinical practice guidance. Higher scores indicate better response quality and closer alignment with clinical practice guidance. Mean scores were calculated for each large language model and each evaluation criterion. |
During the single cross-sectional expert evaluation conducted over approximately 1 month
|
協力者と研究者
研究記録日
主要日程の研究
研究開始 (実際)
一次修了 (実際)
研究の完了 (実際)
試験登録日
最初に提出
QC基準を満たした最初の提出物
最初の投稿 (実際)
学習記録の更新
投稿された最後の更新 (実際)
QC基準を満たした最後の更新が送信されました
最終確認日
詳しくは
本研究に関する用語
追加の関連 MeSH 用語
その他の研究ID番号
- LLM-STUTTER-2026
医薬品およびデバイス情報、研究文書
米国FDA規制医薬品の研究
米国FDA規制機器製品の研究
この情報は、Web サイト clinicaltrials.gov から変更なしで直接取得したものです。研究の詳細を変更、削除、または更新するリクエストがある場合は、register@clinicaltrials.gov。 までご連絡ください。 clinicaltrials.gov に変更が加えられるとすぐに、ウェブサイトでも自動的に更新されます。