이 페이지는 자동 번역되었으며 번역의 정확성을 보장하지 않습니다. 참조하십시오 영문판 원본 텍스트의 경우.

Agreement of ASA Physical Status Classification Among Anesthesia Residents, a Large Language Model, and Senior Anesthesiologist Consensus (ASA-AGREE)

2026년 11월 12일 업데이트: Çiğdem Ünal Kantekin, Kayseri City Hospital

Comparison of ASA Physical Status Classification by Anesthesia Residents, an Artificial Intelligence Large Language Model, and a Two-Anesthesiologist Expert Consensus: A Prospective Three-Rater Validation Study

The ASA Physical Status Classification is widely used in preoperative risk assessment, but agreement between raters is limited, particularly between ASA II and III. This prospective observational study compares the ASA class assigned to the same patients by three types of raters: anesthesia residents working in the preoperative anesthesia clinic, a large language model (LLM), and a consensus of two senior anesthesiologists, which is the reference standard.

Residents complete the standard preoperative evaluation form and assign an ASA class as part of routine care. A copy of the completed form, with patient identifiers and the resident's ASA class removed, is then given independently to the LLM and to the two senior anesthesiologists. No intervention is applied to participants and routine care is not changed.

The primary aim is to measure how closely the residents and the LLM agree with the expert consensus (weighted kappa). Secondary aims are agreement among all three raters, agreement between the two senior anesthesiologists, the direction of misclassification, and how often the LLM matches the consensus when the resident does not.

연구 개요

상태

아직 모집하지 않음

정황

상세 설명

Background The ASA Physical Status Classification is the most widely used tool for describing a patient's preoperative health status. It depends on clinical judgment, and agreement between raters is limited, especially between ASA II and III. In teaching hospitals many preoperative evaluations are done by residents. Large language models can read a structured clinical record and return a classification, and their agreement with anesthesiologists needs to be examined on real preoperative clinic data.

Design and setting Prospective, single-center, observational validation study with three rater groups, conducted in the preoperative anesthesia outpatient clinic of Kayseri City Hospital. Eligible adult patients scheduled for elective surgery are enrolled consecutively after written informed consent.

Procedures The resident takes the history, examines the patient, reviews laboratory results and consultation reports, completes the standard preoperative evaluation form and assigns an ASA class, as in routine care. The completed form is copied digitally. The resident's ASA class is stored in a locked field and is not shown to any other rater until all ratings are complete.

Two further copies are prepared in which patient identifiers (name, archive number, date) and the resident's ASA class, including any ASA checkbox and the free-text assessment, are masked. One copy goes to the LLM and the other to the two senior anesthesiologists.

The LLM is Claude Opus 5.5 (Anthropic), used through the Claude web interface (claude.ai), as a clinician would use it in practice. A dedicated study account is used, with memory and personalization turned off and no custom instructions. Each masked form image is entered in a new chat with the same fixed prompt. The model is queried once per patient, its response is saved verbatim with a screenshot, and the ASA class is entered into the database independently by two investigators. No patient identifiers are sent to the model.

The two senior anesthesiologists (more than 10 years of experience, not involved in the patient's perioperative care) classify each masked form independently. When they disagree, they discuss the case and record a consensus class and the reason. The consensus class is the reference standard. Agreement between the two experts before consensus is reported as a separate result.

Sample size Target enrollment is 330 patients, with 300 analyzable forms expected after allowing for 10% incomplete or excluded forms. The sample size was determined by Monte Carlo simulation for the primary comparison, the difference between two dependent quadratic weighted kappas (LLM vs consensus and resident vs consensus). Assuming a resident-consensus kappa of 0.45 and a clinically relevant difference of 0.15, 300 analyzable patients give 83% power at a two-sided alpha of 0.05.

Statistical analysis Agreement of the resident and of the LLM with the expert consensus is measured with quadratic weighted Cohen's kappa. The primary comparison is the difference between the two kappas, tested with a paired bootstrap 95% confidence interval (2000 resamples of patients). Agreement among all three raters is measured with Fleiss' kappa. Misclassification is described by direction (under- or over-classification) using confusion matrices. The ASA II-III boundary, resident training year and the inter-expert subgroup are pre-specified subgroup analyses. Predictors of resident misclassification are examined with logistic regression. All tests are two-sided at alpha 0.05; secondary comparisons are reported with 95% confidence intervals and treated as exploratory.

연구 유형

관찰

등록 (추정된)

330

연락처 및 위치

이 섹션에서는 연구를 수행하는 사람들의 연락처 정보와 이 연구가 수행되는 장소에 대한 정보를 제공합니다.

연구 연락처

  • 이름: Çiğdem Ünal Kantekin, MD
  • 전화번호: +90 352 315 77 00
  • 이메일: drcgdm@gmail.com

연구 장소

      • Kayseri, 터키 (Türkiye), 38080
        • Kayseri City Hospital, Department of Anesthesiology and Reanimation
        • 연락하다:
          • Çiğdem Ünal Kantekin, MD
          • 전화번호: +90 352 315 77 00
          • 이메일: drcgdm@gmail.com

참여기준

연구원은 적격성 기준이라는 특정 설명에 맞는 사람을 찾습니다. 이러한 기준의 몇 가지 예는 개인의 일반적인 건강 상태 또는 이전 치료입니다.

자격 기준

공부할 수 있는 나이

18년 이상 (성인, 고령자)

건강한 자원 봉사자를 받아들입니다

아니

연구 대상 성별

모두

샘플링 방법

비확률 샘플

연구 인구

Adult patients scheduled for elective surgery who attend the preoperative anesthesia outpatient clinic of Kayseri City Hospital and whose standard preoperative evaluation form is completed by a resident.

설명

Inclusion Criteria:

  • Age 18 years or older
  • Scheduled for elective surgery and evaluated at the preoperative anesthesia clinic
  • Standard preoperative evaluation form completed by a resident, with required laboratory results and consultation reports entered
  • Written informed consent

Exclusion Criteria:

  • Emergency or semi-urgent surgery
  • Incomplete preoperative evaluation form
  • Unable to give informed consent (for example, because of a psychiatric disorder)
  • Refusal to participate

공부 계획

이 섹션에서는 연구 설계 방법과 연구가 측정하는 내용을 포함하여 연구 계획에 대한 세부 정보를 제공합니다.

연구는 어떻게 설계됩니까?

디자인 세부사항

  • 관찰 모델: COHORT
  • 시간 관점: PROSPECTIVE

코호트 및 개입

그룹/코호트
Preoperative anesthesia clinic patients

Adult patients scheduled for elective surgery who are evaluated at the preoperative anesthesia outpatient clinic. The anonymized preoperative form of each patient is rated for ASA class by a resident, an LLM and two senior anesthesiologists.

연구는 무엇을 측정합니까?

주요 결과 측정

결과 측정
측정값 설명
기간
Agreement between resident and expert-consensus ASA classification (quadratic weighted kappa)
기간: Once per patient, from the preoperative clinic evaluation until completion of the blinded ratings (up to 4 weeks)

Quadratic weighted Cohen's kappa between the ASA physical status class (I to V) assigned by the anesthesia resident and the consensus class assigned by two senior anesthesiologists who were blinded to the resident's class. The consensus class is the reference standard. A value of 1 indicates perfect agreement, 0 indicates chance agreement, and negative values indicate agreement below chance.

Once per patient, from the preoperative clinic evaluation until completion of the blinded ratings (up to 4 weeks)
Agreement between large language model and expert-consensus ASA classification (quadratic weighted kappa)
기간: Once per patient, from the preoperative clinic evaluation until completion of the blinded ratings (up to 4 weeks)

Quadratic weighted Cohen's kappa between the ASA class assigned by the large language model from the anonymized preoperative form and the consensus class of two senior anesthesiologists. The model does not see the resident's class or any patient identifier.

Once per patient, from the preoperative clinic evaluation until completion of the blinded ratings (up to 4 weeks)

2차 결과 측정

결과 측정
측정값 설명
기간
Three-rater agreement of ASA classification (Fleiss' kappa)
기간: From the preoperative evaluation to completion of blinded ratings (up to 4 weeks per patient); assessed at the end of data collection (up to 8 months).

Fleiss' kappa across the resident class, the large language model class and the expert consensus class for each patient.

From the preoperative evaluation to completion of blinded ratings (up to 4 weeks per patient); assessed at the end of data collection (up to 8 months).
Agreement between the two senior anesthesiologists before consensus (quadratic weighted kappa)
기간: From the preoperative evaluation to completion of blinded ratings (up to 4 weeks per patient); assessed at the end of data collection (up to 8 months).

Quadratic weighted kappa between the independent ASA classes of senior anesthesiologist A and B, calculated before any discussion between them.

From the preoperative evaluation to completion of blinded ratings (up to 4 weeks per patient); assessed at the end of data collection (up to 8 months).
Agreement between resident and large language model ASA classification (quadratic weighted kappa)
기간: From the preoperative evaluation to completion of blinded ratings (up to 4 weeks per patient); assessed at the end of data collection (up to 8 months).

Quadratic weighted kappa between the ASA class assigned by the resident and the class assigned by the large language model.

From the preoperative evaluation to completion of blinded ratings (up to 4 weeks per patient); assessed at the end of data collection (up to 8 months).
Direction of misclassification relative to expert consensus
기간: From the preoperative evaluation to completion of blinded ratings (up to 4 weeks per patient); assessed at the end of data collection (up to 8 months).

Proportion of patients classified lower (under-classification) or higher (over-classification) than the expert consensus, reported separately for residents and for the large language model. The ASA II-III boundary is analyzed as a pre-specified subgroup.

From the preoperative evaluation to completion of blinded ratings (up to 4 weeks per patient); assessed at the end of data collection (up to 8 months).
Proportion of resident misclassifications in which the large language model matches expert consensus
기간: From the preoperative evaluation to completion of blinded ratings (up to 4 weeks per patient); assessed at the end of data collection (up to 8 months).

Among patients whose resident ASA class differs from the expert consensus, the proportion for whom the large language model class equals the consensus class. This is an exploratory measure of decision-support potential.

From the preoperative evaluation to completion of blinded ratings (up to 4 weeks per patient); assessed at the end of data collection (up to 8 months).
Factors associated with resident misclassification
기간: From the preoperative evaluation to completion of blinded ratings (up to 4 weeks per patient); assessed at the end of data collection (up to 8 months).

Odds ratios from logistic regression of resident misclassification (yes or no) on age, body mass index, number of ASA-relevant comorbidities, resident training year and borderline ASA II-III status.

From the preoperative evaluation to completion of blinded ratings (up to 4 weeks per patient); assessed at the end of data collection (up to 8 months).

공동 작업자 및 조사자

여기에서 이 연구와 관련된 사람과 조직을 찾을 수 있습니다.

스폰서

수사관

  • 수석 연구원: Çiğdem Ünal Kantekin, MD, Kayseri City Hospital

연구 기록 날짜

이 날짜는 ClinicalTrials.gov에 대한 연구 기록 및 요약 결과 제출의 진행 상황을 추적합니다. 연구 기록 및 보고된 결과는 공개 웹사이트에 게시되기 전에 특정 품질 관리 기준을 충족하는지 확인하기 위해 국립 의학 도서관(NLM)에서 검토합니다.

연구 주요 날짜

연구 시작 (추정된)

2027년 1월 1일

기본 완료 (추정된)

2027년 8월 1일

연구 완료 (추정된)

2027년 8월 1일

연구 등록 날짜

최초 제출

2026년 11월 12일

QC 기준을 충족하는 최초 제출

2026년 11월 12일

처음 게시됨 (실제)

2026년 11월 19일

연구 기록 업데이트

마지막 업데이트 게시됨 (실제)

2026년 11월 19일

QC 기준을 충족하는 마지막 업데이트 제출

2026년 11월 12일

마지막으로 확인됨

2026년 11월 1일

추가 정보

이 연구와 관련된 용어

기타 연구 ID 번호

  • ASA-AGREE-2026-01
  • 1208 (기타 식별자: Kayseri City Hospital Non-Interventional Clinical Research Ethics Committee)

개별 참가자 데이터(IPD) 계획

개별 참가자 데이터(IPD)를 공유할 계획입니까?

아니요

IPD 계획 설명

Individual participant data will not be shared outside the study team. The consent form covers coded, confidential use of the data for this study only. Aggregate results will be published.

약물 및 장치 정보, 연구 문서

미국 FDA 규제 의약품 연구

아니

미국 FDA 규제 기기 제품 연구

아니

이 정보는 변경 없이 clinicaltrials.gov 웹사이트에서 직접 가져온 것입니다. 귀하의 연구 세부 정보를 변경, 제거 또는 업데이트하도록 요청하는 경우 register@clinicaltrials.gov. 문의하십시오. 변경 사항이 clinicaltrials.gov에 구현되는 즉시 저희 웹사이트에도 자동으로 업데이트됩니다. .