Evaluation of Large Language Models in Parasternal Block Ultrasound Interpretation

Study Design and Participants This prospective observational study will evaluate the diagnostic accuracy of multimodal large language models (LLMs) in interpreting ultrasound recordings of superficial and deep parasternal intercostal plane blocks (S-PIP and D-PIP).

The study will include 32 adult patients aged 18-85 years, classified as American Society of Anesthesiologists (ASA) physical status I-III, undergoing elective surgery with an ultrasound-guided parasternal intercostal plane block.

Blocks will be performed by experienced anesthesiologists as part of routine clinical practice. Ultrasound recordings will include the local anesthetic injection and its spread within the targeted fascial plane. No additional procedures will be performed for research purposes.

Ultrasound Video Acquisition Ultrasound-guided S-PIP and D-PIP procedures will be recorded in video format, including the injection process.

Videos with inadequate visualization of relevant anatomical structures, needle placement, or local anesthetic spread will be excluded.

All recordings will be anonymized before evaluation. Each eligible recording will be assigned a unique identification code.

Large Language Model Evaluation ChatGPT (OpenAI) and Gemini (Google) will be evaluated using their latest available multimodal versions at the time of assessment. Exact model names, versions, access platforms, and evaluation dates will be documented.

The same ultrasound recording will be independently submitted to both models in MP4 format, provided that direct video analysis is supported by the selected model interfaces.

An identical standardized English-language prompt will be used for every recording and both LLMs. Each recording will be evaluated in a separate conversation to minimize contextual carryover.

Models will receive no patient-identifying information, expert interpretations, or feedback regarding the correctness of their responses.

The following eight questions will be presented in the same order:

1. Identify the anatomical structures visible in this parasternal ultrasound recording.
2. Based on the observed local anesthetic spread, determine whether the solution was deposited in the correct target fascial plane.
3. Determine whether the local anesthetic spread is superficial or deep.
4. If the observed spread is inadequate or occurs outside the target fascial plane, what corrective action would you recommend?
5. Assess whether the needle tip is positioned within the appropriate fascial plane.
6. Identify potential procedural risks involving adjacent structures, including the pericardium and internal mammary vessels.
7. Which dermatomal levels are expected to be covered by this block?
8. If the observed spread is inadequate or occurs outside the target fascial plane, would you recommend an additional injection? Models will be instructed to base their assessments exclusively on the provided ultrasound recording and to explicitly indicate when the available visual information is insufficient for a reliable conclusion.

Responses will be recorded without additional prompting or correction. Response time and other predefined performance parameters will also be documented.

Expert Assessment and Blinding Two experienced anesthesiology professors will participate in the expert assessment process. The first expert (Expert A) will independently review each ultrasound recording and answer the same eight standardized questions. These assessments will serve as the expert reference. LLM-generated responses and Expert A's responses will subsequently be anonymized and randomly coded to conceal their sources.

The second expert (Expert B), blinded to response origin, will independently evaluate the anonymized responses regarding their accuracy, clinical appropriateness, and agreement with the expert reference.

Responses will be assessed using predefined scoring criteria, including content similarity and an overall quality score ranging from 0 to 10.

Outcome Assessment The primary objective is to compare the diagnostic accuracy of ChatGPT and Gemini in identifying anatomical structures and determining whether the local anesthetic solution was correctly deposited within the intended fascial plane during parasternal intercostal plane blocks.

Secondary assessments will include classification of superficial versus deep local anesthetic spread, needle tip position, procedural risk identification, appropriateness of clinical recommendations, predicted dermatomal coverage, and agreement with expert assessments. Model response time, response quality, and reported confidence will also be evaluated.

Data Protection and Clinical Safety All recordings will be de-identified before submission to the LLM platforms. Model-generated interpretations will be used solely for research purposes and will not influence clinical decision-making or patient management.

The study will be conducted in accordance with the approved research protocol and applicable ethical and data protection requirements.