Assessing the clinical applicability of large language models for microwave ablation guidance

Engineering notes

Engineering notes will be added by the aipentium editorial team.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为大语言模型、生成式AI、ChatGPT相关技术、计算机视觉、深度学习等高价值论文补充中文说明。

Original abstract

Purpose To evaluate the performance of five language models (LLMs)—ChatGPT, Gemini, Meta, Claude, and DeepSeek—in providing clinically relevant advice on microwave ablation (MWA), a minimally invasive therapy for tumor treatment. Methods A total of 27 standardized questions covering seven thematic domains of MWA were presented to each LLM using a single-input approach. Responses were independently evaluated by three senior radiologists using three customized 4-point rating scales: Overall Quality Score (OQS), Understandability Score (US), and Implementability Score (IS). The mean of these metrics was calculated as the Mean Quality Score (MQS). Interrater reliability was assessed using intraclass correlation coefficients (ICC). Results 540 ratings were performed. All LLMs successfully generated relevant responses. ChatGPT achieved the highest MQS (3.65 ± 0.54), outperforming Claude (3.47 ± 0.59) and DeepSeek (3.47 ± 0.60), while Gemini (3.00 ± 0.55) and Meta (2.95 ± 0.61) scored significantly lower (p < 0.05). ChatGPT also ranked highest across all subcategories, particularly in Effectiveness and Outcomes (3.73 ± 0.44), Risks and Complications (3.89 ± 0.32), and Post-Procedure Care (3.89 ± 0.32). Interrater reliability was moderate to good (ICC = 0.56–0.75), confirming consistent evaluation. Conclusion All examined LLMs demonstrated the ability to provide meaningful advice on MWA. However, ChatGPT showed the best overall accuracy, clarity, and clinical applicability. Claude and DeepSeek achieved comparable results in specific areas. While these findings highlight the promising potential of LLMs in supporting clinical decision-making, human oversight and domain-specific validation remain essential.

Links and sources

Comments