Enhancing clinical reasoning and diagnostic precision through scaling laws and multi-stage supervised fine-tuning in open-weight medical large language models

Engineering notes

Engineering notes will be added by the aipentium editorial team.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为大语言模型、生成式AI、ChatGPT相关技术、计算机视觉、深度学习等高价值论文补充中文说明。

Original abstract

Abstract The emergence of large language models offers unprecedented opportunities to transform clinical workflows. However, reliance on proprietary closed-source models poses significant risks to data privacy, institutional autonomy, and reasoning transparency. This study investigates the optimization of open-weight architectures, specifically the Llama-3.1-8B and Qwen2.5-14B-Instruct families, to function as specialized diagnostic tools rather than general-purpose conversational agents. Utilizing the MedQA USMLE-style benchmark, we evaluate the synergistic impact of context window scaling and multi-stage supervised fine-tuning within a localized Retrieval-Augmented Generation framework. Our findings identify a critical scaling threshold at 512 tokens, beyond which diagnostic accuracy stabilizes while preserving the computational efficiency required for on-site clinical deployment. Furthermore, we document a pronounced Brevity Shift, in which multi-stage supervised fine-tuning reduced model verbosity by over 99.7%. The models transitioned from long-form reasoning with an average of 465 tokens to single-token outputs for Qwen and near-single-token outputs for Llama. This shift effectively mitigated verbosity compensation, defined as the tendency of models to mask uncertainty through excessive text, while simultaneously improving diagnostic performance, which peaked at 70.0% accuracy for the Qwen architecture on the benchmark task. Additionally, our analysis reveals that systematic fine-tuning substantially reduces option-preference bias, as measured by Total Variation Distance, leading to more objective and consistent model outputs. Conducted under the INFOSTRATEG Strategic Program (PARROT AI) and funded by the National Centre for Research and Development (NCBR), Poland, this work establishes a robust technical framework for developing trustworthy, high-efficiency medical AI systems capable of operating entirely within hospital-controlled infrastructure.

Links and sources

Comments