PD08.07. Multimodal AI-Driven Postoperative Surveillance for Oesophageal Cancer: Automated Patient-Reported Outcome Assessment and Early Complication Warning in a Prospective Study

Engineering notes

Engineering notes will be added by the aipentium editorial team.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为大语言模型、生成式AI、ChatGPT相关技术、计算机视觉、深度学习等高价值论文补充中文说明。

Original abstract

Abstract Topic Esophageal Cancer: Surgical Treatment of Esophageal Cancer – early outcomes and complications Background Postoperative complications following oesophagectomy occur in 25–60% of patients, yet traditional patient-reported outcome (PRO) assessment relies on standardised questionnaires with limited reliability, particularly among elderly patients with low health literacy. We aimed to develop and externally validate a multimodal artificial intelligence (AI) system that automates PRO assessment from natural patient conversations and provides early warning of major postoperative complications. Methods We conducted a prospective observational study at a tertiary cancer center, enrolling consecutive patients undergoing esophagectomy.The development cohort (Centre 1, n=196) and the temporally independent external validation cohort (Centre 2, n=103) were recruited sequentially. Using the validated Patient Symptom Assessment for Oesophageal Cancer (PSA-ESO) instrument, we collected trimodal recordings (video, audio, text) at up to 25 timepoints per patient, yielding 6,813 evaluable assessments. We fine-tuned the Qwen2.5-Omni-7B multimodal large language model with LoRA adaptation for two tasks: automated PRO symptom grading (Task A) and Temporal Transformer-based early warning of Clavien-Dindo grade II or higher complications (Task B). In a prospective implementation substudy (n=62), we evaluated the clinical impact of real-time alerts on time-to-intervention. Results In the external validation cohort, the trimodal PRO assessment achieved a weighted kappa of 0.801 (95% CI 0.73–0.87) and ICC of 0.858 against expert consensus, significantly outperforming audio-text bimodal (kappa 0.754, p<0.0001) and text-only (kappa 0.689, p<0.0001) configurations. The early warning model achieved an AUROC of 0.873 (95% CI 0.82–0.93) with a mean detection lead time of 28.7 hours before clinical diagnosis. The system detected 76.1% of symptom under-reporting cases. In the implementation substudy, real-time alerts reduced median time-to-intervention from 14.2 hours to 6.8 hours (p=0.003) and were associated with shorter ICU stays (3.1 vs 5.4 days, p=0.028). Conclusion An end-to-end multimodal AI system can accurately automate PRO assessment from natural patient conversations and provide clinically meaningful early warning for postoperative complications, with external validation confirming generalisability across cohorts. This conversation-based approach represents a paradigm shift from questionnaire-based PRO evaluation in surgical oncology, with particular relevance for populations with limited health literacy.

Links and sources

Comments