Assessment of ChatGPT responses to nutrition queries of patients with inflammatory bowel disease

Engineering notes

Engineering notes will be added by the aipentium editorial team.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为大语言模型、生成式AI、ChatGPT相关技术、计算机视觉、深度学习等高价值论文补充中文说明。

Original abstract

Considering the critical role of diet in the management of inflammatory bowel disease (IBD), patients with IBD often seek nutrition-related information from various sources. With the increasing application of artificial intelligence (AI) chatbots across multiple fields, their use as a source of patient educational material has also expanded. However, the readability, accuracy, and comprehensibility of data provided by AI chatbots have not yet been fully established. Therefore, this study was conducted to evaluate the readability, scientific accuracy, and comprehensibility of AI-generated responses to the nutritional queries of patients with IBD. Twenty frequently asked questions about nutrition and IBD were retrieved from Google and were posed to Chat Generative Pretrained Transformer (ChatGPT), version 3.5. The readability of ChatGPT-generated responses was assessed using three validated measures: Flesch Reading Ease Score (FRES), the Flesch Kincaid Grade Level (FKGL), and the Simple Measure of Gobbledygook (SMOG). Scientific accuracy and comprehensibility were evaluated by nine nutritionists and dietitians specializing in IBD. Regarding readability, ChatGPT-generated responses were difficult to read, with FRES = 44.54, FKGL = 11.6, and SMOG = 13.8. Additionally, ChatGPT-generated responses to patient questions were generally scientifically accurate (mean score = 4.2 ± 0.5) and comprehensible (mean score 4.3 ± 0.6). However, both scientific accuracy and comprehensibility decreased when questions became more technical and complex. Overall, the assessment of AI-generated responses demonstrated low readability. However, in terms of scientific accuracy and comprehensibility, the AI chatbot provided relatively accurate and understandable information regarding the nutritional information needs of patients with IBD. Further studies are needed to confirm these findings.

Links and sources

Comments