How AI responds to common HIV/AIDS questions: ChatGPT versus DeepSeek

Engineering notes

Engineering notes will be added by the aipentium editorial team.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为大语言模型、生成式AI、ChatGPT相关技术、计算机视觉、深度学习等高价值论文补充中文说明。

Original abstract

Background Artificial intelligence (AI) has garnered significant attention and, to some extent, has even replaced certain search engines as a popular channel for people across regions to access information. HIV/AIDS is a common topic among infectious diseases, and patients or high-risk groups frequently search for information online. Our study evaluated the accuracy of two AI models (ChatGPT-3.5 and DeepSeek-R1) in answering HIV/AIDS-related questions. Methods This was a cross-sectional analytical study comparing two advanced large language models representing Eastern and Western cultural backgrounds, respectively, with a gold standard (an infectious disease specialist). We compiled 26 common questions related to HIV/AIDS and categorized them into four themes (basic knowledge, diagnosis, treatment, and prevention). Three infectious disease experts independently scored the AI models’ responses on a 4-point scale for each answer. There was no statistically significant difference in the scores for HIV/AIDS-related questions between the two AI models: Z = −2.135, two-sided p -value = 0.451 ( p > 0.05). DeepSeek-R1 demonstrated higher accuracy, with 53.85% of its responses rated as “Excellent,” compared to 48.72% for ChatGPT-3.5; there was no significant difference in accuracy between the two AI models when answering HIV/AIDS-related questions ( χ 2 = 0.429, df = 1, p = 0.513). Both AI models achieved high average composite scores (DeepSeek-R1 = 3.50, ChatGPT-3.5 = 3.44, out of a maximum of 4 points). The AI models performed exceptionally well across all domains, though they were relatively weaker in the “Treatment” domain. Conclusion Our study has identified the potential of AI models, particularly ChatGPT-3.5 and DeepSeek-R1, to provide accurate and comprehensive responses to HIV/AIDS-related questions. As free online tools, they can provide personalized, useful medical information to diverse populations across regions. However, medical information generated by AI models should still be used under the supervision and review of healthcare professionals.

Links and sources

Comments