Can Chatbots Serve as Ethical Agents in Healthcare?

Engineering notes

Engineering notes will be added by the aipentium editorial team.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为大语言模型、生成式AI、ChatGPT相关技术、计算机视觉、深度学习等高价值论文补充中文说明。

Original abstract

Generative artificial intelligence has moved clinical conversation from a peripheral interface problem to a core governance problem. Health systems now use or pilot large language model chatbots for patient messaging, triage, documentation, coaching, and informational support, while regulators and standard-setting bodies increasingly require transparency, lifecycle risk management, and human oversight for high-risk medical AI systems [1]-[5]. Yet the most difficult question is not whether chatbots can summarize information, but whether they can legitimately participate in or even replace human ethical judgment when patients cannot decide for themselves. Recent commentary asks whether a chatbot trained on a patient's records, communications, or digital traces could act as a medical surrogate [6]. This paper argues that the answer is no, at least not in the sense recognized by medical ethics, health law, and institutional accountability. I distinguish four layers of delegation in clinical conversation: information support, preference elicitation, moral reasoning, and surrogate authority. The first two can be conditionally authorized under strict governance. The latter two should not be delegated to chatbots as a matter of principle. The reason is not merely technical unreliability. Rather, surrogate medical decision-making is a fiduciary, relational, and institutionally accountable practice that requires answerability, interpretive humility, legal standing, and responsibility-bearing agency. Chatbots may help prepare, clarify, document, and structure human deliberation, but they should not be treated as ethical agents or medical substitute decision-makers. The paper concludes with a governance framework for designers, hospitals, and regulators that separates acceptable supportive uses from prohibited decisional delegation in intensive care, oncology, palliative care, and mental health.

Links and sources

• PDF from original source

Comments