Guideline-Based Accuracy Scores and Educational Quality of Artificial Intelligence-Based Language Model Responses to Search-Result-Derived Korean Pediatric Dental Trauma Queries: A Two-Time-Point Exploratory Study

Engineering notes

Engineering notes will be added by the aipentium editorial team.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为大语言模型、生成式AI、ChatGPT相关技术、计算机视觉、深度学习等高价值论文补充中文说明。

Original abstract

This exploratory study described guideline-based accuracy scores, educational quality, and temporal variation in ChatGPT and Gemini responses to eight searchresult-derived Korean pediatric dental trauma queries. Four tooth-avulsion and four crown-fracture prompts were submitted once to each service in January 2026 and one month later. The queries were treated as a convenience sample of recurrent searchresult content, not a representative set of caregiver questions. Two pediatric dentists scored 32 responses using query-specific key points, predefined major errors, and the Global Quality Score. Inter-rater quadratic weighted kappa was 0.910 for the guideline-based accuracy score and 0.835 for the Global Quality Score. ChatGPT accuracy scores were unchanged in five prompts, increased in one, and decreased in two; Global Quality Score values were unchanged in six and decreased in two. Gemini accuracy scores were unchanged in four, increased in one, and decreased in three; all Global Quality Score values were unchanged. No predefined major errors were identified, but avulsion-related terminology appeared in three of four crown-fragment reattachment responses. Within the sampled queries, clinically important omissions, injury-concept mixing, and temporal variation were observed. The findings do not represent caregiver information needs or establish real-world educational effectiveness, repeatability, or service superiority.

Links and sources

Comments