Abstract
<title>Abstract</title> <p>Background Large language models (LLMs) are increasingly used for clinical information, but their reliability in managing implant-related complications is uncertain. This study compared the diagnostic, management, safety, and communication performance of four LLMs in guideline-informed implant-complication scenarios and assessed their patient-safety event burden. Methods Twenty-four clinical vignettes, covering six complication categories and based on European Association for Osseointegration, International Team for Implantology, and American Association of Oral and Maxillofacial Surgeons consensus documents, were stratified by difficulty and submitted to Gemini 3.1, ChatGPT 5.4, Claude 4.6, and DeepSeek V3.2, resulting in 96 responses. A blinded oral and maxillofacial surgeon and a periodontologist independently evaluated anonymised, randomised responses across five Likert-type domains, with a third expert resolving disagreements. Three binary safety outcomes were also assessed. Results Inter- and intra-rater agreement was substantial to almost perfect. Friedman tests showed significant differences between models across all domains (completeness/actionability, p = 0.001; diagnostic accuracy, p = 0.023), with small-to-moderate effect sizes (Kendall's W = 0.13–0.22). After Holm correction, Gemini 3.1 scored significantly higher than DeepSeek V3.2 in all domains and overall (4.30 vs 3.58; adjusted p = 0.003); no other comparisons were significant. Model differences were significant only in difficult scenarios, where Gemini 3.1 outperformed DeepSeek V3.2 (4.25 vs 3.60; adjusted p = 0.014). Cochran's Q tests found significant differences in major harmful recommendations (p = 0.041) and hallucinated content (p = 0.035), with DeepSeek V3.2 having the highest burden. Conclusions LLMs differed mainly in complex cases and varied in safety profiles. While they may aid clinical education and decision-checking, these exploratory findings suggest they cannot replace expert clinical judgment in implant-complication management.</p>