Artificial; ChatGPT; Comparison; Diagnosis; Head neck; Intelligence; Laryngology; Laryngopharyngeal; Otolaryngology; Reflux; Surgery; Treatment; Voice; Otorhinolaryngology; General Medicine
Abstract :
[en] [en] OBJECTIVES: To study the performance of ChatGPT in the management of laryngology and head and neck (LHN) cases.
METHODS: History and clinical examination of patients consulting at the Otolaryngology-Head and Neck Surgery department were presented to ChatGPT, which was interrogated for differential diagnosis, management, and treatment. The ChatGPT performance was assessed by two blinded board-certified otolaryngologists using the following items of a composite score and the Ottawa Clinic Assessment Tool: differential diagnosis; additional examination; and treatment options. The complexity of clinical cases was evaluated with the Amsterdam Clinical Challenge Scale test.
RESULTS: Forty clinical cases were submitted to ChatGPT, accounting for 14 (35%), 12 (30%), and 14 (35%) easy, moderate and difficult cases, respectively. ChatGPT indicated a significant higher number of additional examinations compared to practitioners (p = 0.001). There was a significant agreement between practitioners and ChatGPT for the indication of some common examinations (audiometry, ultrasonography, biopsy, gastrointestinal endoscopy or videofluoroscopy). ChatGPT never indicated some important additional examinations (PET-CT, voice quality assessment, or impedance-pH monitoring). ChatGPT reported highest performance in the proposition of the primary (90%) or the most plausible differential diagnoses (65%), and the therapeutic options (60-68%). The ChatGPT performance in the indication of additional examinations was lowest.
CONCLUSIONS: ChatGPT is a promising adjunctive tool in LHN practice, providing extensive documentation about disease-related additional examinations, differential diagnoses, and treatments. The ChatGPT is more efficient in diagnosis and treatment, rather than in the selection of the most adequate additional examination.
Disciplines :
Otolaryngology
Author, co-author :
Lechien, Jérome ; Université de Mons - UMONS > Faculté de Psychologie et des Sciences de l'Educatio > Service de Métrologie et Sciences du langage
Georgescu, Bianca M; Division of Laryngology and Broncho-Esophagology, Department of Otolaryngology-Head Neck Surgery, UMONS Research Institute for Health Sciences and Technology, EpiCURA Hospital, University of Mons (UMons), Mons, Belgium
Hans, Stephane; Research Committee of Young Otolaryngologists of the International Federation of Otorhinolaryngological Socities (IFOS), Paris, France ; Department of Otorhinolaryngology and Head and Neck Surgery, School of Medicine, UFR Simone Veil, Foch Hospital, Université Versailles Saint-Quentin-en-Yvelines (Paris Saclay University), Paris, France
Chiesa-Estomba, Carlos M; Research Committee of Young Otolaryngologists of the International Federation of Otorhinolaryngological Socities (IFOS), Paris, France ; Department of Otorhinolaryngology-Head & Neck Surgery, Donostia University Hospital-Biodonostia Research Institute, St. Sebastian, Spain
Language :
English
Title :
ChatGPT performance in laryngology and head and neck surgery: a clinical case-series.
Publication date :
24 October 2023
Journal title :
European Archives of Oto-Rhino-Laryngology
ISSN :
0937-4477
eISSN :
1434-4726
Publisher :
Springer Science and Business Media Deutschland GmbH, Germany
Pernencar C, Saboia I, Dias JC (2022) How far can conversational agents contribute to IBD patient health care—a review of the literature. Front Public Health 10:862432. 10.3389/fpubh.2022.862432 DOI: 10.3389/fpubh.2022.862432
Hill-Yardin EL, Hutchinson MR, Laycock R, Spencer SJ (2023) A Chat(GPT) about the future of scientific publishing. Brain Behav Immun 110:152–154. 10.1016/j.bbi.2023.02.022 DOI: 10.1016/j.bbi.2023.02.022
Choi JH, Hickman KE, Monahan A, Schwarcz D (2023) ChatGPT goes to law school? Minnesota Legal Studies Research Paper No. 23-03
Gupta R, Park JB, Bisht C, Herzog I, Weisberger J, Chao J, Chaiyasate K, Lee ES (2023) Expanding cosmetic plastic surgery research using ChatGPT. Aesthet Surg J. 10.1093/asj/sjad069 DOI: 10.1093/asj/sjad069
Salvagno M, Taccone FS, Gerli AG (2023) Can artificial intelligence help for scientific writing? Crit Care 27(1):75. 10.1186/s13054-023-04380-2 DOI: 10.1186/s13054-023-04380-2
Gercama AJ, de Haan M, van der Vleuten CPM (2000) Reliability of the Amsterdam Clinical Challenge Scale (ACCS): a new instrument to assess the level of difficulty of patient cases in medical education. Med Educ 34(7):519–524 DOI: 10.1046/j.1365-2923.2000.00663.x
Braun LT, Lenzer B, Fischer MR, Schmidmaier R (2019) Complexity of clinical cases in simulated learning environments: proposalfor a scoring system. GMS J Med Educ. 36(6):Doc80. 10.3205/zma001288 DOI: 10.3205/zma001288
Rekman J, Hamstra SJ, Dudek N, Wood T, Seabrook C, Gofton W (2016) A new instrument for assessing resident competence in surgical clinic: the ottawa clinic assessment tool. J Surg Educ 73(4):575–582. 10.1016/j.jsurg.2016.02.003 DOI: 10.1016/j.jsurg.2016.02.003
Fine B, Dhanoa D (2014) Imaging appropriateness criteria: why Canadian family physicians should care. Can Fam Physician 60(3):217–218
Kang SK, Gold HT (2023) How to perform economic evaluation in implementation studies: imaging-specific considerations and comparison of financial models. J Am Coll Radiol 20(3):292–298. 10.1016/j.jacr.2022.11.018 DOI: 10.1016/j.jacr.2022.11.018
Hellings PW, Fokkens WJ, Orlandi R, Adriaensen GF, Alobid I, Baroody FM et al (2023) The EUFOREA pocket guide for chronic rhinosinusitis. Rhinology 61(1):85–89. 10.4193/Rhin22.344 DOI: 10.4193/Rhin22.344
Yoshiyasu Y, Wu F, Dhanda AK, Gorelik D, Takashima M, Ahmed OG (2023) GPT-4 accuracy and completeness against international consensus statement on allergy and rhinology: rhinosinusitis. Int Forum Allergy Rhinol. 10.1002/alr.23201 DOI: 10.1002/alr.23201
Dejonckere PH, Bradley P, Clemente P, Cornut G, Crevier-Buchman L, Friedrich G, Van De Heyning P, Remacle M, Woisard V, Committee on Phoniatrics of the European Laryngological Society (ELS) (2001) A basic protocol for functional assessment of voice pathology, especially for investigating the efficacy of (phonosurgical) treatments and evaluating new assessment techniques. Guideline elaborated by the Committee on Phoniatrics of the European Laryngological Society (ELS). Eur Arch Otorhinolaryngol 258(2):77–82. 10.1007/s004050000299 DOI: 10.1007/s004050000299
Stachler RJ, Francis DO, Schwartz SR, Damask CC, Digoy GP, Krouse HJ, McCoy SJ, Ouellette DR, Patel RR, Reavis CCW, Smith LJ, Smith M, Strode SW, Woo P, Nnacheta LC (2018) Clinical practice guideline: hoarseness (dysphonia) (update). Otolaryngol Head Neck Surg 158(1 suppl):S1–S42. 10.1177/0194599817751030 DOI: 10.1177/0194599817751030
Stuck BA, Beule A, Damm M, Gudziol H, Hüttenbrink KB, Landis BN, Renner B, Sommer JU, Uecker FC, Vent J, Hummel T, Committee on Olfaction and Gustation of the German Society of Otorhinolaryngology, Head and Neck Surgery (2014) Position paper “chemosensory testing for expert opinion in smell disorders.” Laryngorhinootologie 93(5):327–329. 10.1055/s-0033-1364034 DOI: 10.1055/s-0033-1364034
Lechien JR, Vaezi MF, Chan WW, Allen J, Karkos PD, Saussez S, Altman K, Amin MR, Ayad T, Barillari MR et al. (2023) The Dubai definition and diagnostic criteria of laryngopharyngeal reflux: the IFOS Consensus. Laryngoscope
Chiesa-Estomba CM, Lechien JR, Vaira LA, Brunet A, Cammaroto G, Mayo-Yanez M, Sanchez-Barrueco A, Saga-Gutierrez C (2023) Exploring the potential of Chat-GPT as a supportive tool for sialendoscopy clinical decision making and patient information support. Eur Arch Otorhinolaryngol. 10.1007/s00405-023-08104-8 DOI: 10.1007/s00405-023-08104-8
Hoch CC, Wollenberg B, Lüers JC, Knoedler S, Knoedler L, Frank K, Cotofana S, Alfertshofer M (2023) ChatGPT’s quiz skills in different otolaryngology subspecialties: an analysis of 2576 single-choice and multiple-choice board certification preparation questions. Eur Arch Otorhinolaryngol. 10.1007/s00405-023-08051-4 DOI: 10.1007/s00405-023-08051-4
Millstein J, Agarwal A (2023) What can doctors and patients do with ChatGPT? | Expert Opinion. Philadelphia Inquirer
Chaix B, Bibault JE, Pienkowski A, Delamon G, Guillemassé A, Nectoux P, Brouard B (2019) When chatbots meet patients: one-year prospective study of conversations between patients with breast cancer and a chatbot. JMIR Cancer 5(1):e12856. 10.2196/12856 DOI: 10.2196/12856
