Exploring Large Language Models Integration in the Histopathologic Diagnosis of Skin Diseases: A Comparative Study

The integration of large language models (LLMs) into healthcare is rapidly evolving, with the potential to transform patient care and outcomes. Since the introduction of ChatGPT by OpenAI in November 2022, and Gemini by Google, these models have demonstrated remarkable capabilities in generating human-like responses across various tasks. In medicine, ChatGPT has shown promise in analyzing complex medical data, aiding differential diagnosis, and synthesizing information from patient records, medical research, and clinical guidelines. However, the application of LLMs in dermatology and pathology remains limited. This study aims to explore the role of LLMs, specifically ChatGPT-3.5 and Gemini, in diagnosing dermatologic conditions within pathology. It compares their accuracy and concordance with human pathologists and investigates potential advantages, biases, and limitations of integrating LLM tools into pathology decision-making processes.

The study employed a comparative design using 60 real histopathology case scenarios of skin conditions, half neoplastic and half non-neoplastic, selected from a hospital database. Two board-certified pathologists reviewed each case to establish a consensus diagnosis. Cases were included if they had complete histopathological reports and comprehensive patient demographics; cases with incomplete data were excluded. A random sampling method ensured representativeness. The cases were presented to ChatGPT-3.5, Gemini, and an external board-certified pathologist in March 2023. The LLMs received standardized prompts: initially "Hello," followed by "Please provide the most accurate diagnoses from the texts that will be given below." Each case was copy-pasted individually, and the first response was recorded. If no diagnosis was given, the prompt was repeated with a variation. The external pathologist received the same information without histopathological images. Responses were categorized into complete agreement, partial agreement, or no agreement with the original diagnosis, based on whether both the general diagnosis and specific subtype were correctly identified. Statistical analysis was performed to compare agreement levels between LLMs and the pathologist.