MediGPT: Exploring Potentials of Conventional and Large Language Models on Medical Data

Abstract

Medical text classification organizes medical documents into categories to streamline information retrieval and support clinical decision-making. Traditional machine learning techniques, including pre-trained language models, are effective but require extensive domain-specific training data, often underperform across languages, and are costly and complex to deploy on a large scale. In this study, we employed four datasets: Clinical trials on cancer, encompassing 6 million statements from interventional cancer clinical trial protocols; the Illness-dataset, consisting of 22,660 categorized tweets from 2018 and 2019; the Multi-View active learning for short medical text classification in user-generated data, an extended version of the Illness-dataset including 22,660 documents from the same period; and the Symptom2Disease dataset, containing 1,200 data points used to predict diseases based on symptom descriptions. This study uses ChatGPT, particularly its ChatGPT-3.5 and ChatGPT-4 versions, as a viable alternative for classifying medical texts. We investigate essential aspects, including the construction of prompts, the parsing of responses, and the various strategic use of GPT models to optimize outcomes. Through comparative analysis with established methods like pre-trained language model fine-tuning and prompt-tuning, our findings indicate that ChatGPT addresses these challenges efficiently and matches the performance of traditional methods. Furthermore, the enhanced capabilities of the proposed MediGPT (Medical Generative Pre-Trained Transformers) have led to performance improvements of 14.3%, 22.3%, 13.6%, and 13.7% across the datasets, highlighting its adaptability and robustness in diverse medical text scenarios without the need for specialized domain adjustments. This research underscores the capability of ChatGPT to facilitate a versatile AI framework in medical text processing, which could revolutionize medical informatics practices.