Towards Improved Clinical Adoption of AI Segmentation Models: Benchmarking High-Performance Models for Resource-Constrained Settings

Abstract

High-performance medical segmentation models are often benchmarked on high-end GPUs. Such benchmarks do not provide useful performance insights for point-of-care low-end devices. This work, firstly, posits that to achieve improved clinical adoption of AI-powered segmentation models, especially in reduced manpower settings like rural hospitals, we need benchmarks that provide actionable insights on the degree to which high-performance models address five deployment constraints viz: resource-effectiveness for low-end computing devices, clinically acceptable accuracy, clinically compatible execution times, localization of user data, and user-based finetuning. In this work, five state-of-the-art foundation segmentation models and one target-specific model were systematically evaluated on three multi-organ medical datasets. Furthermore, the best-ranking foundation model and target-specific model were benchmarked on three low-end devices. Our findings show that lightweight foundation models provided the best performance trade-off and are easily user-fine-tuned on custom datasets. Target-specific models provide high accuracy out-of-the-box, but may require significant optimisation to deliver comparably fast execution times and user-based finetuning on low-end devices. The methods and results from this research provide actionable insights on high-performance medical segmentation models for low-end computing devices, as a necessary step towards improved adoption in resource-limited clinical settings.