A Comprehensive Benchmarking Analysis of Intrusion Detection for Internet of Medical Things Using CICIoMT2024

Abstract

The Internet of Medical Things (IoMT) is transforming healthcare through connected devices that enable remote monitoring and personalized care, but it also exposes sensitive patient data to growing cyber threats. Intrusion Detection Systems (IDS) are essential for securing these networks, yet challenges such as data imbalance and resource constraints hinder effective deployment. This study conducts a comprehensive benchmarking of machine learning models: Random Forest (RF), Decision Tree (DT), Naive Bayes (NB), Light Gradient Boosting Machine (LightGBM), eXtreme Gradient Boosting (XGBoost) and deep learning models: Convolutional Neural Network (CNN), Multilayer Perceptron (MLP) for IDS using the CICIoMT2024 dataset, which captures diverse IoMT traffic over Wi-Fi and MQTT protocols and includes five major attack categories. We evaluate model performance across both binary and multiclass classification tasks, examining not only detection accuracy through standard metrics but also practical deployment considerations including training time and memory consumption. Our comparative analysis reveals important trade-offs between detection performance and computational efficiency across different model families. The findings provide practical insights for selecting appropriate IDS approaches in resource-constrained IoMT environments, where security must be balanced against the limitations of medical devices. This work contributes to understanding which modeling strategies best handle the unique challenges of IoMT intrusion detection, including severe class imbalance and the need for lightweight, deployable solutions.