Enhancing Brain Tumor Classification: A Comparative Study of Single-Model and Multi-Model Fusion Approaches

Abstract

Brain tumors are the leading cause of death world-wide. Deep learning has been successful in previous tasks like classification. However, it's being limited by the reliance on a single imaging modality which isn't enough, where a single modality can provide higher performance but is unreliable for accurate treatment and diagnosis. This study aims to improve brain tumor classification using deep learning and fusion techniques of multiple modalities. The study employs three fusion approaches: image-level fusion, feature-level fusion, and wavelet-based fusion. Extensive experiments were conducted on the BRATS2020 dataset. Initially, we train and evaluate the performance of 21 baseline models, encompassing 20 CNN-based architectures alongside the vision transformer model. Moreover, we identify the highest-performing models within each class for fusion. Furthermore, inspired by the baseline models, we dive deeper, introducing each modality as input to its respective best-performing model and fusing the outputs for multi-modality model-level fusion. Finally, we employ wavelet-based fusion to optimize information integration, implementing Discrete Wavelet Transform on our dataset. Model-level fusion outperformed image fusion across all evaluation metrics by 1 % accuracy, 4.7% precision, 6.6 % recall, and 0.7% F1-score.