Real-Time Vehicle Detection and Instance Segmentation for ADAS Using the YOLOv11-SEG Model: A Deep Learning Approach

Abstract

This paper introduces YOLOv11-SEG, a real-time vehicle detection with instance segmentation using deep learning for Advanced Driver Assistance Systems (ADAS) and autonomous driving. The C3k2 modules for more accurate feature extraction and C2PSA attention for the treatment of small or occluded objects are added to the YOLO architecture. Trained with a fiveclass vehicle dataset of 6,500 instances, the model achieves box detection precision of 94.5% and segmentation precision of 94.9%, with recall of 94.4% (box) and 93.3% (mask) showing its high sensitivity. mAP@0.5 values of 97.4% (detection) and 97.1% (segmentation) show accurate object localization, while mAP@0.5-0.95 values of 90.4% and 88.5% show robust performance at varying IoU thresholds. Qualitative tests demonstrate consistency in performance in unfavorable conditions of fog, low light, and occlusion, showing the suitability of YOLOv11-SEG for real-world deployment in ADAS. Future work will be focused on accelerating the speed of inference and reducing the computational complexity, enhancing the applicability of the model for safe, efficient autonomous driving.