Enhancing Road Safety: A Comprehensive Driver Behavior Scoring Framework with K-Means Action Segmentation and Deep Learning Behavior Detection

Abstract

The rising concern for road safety due to increasing vehicle numbers requires innovative strategies to address unsafe driving practices. This paper presents a comprehensive Driver Behavior Scoring Framework aimed at improving road safety. It consists of five modules: Video Segmentation, Behavior Classification, Voice and Speech Analysis, Facial Analysis, and Safety Scoring. In Video Segmentation, we propose a novel approach using VGG-16 and k-means for action segmentation. The Behavior Classification module utilizes a CNN-LSTM classifier, which obtained an F1-score of 87.5%. Voice and Speech Analysis use a DistilBERT-based sentiment analysis achieving 93.42% accuracy in sentiment detection and 76.6% accuracy in profanity detection. Facial Analysis detects eye and mouth movements with MAE of 0 and 1.6 respectively, and identifies anger and happiness behaviors with an MAE of 44 using a CNN-based pretrained model. The fifth module introduces a Linear Regression-based safety scoring algorithm trained on survey data, yielding an RMSE of 1.1616.