Facial Expression Recognition with Combined Eye-Mouth ROI Extraction using Facial Landmarks and CNN

Abstract

Facial expression recognition (FER) has become increasingly important in human-computer interaction, affective computing, and social robotics. This paper presents a novel approach to FER that combines Region of Interest (ROI) extraction with a custom Convolutional Neural Network (CNN) architecture. Our method addresses the limitations of traditional approaches by focusing on the most discriminative facial regions: the eye-eyebrow area and the mouth region.The proposed methodology consists of four main stages: (1) preprocessing the image, (2) detection of facial landmarks, (3) extraction and sequencing of ROI regions in an input image, and (4) feature extraction and classification using a custom CNN with three convolutional layers with various filter sizes and an activation function =1.5*X. Extensive experiments on the FER2013 dataset demonstrate the effectiveness of our approach. The proposed ROI-based CNN achieves an accuracy of 72.04%, outperforming conventional methods by focusing on the most expressive regions of the face. Our comparative analysis reveals that ROI extraction significantly improves recognition performance. The activation function (f(x) = 1.5x) provides a better decay flow that amplifies the response of neurons in order to improve the network’s sensitivity to slight variations in facial expressions and avoid dead neurons.