Transparent Machine Vision Techniques For Facial Emotion Detection

Abstract

Emotions are very important when it comes to collective understanding among people. In recent times, emotion recognition has emerged as a crucial facet within the realm of effective computing. Many physiological signals that could be used to notice emotions, getting facial expression images is one of the most natural and cheapest approaches. While humans find recognizing facial expressions for emotion detection effortless, the same task presents a unique challenge for computer algorithms devoid of empathy. Having this question in mind, we want to use machine vision based techniques for getting a better result regarding emotions detection. In this study, facial expression recognition (FER) was classified using the Vision Transformers(ViT), Convulational Neural Networks(CNN), ResNet, VGG16 and VGG19 models. The experimental outcomes show case that our models have the ability of being successfully used in practical settings having been trained on the FER + dataset consisting of 35,887 gray-scale images of the 8 classes, with each image having a resolution of 48×48 pixels. Experimental findings indicate that the Vision Transformers were most effective with an accuracy of 83. Additionally, the CNN model achieved an accuracy of 78 while the VGG19, VGG16 and ResNet achieved accuracy of 72, 71 and 63 respectively. Additionally we applied explainability techniques such as Lime, Integrated gradients and layerwise propagation on each model so as to understand what the model focuses on so as to classify an expression and Lime turned out to be the best XAI technique for our models.