Layer-wise Relevance Backward Propagation for Fake Face Detection and Reconstruction

Abstract

Deep learning techniques involving Generative Adversarial Networks (GANs) enable artificially generated facial images. This alone presents difficulties in terms of authenticity resulting in ethical concerns among institutional applications for digital forensics, security and social media moderation. A question remains today, why would someone fake identity and how can we improve performance of the deep neural networks? This project initiated a novel approach called Layer-wise relevance backward propagation for fake face detection and reconstruction that determines the relevance of each layer to the final prediction by extracting the most important features in the input images while giving the reason as to how the model has come to the decision. Layer wise backward relevance propagation (LBRP) being the most effective explainable technique to give relevance scores, the Convolutional Neural Network (CNN) was trained using human face dataset from flickr-face-hq along with pretrained models models including VGG-16, Res-net, Mobile-net, Vision Transformer.To improve and monitor model performance, accuracy was consider along other evaluation metrics such as precision and finally Machine learning operations (MLOPs) was used for managing the pipeline resources of the model thus an accuracy 99.84% with VGG-16 was obtained along with different visualization of Grad-cam and more to give insights into each prediction made by the model. Therefore due tho this effectiveness from the results obtained during image processing tasks we propose an algorithm that combines several techniques that ensure improved accuracy, precision through incorporation of interpretability using explainable methods involving LBRP and grad-cam for fake face detection and reconstruction of faces to enable capture informative features at different scales, integrating transfer learning and ensembled method for scalable performance.