Motion-Oriented Diffusion Models for Facial Expression Synthesis

Abstract

Facial expression generation in computer vision is essential for improving human-computer interaction by enabling machines to interpret and respond to human emotions effectively. This area has attracted considerable research interest. In this context, we introduce a new approach for generating facial expressions from a single neutral image and a target expression label. Our method, referred to as Motion-Oriented Diffusion Model (MODM), leverages latent diffusion techniques, which are known for their ability to learn complex latent spaces and integrate controlled stochasticity to diversify generated content. The main idea of MODM is separating the embedding space into identity and motion domains, and applying diffusion to the motion latent space only. This strategy enhances our model capability to generate various facial expressions while ensuring that the identity details remain consistent across different expressions. To assess the effectiveness of MODM, we perform qualitative and quantitative evaluations using the MUG facial expression database. The preliminary results demonstrate that MODM can generate realistic videos of the six basic facial expressions, preserving the identity of the input subject while accurately representing different emotional states. Additionally, our study highlights promising directions for potential future research and improvements.