Micro-expression recognition with event cameras

Abstract

Micro-expression recognition is a challenging task due to the subtle and rapid nature of these facial movements. Traditional RGB-based systems often struggle with capturing microexpressions, particularly in low-light or high-speed scenarios. Event-based vision offers a promising alternative, leveraging asynchronous brightness change detection to achieve high temporal resolution and low latency. However, the sparse and unconventional data format of event cameras requires the development of tailored algorithms and architectures. This thesis explores the application of event-based vision for micro-expression recognition, focusing on the integration of advanced deep learning architectures, including Conditional Variational Autoencoders (CVAE), Spiking Neural Networks (SNN), and multi-region of interest (multi-ROI) processing. To facilitate training, synthetic event data was generated from traditional RGB datasets, enabling direct comparison between event-driven and framebased approaches. Our experiments demonstrate that event-driven architectures preserve critical temporal cues, and can achieve promising results compared to conventional methods. The proposed system shows promising results in maintaining accuracy and responsiveness, particularly in conditions where traditional cameras fall short. These findings highlight the potential of combining event-based vision with specialized neural networks to advance micro-expression recognition in real-world applications.