Emotion recognition based on physiological signals for Virtual Reality applications

Abstract

Virtual Reality (VR) Systems have been used in the last years with an increasing frequency because they can be implemented for multiple applications in various fields. Some of these include aerospace, military, psychology, education, and entertainment. A way to increase the sense of presence is to induce emotions through the VE, and since one of the main purposes of VR Systems is to evoke the same emotions as a real experience would, the induction of emotions and emotion recognition could be used to enhance the experience. The emotion of a user can be recognized through the analysis and processing of physiological signals such as Electrocardiogram (ECG) and Electrodermal Activity (EDA) signals. However, very few systems that present online feedback regarding the subject’s emotional state and the possibility of adapting the VE during user experience have been developed. This thesis proposes the development of a Virtual Reality video game that can be dynamically modified according to the physiological signals of a user to regulate his emotional state. The first experiment served for the creation of a database. Previous studies have shown that specific features from these signals, can be used to develop algorithms capable of classifying the emotional states of the subjects into multiple classes or the two emotional dimensions: valence and arousal. Thus, this experiment helped to develop an appropriate Virtual Reality video game for stress induction, a signal acquisition, and conditioning system, a signal processing model and to extract time-domain signal features offline. A statistical analysis was performed to find significant differences between game stages and machine learning algorithms were trained and tested to perform classification offline. A second experiment was performed for the Proof of Concept Validation. For this, a model was created to extract features online and the classification algorithms were re-fitted with the online extracted features. Additionally, to facilitate a completely online process, the signal processing and feature extraction models were embedded on an STM32F446 Nucleo board, a strategy was implemented to dynamically modify the VE of the Virtual Reality video game according to the detected class, and the complete system was tested.