Towards a real-time lightweight facial reconstruction model

Abstract

3D facial reconstruction algorithms are highly effective for diverse uses, including facial recognition, virtual reality, and medical imaging. Yet, the intricacy and computational demands of these methods, coupled with the limited availability of datasets, have confined their use to a specific set of researchers and experts. Furthermore, in response to the demand for resource-efficient solutions, the development of lightweight processes has become a key area of research in computer vision. These models aim to find an equilibrium between model size, computational demands, and accuracy. They offer advantages like efficient use of resources, quicker inference times, and enhanced accessibility. Particularly for 3D facial reconstruction models, lightweight architectures open up possibilities for deployment on less powerful hardware, given that these techniques typically depend on high-performance processors like NVIDIA graphics cards. This thesis presents an overview of 3D face creation, followed by state-of-the-art methods which were analyzed in a comparative table, offering an survey of the fundamental characteristics of each method. As well as that, a benchmark comparison among various leading lightweight models in a facial reconstruction framework, aiming to decrease its computational complexity to enable testing on a mobile device. A quantitative evaluation, such as its losses over the training and testing stages, the inference speed achieved and an evaluation in cutting-edge datasets were presented. Additionally, an analysis on the qualitative aspect, for example, the 3D pose or depth estimation. Those aspects were the base to select a lightweight backbone. Finally, an user interface was developed using Python and Kivy. The model was runned on a constrained-device, such as a single-core of a commercial laptop, to examine its performance. EfficientNetLite was determined as a suitable replacement for the current backbone, since its characteristics and scores obtained over several examinations presented a similar behavior to MobileNet-V1, the default backbone of the facial reconstruction model selected.