A Comprehensive study into digital human faces. emphatic perception using an improved machine vision model for a democratized facial tracking using a facial action code system

Abstract

This thesis explores the flourishing field of digital humans, which has gained prominence in recent years as embodied conversational agents with diverse applications. The challenge lies in achieving realism in graphics, empathic responses, and accurate facial expression replication. While computer facial animation has made strides in creating adaptable, real-time systems, democratized solutions for facial motion capture are limited by cost and accessibility. This research presents a framework integrating artificial intelligence techniques for real-time facial tracking. It is hypothesized that by combining the Facial Action Coding System with machine learning, digital human realism and empathic responses can be enhanced. Key questions and objectives address the feasibility of open-source real-time facial tracking, the impact of integration between the Facial Action Coding System and artificial intelligence, and the balance between photo-realistic quality and expressive nuances. Contributions encompass a general facial capture pipeline proposal, an open-source application, an evaluation model for empathic responses, and a comparative analysis of accessible facial performance capture solutions. Parallel research findings include a protocol for genuine expression capture, insights into emotional regulation in virtual environments, and an evaluation of lightweight backbone models for facial reconstruction. Overall, the research carried out during this thesis holds the potential for improving realism and empathy in digital humans, offering valuable insights and setting the stage for future advancements in the field.