Closing the gap on affordable real-time very low resolution face recognition for automated video surveillance

Abstract

Public and private security is a worldwide problem where efficient and automated video-surveillance technologies have a lot of potential. In an emerging country like Mexico, a functional real-time automated video-surveillance system will have a very positive social, economic, and technological impact. Proposing an open framework for face recognition at very low resolutions which public and private institutions could implement and take advantage of, will ultimately benefit our society and contribute to the state of the art in terms of efficacy and efficiency. Currently, efficient face recognition for automated video surveillance is not present within the reach of public institutions and much less so for the smallest business establishments, such as convenience stores and small offices. To make an impact in this area, the scientific problem that we are focusing on solving is the one of effectively and efficiently extracting robust facial features from Very Low Resolution face images from surveillance footage, to perform the appropriate subspace projection, and perform the posterior face identification using a dataset reference, in order to improve in efficiency terms. In this thesis, we propose solving this problem using our novel method, BinaryFaceNet, with state-of-the-art training methodology and advancements in the Binary Neural Network (BNN) and Lightweight Convolutional Neural Network (CNN) literature. The implementation of our method makes accurate and real-time face recognition available for affordable ARM-based embedded devices, with limited identification and verification performance penalties while achieving an inference performance of less than 90\% latency against state-of-the-art BNNs. We finally discuss the feasibility of implementing BNN technology on extremely limited hardware, the compromises made to achieve maximum efficiency, training stable ultra-compact binarized models, and provide future work directions to complement this proposal. Finally, in our concluding remarks, we summarize the research work done and the research outcomes during the tenure of this thesis project.