Detection of Violent Behavior in Open Environments Using Pose Estimation and Neural Networks

Abstract

People’s safety and security have always been an issue to attend. With the coming of techno- logical advances, part of it has been used to improve safeguards, though other aspects, without precautions, have made people even more vulnerable. People can get their sensitive data stolen or become victims of transaction fraud. These may be crimes done without physical interac- tion, but felonies with physical violence still exist. Some solutions for pedestrian safety are guards, police cars patrolling, sensors and security cameras. Nonetheless, these methods only react when the crime is happening or, even more critical, when it has already occurred, and the damage has been done. Therefore, numerous methods have been implemented using Arti- ficial Intelligence in order to solve this problem. Many approaches to detect violent behavior and action recognition rely on 3D convolutional neural networks (3D CNNs), spatial tempo- ral models, long short term memory networks, pose estimation among other implementations. However, in the current state of the art, how these approaches are used do not work perfectly and are not adapted to an uncontrolled environment. Therefore, a significant contribution from this work was the development of a new solu- tion model that is able to detect violent behavior. This approach focuses on using pedestrian detection, tracking, pose estimation and neural networks to predict pedestrian behavior in video frames. This method uses a time window frame to extract joint angles, given by the pose estimation algorithm, as features for classifying behavior. At the moment of developing this thesis project, there were not many databases with violent behavior videos. The ones that existed were low quality; cluttered were pedestrians cannot be seen clearly, and with unfixed camera angles. Consequently, another important contribution of this work was creating a new database, Kranok-NV, with a total of 3,683 normal and violent videos. This database was used to train and test the solution model. For the evaluation, a protocol was designed using 10-fold cross- validation. With the implemented solution model, accuracy of more than 98% was achieved on the Kranok-NV database. This approach surpassed the performance of state of the art methods for violence detection and action recognition in the developed database. Though this new solution model is able to detect violent and normal behavior, it can be easily extended to classify more types of behaviors. Further work requires to test this approach in emerging databases of videos and optimize specific areas of the solution model. Additionally, the contributions of this work can aid in the development of new approaches.