Real-time armed individual detection in video surveillance usingdeep learning and heuristic approaches

Abstract

This researchaimstoenhancetheautomaticidentificationofarmedindividualsinvideo surveillanceinreal-time.Theproposedmethodologyinvolvesthedevelopmentofalgorithms specifically designedforthedetectionofindividualscarryinghandguns,whichincludepistols and revolvers.Toachievethis,theYOLOv4modelhasbeenselectedtodetectindividuals, handguns, andfaces.Subsequently,real-timeinformationisextractedfromtheYOLOmodel, including boundingboxcoordinates,distances,andintersectionareasbetweenhandgunsand individualswithineachvideoframe.Thisinformationfeedsourheuristicsanddifferentma- chine learning(ML)proposed,facilitatingtherecognitionofarmedindividuals.Severalchal- lenges mustbeaddressed,suchasocclusion,concealedguns,andproximityofindividualsto one another.Itencouragesthedevelopmentandcomparisonofdifferenttypesofsolutions. Theyaremadeupofthreeheuristics,seven-armedpeopledetectors(APD),and44APDto use ineachvideoframe(APD4F). The heuristicsaretheDeterministicMethodofCenters(DMC),theDeterministicMethod of Distances(DMD),andtheDeterministicMethodofIntersections(DMI).Furthermore, the APDmodelsareRandomForestClassifier(RFC-APD),MultilayerPerceptron(MLP- APD), k-Nearest-Neighbors(KNN-APD),SupportVectorMachine(SVM-APD),Logistic Regression(LR-APD),NaiveBayes(NB-APD),andGradientBoostingClassifier(GBC- APD). Thereby,IproposetocreateselectorsfordecidingwhichAPDtouseineachvideo frame (APD4F)toimprovethedetectionresults.Besides,weimplementedtwotypesof APD4Fs, onebasedonaRandomForestClassifier(RFC-APD4F)andanotherinaMultilayer Perceptron (MLP-APD4F).Wedeveloped44APD4FscombiningsubsetsofsixAPDs.The most ofAPD4FoutperformedoftheindependentuseofallAPDs.Amultilayerperceptron- based APD4F,whichcombinesanMLP-APD,aNB-APD,andaLR-APD,presentedthebest performance, achievinganaccuracyof95.84%,arecallof99.28%andanF1scoreof96.07%. This researchalsoproposesasolutiontooptimizetheproblemofdetectingarmedpeople when theweaponisnotvisible.Therefore,weapplyrecurrentneuralnetworks,suchasLong Short TermMemory(LSTM),topredictthecoordinatesoftheguns.Inthisway,itispossible to haveapredictionofarmedpeopleatalltimes.ThemeasurementbetweentheYOLO handgun detectionboundingboxesandtheLSTMpredictionresultedinanIoUof65.23%. When thefirearmdetectionbytheobjectdetectorisinterrupted,theweapon’spositionis generated bytheLSTMmodelsthat,togetherwiththeAPDs,identifythearmedpeople. When theLSTMsdeliveredtheirpredictionstotheAPDs,theNB-APDdemonstratedthe best performance,achievinganaccuracyof80.93%.TheLSTMsallowedtheanalysisof 5,288 recordsofthetestvideothatcouldnotbeanalyzedbeforeduetothelackofknowledge of thegun’sposition.