Smart camera FPGA hardware implementation for semantic segmentation of wildfire imagery

Abstract

In the past few years, the more frequent occurrence of wildfires, which are a result of climate change, has devastated society and the environment. Researchers have explored various technologies to address this issue, including deep learning and computer vision solutions. These techniques have yielded promising results in semantic segmentation for detecting fire using visible and infrared images. However, implementing deep learning neural network models can be challenging, as it often requires energy-intensive hardware such as a GPU or a CPU with large cooling systems to achieve high image processing speeds, making it difficult to use in mobile applications such as drone surveillance. Therefore, to solve the portability problem, an FPGA hardware implementation is proposed to satisfy low power consumption requirements, achieve high accuracy, and enable fast image segmentation using convolutional neural network models for fire detection. This thesis employs a modified UNET model as the base model for fire segmentation. Subsequently, compression techniques reduce the number of operations performed by the model by removing filters from the convolutional layers and reducing the arithmetic precision of the CNN, decreasing inference time and storage requirements and allowing the Vitis AI framework to map the model architecture and parameters onto the FPGA. Finally, the model was evaluated using metrics utilized in prior studies to assess the performance of fire detection segmentation models. Additionally, two fire datasets are used to compare different data types for fire segmentation models, including visible images, a fusion of visible and infrared images generated by a GAN model, fine-tuning of the fusion GAN weights, and the use of visible and infrared images independently to evaluate the impact of visible-infrared information on segmentation performance.