Sensor data fusion for a mobile robot using a neural network algorithm

Mobile robots must be capable to obtain an accurate map of their surroundings and move within it. To detect different materials that might be undetectable to one sensor but not others it is necessary to have at least two sensors, with this is possible to generate a 2D occupancy map that is as close to reality as possible. In this thesis, an artificial neural network is used to fuse data from a tri-sensor (Intel RealSense Stereo Camera, 2D 360° LiDAR-Light Detection and Ranging Sensor and an HC-SR04 Ultrasonic Sensor) setup capable of detecting glass, polished metals, brick walls, wooden panels and other materials typically found in indoor environments. When a map is to be compiled out of different sensor’s data, it is necessary to implement a preprocessing scheme to filter all the outliers in the data for each sensor. Then, run a data fusion algorithm to integrate all the information into a single, more accurate 2D map that considers all sensor’s information. The Robotis Turtlebot 3 Waffle Pi robot is used as an experimental platform along with Robotic Operating System as the main Human Machine Interface to implement the algorithms. Test results show that with the fusion algorithm implemented, it is possible to detect glass and other obstacles invisible to the LiDAR with an estimated root-mean-square error of 4 cm with multiple sensor configurations.