Development of algorithm and architecture design of camera-radar perception system for agriculture & construction machinery

In a smart farming & construction automated working scenario, the machinery needs accurate perception resources to autonomously perform the tasks and even stop them when an object is frontal, or rear detected. Automotive devices like lidars, radars, cameras and other vision systems are being integrated into road vehicles over the last years. There is profound literature available only on on-road area; there exist many other challenges when dealing with off-road vehicles such as agriculture & construction machinery. The understanding of sensor fusion levels to develop an advanced, cheap, robust, and reliable Camera-Radar perception system output is proposed. Radar provides speed with great precision, and it has good performance vs poor weather conditions. Camera delivers high resolution images, color, and depth information. The architecture design is driven by a low-level fusion, it means that relies on the choice of simple mmWave radar and stereo camera together with a smart ECU (Electronic Control Unit). As a result, real-time colored 3D cloud point information is obtained with high resolution from the camera and radar. Object range is also given by the radar. The proposed objective is to carry out testing through real experiments, understand camera+radar object detection capabilities along different conditions. Also, the aim of this investigation is to increase repertory of heavy machinery environmental expectations aligned to most common scenarios and how they would affect perception system output.