Technical paper review on sensors & sensor fusion technology used in autonomous vehicles Essay

This summary report was written based on the knowledge that I obtained by reading two IEEE papers on sensor fusion technology mainly used for autonomous vehicles. The first paper gave me a general understanding of the various techniques used for sensor fusion and the second paper concentrated on using sensor fusion technology to detect hidden/obscured pedestrians. The author of the first paper made sure that we understand the functioning of each of the sensors like radar, camera, lidar used in autonomous vehicles before talking about the data fusion techniques of these sensors. In the introduction part of the paper, the author divides an autonomous car into four categories: the sensors section, perception section (data from the sensors is interpreted individually, combined and given to the next section), planning section (involves the vehicle behavior planning and also the path planning) and finally the control section (used to control the vehicle as per the planning done by previous section). In the second sub-section of the paper, the author introduces us to the important sensor functionalities: (i) Camera: it is analogous to the eyesight of humans and is excellent for texture interpretation. The main disadvantage is that each of the cameras installed produce at least 30-60 frames of the surroundings per second, hence a very high computational power is needed for processing the huge amount of data produced by all the cameras. But it is supposed to be cheaper in cost when compared to radar and lidar. The main applications of the camera include end-to-end autonomous driving, semantic segmentation, and perception. (ii) Radar (Radio Detection and Ranging): Doppler effect is used by radar to measure the velocity of other vehicles and also this principle helps in faster convergence of sensor fusion algorithms. Radars have good resistance against bad weather when compared to camera and lidar. The main disadvantage is that it has poor angular accuracy, even lesser processing speed than the camera and low resolution in the vertical direction. But it has several applications like blind spot warning, localization, adaptive cruise control, collision warning, and collision avoidance. Radars also have the capability to see underneath other vehicles, and spot obscured objects and buildings. (iii) Lidar (Light Detection and Ranging): Lidar makes use of infrared laser beam which is usually pulsed and such pulses are reflected back by objects in the form of a point cloud that helps in representing the objects. Lidars can detect static as well as identify moving objects and have better resolution than radars because of the large number of scan layers in the vertical direction, and the high density of lidar points per layer. The main disadvantage is the high cost, but current trends are moving in the direction to combine lidar with MEMS micromirror that can provide high-density point cloud at the high frame rate downsizing the cost and size of lidars. The third and most important sub-section of the paper is that of sensor fusion, where the author describes how combining data from various sensors help in making accurate decisions. In particular, the paper concentrates on sensor fusion for 3D object detection, occupancy grid mapping, and moving object detection. (i) 3D object detection: to get an optimal solution in terms of hardware complexity, a combination of lidar and camera is used for this purpose. The result of combining an RGB image and 3d point cloud is a 3d box hypothesis. Whereas by using novel approaches like neural networks we could avoid lossy input predictions as each of the point cloud and image are treated by a separate neural network and finally combined into a larger network to detect the object. (ii) Occupancy grid mapping: here the data from camera and lidar are combined for navigation and localization of autonomous vehicles in dynamic environments. The idea here is to create a grid map of the dynamic environment and apply a filter on all the grid cells except on those occupied by an obstacle (which is usually detected by the camera and lidar combination). (iii) Moving object detection and tracking: as this is a very challenging aspect, the data from all the sensors are fused to get better performance. To solve this problem, first, the object detection is done using radar and lidar. Then desired lidar point clouds are combined with a camera-based classifier, before combining all three sensor data together. The combination of all three sensor data helps in generating a list of moving objects. The low-level fusion (radar and lidar) solves mapping and localization. High-level fusion (all three sensor data) solves detection and classification. The intermediate fusion (camera and lidar) data is crucial in localization and tracking. These were a few of the important topics narrated in the first technical paper. The second technical paper chosen for review is a very short paper that proposes a radar-lidar sensor fusion for detecting obstructed pedestrian using a method called occluded depth generation and ROI (Region of Interest) fusion. The general idea behind this paper is that the lidar is used to detect only the outermost objects and the obstructed object (usually obstructed by the outermost object itself) is considered to be within an occluded depth which is detected using a radar. This obstructed object is then estimated to be a pedestrian candidate by means of a method called RADAR human doppler distribution. Finally, by combining the ROIs of both the sensors, the outermost pedestrian and the obstructed pedestrian are both detected. These two IEEE papers helped me understand the sensors and sensor fusion technology used in autonomous vehicles in a better way. This understanding would surely benefit for my upcoming project work. Reference paper 1: https://ieeexplore.ieee.org/document/8612054 Paper title: Sensors and Sensor Fusion in Autonomous Vehicles Authors: Jelena Kocić, Nenad Jovičić and Vujo Drndarević Conference name: 26th Telecommunications Forum (TELFOR) Date of Conference: 20-21 Nov. 2018 Reference paper 2: https://ieeexplore.ieee.org/document/8561087 Paper title: Radar-Lidar Sensor Fusion Scheme Using Occluded Depth Generation for Pedestrian Detection Authors: Seong Kyung Kwon, S. H. Son, Eugin Hyun, Jin-Hee Lee and Jonghun Lee Conference name: International Conference on Computational Science and Computational Intelligence (CSCI) Date of Conference: 14-16 Dec. 2017