No More Dirty Looks

The stakes in some engineering efforts are significantly higher than in others. While it might be alright if your gesture-controlled smart home automation system misfires from time to time, a self-driving vehicle that gets confused while it is out for a spin can lead to a deadly outcome. For this reason, these autonomous vehicles typically have a number of redundant systems to assist with navigation and obstacle avoidance. These systems may feature RGB depth cameras, LiDAR, and other sensing options to collect the most accurate information possible under a wide range of environmental conditions.

However, the fact that a brand new vehicle that just rolled off of the dealer’s lot performs flawlessly does not mean that it will continue to do so after it has spent some time operating under real-world conditions. LiDAR units, for example, are prone to malfunction over time as contaminants are introduced into the sensor’s cover. Unless this situation is noticed and quickly remedied, the vehicle will unknowingly be acting on inaccurate data, which may lead to collisions or other serious consequences.

As the primary systems of self-driving cars continue to improve in performance, it is the secondary systems that deal with situations such as this that will need greater attention. Researchers at the University of Bologna in Italy are actively developing a system called TinyLid that continually monitors LiDAR sensors for contamination. This proved to be a challenging task, as the algorithm needs to run on-vehicle, near the LiDAR sensor, to ensure that problems are caught immediately.

The team’s goal was to develop an algorithm that can classify the type of contaminant that is found on the cover of a LiDAR unit. By knowing the specific issue, it would be possible to suggest a solution that can correct the problem, perhaps even in an automated manner. Toward that goal, they evaluated a number of machine learning algorithms to determine which ones performed well enough, and were also sufficiently lightweight computationally to run at the edge, to be useful for real-world applications.

A RISC-V-based microcontroller unit called GAP8 was selected for the task as it is known to be ultra-efficient, highly performant, and to use very little energy, making it ideal for edge computing applications. A preexisting automotive LiDAR dataset, which specifically labels different types of contamination, was also located for use in training the algorithms. The tested algorithms included classic one-dimensional machine learning models, as well as more advanced two- and three-dimensional models.

The model that offered the best combination of performance and efficiency proved to be a lightweight two-dimensional convolutional neural network. This model was able to achieve a classification F1 score of 0.97. Furthermore, this result was achieved with inference times of only 2.575 milliseconds, making the algorithm suitable for real-time analyses. Resource utilization was shown to be quite light — only 6.8 percent of the microcontroller’s 512 KiB of L2 memory was required for operation.

As a next step, the researchers intend to test additional classifiers on somewhat beefier hardware that is equipped with GPUs to compare their performance with TinyLid. This sort of work will help to ensure that one day our self-driving vehicles will be nearly problem-free.