In Progress

Multimodal Sensor Calibration - Targetless

Online automatic calibration using deep learning and common feature extraction

Overview

Sensor fusion is essential for autonomous driving and autonomous robots, and radar-camera fusion systems have gained popularity due to their complementary sensing capabilities. To address challenges with target-based calibration methods, we introduce a novel approach that leverages deep learning to extract a common feature from raw radar data (Range-Doppler-Angle) and camera images. Our method implicitly utilizes these common features to match identical objects from both data sources. This feature-based approach achieves online targetless calibration between radar and camera systems, with RANSAC and Levenberg-Marquardt optimization for accuracy and robustness.

Key Highlights

Autonomous Driving
Targetless Calibration
Deep Learning
Online Processing
Robust Optimization

Methodology

Deep learning-based common feature extraction from radar Range-Doppler-Angle data and camera images. Online calibration using feature matching and robust optimization.