In Progress

Multimodal Sensor Calibration - Targetless

Online automatic calibration using deep learning and common feature extraction

Overview

Sensor fusion is essential for autonomous driving and autonomous robots, and radar-camera fusion systems have gained popularity due to their complementary sensing capabilities. To address challenges with target-based calibration methods, we introduce a novel approach that leverages deep learning to extract a common feature from raw radar data (Range-Doppler-Angle) and camera images. Our method implicitly utilizes these common features to match identical objects from both data sources. This feature-based approach achieves online targetless calibration between radar and camera systems, with RANSAC and Levenberg-Marquardt optimization for accuracy and robustness.

Key Highlights

Autonomous Driving
Targetless Calibration
Deep Learning
Online Processing
Robust Optimization

Methodology

Deep learning-based common feature extraction from radar Range-Doppler-Angle data and camera images. Online calibration using feature matching and robust optimization.

Technologies

mmWave Radar Camera Deep Learning Common Feature Extraction RANSAC Levenberg-Marquardt

Project Details

Start Date: September 2024
Status: Active

Resources

Related Publications

CalibRefine: Deep Learning-Based Online Automatic Targetless LiDAR-Camera Calibration with Iterative and Attention-Driven Post-Refinement

L. Cheng, L. Guo, T. Zhang, T. Bang, A. Harris, M. Hajij, M. Sartipi, S. Cao

arXiv preprint, 2025

View Paper

Online Targetless Radar-Camera Extrinsic Calibration Based on the Common Features of Radar and Camera

L. Cheng, S. Cao

IEEE National Aerospace and Electronics Conference (NAECON), 2023

View Paper

Back to All Projects