Hybrid Optimization and Learning-Based Techniques for Autonomous Driving
الملخص
Autonomous driving has progressed rapidly in recent years, driven largely by advances in machine learning, computer vision, and probabilistic optimization methods. Despite this progress, the deployment of autonomous systems in complex, safety-critical real- world settings such as urban driving, dynamic traffic scenes, or large-scale outdoor environments remains a major open challenge. Real-world autonomy requires systems that can calibrate and localize themselves reliably, reason about their surroundings with minimal human supervision, and continuously adapt to changing environments. This thesis develops a set of methods that jointly address these challenges by combining geometric optimization with learning-based perception, and by designing label-efficient and self-supervised systems that scale to real-world deployment. In the first part, we study hybrid optimization methods for calibration and localiza- tion, focusing on the tight coupling of classical geometry with learned perception cues. We introduce a fully automatic camera–LiDAR calibration framework that removes the need for targets, initial guesses, or manual intervention by jointly exploiting motion cues and deep 2D–3D correspondences. Building on this idea, we propose a sparse-map localization system that integrates uncertainty-aware semantic predictions into a pose graph, enabling robust operation even in highly occluded urban environments with minimal map information. The second part focuses on online semantic Bird’s-Eye-View (BEV) mapping, where the goal is to infer a semantic map of the surrounding scene directly from frontal-view images in real time. We first investigate multi-modal fusion through BEVCar, which combines cameras and radar to obtain long-range, all-weather BEV maps using low- cost sensor setups. We then reduce the reliance on BEV annotations with SkyEye, a self-supervised method that learns BEV semantics from monocular images while using only frontal-view semantic labels as supervision. Finally, LetsMap removes this remaining supervision by jointly learning geometry and scene representation from raw RGB videos alone, achieving state-of-the-art label-efficient BEV mapping with only a tiny fraction of annotated BEV data. The third part addresses the challenge that modern perception systems rely on thousands of densely annotated images which is a major bottleneck for deploying autonomy at scale. We study how to drastically reduce this dependence by developing label-efficient and continually adapting panoptic segmentation methods that can be trained from data obtained within hours rather than months. SPINO shows that combining foundation-model features with only a handful of labeled images is sufficient to produce high-quality panoptic pseudo-labels at scale. PASTEL extends this idea with normalized-cut–based instance separation and targeted self-training, achieving even stronger performance under strict annotation budgets. Finally, CoDEPS enables continual test-time learning for both depth and panoptic segmentation through a geometry-aware mixing strategy and replay-based stabilization, allowing models to adapt to new environments during deployment without forgetting previous ones. Across these contributions, the thesis shows that combining geometric structure with modern learning techniques enables perception systems that are more robust, more data-efficient, and more adaptable than traditional supervised approaches. We demonstrate these methods extensively on diverse real-world driving datasets and robotic platforms, highlighting their potential to bring scalable, reliable autonomy closer to widespread deployment.
الكلمات المفتاحية
غير متوفر
التخصصات
رسائل وأطاريح مشابهة
- Lidar-based scene understanding for autonomous driving using deep learning Víctor Vaquero Gómez · Universitat Politècnica de Catalunya · 2020
- Leveraging deep learning for visual understanding of videos Clement Xian Ren Tan · Nanyang Technological University · 2024
- Distributed Intelligence for 3D Object Tracking in Autonomous Vehicles, A Multi-Sensor Approach to Object Tracking and Planning for Self-Driving Cars, Real-Time 3D Perception and Path Planning in Autonomous Systems,,Vision-Based Localization and Distributed Architecture for Driverless Cars,A Hybrid Deep Learning Framework for Object Detection and Trajectory Planning in Autonomous Vehicles Manas Madine · University of Massachusetts Amherst · 2023
- Self-Supervised Learning for Geometry Huangying Zhan · University of Adelaide · 2020
- Interaction-Aware Maneuver and Lane Change Prediction Using Deep Learning Architectures for Autonomous Driving Petrit Rama · Kaiserslautern University of Technology · 2026
- Static and Dynamic Affordance Learning in Vision-based Direct Perception for Autonomous Driving Jean Marie Uwabeza Vianney · University of Waterloo · 2020
