Foundation models and large language models for autonomous driving

The deployment of autonomous driving systems (ADS) is advancing globally, yet long-tail scenarios remain a major challenge. This thesis explores integrating foundation models (FMs) and large language models (LLMs) into ADS to enhance robustness, interpretability, and security. A multi-agent LLM framework is proposed for safe vehicle–LLM interaction, alongside an intent-aware pedestrian reasoning system. A generative framework combining LLMs, diffusion, and state-space models enables fine-grained 3D scene completion and traffic reporting. Further, a multimodal disengagement prediction model and secure path-planning architecture are developed. Collectively, this work provides a roadmap for embedding FMs into modular ADS pipelines.