The Geometry of Concepts in Large Language Models
الملخص
This dissertation develops a mathematical foundation for the linear representation hypothesis, the informal idea that high-level semantic concepts are encoded as directions in the representation spaces of large language models. Despite growing empirical evidence, a rigorous formulation of what linear representation means and how the geometry of representation spaces relates to the semantics of language has been lacking. This work addresses this gap through three interrelated studies. First, we formalize the linear representation of binary concepts and introduce the causal inner product, which connects orthogonality to causal separability. Second, we extend the framework to categorical concepts and show that hierarchical relations are encoded through orthogonality. Third, by identifying the geometry of representation spaces as a Bregman (dually flat) geometry induced by KL divergence, we develop dual steering, a method that modifies a target concept while minimizing off-target interference. Together, these contributions provide a unified framework for understanding how semantic structure is encoded in the geometry of representations. They offer both theoretical insights into AI interpretability and practical tools for controlled model steering.
الكلمات المفتاحية
غير متوفر
التخصصات
رسائل وأطاريح مشابهة
- Learning to adapt: From structured domain adaptation to efficient in-context learning for large language models Zihao Xu · Rutgers University · 2026
- Solving Graph Problems with Large Language Models Angelo Zangari · University of Illinois Chicago · 2026
- GENERATIVE LARGE LANGUAGE MODELS FOR TRANSPARENT ARTIFICIAL INTELLIGENCE IN CLINICAL RESEARCH: ENHANCING INTERPRETABILITY THROUGH APPRAISAL AND EXPLANATION Fangwen Zhou · McMaster University · 2025
- Short Text Clustering with Large Language Models Justin Miller · The University of Sydney · 2025
- Foundation models and large language models for autonomous driving Xiangrui Kong · 2025
- Large language models for causal inference Omar Ait Lhouss · 2025
