غير محددة في المصدر 2026 بحاجة إلى مراجعة

Long Text Understanding with Transformer-based Models

زيارة المصدر الأصلي

الملخص

Transformer-based models have achieved remarkable success across a wide range of natural language processing tasks. However, their performance degrades substantially when processing long inputs, including multi-turn dialogues, lengthy documents, and sequences beyond the training context window. These limitations arise not only from computational constraints but also from structural misalignment, where local predictions drift from global semantics, document coherence is fragmented, and positional extrapolation disrupts pretrained attention patterns. This thesis investigates long text understanding in Transformer-based models through the perspective of preserving structural alignment under constrained context regimes. Three complementary frameworks are proposed. First, for multi-turn dialogue understanding, a multi-level semantic distillation framework aligns intent, slot, and domain knowledge across utterance-, token-, and dialogue-level representations using heterogeneous teacher supervision to improve semantic consistency and robustness. Second, for long document understanding, a semantic-preserving chunk-level representation method compresses documents into meaningful chunks through unsupervised keyphrase extraction while retaining fine-grained token information. Third, for length extrapolation beyond the training context window, a training-free inference-time method stabilizes positional generalization by reusing pretrained positional intervals and locally interpolating attention logits. Experiments on multi-turn dialogue benchmarks, long document understanding tasks, and long-context evaluation suites demonstrate the effectiveness and generalizability of the proposed methods. Collectively, these contributions establish a unified framework for robust long text understanding by preserving semantic coherence, structural consistency, and positional stability under context constraints.

الكلمات المفتاحية

غير متوفر

التخصصات

رسائل وأطاريح مشابهة