Large Language Model Evaluation
الملخص
Large language models (LLMs) have become foundational resources in the natural language processing field.The capabilities of LLMs in inference, natural language understanding (NLU), and their pre-trained knowledge are widely utilized in various applications.To fully leverage these capabilities, it is essential to properly evaluate them.However, it is very challenging to evaluate LLMs accurately.For example, when evaluating an LLM's inference capabilities to infer unseen facts from seen ones, it may rely on memorized knowledge from pre-training rather than pure inference, due to the pre-training datasets being curated from web data, which can lead to contamination with related knowledge.Therefore, traditional evaluation frameworks for inference capabilities struggle to capture the pure inference capability of LLMs, independent of their pre-trained knowledge.Furthermore, when evaluating the NLU capabilities of LLMs, we should consider the stability in outputs for various input prompts.Finally, constructing evaluation datasets involves significant costs of financial, time, and human labor.Especially when focusing on multilingual datasets, there is a limitation in the availability of annotators, so most datasets are machine-translated from English.However, due to cultural differences and noise introduced by translation, these evaluations are unsuitable for language-specific evaluation.This disserta-
الكلمات المفتاحية
غير متوفر
التخصصات
رسائل وأطاريح مشابهة
- Can Large Language Models become Innovation Experts? : Emulating Expert Evaluation of Innovation in Startups with Large Language Models Ricarda Nettinger · University of Turku · 2026
- Large Language Model Evaluation Corredor Casares, Joel · Universitat Politècnica de Catalunya · 2025
- Educational question generation and evaluation with large language models Chun Yi Louis Bo · 2024
- Learning to adapt: From structured domain adaptation to efficient in-context learning for large language models Zihao Xu · Rutgers University · 2026
- On the Safety Alignment of Large Language Models Mengxuan Hu · 2026
- Documenting and Improving Prompts for Large Language Models Amandine M. Caut · Dalarna University Library · 2026
