Association knowledge in natural language learning
Abstract
Association is an important feature of natural language originates from the human's cognitive ability to associate concepts.It is the key to unveiling the computational mechanism of natural language, which is closely related to the research of natural language processing (NLP).However, the current dominant learning paradigm, which is based on neural models that combine distributed representations with probabilistic modeling, demonstrates insufficient capabilities in modeling associations in natural language.To compensate for the deficiency, we mathematically formulate the concept of Association Knowledge as the joint distribution over the probabilities of instances and establish a general methodology to incorporate association knowledge into the training architecture of neural models.We delve into Association Knowledge through a series of case studies across various dimensions, including associations among types of knowledge, languages, instances and unstructured information.These case studies span both smaller neural models and large language models.Through our investigations, we demonstrate that explicitly integrating Association Knowledge into neural architectures markedly improves model performance and efficiency.This enhanced capability is evident in diverse scenarios, from improving event detection in lifelong learning settings and facilitating robust cross-lingual translations, to enhancing the detection of long-tail mentions and refining the updates in large language models.Collectively, our findings underscore the pivotal role of Association Knowledge in advancing the state of NLP by fostering more robust and knowledge-aware neural models.vi
Keywords
Not available
Specialisations
Similar theses
- Multidimensional Transfer Learning in Natural Language Processing: Domain, Task, Language, and Expertise Adaptation Nghia Ngo · University of Oregon · 2026
- Deep learning and natural language processing for computer aided diagnosis Sadam Hussain · Tecnológico de Monterrey · 2025
- Neural transfer learning for natural language processing Sebastian Ruder · Dublin Institute of Technology · 2019
- Pre-training model based on the transfer learning in natural language processing Jiayi Tang · Nanyang Technological University · 2019
- Deep Learning Methods for Natural Language Processing Considering Writers and Background World Knowledge 大輔 大葉 · 2023
- Inductive and Transductive Learning with Vision-Language Models in Low-Shot Regimes Maxime Zanella · 2026
