Not stated in source 2022 Needs review

A deep learning odyssey on structured natural language processing

Abstract

Availability of immense amount of unstructured text has surged the demand for intelligent natural language processing (NLP) systems. Specifically, these systems are expected to represent, extract, transfer and utilize knowledge in order to be successful on language related problems. In this work, we have directed our efforts to explore various forms of knowledge, especially linguistic and latent structures, available for a NLP system. Additionally, we investigate the significance of aforementioned structures in various language relevant tasks ranging from highly researched semantic matching to less explored abstract text summarization, and many more. In chapter 2, we study the interaction between explicit linguistic structures and implicit structure revealed by attention mechanisms. This investigation on various semantic matching datasets, especially question-answering, display significant patterns of substitutability between the two types of structures (explicit and implicit), and may affect other modalities. As a result, we develop a multi-view progressive attention mechanism which is general enough to operate on various linguistic structures of text. Moving further, to represent and extract latent structure from a collection of documents, we propose a family of vector-quantization-based topic models (VQ-TMs) in Chapter 3. Specifically, VQ-TMs consider dense and global topic embeddings which are homomorphic to word embeddings. Moreover, the family exploits vector-quantization (VQ) technique to promote discreteness in the overall architecture. Lastly, the learned topics are successfully transferred to a different task of code-generation. VQ-TMs exploit VQ technique while learning topics representation. Although VQ method aids in learning discrete topic embeddings, yet, the method is built upon a geometric intuition. Moreover, VQ-TMs utilizes the learned topic representations as reference knowledge in a downstream task. Some natural questions arise. Is it possible to exploit probabilistic intuition to infuse discreteness while learning topics representation? Additionally, can the learned topics be leveraged to control sentence generation and augment the performance in aspect-aware item recommendation? In order to push the envelope further, we explore these ideas in Chapter 4.

Keywords

Not available

Specialisations

Similar theses