Natural Language Processing for Discussion Platforms

Online participation platforms and digital discussion spaces have become central arenas for public discourse, particularly in times of crisis such as the COVID-19 pandemic. While these environments expand opportunities for civic engagement, they are frequently characterized by incivility, misinformation, and unstructured exchanges. Drawing on the concept of deliberation, defined by the core dimensions of rationality, civility, interactivity and equality, this thesis investigates how Natural Language Processing (NLP) methods can be leveraged to foster more respectful, reasoned, and inclusive online discussions. The dissertation addresses five central aspects: (i) the current approaches in applying NLP to online discussion platforms, (ii) the optimization of smaller language models for efficient real-time deployment, (iii) the integration of NLP-based interventions into discussion platforms, (iv) user perceptions of Artificial Intelligence (AI)-supported moderation and assistance, and (v) the measurable effects of such interventions on discussion quality. First, a systematic review provides a comprehensive overview of existing NLP applications in political and civic online discussions, identifying established approaches and tasks that could potentially be used to support discussions (see Chapter 4). Second, the thesis develops and evaluates methods to improve the efficiency and effectiveness of small Transformer-based models, particularly Bidirectional Encoder Representations from Transformers (BERT), for tasks relevant to online deliberation. This includes adapting pre-training strategies for argument similarity detection (ArgueBERT, see Chapter 5) and optimizing fine-tuning procedures for classification performance (MaxPoolBERT, see Chapter 6). In addition, the Additive deliberative Quality score with Adapters (AQuA) is introduced to assess the deliberative quality of user contributions (see Chapter 7). This model, along with a comment recommendation module based on stance detection, has been integrated into an online discussion platform and tested in a large-scale, real-time experiment (see Chapter 8). The findings provide insights into the technical integration of AI-supported tools into online discussion platforms, as well as into users’ perceptions of such systems and their observed effects in a specific participatory context. By combining perspectives from computer science, political science, and communication science, this interdisciplinary work contributes to the development of AI-supported tools that aim to support deliberative processes in digital public spheres.