Semi-supervised learning using pseudo-labels: A case study in Northern Sámi ASR
الملخص
While Self-Supervised Learning has advanced Automatic Speech Recognition (ASR) for low-resource languages, limited labeled data remains a bottleneck, often preventing reasonable downstream performance. In line with this requirement, Semi-Supervised Learning approaches have gained prominence as providing a means to data-efficient methods that can be helpful, even with limited supervised data. This work investigates the usefulness of semi-supervised learning, specifically Pseudo-Labeling with self-supervised speech models for Northern Sámi, an under-resourced agglutinative language. Within a teacher–student framework, pseudo-labels are incorporated using two strategies: (a) filtering them through WER-based agreement between the models, and (b) using all pseudo-labels generated by a teacher model. Results show that pseudo-labels from the larger teacher foundation model can improve the downstream performance of a smaller one without additional labeled data, yielding 33.30–3.84% relative character error rate reduction (CERR) across three out-of-distribution test sets. The smaller student model is evaluated in a supervised setting (20 hours of labeled data) and semi-supervised settings with 20, 26 and 75 hours of pseudo-labeled data. Notably, the student model can benefit as much, or even more, from pseudo-labels as from labeled data. Further gains can be achieved when both annotated and pseudo-labels are combined in a two-stage fine-tuning process, rather than mixing all data in a single training phase.
الكلمات المفتاحية
غير متوفر
التخصصات
رسائل وأطاريح مشابهة
- A Study on Robustness, Adaptation, and Generalization for Visual and Multimodal Learning Xiasi Wang · 2026
- Self and semi-supervised Learning for vehicle make and model recognition in images Ricardo Nuno Alves Teixeira · 2024
- A study on Human Emotion Recognition in Video Images using Deep Learning Jargalsaikhan Orgil · 2022
- Semi-Supervised Deep Learning using Consistency-Based Methods for Segmentation of Medical Images Axel Rönnberg · KTH Royal Institute of Technology · 2020
- Efficient fine-tuning of Transformer Models Under Hardware Constraints : An Empirical Study of LoRA and Knowledge Distillation Sai Mounika Karasu · Dalarna University Library · 2026
- SISTEM PENILAIAN FORUM DISKUSI MAHASISWA PADA E�LEARNING BERBASIS MODEL BERT Ahmad Habibi ; Endina Putri Purwandari ; Widhia oktoeberza · University of Bengkulu · 2025
