Comparison of sequence classification techniques with BERT for named entity recognition
الملخص
This thesis takes its starting point from the recent advances in Natural Language Processing being developed upon the Transformer model. One of the significant developments recently was the release of a deep bidirectional encoder called BERT that broke several state of the art results at its release. BERT utilises Transfer Learning to improve modelling language dependencies in texts. BERT is used for several different Natural Language Processing tasks, this thesis looks at Named Entity Recognition, sometimes referred to as sequence classification. This thesis compares the model architecture as it was presented in its original paper with a different classifier in the form of a Conditional Random Field. BERT was evaluated on the CoNLL-03 dataset, based on English news articles published by Reuters. The Conditional Random Field classifier overall outperforms the original Feed Forward classifier on the F1-score metric with a small margin of approximately 0.25 percentage points. While the thesis fails to reproduce the original report’s results it compares the two model architectures across the hyperparameters proposed for fine-tuning. Conditional Random Fields proves to perform better scores at most hyperparameter combination and are less sensitive to which parameters were chosen, creating an incentive for its use by reducing the effect of parameter search compared to a Feed Forward layer as the classifier. Comparing the two models shows a lower variance in the results for Conditional Random Fields.
الكلمات المفتاحية
غير متوفر
التخصصات
رسائل وأطاريح مشابهة
- Analysis of fine-tuning techniques in text classification Tobias Gonçalves Pires · 2024
- Microbial named entity recognition using BERT models Brian K Rao · University of Illinois at Urbana-Champaign · 2022
- Automating news classification with large language models: Exploring fine-tuning, dataset size, and architecture Yesilyurt, Burcu · 2025
- Enhancing Domain-Specific Automated Audio Captioning: A Study on Adaptation Techniques and Transfer Learning Charith Raghavaraju · Tampere University · 2023
- Multimodal Integration for Natural Language Classification and Generation Zhihao Zhang · The University of Sydney · 2023
- Cyberbullying identification and classification using transformer models Wachiraporn Tapaopong · 2023
