Feature Selection and Transfer Learning in Network Data: Enhancing Anomaly Detection with Zero-Shot and Few-Shot Learning
The exponential growth of high-dimensional data across domains such as bioinformatics, healthcare, finance, and image processing has heightened the need for effective feature selection (FS) methods. These techniques improve model performance by identifying rel…
Towards the Renormalization of Transformer Language Models
Transformer language models (LMs), the cornerstone of the popular large language models (LLMs), have revolutionised the fields of artificial intelligence (AI) and natural language processing (NLP). However, we still understand relatively little about their com…
Transformer Block Coupling and Self-Attention Flows for Token Evolution
Transformers and Large Language Models have achieved incredible performance in deep learning, especially in natural language processing, and a precise understanding of the internal mechanisms driving their success is essential. We first provide an empirical an…
Higher Order Transformers With Kronecker-Structured Attention
Modern datasets are increasingly high-dimensional and multiway, naturally represented as tensors with multiple indexed variables. Such data arise in numerous applications, including multivariate time series forecasting, 3D medical imaging, and climate modeling…
Transformers and other Attention-Based Algorithms for Biomedicine
[EN] This research explores the application of Transformer-based models and Hybrid Attention Mechanisms across three critical biomedical tasks: meningioma segmentation, epileptic seizure detection, and pathogenicity prediction of genomic variants. For meningio…
Attention Mechanisms in Transformers: A Comparative Survey and Structural Enhancements to Linear Attention
This master’s thesis explores the trade-off between computational efficiency and modeling performance in Transformer architectures by studying and improving attention mechanisms. The work consists of two parts: (1) a comparative empirical study evaluating five…
Hybrid Deep Learning Model: LLM-Enhanced Linear Self-Attention Transformer-CNN for Stock Price Prediction
The unpredictability and inherent complexities of financial markets pose persistent challenges in accurately forecasting stock prices. Recent innovations in deep learning methodologies, specifically the integration of technical analysis with linguistic process…
Leveraging Self-Attention in Transformer Models for Improved Retail Demand Forecasting
Inventory management and operation planning in retail sector involve accurate demand forecasting as a key element of the process. Linear models and tree-based models are typical examples of traditional forecasting models that may not work well in capturing com…
Incorporation of deep learning strategies to improve QOS in VXLAN-based networks
This study examines how to improve Quality of Service (QoS) management in VXLAN-based overlay networks by integrating deep learning (DL) techniques. Due to issues like fluctuating workloads, complicated virtualized settings, and increased data traffic, traditi…
