Multimodal deep learning for decoding visual and neural representation
Abstract
Multimodal deep learning combines different types of data, like images and brain signals. Traditional models often need lots of labeled examples and focus on one type of data, making them less able to handle real-world challenges. This thesis uses new deep learning and multimodal techniques to better process visual and brain data (Electroencephalogram), with three key projects. First, a new way to create image captions without labeled examples tackles mismatches between images and text. Second, it links images and EEG to classify objects by decoding brain activity from seeing images. Third, a three-part model combines images, words, and EEG data to understand how the brain processes what we see. Together, these methods advance multimodal learning, improving image captioning, EEG-based classification, and our understanding of how the brain works with visual information.
Keywords
Not available
Specialisations
Similar theses
- Multimodal Deep Learning for Visual Question Answering: Techniques and Challenges Oualid Ilyes Malfi · 2024
- Evaluating Federated Learning and Multimodal Data for Privacy-Aware Video-Based Fall Detection Dick Kuan ; Martin Dahrén · KTH Royal Institute of Technology · 2026
- Adaptive Deep Learning for Vision and Multimodal Understanding with Heterogeneous Data Youwei Liang · 2026
- Multimodal and explainable Learning using Fusion Architecture Through Exploration (FATE) approach Otmane Amel · 2026
- Data efficient deep multimodal learning Meng Shen · Nanyang Technological University · 2025
- Advancing Computational Pathology with Multimodal Deep Learning Md. Mamunur Rahaman · 2025
