Not stated in source 2024 Needs review

Multimodal deep learning for decoding visual and neural representation

Abstract

Multimodal deep learning combines different types of data, like images and brain signals. Traditional models often need lots of labeled examples and focus on one type of data, making them less able to handle real-world challenges. This thesis uses new deep learning and multimodal techniques to better process visual and brain data (Electroencephalogram), with three key projects. First, a new way to create image captions without labeled examples tackles mismatches between images and text. Second, it links images and EEG to classify objects by decoding brain activity from seeing images. Third, a three-part model combines images, words, and EEG data to understand how the brain processes what we see. Together, these methods advance multimodal learning, improving image captioning, EEG-based classification, and our understanding of how the brain works with visual information.

Keywords

Not available

Specialisations

Similar theses