Dynamic Analysis of Recurrent Neural Networks
الملخص
With the advancement in deep learning research, neural networks have become one of the most powerful tools for artificial intelligence tasks. More specifically, recurrent neural networks (RNNs) have achieved state-of-the-art in tasks such as hand-writing recognition and speech recognition. Despite the success of recurrent neural networks, how and why do neural nets work is still not sufficiently investigated. My work on the dynamical analysis of recurrent neural networks can help understand how the input features are extracted in the recurrent layer, how the RNNs make decisions, and how the chaotic dynamics of RNNs affects its behaviors. Firstly, I investigated the dynamics of recurrent neural networks as autonomous dynamical system in the experiment of a two-joint limb controlling task and compared the empirical result and the theoretical analysis. Secondly, I investigated the dynamics of non-autonomous recurrent neural networks on two benchmark tasks: sequential MNIST recognition task and DNA splice junction classification task. How the hidden states of long-short term memory (LSTM) and gated recurrent unit (GRU) cells learn new features and how the input sequence is extracted are demonstrated with experiments. Finally, based on the understanding of the external and internal dynamics of recurrent units, I proposed several algorithms for recurrent neural network compression. The algorithms demonstrate reasonable performance in compression ratio and are able to sustain the performance of the original models.
الكلمات المفتاحية
غير متوفر
التخصصات
رسائل وأطاريح مشابهة
- Generating Audio Using Recurrent Neural Networks Andrew Pfalz · 2018
- Exploiting temporal context in speech technologies using lstm recurrent neural networks Rubén Zazo Candil · 2018
- Harbour Porpoise Click Train Classification with LSTM Recurrent Neural Networks Filip Ärlemalm · KTH Royal Institute of Technology · 2017
- IMPLEMENTASI MODEL CONVOLUTIONAL RECURRENT NEURAL NETWORK (CRNN) DENGAN BIDIRECTIONAL LONG SHORT-TERM MEMORY (BI-LSTM) UNTUK PENGENALAN EMOSI PADA SUARA ADHIRAJASA M. RADITYA · Lampung University · 2025
- Speech recognition with the use of deep neural networks Δημήτρης Παπακωνσταντίνου · 2018
- Speech Reading with Deep Neural Networks Linnar Billman ; Johan Hullberg · KTH Royal Institute of Technology · 2018
