Tag Archives: Recurrent Neural Networks

Neural Network Papers

Table of Contents

  1. Surveys
  2. Datasets
  3. Programming Frameworks
  4. Learning to Compute
  5. Natural Language Processing
  6. Convolutional Neural Networks
  7. Recurrent Neural Networks
  8. Convolutional Recurrent Neural Networks
  9. Autoencoders
  10. Restricted Boltzmann Machines
  11. Biologically Plausible Learning
  12. Supervised Learning
  13. Unsupervised Learning
  14. Reinforcement Learning
  15. Theory
  16. Quantum Computing
  17. Training Innovations
  18. Numerical Optimization
  19. Numerical Precision
  20. Hardware
  21. Cognitive Architectures
  22. Motion Planning
  23. Computational Creativity
  24. Cryptography
  25. Distributed Computing
  26. Clustering



Programming Frameworks

Learning to Compute

Natural Language Processing

Word Vectors

Sentence and Paragraph Vectors

Character Vectors

Sequence-to-Sequence Learning

Language Understanding

Question Answering, and Conversing



Convolutional Neural Networks

Recurrent Neural Networks

Convolutional Recurrent Neural Networks


Restricted Boltzmann Machines

Biologically Plausible Learning

Supervised Learning

Unsupervised Learning

Reinforcement Learning


Quantum Computing

Training Innovations

Numerical Optimization

Numerical Precision


Cognitive Architectures

Motion Planning

Computational Creativity


Distributed Computing


Opinion Mining with Deep Recurrent Neural Networks

WhitePaper: A hypothesize-and-verify framework for Text Recognition using Deep Recurrent Neural Networks

A hypothesize-and-verify framework for Text Recognition using Deep Recurrent Neural Networks

Deep LSTM is an ideal candidate for text recognition. However text recognition involves some initial image processing steps like segmentation of lines and words which can induce error to the recognition system. Without segmentation, learning very long range context is difficult and becomes computationally intractable. Therefore, alternative soft decisions are needed at the pre-processing level. This paper proposes a hybrid text recognizer using a deep recurrent neural network with multiple layers of abstraction and long range context along with a language model to verify the performance of the deep neural network. In this paper we construct a multi-hypotheses tree architecture with candidate segments of line sequences from different segmentation algorithms at its different branches. The deep neural network is trained on perfectly segmented data and tests each of the candidate segments, generating unicode sequences. In the verification step, these unicode sequences are validated using a sub-string match with the language model and best first search is used to find the best possible combination of alternative hypothesis from the tree structure. Thus the verification framework using language models eliminates wrong segmentation outputs and filters recognition errors.

WhitePaper: Gated Feedback Recurrent Neural Networks

Click to Download WhitePaper

In this work, we propose a novel recurrent neural network (RNN) architecture. The proposed RNN, gated-feedback RNN (GF-RNN), extends the existing approach of stacking multiple recurrent layers by allowing and controlling signals flowing from upper recurrent layers to lower layers using a global gating unit for each pair of layers. The recurrent signals exchanged between layers are gated adaptively based on the previous hidden states and the current input. We evaluated the proposed GF-RNN with different types of recurrent units, such as tanh, long short-term memory and gated recurrent units, on the tasks of character-level language modeling and Python program evaluation. Our empirical evaluation of different RNN units, revealed that in both tasks, the GF-RNN outperforms the conventional approaches to build deep stacked RNNs. We suggest that the improvement arises because the GF-RNN can adaptively assign different layers to different timescales and layer-to-layer interactions (including the top-down ones which are not usually present in a stacked RNN) by learning to gate these interactions.

RecurrentJS: Deep Recurrent Neural Networks and LSTMs in Javascript.

via karpathy/recurrentjs · GitHub.

RecurrentJS is a Javascript library that implements:

  • Deep Recurrent Neural Networks (RNN)
  • Long Short-Term Memory networks (LSTM)
  • In fact, the library is more general because it has functionality to construct arbitrary expression graphs over which the library can perform automatic differentiation similar to what you may find in Theano for Python, or in Torch etc. Currently, the code uses this very general functionality to implement RNN/LSTM, but one can build arbitrary Neural Networks and do automatic backprop.