Learning long-term dependencies with gradient descent is difficult

The conjugate gradient method is an effective method for large-scale unconstrained optimization problems. Recent research has proposed conjugate gradient methods based on secant conditions to establish fast convergence of the methods. However, these ...

Conjugate gradient methods are widely used for solving large-scale unconstrained optimization problems because they do not need the storage of matrices. In this paper, we propose a general form of three-term conjugate gradient methods which always ...

Bidirectional recurrent neural network (BRNN) is a non-causal generalization of recurrent neural networks (RNNs). Due to the problem of vanishing gradients, BRNN cannot learn long-term dependencies efficiently with gradient descent. To tackle the long-...