Long short-term memory

The compact forms of the equations for the forward pass of an LSTM cell with a forget gate are:[1][15]

${\displaystyle {\begin{aligned}f_{t}&=\sigma _{g}(W_{f}x_{t}+U_{f}h_{t-1}+b_{f})\\i_{t}&=\sigma _{g}(W_{i}x_{t}+U_{i}h_{t-1}+b_{i})\\o_{t}&=\sigma _{g}(W_{o}x_{t}+U_{o}h_{t-1}+b_{o})\\{\tilde {c}}_{t}&=\sigma _{c}(W_{c}x_{t}+U_{c}h_{t-1}+b_{c})\\c_{t}&=f_{t}\odot c_{t-1}+i_{t}\odot {\tilde {c}}_{t}\\h_{t}&=o_{t}\odot \sigma _{h}(c_{t})\end{aligned}}}$

where the initial values are ${\displaystyle c_{0}=0}$ and ${\displaystyle h_{0}=0}$ and the operator ${\displaystyle \odot }$ denotes the Hadamard product (element-wise product). The subscript ${\displaystyle t}$ indexes the time step.

• ${\displaystyle x_{t}\in \mathbb {R} ^{d}}$: input vector to the LSTM unit
• ${\displaystyle f_{t}\in {(0,1)}^{h}}$: forget gate's activation vector
• ${\displaystyle i_{t}\in {(0,1)}^{h}}$: input/update gate's activation vector
• ${\displaystyle o_{t}\in {(0,1)}^{h}}$: output gate's activation vector
• ${\displaystyle h_{t}\in {(-1,1)}^{h}}$: hidden state vector also known as output vector of the LSTM unit
• ${\displaystyle {\tilde {c}}_{t}\in {(-1,1)}^{h}}$: cell input activation vector
• ${\displaystyle c_{t}\in \mathbb {R} ^{h}}$: cell state vector
• ${\displaystyle W\in \mathbb {R} ^{h\times d}}$, ${\displaystyle U\in \mathbb {R} ^{h\times h}}$ and ${\displaystyle b\in \mathbb {R} ^{h}}$: weight matrices and bias vector parameters which need to be learned during training

where the superscripts ${\displaystyle d}$ and ${\displaystyle h}$ refer to the number of input features and number of hidden units, respectively.

• ${\displaystyle \sigma _{g}}$: sigmoid function.
• ${\displaystyle \sigma _{c}}$: hyperbolic tangent function.
• ${\displaystyle \sigma _{h}}$: hyperbolic tangent function or, as the peephole LSTM paper[18][19] suggests, ${\displaystyle \sigma _{h}(x)=x}$.

A peephole LSTM unit with input (i.e. ${\displaystyle i}$), output (i.e. ${\displaystyle o}$), and forget (i.e. ${\displaystyle f}$) gates

The figure on the right is a graphical representation of an LSTM unit with peephole connections (i.e. a peephole LSTM).[18][19] Peephole connections allow the gates to access the constant error carousel (CEC), whose activation is the cell state.[18] ${\displaystyle h_{t-1}}$ is not used, ${\displaystyle c_{t-1}}$ is used instead in most places.

${\displaystyle {\begin{aligned}f_{t}&=\sigma _{g}(W_{f}x_{t}+U_{f}c_{t-1}+b_{f})\\i_{t}&=\sigma _{g}(W_{i}x_{t}+U_{i}c_{t-1}+b_{i})\\o_{t}&=\sigma _{g}(W_{o}x_{t}+U_{o}c_{t-1}+b_{o})\\c_{t}&=f_{t}\odot c_{t-1}+i_{t}\odot \sigma _{c}(W_{c}x_{t}+b_{c})\\h_{t}&=o_{t}\odot \sigma _{h}(c_{t})\end{aligned}}}$

Each of the gates can be thought as a "standard" neuron in a feed-forward (or multi-layer) neural network: that is, they compute an activation (using an activation function) of a weighted sum. ${\displaystyle i_{t},o_{t}}$ and ${\displaystyle f_{t}}$ represent the activations of respectively the input, output and forget gates, at time step ${\displaystyle t}$.

The 3 exit arrows from the memory cell ${\displaystyle c}$ to the 3 gates ${\displaystyle i,o}$ and ${\displaystyle f}$ represent the peephole connections. These peephole connections actually denote the contributions of the activation of the memory cell ${\displaystyle c}$ at time step ${\displaystyle t-1}$, i.e. the contribution of ${\displaystyle c_{t-1}}$ (and not ${\displaystyle c_{t}}$, as the picture may suggest). In other words, the gates ${\displaystyle i,o}$ and ${\displaystyle f}$ calculate their activations at time step ${\displaystyle t}$ (i.e., respectively, ${\displaystyle i_{t},o_{t}}$ and ${\displaystyle f_{t}}$) also considering the activation of the memory cell ${\displaystyle c}$ at time step ${\displaystyle t-1}$, i.e. ${\displaystyle c_{t-1}}$.

The single left-to-right arrow exiting the memory cell is not a peephole connection and denotes ${\displaystyle c_{t}}$.

The little circles containing a ${\displaystyle \times }$ symbol represent an element-wise multiplication between its inputs. The big circles containing an S-like curve represent the application of a differentiable function (like the sigmoid function) to a weighted sum.

Peephole convolutional LSTM.[20] The ${\displaystyle *}$ denotes the convolution operator.

${\displaystyle {\begin{aligned}f_{t}&=\sigma _{g}(W_{f}*x_{t}+U_{f}*h_{t-1}+V_{f}\odot c_{t-1}+b_{f})\\i_{t}&=\sigma _{g}(W_{i}*x_{t}+U_{i}*h_{t-1}+V_{i}\odot c_{t-1}+b_{i})\\c_{t}&=f_{t}\odot c_{t-1}+i_{t}\odot \sigma _{c}(W_{c}*x_{t}+U_{c}*h_{t-1}+b_{c})\\o_{t}&=\sigma _{g}(W_{o}*x_{t}+U_{o}*h_{t-1}+V_{o}\odot c_{t}+b_{o})\\h_{t}&=o_{t}\odot \sigma _{h}(c_{t})\end{aligned}}}$

Many applications use stacks of LSTM RNNs[22] and train them by connectionist temporal classification (CTC)[23] to find an RNN weight matrix that maximizes the probability of the label sequences in a training set, given the corresponding input sequences. CTC achieves both alignment and recognition.

Sometimes, it can be advantageous to train (parts of) an LSTM by neuroevolution[24] or by policy gradient methods, especially when there is no "teacher" (that is, training labels).

There have been several successful stories of training, in a non-supervised fashion, RNNs with LSTM units.

In 2018, Bill Gates called it a "huge milestone in advancing artificial intelligence" when bots developed by OpenAI were able to beat humans in the game of Dota 2.[11] OpenAI Five consists of five independent but coordinated neural networks. Each network is trained by a policy gradient method without supervising teacher and contains a single-layer, 1024-unit Long-Short-Term-Memory that sees the current game state and emits actions through several possible action heads.[11]

In 2018, OpenAI also trained a similar LSTM by policy gradients to control a human-like robot hand that manipulates physical objects with unprecedented dexterity.[10]

In 2019, DeepMind's program AlphaStar used a deep LSTM core to excel at the complex video game Starcraft II.[12] This was viewed as significant progress towards Artificial General Intelligence.[12]