LSTM vs GRU gating and trade-offs

WHAT THIS TESTS This probes your understanding of recurrent network gating, why gates were introduced to combat vanishing gradients, and how LSTM and GRU differ structurally and in cost.

A GOOD ANSWER COVERS Both LSTMs and GRUs are gated recurrent units designed to let gradients flow across many time steps, solving the vanishing-gradient problem of vanilla RNNs. An LSTM maintains a separate cell state alongside the hidden state and uses three gates: a forget gate that decides what to drop from the cell, an input gate that decides what new information to write, and an output gate that controls what the cell exposes as the hidden state. This gives fine-grained, expressive control over memory. A GRU simplifies this: it merges the cell and hidden state into one, and uses two gates, an update gate that blends keeping old state versus accepting new, and a reset gate that controls how much past state influences the candidate. Consequently a GRU has fewer parameters and matrix operations per step, so it trains faster, uses less memory, and generalizes better on smaller datasets. LSTMs, with more capacity, can sometimes capture very long or complex dependencies more precisely given enough data. Empirically performance is often comparable, so the choice depends on data size and compute budget.

COMMON WRONG ANSWERS Claiming one is universally superior; saying GRU has more gates than LSTM; forgetting the separate cell state in LSTM; ignoring the parameter and compute difference; treating both as identical to a vanilla RNN.

LIKELY FOLLOW-UPS Which gates does each have? Why does fewer parameters help on small data? How do both fix vanishing gradients? When would you abandon both for a Transformer?

ONE CONCRETE EXAMPLE For a small sensor time-series dataset with limited compute, you pick a GRU because its two gates and merged state train faster and overfit less. For a large language modeling task before Transformers dominated, an LSTM's extra gate and cell state gave the capacity to model longer context, justifying the higher cost.