Recurrent Neural Network (RNN)
A neural network architecture with loops that allow information to persist across time steps, designed for processing sequential data.
How It Works
At each time step, an RNN takes the current input and the previous hidden state, producing an output and a new hidden state. This hidden state acts as a 'memory' that carries information from earlier in the sequence.
Variants
LSTM (Long Short-Term Memory): Adds gates to control what information to remember or forget, solving the vanishing gradient problem. GRU (Gated Recurrent Unit): A simplified LSTM with fewer parameters.
Historical Importance
RNNs were the dominant architecture for NLP and sequence tasks before transformers. They powered early machine translation, speech recognition, and text generation. Transformers largely replaced them due to better parallelization and performance.