Dead Neuron
A neuron in a neural network that always outputs zero (or a constant), contributing nothing to the model's predictions, typically caused by large negative bias or aggressive learning rates.
Dying ReLU Problem
With ReLU activation, if a neuron's input becomes negative for all training examples, its gradient is always zero and it can never recover. This 'dying ReLU' problem can affect up to 40% of neurons in poorly configured networks.
Solutions
Use Leaky ReLU, ELU, or GELU activations. Lower the learning rate. Use batch normalization. Initialize weights carefully (He initialization for ReLU). Monitor the percentage of dead neurons during training.