Overfitting vs Underfitting
The two failure modes of model training: overfitting (memorizing training data, poor generalization) and underfitting (failing to learn patterns, poor everywhere).
Diagnosing
Overfitting: Low training error, high validation error. Gap between them grows with training. Underfitting: High training AND validation error. Model hasn't captured the patterns.
Solutions
Overfitting: More data, regularization, dropout, early stopping, simpler model. Underfitting: Bigger model, more training, fewer constraints, better features. Cross-validation helps identify which problem you have.