Representation Learning
The automatic discovery of useful features and representations from raw data, eliminating the need for manual feature engineering.
How Deep Learning Does It
Each layer of a deep neural network learns progressively more abstract representations. Early layers detect low-level patterns (edges), middle layers detect mid-level features (textures, shapes), and deep layers capture high-level concepts (objects, meanings).
Self-Supervised Learning
Modern representation learning often uses self-supervised objectives: masked language modeling (BERT), next-token prediction (GPT), contrastive learning (SimCLR, CLIP), and masked image modeling (MAE). These learn powerful representations without labeled data.