In-Context Learning
The ability of large language models to learn new tasks at inference time purely from examples provided in the prompt, without any weight updates or fine-tuning.
How It Works
You provide the model with a few input-output examples in the prompt, then give it a new input. The model infers the pattern from the examples and generates the appropriate output. This happens entirely through the forward pass -- no training involved.
Why It's Remarkable
In-context learning emerged as an unexpected capability of large models. GPT-3 was the first to demonstrate it convincingly. It means a single model can perform thousands of different tasks just by changing the prompt.
Limitations
Performance depends heavily on example selection, ordering, and formatting. The context window limits how many examples fit. In-context learning generally doesn't match the performance of fine-tuning but requires zero additional training.