Emergent Ability
A capability that appears in large AI models but is absent in smaller ones, seemingly arising unpredictably as model scale increases.
Examples
Chain-of-thought reasoning, in-context learning, code generation, and multilingual translation all emerged at specific model scales without being explicitly trained for. GPT-3 could do few-shot learning that GPT-2 couldn't.
Debate
Whether emergence is a real phase transition or an artifact of evaluation metrics is actively debated. Some researchers argue that emergence is a mirage caused by non-linear evaluation metrics, while others see it as genuine capability thresholds.
Implications
If emergence is real, predicting what capabilities future models will develop is difficult. This unpredictability is both exciting (new capabilities) and concerning (unintended capabilities) for AI safety.