Explainability (XAI)
Techniques and methods for making AI model decisions understandable to humans, answering the question 'why did the model make this prediction?'
Why It Matters
Regulated industries (healthcare, finance, legal) often require explanations for AI decisions. Debugging models requires understanding what they've learned. Trust in AI systems depends on transparency.
Key Techniques
SHAP: Assigns each feature an importance value based on game theory. LIME: Creates local, interpretable approximations of complex models. Attention visualization: Shows which parts of the input the model focused on. Counterfactual explanations: 'The loan would have been approved if income were $5K higher.'
Inherent vs. Post-Hoc
Some models are inherently interpretable (decision trees, linear regression). Deep neural networks require post-hoc explanation methods. Chain-of-thought reasoning in LLMs offers a form of built-in explainability.