Techniques include feature-importance scores such as SHAP, saliency maps that highlight image regions and simpler surrogate models. Mechanistic interpretability goes further, trying to identify the internal features and circuits of neural networks.
Explanations matter in credit, hiring and healthcare, where people may be entitled to the reasons behind automated decisions. See sparse autoencoders and monosemanticity.