Interpretability of AI Models with XAI
In a world where artificial intelligence and machine learning are becoming increasingly important, the interpretability of the underlying algorithms plays a crucial role. If humans are to take responsibility for an algorithm, they must be able to understand and trust its decisions. This article highlights the relevance and various aspects of algorithm interpretability and introduces the field of Explainable Artificial Intelligence (XAI) as an answer to this challenge.
Classical Statistics vs. Machine Learning
With classical statistical models, such as regression, interpretability is a direct given. The value of the coefficients provides information about how individual features affect the prediction. With the much more complex machine learning models, the achievable prediction quality is often better because more complex information structures in the data can be better represented, but this comes at the expense of direct interpretability. This is where XAI comes into play.
The Goal of XAI
XAI aims to provide answers to the following questions:
- What information is important? (Feature Importance)
- What influence do individual features have? (Direction of effect and type of relationship, non-linearities)
- Why does one obtain a particular predicted value for an individual observation? (Microscopic understanding at the individual level)
In determining Feature Importance, the proportional contribution of each feature to the overall explanatory content of the model is determined to identify the relevance of the different pieces of information contained in the data compared to each other. In the case of correlated information, separation is not always straightforward, so the concept of Shapley values from cooperative game theory is used here. Best practice for tabular data is currently the combination of XGBoost and treeSHAP.
In order to understand the direction of action and the nature of the relationship between the features and the quantity being predicted, different directly interpretable algorithms such as Generalized Linear Regression, Generalized Additive Models (GAM) or tree-based methods can be used in combination with SHAP values.
Based on this, microscopic explanation of individual predicted values is directly possible.
Application of XAI
Whether XAI is required at all and sufficient for the desired interpretability of the algorithm used depends on two factors:
- Is the algorithm used opaque and not directly interpretable, requiring additional XAI models?
- Is the interpretability achieved with XAI still so complex that the identified relationships exceed human comprehension?
Thus, it is clear that XAI will be used if the first question is answered in the affirmative and that a further step, namely the addition of lean and thus easily interpretable surrogate models, may be necessary.
Conclusion
The major advantage of Explainable Artificial Intelligence (XAI) is that it allows users to understand and interpret the set of rules of complex black-box models learned from the data, and thus gain confidence in the AI. This trust is necessary for the use of AI to be ethically and substantively responsible.