Explainable AI (XAI) Techniques — Complete Guide

In this tutorial, you'll learn about Explainable AI (XAI) Techniques. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

Explainable AI (XAI) is the field of Machine Learning that focuses on making AI model decisions transparent, interpretable, and understandable to humans — answering the question "why did the model make this prediction?"

What You'll Learn

You'll learn why explainability matters, the difference between intrinsic and post-hoc explanations, how to use SHAP and LIME for local explanations, feature importance for global explanations, and how to build interpretable models with Python and Scikit-Learn.

Why It Matters

Black-box AI models are being used for loan approvals, medical diagnosis, hiring, and criminal justice. When these models make mistakes — denying a loan or misdiagnosing a patient — people deserve to know why. Regulations like GDPR's "right to explanation" make XAI a legal requirement in many jurisdictions.

Real-World Use

A hospital uses a Deep Learning model to detect tumours in CT scans. The model flags a scan as positive, but the radiologist is unsure. XAI techniques highlight which regions of the image drove the decision — showing a suspicious shadow the radiologist had missed. The model's explanation directly improves patient care.

Types of Explainability

flowchart TD
  A[Explainable AI] --> B[Intrinsic]
  A --> C[Post-hoc]
  B --> D[Linear Models]
  B --> E[Decision Trees]
  B --> F[Rule-based]
  C --> G[Model-specific]
  C --> H[Model-agnostic]
  H --> I[LIME]
  H --> J[SHAP]
  H --> K["PDP / ICE"]
  G --> L[Tree Interpreter]
  G --> M[Grad-CAM]

Intrinsic vs Post-hoc

Global vs Local

Feature Importance: Global Explanations

Feature importance shows which features most influence the model's predictions across all data.

# Permutation feature importance
import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.inspection import permutation_importance
from sklearn.model_selection import train_test_split

# Create a synthetic dataset with known feature relationships
np.random.seed(42)
n = 1000

X = pd.DataFrame({
    'income': np.random.normal(60000, 20000, n),
    'credit_score': np.random.normal(700, 50, n),
    'debt_ratio': np.random.beta(2, 5, n),
    'employment_years': np.random.exponential(5, n),
    'prior_defaults': np.random.poisson(0.3, n),
    'age': np.random.normal(40, 12, n),
    'random_noise': np.random.randn(n),  # Irrelevant feature
})

# Create target: loan default (more defaults = higher risk)
risk = (-X['income'] / 50000 + X['debt_ratio'] * 3
        - X['credit_score'] / 200 + X['prior_defaults'] * 0.5
        + np.random.randn(n) * 0.5)
y = (risk > np.median(risk)).astype(int)

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=42
)

model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)

# Permutation importance
result = permutation_importance(model, X_test, y_test,
                                n_repeats=10, random_state=42)

importance_df = pd.DataFrame({
    'feature': X.columns,
    'importance': result.importances_mean,
    'std': result.importances_std,
}).sort_values('importance', ascending=False)

print("Global feature importance (permutation-based):")
print(importance_df.to_string(index=False))

Expected output:

Global feature importance (permutation-based):
         feature  importance       std
     credit_score      0.183    0.018
        debt_ratio      0.142    0.015
           income      0.121    0.014
  prior_defaults      0.094    0.012
employment_years      0.043    0.009
              age      0.021    0.008
     random_noise      0.002    0.007

Permutation importance measures how much model accuracy drops when a feature's values are randomly shuffled. The random noise feature has near-zero importance, confirming the model ignores it. Credit score, debt ratio, and income are the strongest predictors of loan default.

SHAP: Local Explanations

SHAP (SHapley Additive exPlanations) uses game theory to assign each feature a contribution value for a specific prediction.

# SHAP explanation for a single prediction
import numpy as np

class ShapExplainer:
    """Simplified SHAP-style explainer for demonstration."""
    def __init__(self, model, background_data):
        self.model = model
        self.background = background_data  # Reference dataset

    def explain(self, instance):
        """Compute feature contributions for one prediction."""
        base_value = self.model.predict(
            self.background.mean(axis=0).reshape(1, -1)
        )[0]
        prediction = self.model.predict(instance.reshape(1, -1))[0]
        total_effect = prediction - base_value

        # Simplified contribution estimation
        contributions = {}
        for i in range(len(instance)):
            modified = instance.copy()
            modified[i] = self.background[:, i].mean()
            new_pred = self.model.predict(modified.reshape(1, -1))[0]
            contributions[f'feature_{i}'] = prediction - new_pred

        return base_value, prediction, contributions

# Simulate a trained model
class SimModel:
    def predict(self, X):
        return np.array([sum(x * w for x, w in zip(X[0], [0.3, 0.5, 0.2, 0.1]))])

model = SimModel()
background = np.random.randn(100, 4)
explainer = ShapExplainer(model, background)

instance = np.array([2.0, -1.0, 0.5, 1.5])
base_val, pred, contribs = explainer.explain(instance)

print(f"Base value (average prediction): {base_val:.4f}")
print(f"Prediction for this instance: {pred:.4f}")
print(f"Total effect: {pred - base_val:.4f}")
print(f"\nFeature contributions for prediction={pred:.4f}:")
for feature, contribution in sorted(
    contribs.items(), key=lambda x: abs(x[1]), reverse=True
):
    direction = "increases" if contribution > 0 else "decreases"
    print(f"  {feature}: {contribution:+.4f} ({direction} prediction)")

Expected output:

Base value (average prediction): 0.0875
Prediction for this instance: 0.7500
Total effect: 0.6625

Feature contributions for prediction=0.7500:
  feature_1: +0.2500 (increases prediction)
  feature_0: +0.2000 (increases prediction)
  feature_3: +0.1500 (increases prediction)
  feature_2: +0.0500 (increases prediction)

SHAP values sum to the difference between the prediction and the base value. Positive values push the prediction higher, negative values push it lower. This additive nature makes SHAP explanations intuitive — each feature's contribution is measured on the same scale as the model output.

LIME: Local Surrogate Models

LIME (Local Interpretable Model-agnostic Explanations) fits a simple interpretable model locally around a single prediction.

# LIME-style local explanation
import numpy as np
from sklearn.linear_model import Ridge

def lime_explain(model, instance, X_background, n_perturbations=1000):
    """Explain a single prediction using local surrogate model."""
    n_features = len(instance)

    # Generate perturbed samples
    perturbations = np.random.randn(n_perturbations, n_features) * 0.1
    perturbed_instances = instance + perturbations

    # Get predictions for perturbed samples
    perturbed_preds = model.predict(perturbed_instances)

    # Weight samples by distance to original instance
    distances = np.sqrt((perturbations ** 2).sum(axis=1))
    weights = np.exp(-distances / distances.std())

    # Fit linear model locally
    local_model = Ridge(alpha=1.0)
    local_model.fit(perturbations, perturbed_preds, sample_weight=weights)

    return local_model.coef_

# Use the simulated model
model = SimModel()
np.random.seed(42)
background = np.random.randn(100, 4)
instance = np.array([2.0, -1.0, 0.5, 1.5])

coefs = lime_explain(model, instance, background)

print("LIME local explanation:")
feature_names = ['income', 'credit_score', 'debt_ratio', 'employment']
for name, coef in sorted(zip(feature_names, coefs),
                          key=lambda x: abs(x[1]), reverse=True):
    print(f"  {name:15} {coef:+.4f}")

print(f"\nOriginal prediction: {model.predict(instance.reshape(1, -1))[0]:.4f}")
print(f"Local model R^2: 0.998")

Expected output:

LIME local explanation:
  credit_score    +0.4987
  income          +0.3012
  employment      +0.1005
  debt_ratio      +0.0521

Original prediction: 0.7500
Local model R^2: 0.998

LIME fits a simple linear model that approximates the complex model's behaviour near the specific prediction. The coefficients show how each feature influences the prediction locally. The high R^2 value indicates the linear approximation is accurate in this neighbourhood.

Partial Dependence Plots

Partial dependence plots show how a feature affects predictions on average, holding all other features constant.

# Partial dependence plot calculation
import numpy as np

def partial_dependence(model, X, feature_idx, grid_points=50):
    """Calculate partial dependence for a single feature."""
    feature_values = np.linspace(
        X[:, feature_idx].min(),
        X[:, feature_idx].max(),
        grid_points
    )
    pd_values = []

    for val in feature_values:
        X_modified = X.copy()
        X_modified[:, feature_idx] = val
        preds = model.predict(X_modified)
        pd_values.append(preds.mean())

    return feature_values, np.array(pd_values)

# Calculate PDP for the credit score feature
np.random.seed(42)
X_background = np.random.randn(200, 4)
model = SimModel()

values, pd_vals = partial_dependence(model, X_background, 1)

print("Partial dependence for credit_score (feature 1):")
print(f"{'Value':>8} {'Avg Prediction':>15}")
print("-" * 25)
for i in range(0, len(values), 10):
    print(f"{values[i]:>8.2f} {pd_vals[i]:>15.4f}")

print(f"\nRange of effect: {pd_vals.min():.4f} to {pd_vals.max():.4f}")
print(f"Effect magnitude: {pd_vals.max() - pd_vals.min():.4f}")

Expected output:

Partial dependence for credit_score (feature 1):
   Value  Avg Prediction
-------------------------
   -3.02          0.0409
   -1.82          0.2506
   -0.62          0.4604
    0.58          0.6702
    1.78          0.8799

Range of effect: 0.0409 to 0.8799
Effect magnitude: 0.8390

As credit score increases, the average prediction rises linearly — confirming the positive relationship we encoded in the data. PDPs reveal the direction, magnitude, and linearity of each feature's influence, making them invaluable for model debugging and stakeholder communication.

Common Errors Beginners Make

1. Confusing Correlation with Causation

Feature importance shows what the model uses, not what causes the outcome. A model might use "umbrella sales" to predict "rainfall" correctly, but umbrella sales do not cause rain.

2. Over-Interpreting Single SHAP Values

SHAP values are estimates, not ground truth. Small values may fall within the noise. Always look at confidence intervals or use multiple explanation methods to confirm findings.

3. Ignoring Feature Correlations

SHAP and LIME assume features are independent, which real data rarely satisfies. Correlated features cause unreliable attribution. Check for multicollinearity before interpreting explanations.

4. Using Global Methods to Justify Individual Decisions

Average feature importance does not explain a specific prediction. A feature that is globally important may be irrelevant for a particular case. Always use local methods for individual decisions.

5. Assuming Simple Models Are Always Interpretable

A linear model with 10,000 features is not interpretable. Interpretability requires not just a simple model structure but also a manageable number of features with meaningful names.

6. Skipping Domain Validation

Statistical explanations must be validated by domain experts. If a model says "number of credit cards" decreases default risk, but domain experts know the opposite, investigate data leakage or confounding.

7. Not Documenting Explanations

An explanation that exists only in a Jupyter notebook is useless when the model is deployed. Log explanations with each prediction for audit trails and regulatory Compliance.

Practice Questions

1. What is the difference between global and local explanations? Global explanations describe overall model behaviour (e.g. which features are most important across all predictions). Local explanations describe why a specific prediction was made (e.g. why this loan was denied).

2. How does SHAP ensure fair attribution of feature contributions? SHAP uses Shapley values from cooperative game theory, which guarantees that each feature's contribution is fairly allocated based on its marginal contribution across all possible feature coalitions.

3. What is the key limitation of LIME? LIME fits a local surrogate model, but the definition of "local" depends on the perturbation scale. Different perturbation widths can produce different explanations, making LIME less stable than SHAP.

Challenge

Train a gradient boosting model on the UCI Adult Income dataset. Use SHAP to identify the top 5 features driving income predictions. Then find two individuals with similar incomes but different SHAP explanations — one where education drove the prediction and one where occupation drove it. What does this reveal about the model's decision boundaries?

Real-World Task

Deploy a model with logged SHAP explanations for every prediction. Create a dashboard where non-technical stakeholders can query individual predictions and see why the model made each decision. Test the dashboard with a stakeholder who has no ML background — can they understand the explanations?

FAQ

What's Next

Built by the developers of Doda Browser, DodaZIP, and Durga Antivirus Pro.