KernelSHAP can be misleading with correlated predictors | by Shuyang Xiang | Aug, 2024

A concrete case research

“Like many different permutation-based interpretation strategies, the Shapley worth technique suffers from inclusion of unrealistic information cases when options are correlated. To simulate {that a} function worth is lacking from a coalition, we marginalize the function. ..When options are dependent, then we would pattern function values that don’t make sense for this occasion. ”— Interpretable-ML-Book.

SHAP (SHapley Additive exPlanations) values are designed to pretty allocate the contribution of every function to the prediction made by a machine studying mannequin, based mostly on the idea of Shapley values from cooperative sport concept. The Shapley worth framework has a number of fascinating theoretical properties and might, in precept, deal with any predictive mannequin. Nonetheless, SHAP values can doubtlessly be deceptive, particularly when utilizing the KernelSHAP technique for approximation. When predictors are correlated, these approximations may be imprecise and even have the other signal.

On this weblog submit, I’ll show how the unique SHAP values can differ considerably from approximations made by the SHAP framework, particularly the KernalSHAP and talk about the explanations behind these discrepancies.

Take into account a situation the place we intention to foretell the churn price of rental leases in an workplace constructing, based mostly on two key elements: occupancy price and the speed of reported issues.

The occupancy price considerably impacts the churn price. For example, if the occupancy price is just too low, tenants might depart because of the workplace being underutilized. Conversely, if the occupancy price is just too excessive, tenants would possibly depart due to overcrowding, in search of higher choices elsewhere.

Moreover, let’s assume that the speed of reported issues is very correlated with the occupancy price, particularly that the reported downside price is the sq. of the occupancy price.

We outline the churn price operate as follows:

This operate with respect to the 2 variables may be represented by the next illustrations:

SHAP Values Computed Utilizing Kernel SHAP

We are going to now use the next code to compute the SHAP values of the predictors:

# Outline the dataframe 
churn_df=pd.DataFrame(
{
"occupancy_rate":occupancy_rates,
"reported_problem_rate": reported_problem_rates,
"churn_rate":churn_rates,
}
)
X=churn_df.drop(["churn_rate"],axis=1)
y=churn_df["churn_rate"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state = 42)
# append one speical level
X_test=pd.concat(objs=[X_test, pd.DataFrame({"occupancy_rate":[0.8], "reported_problem_rate":[0.64]})])
# Outline the prediction
def predict_fn(information):
occupancy_rates = information[:, 0]
reported_problem_rates = information[:, 1]
churn_rate= C_base +C_churn*(C_occ* occupancy_rates-reported_problem_rates-0.6)**2 +C_problem*reported_problem_rates
return churn_rate
# Create the SHAP KernelExplainer utilizing the right prediction operate
background_data = shap.pattern(X_train,100)
explainer = shap.KernelExplainer(predict_fn, background_data)
shap_values = explainer(X_test)

The code above performs the next duties:

1. Information Preparation: A DataFrame named churn_df is created with columns occupancy_rate, reported_problem_rate, and churn_rate. Variables and goal (churn_rate ) are then created from and Information is break up into coaching and testing units, with 80% for coaching and 20% for testing. Be aware {that a} particular information level with particular occupancy_rate and reported_problem_rate values is added to the check set X_test.
2. Prediction Operate Definition: A operate predict_fn is outlined to calculate churn price utilizing a particular components involving predefined constants.
3. SHAP Evaluation: A SHAP KernelExplainer is initialized utilizing the prediction operate and background_data samples from X_train. SHAP values for X_test are computed utilizing the explainer.

Beneath, you possibly can see a abstract SHAP bar plot, which represents the typical SHAP values for X_test :

Specifically, we see that on the information level (0.8, 0.64), the SHAP values of the 2 options are 0.10 and -0.03, illustrated by the next pressure plot:

SHAP Values by orignal definition

Let’s take a step again and compute the precise SHAP values step-by-step based on their authentic definition. The overall components for SHAP values is given by:

the place: S is a subset of all function indices excluding i, |S| is the scale of the subset S, M is the overall variety of options, f(XS​∪{xi​}) is the operate evaluated with the options in S with xi current whereas f(XS) is the operate evaluated with the function in S with xi absent.

Now, let’s calculate the SHAP values for 2 options: occupancy price (denoted as x1x_1x1​) and reported downside price (denoted as x2x_2x2​) on the information level (0.8, 0.64). Recall that x1x_1x1​ and x2x_2x2​ are associated by x_1 = x_2².

We have now the SHAP worth for occupancy price on the information level:

and, similary, for the function reported downside price:

First, let’s compute the SHAP worth for the occupancy price on the information level:

1. The primary time period is the expectation of the mannequin’s output when X1​ is mounted at 0.8 and X2​ is averaged over its distribution. Given the connection xx_1 = x_2², this expectation results in the mannequin’s output on the particular level (0.8, 0.64).
2. The second time period is the unconditional expectation of the mannequin’s output, the place each X1 and X2 are averaged over their distributions. This may be computed by averaging the outputs over all information factors within the background dataset.
3. The third time period is the mannequin’s output on the particular level (0.8, 0.64).
4. The ultimate time period is the expectation of the mannequin’s output when X1​ is averaged over its distribution, on condition that X2​ is mounted on the particular level 0.64. Once more, because of the relationship x_1 = x_2²​, this expectation matches the mannequin’s output at (0.8, 0.64), much like step one.

Thus, the SHAP values compute from the unique definition for the 2 options occupancy price and reported downside price on the information level (0.8, 0.64) are -0.0375 and -0.0375, respectively, which is kind of totally different from the values given by Kernel SHAP.

The place comes discrepancies?

As you could have seen, the discrepancy between the 2 strategies primarily arises from the second and fourth steps, the place we have to compute the conditional expectation. This entails calculating the expectation of the mannequin’s output when X1X_1X1​ is conditioned on 0.8.

• Actual SHAP: When computing actual SHAP values, the dependencies between options (reminiscent of x1=x_2² in our instance​) are explicitly accounted for. This ensures correct calculations by contemplating how function interactions affect the mannequin’s output.
• Kernel SHAP: By default, Kernel SHAP assumes function independence, which might result in inaccurate SHAP values when options are literally dependent. In response to the paper A Unified Approach to Interpreting Model Predictions, this assumption is a simplification. In observe, options are sometimes correlated, making it difficult to attain correct approximations when utilizing Kernel SHAP.

Sadly, computing SHAP values straight based mostly on their authentic definition may be computationally costly. Listed here are some various approaches to contemplate:

TreeSHAP

• Designed particularly for tree-based fashions like random forests and gradient boosting machines, TreeSHAP effectively computes SHAP values whereas successfully managing function dependencies.
• This technique is optimized for tree ensembles, making it sooner and extra scalable in comparison with conventional SHAP computations.
• When utilizing TreeSHAP inside the SHAP framework, set the parameter feature_perturbation = "interventional" to account for function dependencies precisely.

Extending Kernel SHAP for Dependent Options

• To handle function dependencies, this paper entails extending Kernel SHAP. One technique is to imagine that the function vector follows a multivariate Gaussian distribution. On this method:
• Conditional distributions are modeled as multivariate Gaussian distributions.
• Samples are generated from these conditional Gaussian distributions utilizing estimates from the coaching information.
• The integral within the approximation is computed based mostly on these samples.
• This technique assumes a multivariate Gaussian distribution for options, which can not all the time be relevant in real-world situations the place options can exhibit totally different dependency constructions.

Enhancing Kernel SHAP Accuracy

• Description: Improve the accuracy of Kernel SHAP by guaranteeing that the background dataset used for approximation is consultant of the particular information distribution with independant options.

By using these strategies, you possibly can deal with the computational challenges related to calculating SHAP values and improve their accuracy in sensible purposes. Nonetheless, it is very important observe that no single answer is universally optimum for all situations.

On this weblog submit, we’ve explored how SHAP values, regardless of their sturdy theoretical basis and flexibility throughout numerous predictive fashions, can undergo from accuracy points when predictors are correlated, significantly when approximations like KernelSHAP are employed. Understanding these limitations is essential for successfully deciphering SHAP values. By recognizing the potential discrepancies and choosing probably the most appropriate approximation strategies, we will obtain extra correct and dependable function attribution in our fashions.