shap.KernelExplainer

class shap.KernelExplainer(model, data, feature_names=None, link='identity', **kwargs)

Uses the Kernel SHAP method to explain the output of any function.

Kernel SHAP is a method that uses a special weighted linear regression to compute the importance of each feature. The computed importance values are Shapley values from game theory and also coefficients from a local linear regression.

Parameters:

modelfunction or iml.Model

User supplied function that takes a matrix of samples (# samples x # features) and computes the output of the model for those samples. The output can be a vector (# samples) or a matrix (# samples x # model outputs).

datanumpy.array or pandas.DataFrame or shap.common.DenseData or any scipy.sparse matrix

The background dataset to use for integrating out features. To determine the impact of a feature, that feature is set to “missing” and the change in the model output is observed. Since most models aren’t designed to handle arbitrary missing data at test time, we simulate “missing” by replacing the feature with the values it takes in the background dataset. So if the background dataset is a simple sample of all zeros, then we would approximate a feature being missing by setting it to zero. For small problems, this background dataset can be the whole training set, but for larger problems consider using a single reference value or using the kmeans function to summarize the dataset. Note: for the sparse case, we accept any sparse matrix but convert to lil format for performance.

feature_nameslist

The names of the features in the background dataset. If the background dataset is supplied as a pandas.DataFrame, then feature_names can be set to None (default), and the feature names will be taken as the column names of the dataframe.

link“identity” or “logit”

A generalized linear model link to connect the feature importance values to the model output. Since the feature importance values, phi, sum up to the model output, it often makes sense to connect them to the output with a link function where link(output) = sum(phi). Default is “identity” (a no-op). If the model output is a probability, then “logit” can be used to transform the SHAP values into log-odds units.

Examples

See Kernel Explainer Examples.

__init__(model, data, feature_names=None, link='identity', **kwargs)

Build a new explainer for the passed model.

Parameters:

modelobject or function

User supplied function or model object that takes a dataset of samples and computes the output of the model for those samples.

maskerfunction, numpy.array, pandas.DataFrame, tokenizer, None, or a list of these for each model input

The function used to “mask” out hidden features of the form masked_args = masker(*model_args, mask=mask). It takes input in the same form as the model, but for just a single sample with a binary mask, then returns an iterable of masked samples. These masked samples will then be evaluated using the model function and the outputs averaged. As a shortcut for the standard masking using by SHAP you can pass a background data matrix instead of a function and that matrix will be used for masking. Domain specific masking functions are available in shap such as shap.ImageMasker for images and shap.TokenMasker for text. In addition to determining how to replace hidden features, the masker can also constrain the rules of the cooperative game used to explain the model. For example shap.TabularMasker(data, hclustering=”correlation”) will enforce a hierarchical clustering of coalitions for the game (in this special case the attributions are known as the Owen values).

linkfunction

The link function used to map between the output units of the model and the SHAP value units. By default it is shap.links.identity, but shap.links.logit can be useful so that expectations are computed in probability units while explanations remain in the (more naturally additive) log-odds units. For more details on how link functions work see any overview of link functions for generalized linear models.

algorithm“auto”, “permutation”, “partition”, “tree”, or “linear”

The algorithm used to estimate the Shapley values. There are many different algorithms that can be used to estimate the Shapley values (and the related value for constrained games), each of these algorithms have various tradeoffs and are preferable in different situations. By default the “auto” options attempts to make the best choice given the passed model and masker, but this choice can always be overridden by passing the name of a specific algorithm. The type of algorithm used will determine what type of subclass object is returned by this constructor, and you can also build those subclasses directly if you prefer or need more fine grained control over their options.

output_namesNone or list of strings

The names of the model outputs. For example if the model is an image classifier, then output_names would be the names of all the output classes. This parameter is optional. When output_names is None then the Explanation objects produced by this explainer will not have any output_names, which could effect downstream plots.

seed: None or int

seed for reproducibility

Methods

__init__(model, data[, feature_names, link])	Build a new explainer for the passed model.
addsample(x, m, w)
allocate()
explain(incoming_instance, **kwargs)
explain_row(*row_args, max_evals, ...)	Explains a single row and returns the tuple (row_values, row_expected_values, row_mask_shapes, main_effects).
load(in_file[, model_loader, masker_loader, ...])	Load an Explainer from the given file stream.
not_equal(i, j)
run()
save(out_file[, model_saver, masker_saver])	Write the explainer to the given file stream.
shap_values(X, **kwargs)	Estimate the SHAP values for a set of samples.
solve(fraction_evaluated, dim)
supports_model_with_masker(model, masker)	Determines if this explainer can handle the given model.
varying_groups(x)

explain_row(*row_args, max_evals, main_effects, error_bounds, outputs, silent, **kwargs)

Explains a single row and returns the tuple (row_values, row_expected_values, row_mask_shapes, main_effects).

This is an abstract method meant to be implemented by each subclass.

Returns:

tuple

A tuple of (row_values, row_expected_values, row_mask_shapes), where row_values is an array of the attribution values for each sample, row_expected_values is an array (or single value) representing the expected value of the model for each sample (which is the same for all samples unless there are fixed inputs present, like labels when explaining the loss), and row_mask_shapes is a list of all the input shapes (since the row_values is always flattened),

classmethod load(in_file, model_loader=<bound method Model.load of <class 'shap.models._model.Model'>>, masker_loader=<bound method Serializable.load of <class 'shap.maskers._masker.Masker'>>, instantiate=True)

Load an Explainer from the given file stream.

Parameters:

in_fileThe file stream to load objects from.

save(out_file, model_saver='.save', masker_saver='.save')

Write the explainer to the given file stream.

shap_values(X, **kwargs)

Estimate the SHAP values for a set of samples.

Parameters:

Xnumpy.array or pandas.DataFrame or any scipy.sparse matrix

A matrix of samples (# samples x # features) on which to explain the model’s output.

nsamples“auto” or int

Number of times to re-evaluate the model when explaining each prediction. More samples lead to lower variance estimates of the SHAP values. The “auto” setting uses nsamples = 2 * X.shape[1] + 2048.

l1_reg“num_features(int)”, “auto” (default for now, but deprecated), “aic”, “bic”, or float

The l1 regularization to use for feature selection. The estimation procedure is based on a debiased lasso.

• “num_features(int)” selects a fixed number of top features.

• “aic” and “bic” options use the AIC and BIC rules for regularization.

• Passing a float directly sets the “alpha” parameter of the sklearn.linear_model.Lasso model used for feature selection.

• “auto” (default for now but deprecated): uses “aic” when less than 20% of the possible sample space is enumerated, otherwise it uses no regularization.

Note: The default behaviour will change in a future version to be "num_features(10)". Pass this value explicitly to silence the DeprecationWarning.

silent: bool

If True, hide tqdm progress bar. Default False.

gc_collectbool

Run garbage collection after each explanation round. Sometime needed for memory intensive explanations (default False).

Returns:

np.array or list

Estimated SHAP values, usually of shape (# samples x # features).

Each row sums to the difference between the model output for that sample and the expected value of the model output (which is stored as the expected_value attribute of the explainer).

The type and shape of the return value depends on the number of model inputs and outputs:

• one input, one output: array of shape (#num_samples, *X.shape[1:]).

• one input, multiple outputs: array of shape (#num_samples, *X.shape[1:], #num_outputs)

• multiple inputs: list of arrays of corresponding shape above.

Changed in version 0.45.0: Return type for models with multiple outputs and one input changed from list to np.ndarray.

static supports_model_with_masker(model, masker)

Determines if this explainer can handle the given model.

This is an abstract static method meant to be implemented by each subclass.