shap.PermutationExplainer

class shap.PermutationExplainer(model, masker, link=CPUDispatcher(<function identity>), feature_names=None, linearize_link=True, seed=None, **call_args)

This method approximates the Shapley values by iterating through permutations of the inputs.

This is a model agnostic explainer that guarantees local accuracy (additivity) by iterating completely through an entire permutation of the features in both forward and reverse directions (antithetic sampling). If we do this once, then we get the exact SHAP values for models with up to second order interaction effects. We can iterate this many times over many random permutations to get better SHAP value estimates for models with higher order interactions. This sequential ordering formulation also allows for easy reuse of model evaluations and the ability to efficiently avoid evaluating the model when the background values for a feature are the same as the current input value. We can also account for hierarchical data structures with partition trees, something not currently implemented for KernalExplainer or SamplingExplainer.

__init__(model, masker, link=CPUDispatcher(<function identity>), feature_names=None, linearize_link=True, seed=None, **call_args)

Build an explainers.Permutation object for the given model using the given masker object.

Parameters:

modelfunction

A callable python object that executes the model given a set of input data samples.

maskerfunction or numpy.array or pandas.DataFrame

A callable python object used to “mask” out hidden features of the form masker(binary_mask, x). It takes a single input sample and a binary mask and returns a matrix of masked samples. These masked samples are evaluated using the model function and the outputs are then averaged. As a shortcut for the standard masking using by SHAP you can pass a background data matrix instead of a function and that matrix will be used for masking. To use a clustering game structure you can pass a shap.maskers.Tabular(data, clustering="correlation") object.

seed: None or int

Seed for reproducibility

**call_argsvalid argument to the __call__ method

These arguments are saved and passed to the __call__ method as the new default values for these arguments.

Methods

__init__(model, masker[, link, ...])	Build an explainers.Permutation object for the given model using the given masker object.
explain_row(*row_args, max_evals, ...)	Explains a single row and returns the tuple (row_values, row_expected_values, row_mask_shapes).
load(in_file[, model_loader, masker_loader, ...])	Load an Explainer from the given file stream.
save(out_file[, model_saver, masker_saver])	Write the explainer to the given file stream.
shap_values(X[, npermutations, ...])	Legacy interface to estimate the SHAP values for a set of samples.
supports_model_with_masker(model, masker)	Determines if this explainer can handle the given model.

explain_row(*row_args, max_evals, main_effects, error_bounds, batch_size, outputs, silent)

Explains a single row and returns the tuple (row_values, row_expected_values, row_mask_shapes).

classmethod load(in_file, model_loader=<bound method Model.load of <class 'shap.models._model.Model'>>, masker_loader=<bound method Serializable.load of <class 'shap.maskers._masker.Masker'>>, instantiate=True)

Load an Explainer from the given file stream.

Parameters:

in_fileThe file stream to load objects from.

save(out_file, model_saver='.save', masker_saver='.save')

Write the explainer to the given file stream.

shap_values(X, npermutations=10, main_effects=False, error_bounds=False, batch_evals=True, silent=False)

Legacy interface to estimate the SHAP values for a set of samples.

Parameters:

Xnumpy.array or pandas.DataFrame or any scipy.sparse matrix

A matrix of samples (# samples x # features) on which to explain the model’s output.

npermutationsint

Number of times to cycle through all the features, re-evaluating the model at each step. Each cycle evaluates the model function 2 * (# features + 1) times on a data matrix of (# background data samples) rows. An exception to this is when PermutationExplainer can avoid evaluating the model because a feature’s value is the same in X and the background dataset (which is common for example with sparse features).

Returns:

array or list

For models with a single output this returns a matrix of SHAP values (# samples x # features). Each row sums to the difference between the model output for that sample and the expected value of the model output (which is stored as expected_value attribute of the explainer). For models with vector outputs this returns a list of such matrices, one for each output.

static supports_model_with_masker(model, masker)

Determines if this explainer can handle the given model.

This is an abstract static method meant to be implemented by each subclass.