SHAP with Python (Code and Explanations)

SHAP is the most powerful Python package for understanding and debugging your machine learning models. It can be used to explain both individual predictions and trends across multiple predictions. We explore how by walking through the code and explanations for the SHAP waterfall plot, force plot, absolute mean plot, beeswarm plot and dependence plots.

*NOTE*: You will now get the XAI course for free if you sign up (not the SHAP course)

Read the companion article (no-paywall link):

[adataodyssey]

*NOTE*: You will now get the XAI course for free if you sign up (not the SHAP course)

[pilarangelicarodriguezcaba]

really easy to understand, a lot better than the offician documentation from shap plots

[ShotClockHoops]

This is the best way to explain explanations 😁

I am interested to see a video of yours with more complex models like Deep Neural Networks on Signal Data and how can we use SHAP on that.

Great work!

[cutestbear]

thank you for the awesome video~ really like the way you explain everything thoroughly and meticulously. really friendly to people like us who have just begun our journey into data science

[yaelgut]

It was so clearly and well explained, thank you!

[thegerman]

Thank you so much for this awesome video! I'm currently writing a term paper about this topic and other machine learning explainability techniques. This helped me out a lot while creating my examples!
Kind regards from Germany!

[rafaelagd]

Great video! Could you comment on the future of SHAP? It seems the project was abandoned. The latest commit is from June 2022 and there is a pile of 1.5k issues. I couldn't
find much information about it and the other packages seem to depend on it. So there may be no alternative.

[soniaspisak]

Hi, I'm struggling with explaining GRU and LSTM models with SHAP. Encouraged by your videos, I am considering buying the course, but does it cover working with 3D data? Is even possible to implement SHAP and obtain reliable plots (without flattening the data) for time-series models?

[ShrijaSheth]

I tried XGBoost for a different dataset but it did not give a good scatter plot nor a red line significant to separate the observations. So which other model should one use if the number of features are 870?

[possakornkittipipatthanapo]

Hi Shapley value is very amazing in various interpretation and model understanding. However, I didn't see application related to the multi model like visual language model for example CLIP. Could you please provide any explanation or reference to further research?

[adeauy]

Nice video! the plots will be different for keras model right? i follow your codes but it seems that it wont work for neural network model tho.

[Gustavo-nnzc]

Hi @adataodyssey, great video, thanks! Is there a way to use SHAP for ARIMA/SARIMA?

[tamojitmaiti]

This is so clear and concise! Thank you!

[NasirUddin-imzb]

When i was running my code i had this issues, regading shap: FutureWarning: In the future `np.long` will be defined as the corresponding NumPy scalar.
long_ = _make_signed(np.long), I did pip install 1.20.0, 1.24.2, 1.22.2 so on, no of them work, what can i do, if you can suggest me something it will be great.

[fouedhamouda]

Thanks,  
can I use Shap with GAN model?

[markfedenia]

I see that cuML computes Shapley values, however it does not look like the Explainer object is compatible with shap. Do you know if there is any way to use the cuML Explainer object and model with the shap package (by the way, excellent videos)

[DarkKnight_]

Hi Connor, you mentioned on the limitation of the SHAP values that "highly correlated features are a problem when using shap values technique", but on this video the heat map shows that features are highly correlated?

[apogounte]

Hi! Interesting video! Just wanted to mention that if you just run shap.plots.waterfall(shap_values[0]), you never get on the y-axis, the actual names of the features, but you get instead feature 5, feature 2, etc. Is there a quick fix?

[famin]

Can't thank you enough. You solve my problem.

[mayuribhandari]

I have subscribed to newsletter but not getting access to XAI course