oneTaken/awesome_deep_learning_interpretability：深度学习近年来关于神经网络模型解释性的相关高引用/顶会论文(

技术解读

以下内容由 GitRead 中文社区编辑团队基于项目 README 与官方资料整理，结合中文开发者的常见使用场景补充技术选型建议。具体以项目官方文档为准。

GitHub 仓库技术分析：oneTaken/awesome_deep_learning_interpretability

项目背景与痛点

在人工智能领域，深度学习模型的「黑箱」问题一直是研究者和开发者关注的一个重要课题。随着深度学习模型变得日益复杂，如何解析这些模型的内部运作机制成为了解决模型可解释性和信任度的关键。项目 awesome_deep_learning_interpretability 就是为了解决这个问题而诞生的。它汇集了一系列高引用和顶会论文，这些论文的研究方向涵盖了深度学习模型的解释性和可解释性领域，为开发者提供了理论基础和技术实现的双重参考。

核心技术揭秘

技术架构与实现

该项目的核心在于解决了模型解释性的两大关键技术：模型解释算法和结果可视化。通过系统地引用大量顶尖论文和开源代码，项目构建了一个涵盖从解释算法到可视化工具的完整技术栈。

graph TD A[深度学习模型] --> B[模型解释算法] B --> C{"处理方式"} C -->|"表示学习"| D[归因方法] C -->|"因果推理"| E[Shapley值] C -->|"可视化"| F[Shap图] D --> G[局部归因] E --> H[全局解释] F --> I[结果渲染]

核心算法

归因方法是解释深度学习模型的关键技术之一。这些方法通过计算输入特征对模型输出的贡献，提供了模型内部运作的直观视角。其中，局部归因（Local Attribution）侧重于单一数据点的解释，而全局解释（Global Explanation）则用于理解模型对整体数据分布的响应模式。Shapley值提供了从博弈论角度解析模型贡献的一种方法，而Shap图（SHAP值可视化）则是模型解释结果可视化的重要工具。

功能亮点与差异

该项目的最大亮点在于其全面性和开放性。通过汇集知名论文和开源代码，它为研究者和开发者提供了一个一站式的工作平台，加速了模型解释性研究的技术落地。此外，不同于其他单一或专一方向的研究资源，这个项目提供了从算法理论到实现应用的全方位支持。

应用场景与落地建议

在生产环境中应用该技术栈需要注意几个方面。首先，选择合适的解释算法依赖于具体的应用场景和数据特性。对于复杂的模型，局部归因和全局解释相结合的方式通常会比较合理。其次，合理使用可视化工具，不仅能帮助技术团队更好地解释模型，还能增强业务团队对模型决策的信任度。

对于某些特定应用，例如金融风控模型和医疗诊断模型，解释性更强的模型被更多地使用，因为这些领域要求模型的决策过程透明且可信任。这意味着，项目中的解释性工具可以有效展示模型决策的逻辑，增强最终用户对决策过程的理解和信任。

综合评价

优点

全面资源：涵盖多种解释性算法，提供从基础理论到实现应用的多层次学习资源。
开放性：大量引用开源工具和代码，方便开发者重复实现和验证。

缺点

复杂性：核心算法和技术实现对于初学者来说可能有较高的学习门槛。
依赖性：不同应用场景下的适用性参差不齐，选择适合的工具和算法需要一定的专业知识。

综上所述，项目 awesome_deep_learning_interpretability 是一个非常好的资源平台，在深度学习模型解释性的研究和应用中，它不仅提供了一个系统的学习框架，也加快了模型解释性的实际应用。

关于

awesome awesome-list chainer computer-vision cvpr deep-learning eccv iccv iclr icml interpretability keras matlab neural-network neurips nlp papers pytorch tensorflow torch

MIT License
768 Stars
113 Forks
27 Watching

简要分析

热度分

1021

价值分

497

活跃状态

活跃

GitHub

贡献者

oneTaken

haofanwang

awesome_deep_learning_interpretability

深度学习近年来关于模型解释性的相关论文。

按引用次数排序可见引用排序

159篇论文pdf(有2篇需要上scihub找)上传到腾讯微云。

不定期更新。

Year	Publication	Paper	Citation	code
2020	CVPR	Explaining Knowledge Distillation by Quantifying the Knowledge	81
2020	CVPR	High-frequency Component Helps Explain the Generalization of Convolutional Neural Networks	289
2020	CVPRW	Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks	414	Pytorch
2020	ICLR	Knowledge consistency between neural networks and beyond	28
2020	ICLR	Interpretable Complex-Valued Neural Networks for Privacy Protection	23
2019	AI	Explanation in artificial intelligence: Insights from the social sciences	3248
2019	NMI	Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead	3505
2019	NeurIPS	Can you trust your model’s uncertainty? Evaluating predictive uncertainty under dataset shift	1052	-
2019	NeurIPS	This looks like that: deep learning for interpretable image recognition	665	Pytorch
2019	NeurIPS	A benchmark for interpretability methods in deep neural networks	413
2019	NeurIPS	Full-gradient representation for neural network visualization	155
2019	NeurIPS	On the (In) fidelity and Sensitivity of Explanations	226
2019	NeurIPS	Towards Automatic Concept-based Explanations	342	Tensorflow
2019	NeurIPS	CXPlain: Causal explanations for model interpretation under uncertainty	133
2019	CVPR	Interpreting CNNs via Decision Trees	293
2019	CVPR	From Recognition to Cognition: Visual Commonsense Reasoning	544	Pytorch
2019	CVPR	Attention branch network: Learning of attention mechanism for visual explanation	371
2019	CVPR	Interpretable and fine-grained visual explanations for convolutional neural networks	116
2019	CVPR	Learning to Explain with Complemental Examples	36
2019	CVPR	Revealing Scenes by Inverting Structure from Motion Reconstructions	84	Tensorflow
2019	CVPR	Multimodal Explanations by Predicting Counterfactuality in Videos	26
2019	CVPR	Visualizing the Resilience of Deep Convolutional Network Interpretations	2
2019	ICCV	U-CAM: Visual Explanation using Uncertainty based Class Activation Maps	61
2019	ICCV	Towards Interpretable Face Recognition	66
2019	ICCV	Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded	163
2019	ICCV	Understanding Deep Networks via Extremal Perturbations and Smooth Masks	276	Pytorch
2019	ICCV	Explaining Neural Networks Semantically and Quantitatively	49
2019	ICLR	Hierarchical interpretations for neural network predictions	111	Pytorch
2019	ICLR	How Important Is a Neuron?	101
2019	ICLR	Visual Explanation by Interpretation: Improving Visual Feedback Capabilities of Deep Neural Networks	56
2018	ICML	Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamples	169	Pytorch
2019	ICML	Towards A Deep and Unified Understanding of Deep Neural Models in NLP	80	Pytorch
2019	ICAIS	Interpreting black box predictions using fisher kernels	80
2019	ACMFAT	Explaining explanations in AI	558
2019	AAAI	Interpretation of neural networks is fragile	597	Tensorflow
2019	AAAI	Classifier-agnostic saliency map extraction	23
2019	AAAI	Can You Explain That? Lucid Explanations Help Human-AI Collaborative Image Retrieval	11
2019	AAAIW	Unsupervised Learning of Neural Networks to Explain Neural Networks	28
2019	AAAIW	Network Transplanting	4
2019	CSUR	A Survey of Methods for Explaining Black Box Models	3088
2019	JVCIR	Interpretable convolutional neural networks via feedforward design	134	Keras
2019	ExplainAI	The (Un)reliability of saliency methods	515
2019	ACL	Attention is not Explanation	920
2019	EMNLP	Attention is not not Explanation	667
2019	arxiv	Attention Interpretability Across NLP Tasks	129
2019	arxiv	Interpretable CNNs	2
2018	ICLR	Towards better understanding of gradient-based attribution methods for deep neural networks	775
2018	ICLR	Learning how to explain neural networks: PatternNet and PatternAttribution	342
2018	ICLR	On the importance of single directions for generalization	282	Pytorch
2018	ICLR	Detecting statistical interactions from neural network weights	148	Pytorch
2018	ICLR	Interpretable counting for visual question answering	55	Pytorch
2018	CVPR	Interpretable Convolutional Neural Networks	677
2018	CVPR	Tell me where to look: Guided attention inference network	454	Chainer
2018	CVPR	Multimodal Explanations: Justifying Decisions and Pointing to the Evidence	349	Caffe
2018	CVPR	Transparency by design: Closing the gap between performance and interpretability in visual reasoning	180	Pytorch
2018	CVPR	Net2vec: Quantifying and explaining how concepts are encoded by filters in deep neural networks	186
2018	CVPR	What have we learned from deep representations for action recognition?	52
2018	CVPR	Learning to Act Properly: Predicting and Explaining Affordances from Images	57
2018	CVPR	Teaching Categories to Human Learners with Visual Explanations	64	Pytorch
2018	CVPR	What do deep networks like to see?	36
2018	CVPR	Interpret Neural Networks by Identifying Critical Data Routing Paths	73	Tensorflow
2018	ECCV	Deep clustering for unsupervised learning of visual features	2056	Pytorch
2018	ECCV	Explainable neural computation via stack neural module networks	164	Tensorflow
2018	ECCV	Grounding visual explanations	184
2018	ECCV	Textual explanations for self-driving vehicles	196
2018	ECCV	Interpretable basis decomposition for visual explanation	228	Pytorch
2018	ECCV	Convnets and imagenet beyond accuracy: Understanding mistakes and uncovering biases	147
2018	ECCV	Vqa-e: Explaining, elaborating, and enhancing your answers for visual questions	71
2018	ECCV	Choose Your Neuron: Incorporating Domain Knowledge through Neuron-Importance	41	Pytorch
2018	ECCV	Diverse feature visualizations reveal invariances in early layers of deep neural networks	23	Tensorflow
2018	ECCV	ExplainGAN: Model Explanation via Decision Boundary Crossing Transformations	36
2018	ICML	Interpretability beyond feature attribution: Quantitative testing with concept activation vectors	1130	Tensorflow
2018	ICML	Learning to explain: An information-theoretic perspective on model interpretation	421
2018	ACL	Did the Model Understand the Question?	171	Tensorflow
2018	FITEE	Visual interpretability for deep learning: a survey	731
2018	NeurIPS	Sanity Checks for Saliency Maps	1353
2018	NeurIPS	Explanations based on the missing: Towards contrastive explanations with pertinent negatives	443	Tensorflow
2018	NeurIPS	Towards robust interpretability with self-explaining neural networks	648	Pytorch
2018	NeurIPS	Attacks meet interpretability: Attribute-steered detection of adversarial samples	142
2018	NeurIPS	DeepPINK: reproducible feature selection in deep neural networks	125	Keras
2018	NeurIPS	Representer point selection for explaining deep neural networks	182	Tensorflow
2018	NeurIPS Workshop	Interpretable convolutional filters with sincNet	97
2018	AAAI	Anchors: High-precision model-agnostic explanations	1517
2018	AAAI	Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients	537	Tensorflow
2018	AAAI	Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions	396	Tensorflow
2018	AAAI	Interpreting CNN Knowledge via an Explanatory Graph	199	Matlab
2018	AAAI	Examining CNN Representations with respect to Dataset Bias	88
2018	WACV	Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks	1459
2018	IJCV	Top-down neural attention by excitation backprop	778
2018	TPAMI	Interpreting deep visual representations via network dissection	252
2018	DSP	Methods for interpreting and understanding deep neural networks	2046
2018	Access	Peeking inside the black-box: A survey on Explainable Artificial Intelligence (XAI)	3110
2018	JAIR	Learning Explanatory Rules from Noisy Data	440	Tensorflow
2018	MIPRO	Explainable artificial intelligence: A survey	794
2018	BMVC	Rise: Randomized input sampling for explanation of black-box models	657
2018	arxiv	Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation	194
2018	arxiv	Manipulating and measuring model interpretability	496
2018	arxiv	How convolutional neural network see the world-A survey of convolutional neural network visualization methods	211
2018	arxiv	Revisiting the importance of individual units in cnns via ablation	93
2018	arxiv	Computationally Efficient Measures of Internal Neuron Importance	10
2017	ICML	Understanding Black-box Predictions via Influence Functions	2062	Pytorch
2017	ICML	Axiomatic attribution for deep networks	3654	Keras
2017	ICML	Learning Important Features Through Propagating Activation Differences	2835
2017	ICLR	Visualizing deep neural network decisions: Prediction difference analysis	674	Caffe
2017	ICLR	Exploring LOTS in Deep Neural Networks	34
2017	NeurIPS	A Unified Approach to Interpreting Model Predictions	11511
2017	NeurIPS	Real time image saliency for black box classifiers	483	Pytorch
2017	NeurIPS	SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability	473
2017	CVPR	Mining Object Parts from CNNs via Active Question-Answering	29
2017	CVPR	Network dissection: Quantifying interpretability of deep visual representations	1254
2017	CVPR	Improving Interpretability of Deep Neural Networks with Semantic Information	118
2017	CVPR	MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network	307	Torch
2017	CVPR	Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering	1686
2017	CVPR	Knowing when to look: Adaptive attention via a visual sentinel for image captioning	1392	Torch
2017	CVPRW	Interpretable 3d human action analysis with temporal convolutional networks	539
2017	ICCV	Grad-cam: Visual explanations from deep networks via gradient-based localization	13006	Pytorch
2017	ICCV	Interpretable Explanations of Black Boxes by Meaningful Perturbation	1293	Pytorch
2017	ICCV	Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention	323
2017	ICCV	Understanding and comparing deep neural networks for age and gender classification	130
2017	ICCV	Learning to disambiguate by asking discriminative questions	26
2017	IJCAI	Right for the right reasons: Training differentiable models by constraining their explanations	429
2017	IJCAI	Understanding and improving convolutional neural networks via concatenated rectified linear units	510	Caffe
2017	AAAI	Growing Interpretable Part Graphs on ConvNets via Multi-Shot Learning	67	Matlab
2017	ACL	Visualizing and Understanding Neural Machine Translation	179
2017	EMNLP	A causal framework for explaining the predictions of black-box sequence-to-sequence models	192
2017	CVPR Workshop	Looking under the hood: Deep neural network visualization to interpret whole-slide image analysis outcomes for colorectal polyps	47
2017	survey	Interpretability of deep learning models: a survey of results	345
2017	arxiv	SmoothGrad: removing noise by adding noise	1479
2017	arxiv	Interpretable & explorable approximations of black box models	259
2017	arxiv	Distilling a neural network into a soft decision tree	520	Pytorch
2017	arxiv	Towards interpretable deep neural networks by leveraging adversarial examples	111
2017	arxiv	Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models	1279
2017	arxiv	Contextual Explanation Networks	77	Pytorch
2017	arxiv	Challenges for transparency	142
2017	ACMSOPP	Deepxplore: Automated whitebox testing of deep learning systems	1144
2017	CEURW	What does explainable AI really mean? A new conceptualization of perspectives	518
2017	TVCG	ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models	346
2016	NeurIPS	Synthesizing the preferred inputs for neurons in neural networks via deep generator networks	659	Caffe
2016	NeurIPS	Understanding the effective receptive field in deep convolutional neural networks	1356
2016	CVPR	Inverting Visual Representations with Convolutional Networks	626
2016	CVPR	Visualizing and Understanding Deep Texture Representations	147
2016	CVPR	Analyzing Classifiers: Fisher Vectors and Deep Neural Networks	191
2016	ECCV	Generating Visual Explanations	613	Caffe
2016	ECCV	Design of kernels in convolutional neural networks for image classification	24
2016	ICML	Understanding and improving convolutional neural networks via concatenated rectified linear units	510
2016	ICML	Visualizing and comparing AlexNet and VGG using deconvolutional layers	126
2016	EMNLP	Rationalizing Neural Predictions	738	Pytorch
2016	IJCV	Visualizing deep convolutional neural networks using natural pre-images	508	Matlab
2016	IJCV	Visualizing Object Detection Features	38	Caffe
2016	KDD	Why should i trust you?: Explaining the predictions of any classifier	11742
2016	TVCG	Visualizing the hidden activity of artificial neural networks	309
2016	TVCG	Towards better analysis of deep convolutional neural networks	474
2016	NAACL	Visualizing and understanding neural models in nlp	650	Torch
2016	arxiv	Understanding neural networks through representation erasure)	492
2016	arxiv	Grad-CAM: Why did you say that?	398
2016	arxiv	Investigating the influence of noise and distractors on the interpretation of neural networks	108
2016	arxiv	Attentive Explanations: Justifying Decisions and Pointing to the Evidence	88
2016	arxiv	The Mythos of Model Interpretability	3786
2016	arxiv	Multifaceted feature visualization: Uncovering the different types of features learned by each neuron in deep neural networks	317
2015	ICLR	Striving for Simplicity: The All Convolutional Net	4645	Pytorch
2015	CVPR	Understanding deep image representations by inverting them	1942	Matlab
2015	ICCV	Understanding deep features with computer-generated imagery	156	Caffe
2015	ICML Workshop	Understanding Neural Networks Through Deep Visualization	2038	Tensorflow
2015	AAS	Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model	749
2014	ECCV	Visualizing and Understanding Convolutional Networks	18604	Pytorch
2014	ICLR	Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps	6142	Pytorch
2013	ICCV	Hoggles: Visualizing object detection features	352

论文talk

暂无评论

awesome_deep_learning_interpretability

技术解读

GitHub 仓库技术分析：oneTaken/awesome_deep_learning_interpretability

项目背景与痛点

核心技术揭秘

技术架构与实现

核心算法

功能亮点与差异

应用场景与落地建议

综合评价

优点

缺点

相关仓库

关于

简要分析

贡献者

awesome_deep_learning_interpretability

评论