Edition #9: How visualizations help make sense of deep learning
Using viz to answer why/how/what questions in ML/AI
🗃️ How visualizations make sense of deep learning
When viz are used to make sense of deep learning models, whether for exploration, interpretation or diagnosis, they’re often used to address one of these questions:
why - why does the model work (interpretability)
how - how well might the model work (on different subsets of data)
what - what’s the performance of the model
This edition we will look at some notable viz tools for these purposes.
🍱 Interpretability
Besides generic packages like SHAP and LIME, many interpretability tools are domain-specific.
👀 Interpretability models in Vision
CNN explainer
As a standard-alone application, it explains via interactive graphics how different Conv layers work generically.
This package applies various image models explanation methods (such as Grad CAM, occlusion sensitivity, smoothGrad, and integrated gradients), and can be applied either during training as a callback or on a loaded model
Feature visualization of CNN layers. The Lucid package doesn’t support tf2 yet - its Lucent counterpart for PyTorch might be more updated.
💬 NLP
RNNvis
A prototype tool to understand the hidden state of RNN by co-clustering the bipartite graph of hidden states and the word space.
BertViz
A python package offering the birds’ eyes view of attention across layers and headers, as well as query and key values of a particular attention head.
A python package with several interactive illustrations of NLP models, such as the one below which uses non-negative matrix factorization to visualize how groups of neurons spike in response to inputs.
With an example on Question-Answering system, this tool demonstrated its usage for scalable error analysis through a querying language and counterfactual inspections.
LiT (language interpretability tool)
This browser-based interface can be used to explore the behaviors of multiple NLP models, understand what kind of example does the NLP model performs poorly on, and if the prediction is attributed to adversarial behavior or due to training data priors. It has comparison mode for different models/examples. What’s special about it is being framework agnostic (support Tensorflow/PyTorch and others).
🥊 How well might the model work?
This mainly could involve evaluating the model on different slices of data, individual datapoint, evaluate fairness, and adversarial robustness.
What-if tools
A small multiples galore with a UI that can be integrated with notebook or TensorBoard. A slice-and-dice tool to segment various features and inference scores. There’s some functionality to optimize for fairness. It offers counterfactual for individual data points, but doesn’t seem to provide it for groups of data.
Tf data validation
It visualizes the distribution stats of the training data and can detect drifts between training and serving data.
🎯 Performance monitoring
Tensorboard
Besides measuring performance metrics, having plugins such as the embedding projector, also has a debugger mode which helps identify numerical issues.
The Embedding Projector
Ludwig’s comparison Viz
Ludwig is an autoDL tool aiming to make deep learning more accessible. Besides its large selection of models supported, It also has some unique visualization options (with a slight focus on classification). Those options include multiple ways of comparing classification models – such as when both models got the prediction right, both got the prediction wrong, and one model got it right, visualize multiclass probabilities of predictions for each class, and confidence threshold surface.
HiPlot
Interactive parallel plots showing which combination of parameters render the best outcome
🚄 Visualizing Training Data
Being able to debug training data is equally important. Here’re some tools to understand the characters of dataset better.
Facets
Overall, we see the use cases of some viz tools. It’ll be exciting to see what comes next in the interplay between DL and Viz.
📚 Recent Reads
This is a section I share about the books, papers, or blogs on data visualization, data science, or others.
Deep Learning with Python by Francois Chollet
This book is good for developing some intuition on the inner working of deep learning algorithms. It’s practical and accessible. As with all practical books, one needs to try the techniques out to internalize what works and what doesn’t in different scenarios.
Data Science on GCP by Valliappa Lakshmanan
This book cover examples of using different gcp tools such as dataflow, dataproc and AI platform etc to create and implement models at scale. I read it mainly because my workplace uses gcp.