PNNL

Abstract

In this paper, we propose a visual analytics workflow to help data scientists explore, diagnose, and understand the decisions made by a binary classifier. The approach leverages “instance-level explanations”, measures of local feature relevance that explain single instances and uses them to build a set of visual representations that guide the users in their investigation. The workflow is based on three main visual representations and steps: one based on aggregate statistics to see how data distributes across correct or incorrect decisions; one based on explanations to understand which features are used to make these decisions; and one based on raw data, to derive insights on potential root causes for the observed patterns. The work is validated through a long-term collaboration with a group of healthcare professionals who used our method to make sense of machine learning models they developed. The case studies from this collaboration demonstrate that the proposed method helps experts derive useful knowledge about the model and the phenomena it describes and also generates useful hypotheses on how a model can be improved.