HypoML: Visual Analysis for Hypothesis-based Evaluation of Machine Learning Models

Qianwen Wang, William Alexander, Jack Pegg, Huamin Qu, Min Chen

Introduction: The Challenge of Model Interpretation

Evaluating machine learning (ML) models typically falls into two categories: black-box analysis, which focuses on aggregate performance metrics like accuracy, and white-box analysis, which involves inspecting internal states such as neuron activations.

While aggregate metrics provide a high-level view, they fail to explain why a model succeeds or fails. Conversely, white-box methods can be overwhelming due to the complexity of modern neural networks, where millions of neurons rarely correspond to clear, semantically interpretable concepts.

Traditional white-box analysis often involves inspecting complex internal activations, which can be difficult to interpret semantically.

Developers often have intuitive "hunches" about model behavior—for instance, suspecting a failure to handle rotated objects—but lack a formal, automated method to validate these hypotheses across entire datasets. HypoML bridges this gap by providing a structured framework for testing "concept-based hypotheses."

The HypoML Framework

HypoML introduces a visual analytics workflow that transforms raw performance data into interpretable conclusions about a model's internal knowledge. The core methodology relies on a comparative testing paradigm.

1. Comparative Testing Workflow

To evaluate a specific concept (e.g., "rotation normalization"), the system utilizes a four-step process:

  1. Hypotheses Formulation: Defining what it means for a model to "know" a concept.
  2. Structured Testing: Comparing a base model ($M$) with an augmented model ($M+$) trained on data that explicitly includes the target concept as "extra information."
  3. Statistical Analysis: Performing six pairs of statistical comparisons (using paired t-tests) between the results of the models on standard and augmented data.
  4. Logical Inference: Feeding statistical outcomes into a reasoning engine to support or reject high-level hypotheses.
The HypoML workflow: from defining hypotheses about concepts to logical inference based on statistical comparisons.

2. Statistical and Logical Engine

The framework evaluates 12 predefined hypotheses ($H_1$ through $H_{12}$) addressing whether a concept is useful, if it has already been learned implicitly, or if the model relies too heavily on the provided extra information. For example, if $M+$ performs significantly better than $M$ when both are provided with extra information, the system concludes that the concept is "useful" ($H_1$).

3. Visual Interface

The tool features a matrix-based visualization mapping statistical comparisons to the 12 hypotheses. This "logic flow" design allows users to trace how specific p-values lead to the support, rejection, or "unproven" status of hypotheses, explicitly accounting for statistical uncertainty.

Evaluation and Key Insights

The researchers evaluated HypoML using the Fashion MNIST dataset, creating concept-based variants for rotation, scaling, and pixel intensity.

  • Rotation Invariance: The analysis confirmed that standard CNNs are not naturally rotation-invariant. $M+$ performed significantly better, proving the concept was useful ($H_1$) and that the original model had not learned it adequately ($H_3$ rejected).
  • Intensity Normalization: Interestingly, the base model ($M$) showed improved performance when intensity info was present in test data ($H_5$ supported), even though it wasn't trained to use it. This suggests the "extra info" reinforced existing signals the model was already utilizing.
  • The "M vs M+" Paradigm: By comparing a model trained with the answer against one trained without it, HypoML creates a controlled environment to quantify the value of "knowing" a specific concept.

Limitations and Future Directions

While powerful, HypoML is currently limited to 12 fixed hypotheses and requires concepts to be explicitly encodable as extra data—a task that is straightforward for geometric properties like rotation but challenging for abstract concepts like "style." Future work aims to allow user-defined logical rules and better integration with instance-level inspection tools to identify specific failure cases.

Summary updated

BibTeX
@article{wang2021hypoml,
  author = {Qianwen Wang and William Alexander and Jack Pegg and Huamin Qu and Min Chen},
  title = {{HypoML: Visual Analysis for Hypothesis-based Evaluation of Machine Learning Models}},
  journal = {IEEE Trans. Vis. Comput. Graph.},
  year = {2021},
  doi = {10.1109/tvcg.2020.3030449},
}
Figure