Auditable Hierarchical Inference with Multimodal Evidence and Bayesian Active Inquiry

Yucheng Liu, Everett X. WangarXiv 2026 · arXiv

This paper presents a framework for moving beyond "black-box" medical classifiers toward an auditable, sequential inquiry system. By separating objective multimodal evidence from active questioning, the authors demonstrate how a system can build a coarse diagnostic prior and then refine it through cost-sensitive Bayesian inquiry.

TL;DR

The researchers developed a synthetic framework that mimics clinical reasoning by first analyzing objective data (tongue, pulse, audio) to form a "syndrome family" prior, then using cost-sensitive active inquiry to narrow down specific syndrome terms. While the system significantly improved accuracy through questioning, it highlighted the difficulty of balancing diagnostic precision with the "burden" of asking too many questions.

What Problem Does This Paper Address?

Most multimodal AI classifiers assume that all relevant data—images, vitals, history—is available at once. In reality, clinical inquiry is a conversation. A doctor observes a patient (passive/objective evidence) and then decides which questions are worth asking (active inquiry).

The authors address four critical gaps in current medical AI:

  1. Auditability: Can we see exactly how a prior belief was formed and how each new answer changed it?
  2. Sequential Logic: Can the system decide which observation to acquire next based on current uncertainty?
  3. Hierarchy: Can the system maintain uncertainty across a hierarchy (e.g., knowing the "family" of a disease even if the specific "term" is unclear)?
  4. Abstention: Does the system know when to stop or refuse to predict because evidence is insufficient?

Methods

The study uses a modular architecture that separates "authorities":

  • Objective-Inference Module: A multimodal model that processes synthetic tongue images, acoustic data (coughing, breathing), and a complex "pressure–space–time pulse tensor" (representing 28 distinct pulse qualities). This module predicts one of eight "syndrome families."
  • Knowledge Core: A structured, provenance-bearing resource that links symptoms to syndrome terms. It acts as the "brain" for the inquiry phase.
  • Bayesian Active Inquiry: This layer uses Cost-Sensitive Expected Information Gain (cost-EIG). It doesn't just pick the most informative question; it balances information gain against the "cost" or burden of asking that question.
  • Safety Firewall: Seven specific "warning-sign" questions are isolated. Their answers can trigger safety actions but are strictly prohibited from altering the diagnostic posterior, preventing "safety noise" from corrupting the statistical model.

Datasets

The study relies entirely on a synthetic dataset designed to simulate Traditional Chinese Medicine (TCM) syndrome differentiation.

  • Scale: 2,400 training cases, 600 development cases, and a "frozen" 600-case confirmation split.
  • Labels: 8 syndrome families and 33 fine-grained syndrome terms.
  • Nature: The data includes synthetic tongue images, audio WAV files, and pulse tensors. This "bounded synthetic setting" allows the authors to inject failures and test the system's limits without the ethical risks of real-world clinical trials.

Key Insights and Ideas

  • The "Abstention" Invariant: One of the most robust features is a workflow rule: if no objective modalities are present, the system must abstain. It is not allowed to "manufacture" a prior out of thin air.
  • Personalization vs. Prior: The system showed a "personalization effect" where the objective evidence changed the first question asked in 8.5% of cases. This suggests the multimodal prior successfully guides the inquiry path.
  • Failure Injection as a Feature: The authors didn't just test success; they intentionally broke the system. For example, permuting the "Knowledge Core" (linking the wrong symptoms to terms) caused the largest drop in accuracy, proving that the system’s performance is genuinely tied to its underlying knowledge structure rather than just pattern matching.
  • Hierarchy Matters: The system was much better at identifying the "family" (95.5% accuracy) than the "fine term" (42.5%). This highlights that even "smart" inquiry has limits when distinctions are subtle.

Key Results

The "frozen" confirmation (a one-time test on unseen data) yielded several descriptive findings:

  • Accuracy Gains: The objective-only model started at 87.0% family accuracy. Adding the cost-EIG inquiry boosted this to 95.5%.
  • Policy Comparison: Cost-EIG tied with unpenalized EIG for the best fine-term accuracy (42.5%) but did so using fewer questions on average.
Table 1: Policy comparisons showing accuracy and question counts.
  • The Efficiency Failure: Interestingly, the "question-efficiency" hypothesis failed. While cost-EIG was better than random or greedy EIG, it actually asked more questions (9.795) than a simple fixed-order baseline (9.352). This suggests that being "cost-sensitive" in selection doesn't automatically result in the shortest path to a conclusion.
  • Robustness: The system handled missing audio well, retaining 94.0% family accuracy by routing through specialized "expert" sub-models.
Table 2: Targeted, descriptive confirmation stress tests. Each row changes one declared interface relative to nominal cost-EIG. “Term-map permutation” permutes fine-term answer-likelihood rows and is not arbitrary Knowledge Core corruption. All-objective-missing is omitted because it was evaluated as a workflow invariant rather than a case-derived metric.

Limitations and Future Work

The authors are careful to state that this is a synthetic workflow result, not a claim of clinical validity.

  • Synthetic Gap: Because all data is synthetic, the cross-modal coherence might be overstated compared to messy real-world clinical data.
  • No Natural Language: The current system uses structured, typed answers. A future version would need to integrate Large Language Models (LLMs) to handle free-text patient responses without letting the LLM "hallucinate" diagnostic probabilities.
  • Fixed Order Baseline: The fixed-order comparison used an unoptimized sequence. A more competitive baseline (like a professionally designed medical intake form) might be harder to beat.
  • Expert Validation: The underlying ontology and question contracts have not yet been validated by external medical experts.

Summary updated

Abstract

Most multimodal classifiers assume that all useful evidence is available upfront, whereas clinical inquiry is sequential. We present a synthetic evaluation framework separating objective multimodal inference from active inquiry. An objective model maps synthetic tongue, bilateral multipressure pulse, and acoustic observations to eight syndrome families. A provenance-bearing Knowledge Core and training-only answer model support Bayesian narrowing over 33 fine terms using 25 typed questions, while seven safety questions remain outside the diagnostic posterior. Weak shared symptom factors limit fine-term influence on objective phenotypes, and an objective-only probe tests leakage. In a preregistered 600-case synthetic evaluation, objective family accuracy was 87.0%. Cost-sensitive expected information gain (cost-EIG) raised family accuracy to 95.5% and exact fine-term accuracy to 42.5%, versus 92.7% and 33.0% for an unoptimized fixed-order baseline. Selective family accuracy reached 96.9% at 95.3% coverage. However, cost-EIG asked more questions on average than fixed order (9.795 versus 9.352). Missing-audio routing retained 94.0% family accuracy, and a workflow invariant enforced abstention when all objective modalities were absent. Permuting fine-term answer likelihoods reduced family accuracy to 78.7%. Five of six frozen checks passed, but the question-efficiency hypothesis failed. Comparisons are descriptive because only aggregate, not paired case-level, outputs were retained. These results establish a reproducible synthetic workflow, not clinical diagnostic or treatment validity.
BibTeX
@misc{liu2026auditable,
  author = {Yucheng Liu and Everett X. Wang},
  title = {{Auditable Hierarchical Inference with Multimodal Evidence and Bayesian Active Inquiry}},
  year = {2026},
  eprint = {2610.38774},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.38774},
}
Figure