Abstract
Biomedical data harmonization is essential for enabling exploratory analyses and meta-studies, but the process of schema matching - identifying semantic correspondences between elements of disparate datasets (schemas) - remains a labor-intensive and error-prone task. Even state-of-the-art automated methods often yield low accuracy when applied to biomedical schemas due to the large number of attributes and nuanced semantic differences between them. We present BDIViz, a novel visual analytics system designed to streamline the schema matching process for biomedical data. Through formative studies with domain experts, we identified key requirements for an effective solution and developed interactive visualization techniques that address both scalability challenges and semantic ambiguity. BDIViz employs an ensemble approach that combines multiple matching methods with LLM-based validation, summarizes matches through interactive heatmaps, and provides coordinated views that enable users to quickly compare attributes and their values. Our method-agnostic design allows the system to integrate various schema matching algorithms and adapt to application-specific needs. Through two biomedical case studies and a within-subject user study with domain experts, we demonstrate that BDIViz significantly improves matching accuracy while reducing cognitive load and curation time compared to baseline approaches.
1. Introduction
In biomedical research, data is frequently siloed across disparate repositories, each employing unique naming conventions and structures. To perform large-scale meta-studies, researchers must harmonize these datasets—a process known as schema matching. For instance, determining if age_at_index in one dataset corresponds to age_at_diagnosis in another is a critical yet complex task.
Traditional automated tools often struggle with the nuanced terminology of biomedical data. Consequently, experts frequently resort to manual mapping, a process that is tedious, error-prone, and fails to scale as schemas grow to include hundreds of attributes. BDIViz addresses these challenges by introducing a human-in-the-loop system that facilitates collaboration between AI and domain experts.
2. System Architecture
BDIViz is designed as a comprehensive pipeline that bridges automated matching with expert curation. The system architecture consists of an automated backend and a human-in-the-loop frontend.
The backend utilizes a Matcher Ensemble approach. Rather than relying on a single algorithm, BDIViz combines multiple matchers (e.g., contrastive learning, Jaccard distance). The system dynamically adjusts matcher weights based on user feedback, such as accepting or rejecting proposed matches. Furthermore, an LLM Agent provides Retrieval-Augmented Generation (RAG) to offer "Chain-of-Thought" explanations, clarifying the rationale behind suggested matches (e.g., identifying shared cancer stage codes).
3. Interactive Visualization Interface
The BDIViz frontend is designed to handle the scalability and semantic ambiguity inherent in biomedical schemas. It provides several coordinated views to support expert decision-making.
Key interface components include:
- Interactive Heatmap: A matrix-based view that summarizes potential matches, offering better scalability than traditional node-link diagrams.
- Agent Panel: Displays LLM-powered validation results, including confidence scores and natural language explanations for semantic, value-based, and historical matches.
- Value Distribution Views: Coordinated histograms and tables allow users to compare actual data values side-by-side, providing empirical evidence for matches that may have ambiguous names.
- Control Panel: Enables users to filter matches by confidence thresholds and track their history of actions.
4. Evaluation and Results
The system was evaluated using high-stakes biomedical contexts, including the Genomic Data Commons (GDC) schema (700+ attributes) and datasets from the CPTAC study. A within-subject user study with 12 domain experts compared BDIViz against traditional manual methods (e.g., Excel or Python scripts).
The results demonstrated significant improvements:
- Accuracy: BDIViz improved matching accuracy by 50%.
- Efficiency: Task completion time was reduced by over 14%.
- Cognitive Load: NASA-TLX scores indicated a substantial decrease in frustration and mental demand.
- Robustness: The ensemble approach achieved a 93.8% Precision@10, outperforming any single matcher.
In a real-world case study, an expert using BDIViz identified 10 additional correct matches in just 15 minutes—matches that had been overlooked during a previous months-long manual curation effort. Additionally, the system's "Easy Match" heuristic successfully captured 35% of correct matches automatically, significantly reducing initial workload.
5. Discussion and Future Work
While BDIViz significantly enhances the schema matching workflow, it remains dependent on the underlying matcher ensemble; if no algorithm identifies a potential link, it may result in a false negative. Additionally, while LLM explanations are highly valued by experts, they require vigilance to guard against potential hallucinations.
Future work will focus on supporting "many-to-one" harmonization—merging multiple source datasets into a single target schema simultaneously—and refining search mechanisms to further reduce false negatives in extremely large schemas.