Abstract
Multi-instance multi-label (MIML) learning is a powerful paradigm for objects described by multiple instances (e.g., image patches) and simultaneously associated with multiple labels. Existing MIML methods suffer from relatively low accuracy and training efficiency due to several issues: neglected inter-label correlations, the inability to learn inter-instance correlations directly due to missing instance labels, and the requirement of multiple stages to learn diverse correlations. To resolve these issues, we propose Broad Multi-Instance Multi-Label Learning (BMIML), a single-stage framework. BMIML incorporates three innovative modules: Auto-Weighted Label Enhancement Learning (AWLEL) to capture inter-label correlations; Scalable Multi-Instance Probabilistic Regression (SMIPR) to effectively estimate inter-instance correlations using only object labels; and Interactive Decision Optimization (IDO) to combine and optimize results into a single-stage framework. Experiments demonstrate that BMIML is highly competitive in accuracy and significantly faster than existing methods, even for large medical image datasets exceeding 90,000 images.
Introduction: The Complexity of Medical Image Diagnosis
Medical image diagnosis is rarely a simple "one image, one label" task. A single pathology slide or X-ray often contains multiple regions of interest (instances) and can be associated with several different clinical findings (multi-label). This is known as Multi-Instance Multi-Label (MIML) learning.
Existing MIML methods face significant hurdles. They often ignore how different labels relate to each other (e.g., certain diseases often co-occur) and struggle with missing local labels for individual image patches. Furthermore, most current frameworks are multi-stage and computationally expensive, making them slow to train on the large-scale datasets required for modern clinical AI. As shown in the comparison below, BMIML is uniquely designed to capture a comprehensive set of inter-correlations that other methods neglect.
The BMIML Framework
BMIML is built upon the Broad Learning System (BLS)—a flat network architecture known for its extreme efficiency compared to deep "tall" networks. The framework integrates global and local views into a single-stage optimization process.
Core Modules
- Auto-Weighted Label Enhancement Learning (AWLEL): This module captures inter-label correlations using a retargeting layer to adaptively adjust label weights, helping the model handle imbalanced data and focus on hard-to-distinguish classes.
- Scalable Multi-Instance Probabilistic Regression (SMIPR): This module estimates inter-instance correlations. It uses a clustering approach to group similar image bags, allowing the model to estimate the probability that a specific patch belongs to a certain class using only the overall image label.
- Interactive Decision Optimization (IDO): This module fuses the global view (AWLEL) and the local view (SMIPR) into a single-stage optimization process, preventing information loss typical of disjoint multi-stage steps.
Experimental Results and Scalability
The model was evaluated on five diverse medical imaging datasets, including the NIH Chest X-ray dataset (over 112,000 images) and the ODR retinal fundus dataset.
- Accuracy: BMIML outperformed seven state-of-the-art methods (including DeepMIML and MIMLSVM) across almost all metrics. On the NIH dataset, it achieved an Average Precision of 0.661, significantly higher than competitors.
- Efficiency: BMIML was found to be 16.56% to 99.99% faster than existing algorithms. For example, on the ODR dataset, BMIML trained in roughly 19 hours, while some competitors required over 72 hours or failed to return results entirely.
- Big Data Suitability: Unlike many competitors that "timed out" on the 112K-image NIH dataset, BMIML maintained stable performance and fast training times, showing a clear trend of improving performance as data size increased.
Conclusion and Future Work
BMIML provides a novel, efficient, and accurate framework for complex medical image classification. By integrating diverse inter-correlations into a single-stage broad learning architecture, it overcomes the limitations of traditional multi-stage MIML methods. Future research will explore more sophisticated instance generation techniques, such as ROI detectors, and test the framework's generalization on other complex multi-label data types like video annotation.
Summary updated